Running S3 Object Storage Locally with MinIO

Object storage is amazingly versatile. We normally associate it with AWS S3, but Object Storage can also be run on your private network with MinIO. In this article, I’ll show you how to install MinIO using Docker and also on a Mac using homebrew.

MinIO has an API which is compatible with Amazon S3. This will allow you to develop applications locally and later, easily port them to AWS or you can use MinIO as your primary object store in production.

Prerequisites

You can code along with this tutorial on any operating system If you have Docker and docker-compose.

If you have a Mac you will optionally have to have homebrew installed.

If you want to run the demo application at the end of the tutorial, you will need NodeJS installed.

Installation

Install MinIO using Homebrew

Homebrew is the fastest way to get MinIO up and running if you have a Mac. If you don’t have a Mac, follow the Docker tutorial below.

brew install minio/stable/minio

I’m also going to install the mc CLI tool

brew install minio/stable/mc

Create a MinIO server instance

Creating a new server instance is quite simple. Run the command below in a terminal.

minio server /path/to/some/dir

The first time you run this command, you’re going to get nagged about default ACCESS_KEY and SECRET_KEY. Let’s change them now.

Hit Ctrl+c to shut down the server.

export ACCESS_KEY=admin
export ACCESS_SECRET=password

Now run the server again

minio server /path/to/some/dir

Installing MinIO using Docker

This is my favoured approach. I have a Synology NAS, which I use to host Docker Containers and more. For MinIO it gives me plenty of storage space and access to S3 across my network.

In this tutorial, I will use a docker-compose file so my instructions will work on any operating system you may be running. If you don’t have Docking installed, take a look at https://www.docker.com/

Create a new folder and create a docker-compose.yml file with the following contents.

Save the file and then the terminal, inside the same folder where the docker-compose file is located; run

docker-compose up

The above command will bring 4 MinIO containers as a cluster. Each will have admin/password as credentials and the web interface accessible on

http://localhost:9001
http://localhost:9002
http://localhost:9003
http://localhost:9004

Any data that is written or deleted from any one of the above instances will be replicated to the other 3.

In the subsequent sections, I will discuss accessing MinIO on port 9000 as that is the port created when installed using homebrew. If you have installed MinIO using Docker, just change port 9000 for one of the ports listed above.

Other operating systems

The MinIO docker hub repo contains further details for installing MinIO. It also covers other operating systems such as Linux and Windows. https://registry.hub.docker.com/r/minio/minio/

The MinIO Web Interface

This section of the tutorial assumes you used homebrew to install MinIO. If you used Docker, simply change the port number 9000 to 9001

In the terminal window where you started the MinIO server, you will see a list of web endpoints. You can use any of them. I will use http://localhost:9000 Visit this address in your browser.

MinIO Web UI login screen

Enter admin/password for the ACCESS Key/Secret Key

MinIO Web Browser Interface

The red button at the bottom right allows you to create new Buckets and upload files. I am going to create a new bucket called testing and upload a file.

MinIO Web Browser after creating a bucket and file

The file I uploaded is from https://www.bensound.com. I don’t think I need to give credit in this case, but I think it’s polite.

The ellipse to the right of the file allows Downloading, Deleting and time-limited link sharing.

mc CLI tool

Let’s take a look at the MinIO CLI tool, mc

Creating an alias

Before we can use mc, we need to create an alias to the MinIO server endpoint.

In the same terminal where your MinIO instance is running hit Ctrl+c to close the server. Then enter the following command in the terminal

mc alias set localS3 http://192.168.30.100:9000

I have used localS3 as the alias to the MinIO instance URL. You will need to change the IP address or use http://localhost:9000

Now start the server again

Listing buckets and files in the CLI

Keep the server running and open another terminal window.

Enter the following command

mc ls localS3

You should see the bucket testing that was created in the browser. The above command lists everything in the root of the data store.

Listing buckets

To view the file that was uploaded

mc ls localS3/testing

and you should see

Listing files inside buckets

Deleting files and buckets with mc

The two commands used to remove buckets and files are

• rm : Removes a file
• rb : Removes a bucket

I am going to remove the file first

mc rm locals3/testing/bensound-goinghigher.mp3

removing a file from a bucket

and now to remove the bucket

mc rb localS3/testing

removing a bucket

If you again issue the command mc ls localS3 you will find that no results are returned.

Further mc commands and built in help

There is a lot more you can do with mc the built-in documentation is really helpful. To access the docs simply type mc —-help to get help on a specific command like ls type mc ls --help

Using AWS-SDK with MinIO

As MinIO has an API compatible with AWS, it’s possible to use the AWS-SDK in your code. Even if your planning to use MinIO in production, using the AWS-SDK allows for a seamless transition to AWS at a future date.

I’m going to demo the AWS-SDK in a small node app.

In a terminal, issue the following commands

mkdir minio_demo
cd minio_demo
npm init -y
npm install aws-sdk

Create a new file app.js, open it in your editor, and add the following code.

Change the endpoint to suit your MinIO store, if you followed the Docker section you can use any port between 9001 and 9004

If you run the app with the following command

node app.js

You should get the following result.

The result from running the demo node app

The above code creates a new bucket called awstest. It then creates an object with the contents Hello from MinIO! and uploads it to the bucket.

After creating the bucket and object, the SDK is used again to download the object and save it to a local file in your app folder. Take a look, there should be a file called test_from_minio.txt

Summary

You have learned that MinIO is an Object Storage server with an AWS compatible API.

You can now install MinIO using Docker or homebrew on a Mac

You have installed mc the MinIO CLI and seen how it can be used to manage your MinIO instance.

You have seen how the aws-sdk can be used in a NodeJS script to interact with MinIO