Running S3 Object Storage Locally with MinIO
Object storage is amazingly versatile. We normally associate it with AWS S3, but Object Storage can also be run on your private network with MinIO. In this article, I’ll show you how to install MinIO using Docker and also on a Mac using homebrew.
MinIO has an API which is compatible with Amazon S3. This will allow you to develop applications locally and later, easily port them to AWS or you can use MinIO as your primary object store in production.
Prerequisites
You can code along with this tutorial on any operating system If you have Docker and docker-compose.
If you have a Mac you will optionally have to have homebrew installed.
If you want to run the demo application at the end of the tutorial, you will need NodeJS installed.
Installation
Install MinIO using Homebrew
Homebrew is the fastest way to get MinIO up and running if you have a Mac. If you don’t have a Mac, follow the Docker tutorial below.
brew install minio/stable/minio
I’m also going to install the mc
CLI tool
brew install minio/stable/mc
Create a MinIO server instance
Creating a new server instance is quite simple. Run the command below in a terminal.
minio server /path/to/some/dir
The first time you run this command, you’re going to get nagged about default ACCESS_KEY
and SECRET_KEY
. Let’s change them now.
Hit Ctrl+c
to shut down the server.
export ACCESS_KEY=admin
export ACCESS_SECRET=password
Now run the server again
minio server /path/to/some/dir
Installing MinIO using Docker
This is my favoured approach. I have a Synology NAS, which I use to host Docker Containers and more. For MinIO it gives me plenty of storage space and access to S3 across my network.
In this tutorial, I will use a docker-compose file so my instructions will work on any operating system you may be running. If you don’t have Docking installed, take a look at https://www.docker.com/
Create a new folder and create a docker-compose.yml
file with the following contents.
Save the file and then the terminal, inside the same folder where the docker-compose file is located; run
docker-compose up
The above command will bring 4 MinIO containers as a cluster. Each will have admin/password
as credentials and the web interface accessible on
http://localhost:9001
http://localhost:9002
http://localhost:9003
http://localhost:9004
Any data that is written or deleted from any one of the above instances will be replicated to the other 3.
In the subsequent sections, I will discuss accessing MinIO on port 9000
as that is the port created when installed using homebrew
. If you have installed MinIO using Docker, just change port 9000 for one of the ports listed above.
Other operating systems
The MinIO docker hub repo contains further details for installing MinIO. It also covers other operating systems such as Linux and Windows. https://registry.hub.docker.com/r/minio/minio/
The MinIO Web Interface
This section of the tutorial assumes you used homebrew to install MinIO. If you used Docker, simply change the port number 9000
to 9001
In the terminal window where you started the MinIO server, you will see a list of web endpoints. You can use any of them. I will use http://localhost:9000
Visit this address in your browser.
Enter admin/password
for the ACCESS Key/Secret Key
The red button at the bottom right allows you to create new Buckets and upload files. I am going to create a new bucket called testing
and upload a file.
The file I uploaded is from https://www.bensound.com. I don’t think I need to give credit in this case, but I think it’s polite.
The ellipse to the right of the file allows Downloading
, Deleting
and time-limited link sharing
.
mc CLI tool
Let’s take a look at the MinIO CLI tool, mc
Creating an alias
Before we can use mc
, we need to create an alias to the MinIO server endpoint.
In the same terminal where your MinIO instance is running hit Ctrl+c to close the server. Then enter the following command in the terminal
mc alias set localS3 http://192.168.30.100:9000
I have used localS3
as the alias to the MinIO instance URL. You will need to change the IP address or use http://localhost:9000
Now start the server again
Listing buckets and files in the CLI
Keep the server running and open another terminal window.
Enter the following command
mc ls localS3
You should see the bucket testing
that was created in the browser. The above command lists everything in the root of the data store.
To view the file that was uploaded
mc ls localS3/testing
and you should see
Deleting files and buckets with mc
The two commands used to remove buckets and files are
rm
: Removes a filerb
: Removes a bucket
I am going to remove the file first
mc rm locals3/testing/bensound-goinghigher.mp3
and now to remove the bucket
mc rb localS3/testing
If you again issue the command mc ls localS3
you will find that no results are returned.
Further mc commands and built in help
There is a lot more you can do with mc
the built-in documentation is really helpful. To access the docs simply type mc —-help
to get help on a specific command like ls
type mc ls --help
Using AWS-SDK
with MinIO
As MinIO has an API compatible with AWS, it’s possible to use the AWS-SDK
in your code. Even if your planning to use MinIO in production, using the AWS-SDK allows for a seamless transition to AWS at a future date.
I’m going to demo the AWS-SDK in a small node app.
In a terminal, issue the following commands
mkdir minio_demo
cd minio_demo
npm init -y
npm install aws-sdk
Create a new file app.js
, open it in your editor, and add the following code.
Change the endpoint to suit your MinIO store, if you followed the Docker section you can use any port between 9001
and 9004
If you run the app with the following command
node app.js
You should get the following result.
The above code creates a new bucket called awstest
. It then creates an object with the contents Hello from MinIO!
and uploads it to the bucket.
After creating the bucket and object, the SDK is used again to download the object and save it to a local file in your app folder. Take a look, there should be a file called test_from_minio.txt
Summary
You have learned that MinIO is an Object Storage server with an AWS compatible API.
You can now install MinIO using Docker or homebrew on a Mac
You have installed mc
the MinIO CLI and seen how it can be used to manage your MinIO instance.
You have seen how the aws-sdk can be used in a NodeJS script to interact with MinIO