How to Setup a 4-Node Distributed MinIO Cluster on CentOS 7

MinIO is a highly performant, open-source, S3-compatible object storage server. When deployed in distributed mode across multiple nodes, MinIO pools storage drives into a single high-availability object store, protecting data against hardware failures using erasure coding.

In this guide, I will walk you through setting up a 4-node distributed MinIO cluster on CentOS 7, configuring dedicated XFS storage drives, establishing systemd service configuration, managing users via the MinIO Client (mc), and mounting S3 buckets locally using s3fs-fuse.

Prerequisites

You will need four CentOS 7 virtual machines with root access. Each VM requires a dedicated secondary disk (/dev/sdb) for object storage.

Cluster Node Inventory

---

Step 1: Base System Update and Package Installation

First, update the system packages and install required utilities, including open-vm-tools and network monitoring tools.

yum update -y
yum install epel-release -y
yum install open-vm-tools bind-utils htop iftop net-tools wget -y

Disable SELinux in /etc/selinux/config:

sed -i 's/^SELINUX=.*/SELINUX=disabled/g' /etc/selinux/config

Reboot the server for changes to take effect:

reboot

---

Step 2: Disk Partitioning and XFS Filesystem Formatting

MinIO requires dedicated, unformatted drives to store object data. We will format the secondary drive /dev/sdb with the XFS filesystem and mount it at /data.

Execute the following commands on all four nodes:

# Create a new partition on /dev/sdb
parted -s /dev/sdb mklabel gpt mkpart primary xfs 0% 100%

# Format partition as XFS
mkfs.xfs -f /dev/sdb1

# Create mount directory
mkdir -p /data

# Add persistent mount entry to /etc/fstab
echo "/dev/sdb1 /data xfs defaults,noatime,nofail 0 0" >> /etc/fstab

# Mount the drive and verify
mount /data
df -h /data

---

Step 3: Firewall and Hosts Configuration

MinIO communicates over port 9000 by default. Open port 9000/tcp in the firewall on all nodes:

firewall-cmd --zone=public --add-port=9000/tcp --permanent
firewall-cmd --reload && firewall-cmd --list-all

Next, configure /etc/hosts on all four nodes so they can resolve each other's hostnames correctly:

cat >> /etc/hosts << "EOF"
192.16.10.10 west-dc1-minserver1.induslevel.com
192.16.10.11 west-dc1-minserver2.induslevel.com
192.16.20.10 east-dc1-minserver1.induslevel.com
192.16.20.11 east-dc1-minserver2.induslevel.com
EOF

---

Step 4: Install MinIO Server Binary

Download the official MinIO binary and place it in /usr/local/bin:

wget https://dl.min.io/server/minio/release/linux-amd64/minio -O /usr/local/bin/minio
chmod +x /usr/local/bin/minio

---

Step 5: Configure Systemd Service and Cluster Environment

We need to create a persistent systemd service file and an environment configuration file containing our credentials and cluster endpoints.

Create Environment File

Execute the following on all four nodes (ensure you use strong, matching access and secret keys):

cat > /etc/default/minio << "EOF"
MINIO_OPTS="http://west-dc1-minserver1.induslevel.com:9000/data http://west-dc1-minserver2.induslevel.com:9000/data http://east-dc1-minserver1.induslevel.com:9000/data http://east-dc1-minserver2.induslevel.com:9000/data"
MINIO_ACCESS_KEY="AKaHEgQ4II0S7BjT6DjAUDA4BZ"
MINIO_SECRET_KEY="SKFzHq5iDoQgF7gyPYRFhzNMYSvY6ZFMpG"
EOF

Create Systemd Service File

cat > /lib/systemd/system/minio.service << "EOF"
[Unit]
Description=MinIO
Documentation=https://docs.min.io
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/minio

[Service]
WorkingDirectory=/usr/local/
User=root
Group=root
EnvironmentFile=/etc/default/minio
ExecStart=/usr/local/bin/minio server $MINIO_OPTS
Restart=always
LimitNOFILE=65536
TimeoutStopSec=infinity
SendSIGKILL=no

[Install]
WantedBy=multi-user.target
EOF

Enable and Start MinIO Service

systemctl daemon-reload
systemctl enable minio
systemctl start minio.service && systemctl status minio.service

---

Step 6: Installing and Using MinIO Client (mc)

To manage buckets and users via the CLI, install the MinIO Client (mc):

wget https://dl.min.io/client/mc/release/linux-amd64/mc -O /usr/local/bin/mc
chmod +x /usr/local/bin/mc

Configure mc to connect to your local cluster:

mc alias set minio http://192.16.10.10:9000 AKaHEgQ4II0S7BjT6DjAUDA4BZ SKFzHq5iDoQgF7gyPYRFhzNMYSvY6ZFMpG

Create a new standard user for application access:

mc admin user add minio wazeem Temp123

---

Step 7: Mounting S3 Buckets Locally via s3fs-fuse

If you have applications that require POSIX filesystem access to your S3 buckets, you can mount them locally using s3fs-fuse.

yum install s3fs-fuse -y

Create the credentials file:

echo "AKaHEgQ4II0S7BjT6DjAUDA4BZ:SKFzHq5iDoQgF7gyPYRFhzNMYSvY6ZFMpG" > /etc/s3fs-passwd
chmod 600 /etc/s3fs-passwd

Mount the bucket to /export/home/chroot/:

mkdir -p /export/home/chroot/
s3fs sftp /export/home/chroot/ -o passwd_file=/etc/s3fs-passwd -o url=http://127.0.0.1:9000/ -o use_path_request_style -o use_cache=/tmp -o umask=227

Persistent /etc/fstab Entry

To ensure the bucket mounts automatically on boot, add the following line to /etc/fstab:

sftp /export/home/chroot fuse.s3fs _netdev,passwd_file=/etc/s3fs-passwd,url=http://127.0.0.1:9000,use_path_request_style,use_cache=/tmp,umask=227 0 0

Your distributed MinIO object storage cluster is now fully operational, resilient, and accessible via both S3 APIs and local filesystem mounts!

References

• Setup a 4-Node Distributed MinIO Cluster