MooseFS: Build and Installation Guide

I recently learned about Distributed File Systems (DFS) and the benefits they could bring to an organization whose needs require redundant and highly available information across their systems. As part of a class project, I had to look into MooseFS, a fault-tolerant, network based DFS that can be mounted to virtual disks on client machines. Below is a guide on how to set this infrastructure up using CentOS on a practice “how-to” level, rather than a full-scale production.

NOTE: I have not configured MooseFS on a production scale, however modifying the specs within this guide should allow for that capability. I would still recommend checking out the MooseFS official documentation to be sure.

What you need:

The below specs are guidelines for a practice environment and can be set to whatever you need them to be:

  • 4 CentOS servers of default storage (16-20GB) placed on your LAN
  • 2 CentOS servers with two HDDs (16-20GB & ~15GB) attached and placed on your LAN, keeping one of the drives un-configured during OS installation

Networking:

The following are the hostnames and IP addresses I have used for this guide. Feel free to use whatever you would like, however make sure you document what you choose for easier installation moving forward:

  • moosefs-master10.0.5.20
  • moosefs-meta10.0.5.21
  • Moosefs-chunk01 (2 HDDs) – 10.0.5.22
  • moosefs-chunk02 (2 HDDs) – 10.0.5.23
  • linux0110.0.5.91
  • linux0210.0.5.92

Brief Architecture Overview

https://moosefs.com/

Master Server – the machines managing the whole file system, storing metadata for every file and folder; central point of the cluster.

Chunkservers (Data Servers) – the machines that store files’ data in “chunks”. They serve data for clients and synchronize among themselves when necessary.

Metaloggers (Backup Server) – the machines that store metadata changelogs. They can be easily made a master server in the event of master server failure.

Client Machine – machines that utilize the MooseFS TCP/IP network protocol and mount the filesystem being provided by the MooseFS cluster.

Setup MooseFS Master Server (moosefs-master)

Configure MooseFS Repository

curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
curl "http://ppa.moosefs.com/MooseFS-3-el7.repo" > /etc/yum.repos.d/MooseFS.repo

Install MooseFS Master Server and Web GUI

yum install moosefs-master moosefs-cgi moosefs-cgiserv moosefs-cli

Start MooseFS Master Server and Web GUI as a service

systemctl enable moosefs-master
systemctl start moosefs-master
systemctl enable moosefs-cgiserv
systemctl start moosefs-cgiserv

Add Firewall Rules for MooseFS services

firewall-cmd --zone=public --add-service=http --permanent
firewall-cmd --zone=public --add-port=9425/tcp --permanent
firewall-cmd --zone=public --add-port=9419/tcp --permanent
firewall-cmd --zone=public --add-port=9420/tcp --permanent
firewall-cmd --zone=public --add-port=9421/tcp --permanent
firewall-cmd --reload

Access Web GUI via <IPofMasterServer>:9425 in your Browser

Functioning MooseFS Dashboard

Setup MooseFS Metalogger (moosefs-meta)

Configure MooseFS Repository

curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
curl "http://ppa.moosefs.com/MooseFS-3-el7.repo" > /etc/yum.repos.d/MooseFS.repo

Install MooseFS Metalogger

yum install moosefs-metalogger

Edit MooseFS Metalogger to point to DNS name of Master Server

cd /etc/mfs
nano mfsmetalogger.cfg (change name of desired master to DNS name)

Start MooseFS Metalogger as a service

systemctl enable moosefs-metalogger
systemctl start moosefs-metalogger

Setup MooseFS Chunkservers (moosefs-chunk01)

Configure MooseFS Repository

curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
curl "http://ppa.moosefs.com/MooseFS-3-el7.repo" > /etc/yum.repos.d/MooseFS.repo

Install MooseFS Chunkserver

yum install moosefs-chunkserver

Edit MooseFS Chunkserver to point to DNS name of Master Server

cd /etc/mfs
nano mfschunkserver.cfg (change name of desired master to DNS name)

Create/Format Partition on 2nd HDD for Chunkserver

parted -l 
parted --align optimal /dev/sdb
(parted) mklabel gpt
(parted) mkpart mfschunks1 0% 100%
(parted) q

Install Dependency:

yum install xfsprogsls
mkfs.xfs /dev/sdb1

Add the following to /etc/fstab:

/dev/sdb1/   /mnt/mfschunks1   xfs   defaults   0   0

Make Mount Directory and Mount New Partition:

mkdir /mnt/mfschunks1
mount /mnt/mfschunks1

Change ownership so MooseFS can make changes:

chown mfs:mfs /mnt/mfschunks1
chmod 770 /mnt/mfschunks1

Append the following to /etc/mfs/mfshdd.cfg:

/mnt/mfschunks1

Start MooseFS Chunkserver as a service

systemctl enable moosefs-chunkserver
systemctl start moosefs-chunkserver

NOTE: Repeat the above steps for ALL additional chunkservers you wish to add. For this guide, we only have moosefs-chunk02 left to do.

Chunkservers configured and appearing in MooseFS Web GUI

Configure Clients (linux01 & linux02)

Configure MooseFS Repository

curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
curl "http://ppa.moosefs.com/MooseFS-3-el7.repo" > /etc/yum.repos.d/MooseFS.repo

Install MooseFS Client and Dependencies

yum install fuse fuse-devel
yum install moosefs-client

Mount MooseFS Filesystem to a Created Directory

mkdir -p /mnt/mfs
mfsmount /mnt/mfs -H moosefs-master
Successful mount message targeting MooseFS Master hostname

NOTE: Repeat for ALL other desired Linux Clients. For this guide, we only have linux02 left to do.

Both Linux (CentOS) clients connected and displayed in MooseFS Web GUI

Use MooseFS!

MooseFS should now be completely setup. To test this, we will be adding a test file and looking for it to appear on both linux01 and linux02.

Make a file within the mounted directory on linux01:

cd /mnt/mfs

Add contents to that file:

echo “test” > test1

Check the mounted directory on linux02 to see it replicated:

If you have a replicated file, you are all set!

Sources

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s