GlusterFS – a clustered filesystem

High availability requires shared storage of files

I’m using CentOS Linux 7.9 in a 2 node cluster, for this article i chose to user version 9 of GlusterFS. Each node have 2 network cards, one for LAN (192.168.1.22-23) and one for internal communication (10.0.0.1-2). The nodes have been assigned an extra hard drive (sdb) to be used by Gluster.

Start by installing the required packages on both nodes

yum update
yum install centos-release-gluster9.noarch
yum install glusterfs-server

Add hostnames to your system configuration

192.168.1.22	gluster1
192.168.1.23	gluster2

10.0.0.1	gluster1-int
10.0.0.2	gluster2-int

Create partitions to be used as “bricks”. I’m using XFS filesystem since it is CentOS standard but ext4 can also be used.

pvcreate /dev/sdb
vgcreate vg_gluster /dev/sdb
lvcreate -L 500m -n brick1 vg_gluster
mkfs.xfs /dev/vg_gluster/brick1

Create mountpoint and mount the brick

mkdir -p /bricks/brick1
echo "/dev/vg_gluster/brick1  /bricks/brick1    xfs     defaults    0 0" >> /etc/fstab
mount -a

Enable and start the Gluster service

systemctl enable glusterd.service
systemctl start glusterd.service

firewall-cmd --zone=public --add-port=24007-24008/tcp --permanent
firewall-cmd --zone=public --add-port=24009/tcp --permanent
firewall-cmd --reload

Add the second node to your cluster and check the status.

gluster peer probe gluster2-int
gluster peer status

Create the cluster filesystem and view cluster info.

mkdir /bricks/brick1/brick
gluster volume create glustervol1 replica 2 transport tcp gluster1-int:/bricks/brick1/brick gluster2-int:/bricks/brick1/brick
gluster volume info all

Mount your new filesystem using FUSE (Filesystem in USErspace), the following example is for the first node, change the hostname when adding to the second node.

echo "gluster1-int:/glustervol1 /mnt glusterfs defaults,_netdev 0 0" >> /etc/fstab
mount -a

Glossary

  • A trusted pool refers collectively to the hosts in a given Gluster Cluster.
  • A node or “server” refers to any server that is part of a trusted pool. In general, this assumes all nodes are in the same trusted pool.
  • A brick is used to refer to any device (really this means filesystem) that is being used for Gluster storage.
  • The term Global Namespace is a fancy way of saying a Gluster volume
  • A Gluster volume is a collection of one or more bricks.