Link Search Menu Expand Document
LZone

GlusterFS Cheat Sheet

The following gives a quick overview on the different GlusterFS administration tools.

Add/Remove Peers

# Add peer
gluster peer probe <host name>

# Remove peer
gluster peer detach <host name>

You can list the status of all known peers by running.

gluster peer status

gluster volume info all

Check free space per Node

gluster volume status <volume> detail

Mounting Volumes

You can do standard Unix mounting

mount -t glusterfs server1:/volume /mnt/volume

which has the disadvantage of specifying one server IP. If this server is down you can’t mount the volume even though it is available. What is important to know is that the given server is only used to fetch a volume info file, which itself lists all servers providing this volume. So the volume info file doesn’t need to be on the volume servers. Also remember as with NFS consider noatime mount options when you have many small files accessed often.

Problems on Ubuntu 12.04

When adding GlusterFS share to /etc/fstab do not forget to add “_netdev” to the mount options. Otherwise on next boot your system will just hang!

Actually there doesn’t seem to be a timeout. That would be nice too.

As a side-note: do not forget that Ubuntu 12.04 doesn’t care about the “_netdev” even. So network is not guaranteed to be up when mounting. So an additional upstart task or init script is needed anyway. But you need “_netdev” to prevent hanging on boot.

Monitoring

Nagios

Best use the glfs-health.sh script from http://www.sirgroane.net/2010/04/monitoring-gluster-with-nagios/ but you can also write a Nagios check just based on the GlusterFS reported cluster status you get from “gluster peer info”.

Munin

You can enable monitoring using Munin for example to track protocol command statistics with this Munin plugin https://github.com/acrollet/munin-glusterfs.

Healing Split Brain

Check a helper Python script from here: http://www.gluster.org/2012/06/healing-split-brain/

Suggested Readings