Introduction
This howto will cover how to create a DRBD disk set with GFS concurrent disk access setup.
Why DRBD?
DRBD disk mirroring benefit as a shared SAN disk to archive higher service resilience with low cost hardware component. Alternately, it may be configured for backup purpose with it’s flexibility.
Prerequisite
1. RedHat Cluster Suit and GFS packages
cman-2.0.115-1.el5 gfs2-utils-0.1.62-1.el5 gfs-utils-0.1.20-1.el5 kmod-gfs-0.1.34-2.el5
2. drbd83 package
yum install drbd83
3. Create a partition on both node with using same size, we use [sda5] for this show case.
Disk /dev/sda: 499.5 GB, 499558383616 bytes 255 heads, 63 sectors/track, 60734 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 13 104391 83 Linux /dev/sda2 14 6540 52428127+ 8e Linux LVM /dev/sda3 6541 7062 4192965 82 Linux swap / Solaris /dev/sda4 7063 60734 431120340 5 Extended /dev/sda5 7063 19221 97667136 83 Linux
Prepare the configuration file /etc/drbd.conf
global { # Participate in DRBD’s online usage counter <URL:http://usage.drbd.org> usage-count no; } common { # Protocol A: write IO is reported as completed, if it has reached local disk and local TCP send buffer. # Protocol B: write IO is reported as completed, if it has reached local disk and remote buffer cache. # Protocol C: write IO is reported as completed, if it has reached both local and remote disk. Protocol C; } # Define the first disk name "r0" resource r0 { startup { # Wait for connection timeout. The init script drbd blocks the boot process until the DRBD resources are connected. When the cluster manager starts later, it does not see a resource with internal split-brain. In case you want to limit the wait time, do it here. Default is 0, which means unlimited. The unit is seconds. wfc-timeout 20; # Wait for connection timeout, if this node was a degraded cluster. In case a degraded cluster (= cluster with only one node left) is rebooted, this timeout value is used instead of wfc-timeout, because the peer is less likely to show up in time, if it had been dead before. Value 0 means unlimited. degr-wfc-timeout 120; # Sets on which node the device should be promoted to primary role by the init script. The node-name might either be a host name or the key word both. When this option is not set the devices stay in secondary role on both nodes. Usually one delegates the role assignment to a cluster manager (e.g. heartbeat, GFS). #become-primary-on both; ### Remark it during initiate setup, turn this on once the resource disk created. } disk { # In the event of I/O problem, detach it. on-io-error detach; } net { # Define the shared secret used in peer authentication. cram-hmac-alg "md5"; shared-secret "$1$Kdk9IEna$1D1.AKIO1nAhF0H86Z1zH0"; # With this option set you may assign primary role to both nodes. You only should use this option if you use a shared storage file system on top of DRBD. At the time of writing the only ones are: OCFS2 and GFS. If you use this option with any other file system, you are going to crash your nodes and to corrupt your data! allow-two-primaries; # Spit-brain policy for GFS after-sb-0pri discard-zero-changes; after-sb-1pri discard-secondary; after-sb-2pri disconnect; } syncer { rate 12M; # Transfer rate for drbd, 100Mb ethernet maximum is 12.5M in general. } on node1.localdomain { device /dev/drbd0; disk /dev/sda5; address 10.1.10.1:7789; meta-disk internal; } on node2.localdomain { device /dev/drbd0; disk /dev/sda5; address 10.1.10.2:7789; meta-disk internal; } }
Create DRBD disk resource
On both node1 and node2
drbdadm create-md r0 #Writing meta data... #initializing activity log #NOT initialized bitmap #New drbd meta data block successfully created.
Start DRBD Daemon on both node1 and node2
/etc/rc.d/init.d/drbd start Starting DRBD resources: [ d(r0) s(r0) n(r0) ]....
Check the role status, you should see both node is Secondary.
drbdadm role r0 Secondary/Secondary
Outdate it on node2 to initiate fresh sync
drbdadm outdate r0
Monitor the syncer status
cat /proc/drbd version: 8.3.2 (api:88/proto:86-90) GIT-hash: dd7985327f146f33b86d4bff5ca8c94234ce840e build by mockbuild@v20z-x86-64.home.local, 2009-08-29 14:07:55 0: cs:SyncTarget ro:Secondary/Secondary ds:Inconsistent/UpToDate C r---- ns:0 nr:1275904 dw:1275904 dr:0 al:0 bm:77 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:38732684 [>....................] sync'ed: 3.2% (37824/39068)M
or by command
drbd-overview
Once sync completed, check the role on node1 status again, state changed to Primary
drbdadm role r0 Primary/Secondary
Check Point
With successfully created a /dev/drbd0[r0] and the role changed to Primary on node1, so you’re able to access the drbd drive now and format your partition. Bear in mind when the state changed to “Secondary/Secondary”, that mean you cannot do any I/O operation on that resource.
Turn on Primary role on both node for GFS
Uncomment in /etc/drbd.conf
allow-two-primaries;
or
Enter the following command on both node1 and node2
drbdadm primary r0
Check the role status, you should see both node is Primary now.
drbdadm role r0 Primary/Primary
Format it as GFS
mkfs.gfs -t cluster0:r0 -j 2 /dev/drbd0
Mount it on both node , That’s it!
mount -t gfs /dev/drbd0 /mnt/somewhere
Uncomment in /etc/drbd.conf
1 allow-two-primaries;
or become-primary-on both?
allow-two-primaries is global parameter for nodes,
become-primary-on both, disk resource specific.