IO Fencing

I/O Fecncing :is mechanism to prevent uncoordinated access to shared storage.

SCSI_3 Persistant —–> accepting I/O ‘s from multiple nodes/hosts
SCSI-3 PR enables access for multiple nodes to a device and simultaneously blocks access for other nodes.
SCSI-3 PR uses a concept of registration and reservation. Each system registers its own “key” with a SCSI-3 device

I/O Fencing Service/Driver:
I/O fencing uses GAB port b for communication.Fencing is started with the vxfenconfig -c command.
The fencing driver vxfen is started during system start up using /etc/rc2.d/s97vxfen.

Configuration Files for I/O fencing:
1./etc/vxfentab
2./etc/vxfendg
3./etc/vxfenmode

I/O fencing Implementation on Existing clusters
1.Configure coordinator diskgroup using SCSI-3 Persistent Reservations LUNS .(Most of the SAN supports SCSI-3 PR)
vxdisksetup -i c2t1d0
vxdisksetup -i c2t1d1
vxdisksetup -i c2t1d2
vxdg init fendg disk1=c2t1d0s2
vxdg -g fendg adddisk disk2=c2t1d1s2
vxdg -g fendg adddisk disk3=c2t1d2s2

2.Set coordinator flag to the diskgroup
#vxdg -g fendg set coordinator=on

3.Fencing Supports DMP for coordinator and data disks.Set policy in /etc/vxfenmode.
Perform the below on each cluster nodes.
# mv /etc/vxfenmode /etc/vxfenmode.previous
# cp /etc/vxfen.d/vxfenmode_scis3_dmp /etc/vxfenmode
# cat /etc/vxfenmode
vxfen_mode=scsi3
scsi3_disk_policy=dmp

4.Deport the diskgroup. “-t” turns off automatic importing when system starts.
# vxdg deport fendg
# vxdg -t import fendg
# vxdg deport fendg

5.Create /etc/vxfendg on all cluster nodes
# echo “fendg” > /etc/vxfendg

6.Test the diskgroup for I/O fencing compatibility.Option “-r” will run the test in non-destructive mode.
Arena-Node2#/opt/VRTSvcs/vxfen/bin/vxfentsthdw -r -g fendg

VERITAS vxfentsthdw version 5.1 Solaris

The utility vxfentsthdw works on the two nodes of the cluster.
The utility verifies that the shared storage one intends to use is
configured to support I/O fencing. It issues a series of vxfenadm
commands to setup SCSI-3 registrations on the disk, verifies the
registrations on the disk, and removes the registrations from the disk.

The logfile generated for vxfentsthdw is /var/VRTSvcs/log/vxfen/vxfentsthdw.log.11074

Enter the first node of the cluster: node1
Enter the second node of the cluster:node2

********************************************

Testing node1 /dev/vx/rdmp/disk_4s2 node2 /dev/vx/rdmp/disk_7s2

Evaluate the disk before testing …………………… No Pre-existing keys
RegisterIgnoreKeys on disk /dev/vx/rdmp/disk_4s2 from node node1 ……. Passed
Verify registrations for disk /dev/vx/rdmp/disk_4s2 on node node1 …… Passed
RegisterIgnoreKeys on disk /dev/vx/rdmp/disk_7s2 from node node2 ……. Passed
Verify registrations for disk /dev/vx/rdmp/disk_7s2 on node node2 …… Passed
Unregister keys on disk /dev/vx/rdmp/disk_4s2 from node node1 ………. Passed
Verify registrations for disk /dev/vx/rdmp/disk_7s2 on node node2 …… Failed

Unregistration test for disk failed on node node2.
Unregistration from one node is causing unregistration of keys from the other node.
Disk is not SCSI-3 compliant on node node2.
Execute the utility vxfentsthdw again and if failure persists contact
the vendor for support in enabling SCSI-3 persistent reservations
Removing test keys and temporary files, if any…
The above test should be password in a order to use the diskgroup as coordinator DG.
Note:As the above results,my test is failed due to SCSI-3 PR issue.

7.Start the fencing driver using script on all cluster nodes.
# /sbin/vxfen-startup
Node:/etc/vxfentab  will be updated whenever fencing driver re-initializes.

8.Close the cluster configuration on all the cluster nodes.
# haconf -dump -makero

9.Perform the offline configuration (i.e modify main.cf using vi editor) to add “UseFence = SCSI3″ line under the bottom of cluster Name
cluster UNIXARENA
UserNames = { admin = “ABCDGFRFSLK.” }
Administrators = { admin }
ClusterAddress = “192.168.2.7″
UseFence = SCSI3
)

10.Stop VCS on all cluster nodes.
# hastop -all

11.Restart VCS on all the cluster nodes.
# hastart

12.Display the fencing membership status
# vxfenadm -d
I/O Fencing Cluster Information:
================================
Fencing Protocol Version: 201
Fencing Mode: SCSI3
Fencing SCSI3 Disk Policy: dmp
Cluster Members:
0 (karri1)
1 (karri2)
RFSM State Information:
node 0 in state 8 (running)
node 1 in state 8 (running)

For your information,we can configure I/O fencing using CLI option as well,
./installvcs -fencing
=========================================================
vxfenadm utility
Administrators can use the vxfenadm command to troubleshoot and test fencing configurations. The command’s options for use by administrators are:

 

-g read and display keys
 -i read SCSI inquiry information from device
 -m register with disks
 -n make a reservation with disks
 -p remove registrations made by other systems
 -r read reservations
 -x remove registrations