Solaris Multipathing

Configuration files

For Solaris 10 and earlier

/kernel/drv/fp.conf Fiber channel port configuration file (For Fiber Channel Devices)
/kernel/drv/mpt.conf mpt scsi/SAS HBA controller driver configuration file (For SAS Devices)
/kernel/drv/scsi_vhci.conf MPXIO operation configuration file (Used to define loadbalancing and failback modes and configure 3rd party devices)
/kernel/drv/iscsi_conf iSCSI MPxIO configuration file (For iSCSI devices)

For Solaris 11
In case of Solaris 11, files in the directory /kernel/drv/ and /etc/driver/drv are the sample default configuration files delivered with OS (not used by MPxIO). Do not edit these files. The location of the configuration files is /etc/driver/drv/. The 4 configuration files used in case of Solaris 11 are the same, performing the same task. Although the syntax may be different.

/etc/driver/drv/fp.conf
/etc/driver/drv/mpt.conf
/etc/driver/drv/scsi_vhci.conf
/etc/driver/drv/iscsi_conf

It is best practice to edit these files using the stmsboot utility, rather than editing them manually.

Enabling/Disabling MPxIO

The syntax of the command stmsboot used to enable/disable MPxIO is :

stmsboot [-D fp|mpt] [-d | -e | -u | -L | -l controller_number]

where:
-D fp|mpt -> to select the protocol where you want to enable/disable mpxio
-e -> to enable STMS on all HBA
-d -> to disable STMS on all HBA
-u -> to update vfstable and dump config after manual file configuration
-l or -L unchanged - see the manpage of stms for a full description.

Few examples of using stmsboot command are :

# stmsboot -e To enable MPxIO or all devices (globally)
# stmsboot -D fp -e To enable MPxIO for fiber channel devices
# stmsboot -d To disable MPxIO for all devices (globally)

How to confirm MPxIO is enabled

To make sure that MPxIO is enabled for a LUN, simply run the format command and check for the /scsi_vhci in the logical path of the LUN.

# format
4. c6t60060E800561CF00000061CF00000002d0 
          /scsi_vhci/ssd@g60060e800561cf00000061cf00000002

Another way is to check the “mpxio-disable” parameter in the configuration files :

Solaris 7,8,9

# egrep "^mpxio-disable=" /kernel/drv/*.conf
/kernel/drv/scsi_vhci.conf:mpxio-disable="no";

Solaris 10

# egrep "^mpxio-disable=" /kernel/drv/*.conf
/kernel/drv/fp.conf:mpxio-disable="no";

Solaris 11

# egrep "^mpxio-disable=" /etc/driver/drv/*.conf
/etc/driver/drv/fp.conf:mpxio-disable="no";
For Solaris 10 and 11, MPxIO is enabled by default on X86 platforms.

Verifying that the scsi_vhci module is loaded

scsi_vhci is the virtual host controller interconnect driver used to manage path of storage LUN. If you find that the scsi_vhci module is not loaded, then most likely MPxIO is not enabled.

Solaris[TM] 7,8,9,10,11
# modinfo -c |grep scsi_vhci
 21          1 scsi_vhci                        LOADED/INSTALLED

Verifying Health of paths

We can check the multiple paths to the storage array LUNs by using mpathadm or luxadm command. The various states for the paths are :

ONLINE/primary
ONLINE/secondary
STANDBY/secondary
OFFLINE/*
UNAVAILABLE/*

mpathadm (supported for FC/SAS/iSCSI) [Works on Solaris 10, 11]
luxadm (supported on FC only) [Works on Solaris 7,8,9,10,11]

# mpathadm list LU
        /dev/rdsk/c9t60080E50001C0300000006464D8C9044d0s2
                Total Path Count: 2 
                Operational Path Count: 2

To check the health of the paths using mpathadm :

# mpathadm show LU  /dev/rdsk/c9t60080E50001C0300000006464D8C9044d0s2

Logical Unit:  /dev/rdsk/c9t60080E50001C0300000006464D8C9044d0s2
        mpath-support:  libmpscsi_vhci.so
        Vendor:  SUN     
        Product:  LCSM100_F
        Revision:  0777
        Name Type:  unknown type
        Name:  60080e50001c0300000006464d8c9044
        Asymmetric:  yes
        Current Load Balance:  round-robin
        Logical Unit Group ID:  NA
        Auto Failback:  on
        Auto Probing:  NA

        Paths:  
                Initiator Port Name:  210000e08b07922c
                Target Port Name:  203d0080e51c2cc4
                Override Path:  NA
                Path State:  OK
                Disabled:  no

                Initiator Port Name:  210000e08b07f32c
                Target Port Name:  203e0080e51c2cc4
                Override Path:  NA
                Path State:  OK
                Disabled:  no
.........

To check health of paths using luxadm :

# luxadm probe
                Found Fibre Channel device(s):
                  ......
                Logical Path:/dev/rdsk/c6t60060E800561CF00000061CF00000002d0s2
                Node WWN:203e0080e51c2cc4  Device Type:Disk device
                   ......
# luxadm display 203e0080e51c2cc4
DEVICE PROPERTIES for disk: 203e0080e51c2cc4
  Vendor:               SUN     
  Product ID:           LCSM100_F ------------> this is a 2540 or 2540-M2 array
  Revision:             0777
  Serial Num:           SV04720931      
  Unformatted capacity: 5120.000 MBytes
  Write Cache:          Enabled
  Read Cache:           Enabled
    Minimum prefetch:   0x3
    Maximum prefetch:   0x0
  Device Type:          Disk device
  Path(s):

  /dev/rdsk/c9t60080E50001C0300000006464D8C9044d0s2
  /devices/scsi_vhci/ssd@g60080e50001c0300000006464d8c9044:c,raw
   Controller           /devices/pci@1d,700000/SUNW,qlc@1/fp@0,0
    Device Address              203d0080e51c2cc4,0
    Host controller port WWN    210000e08b07922c
    Class                       primary
    State                       ONLINE
   Controller           /devices/pci@1c,600000/SUNW,qlc@1/fp@0,0
    Device Address              203e0080e51c2cc4,0
    Host controller port WWN    210000e08b07f32c
    Class                       secondary
    State                       ONLINE

Load Balancing policies

MPxIO also provides the load balancing capability to the I/O. There are basically 3 options to configure load balancing :

Round Robin Selects all paths in a round robin way.
Logical Block I/Os within the range specified by range_size parameter would go on the same path. Improves performance on some storage arrays which have per path caching of data.
None No load balancing – only one path is used.

The load balancing policy can be set in the configuration file /kernel/drv/scsi_vhci.conf for solaris 10 and earlier or in the /etc/driver/drv/scsi_vhci.conf file for solaris 11.

load-balance="round-robin";

Auto-failback policy

Auto-failback policy enables the re-enable the failed path after it has recovered from the failure. By default the auto-failback is enabled. There are 2 ways to enable the auto-failback :

Non-persistent way
By this method, the auto-failback option does not persist across reboots.

# mpathadm modify mpath-support --autofailback on libmpscsi_vhci.so
# mpathadm modify mpath-support -a on libmpscsi_vhci.so

Persistent way
For the changes to persist across reboots set the auto-failback parameter in the configuration file /kernel/drv/scsi_vhci.conf for solaris 10 and earlier or in the /etc/driver/drv/scsi_vhci.conf file for solaris 11. This requires a reconfiguration reboot.

auto-failback="enable";

Verifying auto-failback
To confirm that the auto-failback is enabled use :

# mpathadm show mpath-support libmpscsi_vhci.so | grep -i failback
           Supported Auto Failback Config:  1
           Auto Failback:  on  <---<<<
           Failback Polling Rate (current/max):  0/0

Some IMP notes about SAN Multipating

UseFul SAN Commands

Brief about Emulex / Luxadm

Brief About HBA

HBA Upgrade

HBA Hard reset / HBA Brands / HBA-Lpfc drivers upgrade

Powerpath-upgrade

powermt-HBA