Multipath on Linux Servers
In Today's post, we'll go over configuring multipath on Linux Servers. This is applicable to RedHat 5.5, 5.6 and Oracle Enterprise Linux (OEL) 5.6. Also, this is in working with QLOGIC HBA's, it will be different with other HBA's!
We will be configuring LUNs presented by
a 3PAR or EMC SAN for use on the above Unix servers.
Necessary Information:
The person that provisioned
the storage has to provide the worldwide ID (WWID) for each LUN assigned to the
Linux server. This is critical when multiple LUNs are to be assigned at the same time as the WWID is
uniquely assigned to each LUN.
Note that WWIDs should only have lower case characters(!)
The
Linux server should have a copy of the Qlogic LUN scanning software installed
at /usr/local/bin/. Otherwise, a copy should be made of the ql-dynamic-tgt-lun-disc.sh script.
Depending upon the version of Linux you
are using, you will need to use a specific version of a generic copy of the
multipath.conf file.
These versions do not include support for EMC LUNs at this time. A copy of this file should be placed in the
/etc/ directory on the Linux server being configured.
Configuring
multipath:
You have to be root in order to execute the following commands:
[root@linux01 ~]# scsi_id -g -u -s /block/sda
2) Start up the multipathd daemon process.
[root@linux01 ~]# service multipathd start
3) Enable multipathd to start on subsequent boots.
[root@linux01 ~]# chkconfig --add multipathd
[root@linux01 ~]# chkconfig multipathd on
(chkconfig provides a simple command-line tool for maintaining the
/etc/rc[0-6].d directory hierarchy by relieving system administrators
of the task of directly manipulating the numerous symbolic
links in those directories.)
4) Execute the following commands to scan for LUNs on the Linux machine:
[root@linux01 ~]# ql-dynamic-tgt-lun-disc.sh -cl
[root@linux01 ~]# ql-dynamic-tgt-lun-disc.sh -s
5) Edit the multipath.conf file to include the internal SCSI disk’s unique SCSI ID.
WAS: wwid # PERC controller from /block/sda
IS: wwid <hex string from the scsi_id output> # PERC controller from /block/sda
6) Execute the following command to verify that the LUNs are visible. If the LUNs are from a 3PAR, you should see the string “3PARdata” visible for each LUN in the output.
You should see one “mpath?” where ‘?’ is a different letter for every LUN detected.
[root@linux01 ~]# multipath –ll
7) Edit the multipath.conf file again to insert the WWIDs for each LUN as shown by the previous command and assign alias names to the LUNs. This requires adding the following to this file:
multipaths {
multipath {
# <some descriptive
comment>
wwid <WWID of the LUN>
alias <some descriptive
alias name>
}
}
The portion above that is italicized should exist for every existing LUN.
Note that the WWID of the LUN should be provided by the person that provisioned the storage but the hex string should be proceeded by the character ‘3’. This resulting WWID can be compared to the output of the “multipath –ll" command for correctness.
The alias names should be descriptive such as “grid1” for the first Oracle ASM LUN, “data1” for the first Oracle data LUN, or “fra1” for the first Oracle flash recovery area.
8) Restart multipathd to reread the multipath.conf file.
[root@linux01 ~]# service multipathd restart
9) Executing the “multipath –ll" command should now display the alias names.
[root@linux01 ~]# multipath –ll
10) You should also verify that the LUNs now exist in the /dev/mapper/ directory. You should see each LUN specified by it’s alias name (e.g. grid1).
[root@linux01 ~]# cd /dev/mapper
[root@linux01 ~]# ll
11) If these LUNs are being created for Oracle, I will publish another article at a later date.
12) If additional LUNs are added, execute the following:
# ql-dynamic-tgt-lun-disc.sh -s
# multipath –ll
Edit the multipath.conf file again to insert the WWIDs for each new LUN as reflected by the previous command and assign alias names to the new LUNs.
[root@linux01 ~]# service multipathd restart
13) If you wish to rename an alias, execute the following:
Edit the multipath.conf file to insert the new alias name(s) assigned to the LUN.
[root@linux01 ~]# service multipathd restart
Verify that the alias(s) have changed by looking at the /dev/mapper directory.
Cleaning up failed/faulty (dead) paths in multipath:
This situation can occur if LUNs are removed or in case of a hardware failure.
1) Flush the device (e.g. /dev/sdx) from the multipath service:
[root@linux01 ~]# multipath -f sdx
2) Flush any outstanding I/O to the device:
[root@linux01 ~]# blockdev --flushbufs /dev/sdx
3) Remove the device file from the SCSI subsystem:
[root@linux01 ~]# echo 1>/sys/block/sdx/device/delete
4) Verify that the device file is gone from multipath:
[root@linux01 ~]# multipath -ll | grep –i sdx
