Red Hat ENTERPRISE LINUX 5 - DM MULTIPATH, Configuration And Administration Manual

Page 1: ...Red Hat Enterprise Linux 5 DM Multipath DM Multipath Configuration and Administration ...

Page 2: ... available at http www opencontent org openpub Red Hat and the Red Hat Shadow Man logo are registered trademarks of Red Hat Inc in the United States and other countries All other trademarks referenced herein are the property of their respective owners 1801 Varsity Drive Raleigh NC 27606 2072 USA Phone 1 919 754 3700 Phone 888 733 4281 Fax 1 919 754 3701 PO Box 13588 Research Triangle Park NC 27709...

Page 3: ... Multipathing Database 10 4 The DM Multipath Configuration File 11 4 1 Configuration File Overview 11 4 2 Configuration File Blacklist 12 4 2 1 Blacklisting by WWID 12 4 2 2 Blacklisting By Device Name 12 4 2 3 Blacklisting By Device Type 13 4 2 4 Blacklist Exceptions 13 4 3 Configuration File Defaults 14 4 4 Multipaths Device Configuration Attributes 17 4 5 Configuration File Devices 18 5 DM Mult...

Page 4: ...iv ...

Page 5: ...ter Provides information about installing configuring and managing Red Hat Cluster components Logical Volume Manager Administration Provides a description of the Logical Volume Manager LVM including information on running LVM in a clustered environment Global File System Configuration and Administration Provides information about installing configuring and maintaining Red Hat GFS Red Hat Global Fi...

Page 6: ...the set is installed on your system If not alternative but equivalent typefaces are displayed Note Red Hat Enterprise Linux 5 and later includes the Liberation Fonts set by default 4 1 Typographic Conventions Four typographic conventions are used to call attention to specific words and phrases These conventions and the circumstances they apply to are as follows Mono spaced Bold Used to highlight s...

Page 7: ... the Text to copy field and then click the Copy button Now switch back to your document and choose Edit Paste from the gedit menu bar The above text includes application names system wide menu names and items application specific menu names and buttons and text found within a GUI interface all presented in Proportional Bold and all distinguishable by context Note the shorthand used to indicate tra...

Page 8: ...lti line data types are set off visually from the surrounding text Output sent to a terminal is set in Mono spaced Roman and presented thus books Desktop documentation drafts mss photos stuff svn books_tests Desktop1 downloads images notes scripts svgs Source code listings are also set in Mono spaced Roman but are presented and highlighted as follows package org jboss book jca ex1 import javax nam...

Page 9: ...at makes your life easier Important Important boxes detail things that are easily missed configuration changes that only apply to the current session or services that need restarting before an update will apply Ignoring Important boxes won t cause data loss but may cause irritation and frustration Warning A Warning should not be ignored Ignoring warnings will most likely cause data loss ...

Page 10: ...x ...

Page 11: ...h the cable switch or controller fails DM Multipath switches to an alternate path Improved Performance DM Multipath can be configured in active active mode where I O is spread over the paths in a round robin fashion In some configurations DM Multipath can detect loading on the I O paths and dynamically re balance the load Figure 1 1 Active Passive Multipath Configuration with One RAID Device shows...

Page 12: ...mple shown in Figure 1 2 Active Passive Multipath Configuration with Two RAID Devices there are two I O paths to each RAID device just as there are in the example shown in Figure 1 1 Active Passive Multipath Configuration with One RAID Device With DM Multipath configured a failure at any of the points of the I O path to either of the RAID devices will cause DM Multipath to switch to the alternate ...

Page 13: ...ath Configuration File Some storage arrays require special handling of I O errors and path switching These require separate hardware handler kernel modules 1 3 DM Multipath Components Table 1 1 DM Multipath Components describes the components of DM Multipath Component Description dm multipath kernel module Reroutes I O and supports failover for paths and path groups multipath command Lists and con...

Page 14: ...default settings that are suitable for common multipath configurations Setting up DM multipath is often a simple procedure The basic procedure for configuring your system with DM Multipath is as follows 1 Install device mapper multipath rpm 2 Edit the multipath conf configuration file comment out the default blacklist change any of the existing defaults as needed save the configuration file 3 Star...

Page 15: ...devices to access the multipathed devices for example when creating logical volumes The devices in dev mpath are provided as a convenience so that all multipathed devices can be seen in one directory These devices are created by the udev device manager and may not be available on startup when the system needs to access them Do not use these devices for creating logical volumes or filesystems Any d...

Page 16: ...0 You can use the resulting LVM physical device when you create an LVM volume group just as you would use any other LVM physical device Note If you attempt to create an LVM physical volume on a whole device on which you have configured partitions the pvcreate command will fail Note that the Anaconda and Kickstart installation programs create empty partition tables if you do not specify otherwise f...

Page 17: ...nt it out to enable multipathing blacklist devnode After commenting out those lines this section appears as follows blacklist devnode 2 The default settings for DM Multipath are compiled in to the system and do not need to be explicitly set in the etc multipath conf file The default value of path_grouping_policy is set to failover so in this example you do not need to change the default value For ...

Page 18: ...s have local SCSI cards for their internal disks DM Multipath is not recommended for these devices The following procedure shows how to modify the multipath configuration file to ignore the local disks when configuring multipath 1 Determine which disks are the internal disks and mark them as the ones to blacklist In this example dev sda is the internal disk Note that as originally configured in th...

Page 19: ...t would not be safe procedure since dev sda is not guaranteed to be the same on reboot To blacklist individual devices you can blacklist using the WWID of that device Note that in the output to the multipath v2 command the WWID of the dev sda device is SIBM ESXSST336732LC____F3ET0EP0Q000072428BX1 To blacklist this device include the following in the etc multipath conf file blacklist wwid SIBM ESXS...

Page 20: ...0 3 sde 8 64 _ 3 0 0 3 sdi 8 128 3 3 Adding Devices to the Multipathing Database By default DM Multipath includes support for the most common storage arrays that support DM Multipath The default configuration values including supported devices can be found in the multipath conf defaults file If you need to add a storage device that is not supported by default as a known multipath device edit the e...

Page 21: ... need for your configuration or that you wish to change from the default values specified in the multipath conf defaults file If there are sections of the file that are not relevant to your environment or for which you do not need to override the default values you can leave them commented out as they are in the initial file The configuration file allows regular expression description syntax An an...

Page 22: ... in Section 3 1 Setting Up DM Multipath After commenting out the universal blacklist you can specify general device types and individual devices to blacklist You can blacklist devices according to the following criteria By WWID as described in Section 4 2 1 Blacklisting by WWID By device name as described in Section 4 2 2 Blacklisting By Device Name By device type as described in Section 4 2 3 Bla...

Page 23: ...thing on any of these devices you would need to specify them in the blacklist_exceptions section of the configuration file as described in Section 4 2 4 Blacklist Exceptions blacklist devnode ram raw loop fd md dm sr scd st 0 9 devnode hd a z 4 2 3 Blacklisting By Device Type You can specify specific device types in the blacklist section of the configuration file with a device section The followin...

Page 24: ...r_friendly_names yes This overwrites the default value of the user_friendly_names parameter The configuration file includes a template of configuration defaults This section is commented out as follows defaults udev_dir dev polling_interval 10 selector round robin 0 path_grouping_policy multibus getuid_callout sbin scsi_id g u s block n prio_callout bin true path_checker readsector0 rr_min_io 100 ...

Page 25: ...y group per path priority value group_by_node_name 1 priority group per target node name The default value is failover getuid_callout Specifies the default program and arguments to call out to obtain a unique path identifier An absolute path is required The default value is sbin scsi_id g u s prio_callout Specifies the the default program and arguments to call out to obtain a path priority value F...

Page 26: ...The default value is null flush_on_last_del RHEL 5 3 and later If set to yes the multipathd daemon will disable queueing when the last path to a device has been deleted The default value is no queue_without_daemon RHEL 5 3 and later If set to no the multipathd daemon will disable queueing for all devices when it is shut down The default value is yes user_friendly_names If set to yes specifies that...

Page 27: ...per detected serial number group_by_prio 1 priority group per path priority value group_by_node_name 1 priority group per target node name path_selector Specifies the default algorithm to use in determining what path to use for the next I O operation failback Specifies path group failback A value of 0 or immediate specifies that as soon as there is a path group with a higher priority than the curr...

Page 28: ...lue is determined by the process Table 4 2 Multipath Attributes The following example shows multipath attributes specified in the configuration file for two specific multipath devices The first device has a WWID of 3600508b4000156d70001200000b0000 and a symbolic name of yellow The second multipath device in the example has a WWID of 1DEC_____321816758474 and a symbolic name of red In this example ...

Page 29: ..._policy to multibus Other parameters you may need to set are no_path_retry and rr_min_io as described in Table 4 3 Device Attributes If the device is active passive but it automatically switches paths with I O to the passive path you need to change the checker function to one that does not send IO to the path to test if it is working otherwise your device will keep failing over This almost always ...

Page 30: ...o requests to a path before calling selector to choose the next path the number of requests to send is determined by rr_min_io times the path s priority as determined by the prio_callout program Currently there are priority callouts only for devices that use the group_by_prio path grouping policy which means that all the paths in a path group will always have the same priority If set to uniform al...

Page 31: ...ast path to a device has been deleted The default value is no product_blacklist Specifies a regular expression used to blacklist devices by product Table 4 3 Device Attributes The following example shows a device entry in the multipath configuration file device vendor COMPAQ product MSA1000 path_grouping_policy multibus path_checker tur rr_weight priorities ...

Page 32: ...22 ...

Page 33: ...roubleshooting with the multipathd Interactive Console 5 1 Resizing an Online Multipath Device If you need to resize an online multipath device use the following procedure 1 Resize your physical device 2 Use the following command to find the paths to the LUN multipath l 3 Resize your paths For SCSI devices writing a 1 to the rescan file for the device causes the SCSI driver to rescan as in the fol...

Page 34: ...e multipathd daemon is not running which should not be a problem for the vast majority of multipath users 5 4 Issues with queue_if_no_path feature If features 1 queue_if_no_path is specified in the etc multipath conf file then any process that issues I O will hang until one or more paths are restored To avoid this set the no_path_retry N parameter in the etc multipath conf file where N is the numb...

Page 35: ...or I O the status of the path is ready or active If the path is down the status is faulty or failed The path status is updated periodically by the multipathd daemon based on the polling interval defined in the etc multipath conf file The dm status is similar to the path status but from the kernel s point of view The dm status has two states failed which is analogous to faulty and active which cove...

Page 36: ...3 0 0 6 sdc 8 64 active ready 5 7 Multipath Command Options Table 5 1 Useful multipath Command Options describes some options of the multipath command that you may find useful Option Description l Display the current multipath configuration gathered from sysfs and the device mapper ll Display the current multipath configuration gathered from sysfs the device mapper and all other available componen...

Page 37: ...this command you can enter help to get a list of available commands you can enter a interactive command or you can enter CTRL D to quit The multipathd interactive console can be used to troubleshoot problems you may be having with your system For example the following command sequence displays the multipath configuration including the defaults before exiting the console multipathd k show config CT...

Page 38: ...28 ...

Page 39: ...ins partitions you cannot create a logical volume using the whole device without removing the partitions Resolves 513330 Elaborates documentation to address issues with queue_if_no_path features option Resolves 510707 Adds a procedure for resizing online multipath devices Resolves 498986 Fixes minor typo Resolves 498988 Fixes minor typo Revision 1 0 Thu Jan 29 2009 ...

Page 40: ...30 ...

Page 41: ...15 17 19 path_selector parameter 17 19 polling interval parameter 15 prio_callout parameter 15 19 product parameter 19 product_blacklist parameter 19 queue_without_daemon parameter 15 rr_min_io parameter 15 17 rr_weight parameter 15 17 19 selector parameter 15 udev_dir parameter 15 uid parameter 15 17 user_friendly_names parameter 14 15 vendor parameter 19 wwid parameter 17 configuring DM Multipat...

Page 42: ...1 multipathd command 27 interactive console 27 multipathd daemon 3 multipathd start command 7 multipaths section multipath conf file 17 N no_path_retry parameter 15 17 19 P path_checker parameter 15 19 path_grouping_policy parameter 15 17 19 path_selector parameter 17 19 polling_interval parameter 15 prio_callout parameter 15 19 product parameter 19 product_blacklist parameter 19 Q queue_without_d...