Novell LINUX ENTERPRISE SERVER 10 - STORAGE ADMINISTRATION GUIDE 7-2007, Administration Manual

Page 1: ...Novell w w w n o v e l l c o m novdocx en 6 April 2007 SLES 10 Storage Administration Guide SUSE Linux Enterprise Server 1 0 J u l y 2 0 0 7 S T O R A G E A D M I N I S T R A T I O N G U I D E...

Page 2: ...or re export to entities on the current U S export exclusion lists or to any embargoed or terrorist countries as specified in the U S export laws You agree to not use deliverables for prohibited nucle...

Page 3: ...this document is copied distributed and or modified from the following document under the terms specified in the document s license EVMS User Guide January 18 2005 Copyright 2005 IBM License Informati...

Page 4: ...novdocx en 6 April 2007...

Page 5: ...Restart the Server 26 2 2 7 Verify that EVMS Manages the Boot Swap and Root Partitions 26 2 3 Configuring LVM Devices to Use EVMS 27 2 4 Using EVMS with iSCSI Volumes 27 2 5 Using the ELILO Loader Fil...

Page 6: ...the Storage Subsystem to etc multipath conf 50 5 8 Configuring User Friendly Names in etc multipath conf 50 5 9 Tuning the Failover for Specific Host Bus Adapters 51 5 10 Configuring Multipath I O for...

Page 7: ...nding RAID 6 81 7 1 2 Creating a RAID 6 82 7 2 Creating Nested RAID 10 Devices with mdadm 82 7 2 1 Understanding Nested RAID Devices 82 7 2 2 Creating Nested RAID 10 1 0 with mdadm 83 7 2 3 Creating N...

Page 8: ...ormation 105 10 Troubleshooting Storage Issues 107 10 1 Is DM MPIO Available for the Boot Partition 107 10 2 Rescue System Cannot Find Devices That Are Managed by EVMS 107 10 3 Volumes on EVMS Devices...

Page 9: ...age 101 Chapter 10 Troubleshooting Storage Issues on page 107 Audience This guide is intended for system administrators Feedback We want to hear your comments and suggestions about this manual and the...

Page 10: ...10 SLES 10 Storage Administration Guide novdocx en 6 April 2007 A trademark symbol TM etc denotes a Novell trademark An asterisk denotes a third party trademark...

Page 11: ...r A plug in framework for flexible extensibility and customization Plug ins to extend functionality for new or evolving storage managers Support for foreign partition formats Cluster aware 1 2 Plug In...

Page 12: ...VM LVM2 for containers and region MD for RAIDs and DM for multipath I O EVMS Features Manages EVMS features Drive linking linear concatenation Bad Block Relocation BBR and Snapshot File System Interfa...

Page 13: ...iner managed by the Cluster Resource Manager It is accessible to all nodes of a cluster An administrator can configure the storage objects in the cluster container from any node in the cluster Cluster...

Page 14: ...Guide novdocx en 6 April 2007 A software RAID device dev md1 dev evms md md1 An LVM volume dev lvm_group lvm_volume dev evms lvm lvm_group lvm_volume Storage Object Standard Location the Device Node...

Page 15: ...on page 15 Section 2 1 2 During the Server Install on page 17 Section 2 1 3 After the Server Install on page 20 2 1 1 Before the Install System Device on page 15 Device Size Limits on page 16 Data Lo...

Page 16: ...nd Administration Guide http www novell com documentation sles10 Data Loss Considerations for the System Device This install requires that you delete the default partitioning settings created by the i...

Page 17: ...ioner dialog box 2d Select Expert Delete Partition Table and Disk Label then click Yes twice to continue through the Warning advisories This deletes the recommended partitions and the partition table...

Page 18: ...ce Swap is not required for systems with more than 1 GB of RAM You must have at least 1 GB of virtual memory RAM plus swap during the install but if the swap is more than 2 GB you might not be able to...

Page 19: ...ame such as sys_lx 7d In the Size field click Max to set the size of the system volume as the remaining space available in the lvm system partition 7e Specify the mount point as root volume 7f Click O...

Page 20: ...eps you disable boot lvm and boot md then enable boot evms In effect this forces EVMS to scan all the partitions at boot time including the boot partition and it activates boot under the dev evms dire...

Page 21: ...Services Disable the boot lvm and boot md services so they do not run at boot time runlevel B EVMS now handles the boot 1 In YaST click System System Services Runlevel Expert Mode 2 Select boot lvm 3...

Page 22: ...install If you do not configure the device to use EVMS you must manage the device and all of its volumes with its current volume manager the default is LVM and free space on the device cannot be used...

Page 23: ...t at this time NOTE Effective in SUSE Linux Enterprise 10 the dev directory is on tmpfs and the device nodes are automatically re created on boot It is no longer necessary to modify the etc init d boo...

Page 24: ...il the server is restarted Do not restart at this time 6 Continue with Section 2 2 4 Edit the Boot Loader File on page 24 2 2 4 Edit the Boot Loader File When you boot the system the kernel reads the...

Page 25: ...page 26 Using a Text Editor To edit the boot loader file in a text editor 1 Log in as the root user or equivalent 2 Open the boot loader file in a text editor For Grub this is the boot grub menu 1st f...

Page 26: ...hese two partitions dev evms sda1 dev evms sda2 3 At a terminal console prompt enter the following to update the initrd image with the dev evms path information for the root partition sbin mkinitrd f...

Page 27: ...s dev evms lvm2 sdb1 reiserfs defaults 1 2 IMPORTANT Make sure not to leave any stray characters or spaces in the line With these changes each time your system boots your file system is mounted using...

Page 28: ...le system type for the boot partition to vfat then choose Fstab Options and set the Arbitrary option value to umask 077 to ensure that the partition is accessible only to administrators WARNING Whenev...

Page 29: ...UI http evms sourceforge net user_guide GUI in the EVMS User Guide at the EVMS project on SourceForge net evmsn Starts the text mode interface for EVMS Ncurses For information about features in this i...

Page 30: ...30 SLES 10 Storage Administration Guide novdocx en 6 April 2007...

Page 31: ...ng rules based on certain buses drive types or file systems For information about how to define your own rules for udev see Writing udev Rules http reactivated net writing_udev_rules html Along with t...

Page 32: ...e dev disk by uuid directory For example a UUID looks like this e014e482 1c2d 4d09 84ec 61b3aefde77a 3 3 Using UUIDs in the Boot Loader and etc fstab File x86 After the install you can optionally use...

Page 33: ...line entry from dev sda1 reiserfs acl user_xattr 1 1 to UUID e014e482 1c2d 4d09 84ec 61b3aefde77a reiserfs acl user_xattr 1 1 IMPORTANT Make sure to make a backup copy of the fstab file before you be...

Page 34: ...tr 1 1 IMPORTANT Make sure to make a backup copy of the boot efi SuSE elilo conf file before you begin and do not leave stray characters or spaces in the file 3 5 Additional Information For more infor...

Page 35: ...Segment Managers The most commonly used segment manager is the DOS Segment Manager The following table describes the segment managers available in EVMS Table 4 1 EVMS Segment Managers Segment Manager...

Page 36: ...o the limit also applies to the md plug in for EVMS Software RAID devices you create with EVMS can be larger than 2 TB of course because the md driver plug in manages the disks underneath that storage...

Page 37: ...ibility volume For example a new disk sdb would show up as dev evms sdb Delete it from the Volumes list to force the disk to show up in Available Objects then create segments as desired 4 2 3 Adding a...

Page 38: ...ger for the device you want to manage then click Next DOS Segment Manager the most common choice GPT Segment Manager for IA 64 platforms Cluster Segment Manager available only if it is a viable option...

Page 39: ...volume on the server By default this field is empty Mount read only Select the check box to enable this option It is deselected disabled by default If this option is enabled files and directories cann...

Page 40: ...to the file system then enters the metadata in the journal This is the default Journal Writes data twice once to the journal then to the file system Writeback Writes data to the file system and write...

Page 41: ...onfigure multipathing for the devices before creating software RAIDs or file system volumes on the devices For information see Chapter 5 Managing Multipath I O for Devices on page 43 If you want to co...

Page 42: ...42 SLES 10 Storage Administration Guide novdocx en 6 April 2007...

Page 43: ...the Root Device on page 51 Section 5 11 Configuring Multipath I O for an Existing Software RAID on page 52 Section 5 12 Using Multipathed Devices on page 53 Section 5 13 Viewing Multipath I O Status o...

Page 44: ...reboot When you plan to create software RAID devices multipathing should be configured for the devices you plan to use prior to creating the software RAID devices because multipathing runs underneath...

Page 45: ...the etc multipath conf file 5 2 2 Multipath I O Management Tools The multipath tools user space package takes care of automatic path discovery and grouping It automatically tests the path periodically...

Page 46: ...the devices correctly by ID instead of by device node path For instructions see Section 5 4 4 Configuring mdadm conf and lvm conf to Scan Devices by UUID on page 48 To use software RAID with mdadm the...

Page 47: ...tipathed Devices on page 48 Section 5 4 3 Configuring the Server for Multipathing on page 48 Section 5 4 4 Configuring mdadm conf and lvm conf to Scan Devices by UUID on page 48 5 4 1 Preparing SAN De...

Page 48: ...he variable INITRD_MODULES in the file etc sysconfig kernel For example if your system contains a RAID controller accessed by the cciss driver and multipath I O devices connected to a Qlogic controlle...

Page 49: ...5 5 1 YaST 1 In YaST click System System Services Runlevel Simple Mode 2 Select multipathd then click Enable 3 Click OK to acknowledge the service startup message 4 Click Finish then click Yes The ch...

Page 50: ...active configuration all paths end up in the same group To model active passive configuration the paths that should not be active in parallel are placed in several distinct priority groups This norma...

Page 51: ...ilover modify the driver s options in the etc modprobe conf local file Refer to the HBA vendor s documentation for information about how to disable failover settings for your driver For example for th...

Page 52: ...following instructions assume the software RAID device is dev md0 which is its device name as recognized by the kernel Make sure to modify the instructions for the device name of your software RAID 1...

Page 53: ...us Enter mdadm detail dev md0 For example Number Major Minor RaidDevice State 0 253 0 0 active sync dev dm 0 1 253 1 1 active sync dev dm 1 2 253 2 2 active sync dev dm 2 5 12 Using Multipathed Device...

Page 54: ...s a reboot 5 13 Viewing Multipath I O Status Querying the multipath I O status outputs the current status of the multipath maps The multipath l option displays the current path status as of the last t...

Page 55: ...o the new storage Refer to the vendor s documentation for details 2 On the Linux system scan the SAN at a low level to discover the new devices At a terminal console prompt enter echo 1 sys class fc_h...

Page 56: ...that multipathd is running and works for your scenario Otherwise I O might be stalled forever on the affected multipathed device until reboot or until you manually return to failover instead of queuin...

Page 57: ...auses all queued I O to fail and propagates the error to the calling application 2 Reactivate queueing by entering the following command at a terminal console prompt dmsetup message mapname 0 queue_if...

Page 58: ...58 SLES 10 Storage Administration Guide novdocx en 6 April 2007...

Page 59: ...on page 60 Section 6 1 3 Comparison of RAID Performance on page 61 Section 6 1 4 Comparison of Disk Fault Tolerance on page 61 Section 6 1 5 Configuration Options for RAIDs on page 62 Section 6 1 6 Gu...

Page 60: ...dancy Any disk failure causes all data in the RAID to be lost 1 Mirrors data by copying blocks of one disk to another and keeping them in continuous synchronization If disks are different sizes the sm...

Page 61: ...disk fails performance is degraded while the RAID uses the parity to reconstruct data for the replacement disk Provides slightly less data redundancy than mirroring because it uses parity to reconstru...

Page 62: ...ributed parity to reconstruct data and parity on the failed disk Option Description Spare Disk For RAIDs 1 4 and 5 you can optionally specify a device segment or region to use as the replacement for a...

Page 63: ...to match disks and controllers with similar technologies performance and capacity Use a low number of drives on each controller to maximize throughput IMPORTANT As with any hardware solution using the...

Page 64: ...the default setting and is considered the fastest method for large reads Stripes wrap to follow the parity The parity s position in the striping sequence moves in a round robin fashion from last to fi...

Page 65: ...or multiple storage devices Areas can be of different sizes Snapshots Snapshots of a file system at a particular point in time even while the system is active thereby allowing a consistent backup The...

Page 66: ...c Specify the amount of space to use for the segment 4d Specify the segment options then click Create 5 Create and configure a software RAID Device 5a Select Action Create Region to open the Create St...

Page 67: ...ettings as desired For RAIDs 1 4 or 5 optionally specify a device to use as the spare disk for the RAID The default is none For RAIDs 0 4 or 5 specify the chunk stripe size in KB The default is 32 KB...

Page 68: ...d in Step 5 6c Specify a name for the device Use standard ASCII characters and naming conventions Spaces are allowed 6d Click Done 7 Create a file system on the RAID device you created 7a Select Actio...

Page 69: ...tion 6 3 2 Adding Segments to a RAID 4 or 5 on page 70 6 3 1 Adding Mirrors to a RAID 1 Device In a RAID 1 device each member segment contains its own copy of all of the data stored in the RAID You ca...

Page 70: ...and 5 can tolerate at least one disk failure Any given RAID can have one spare disk designated for it but the spare itself can serve as the designated spare for one RAID for multiple RAIDs or for all...

Page 71: ...pare Disk to a Region the addspare plug in for the EVMS GUI 3 Select the RAID device you want to manage from the list of Regions then click Next 4 Select the device to use as the spare disk 5 Click Ad...

Page 72: ...mode until you configure and add a spare When you add the spare the MD driver detects the RAID s degraded mode automatically activates the spare as a member of the RAID then begins synchronizing RAID...

Page 73: ...sign a spare device to the RAID so that it can be automatically added to the array and replace the failed device 6 5 3 Replacing a Failed Device with a Spare When a component device fails the md drive...

Page 74: ...the remfaulty plug in the EVMS GUI 2 Select the RAID device you want to manage from the list of Regions then click Next 3 Select the failed disk 4 Click Remove 6 6 Monitoring Status for a RAID Sectio...

Page 75: ...u have two RAIDs defined with labels of raid5 and raid4 md0 active raid5 sdg1 0 sdk1 4 sdj1 3 sdi1 2 device active not active RAID label you specified storage object RAID order The RAID is active and...

Page 76: ...tive sync dev sdk1 Example 2 Spare Disk Replaces the Failed Disk In the following mdadm report only 4 of the 5 disks are active and in good condition Active Devices 4 Working Devices 5 The failed disk...

Page 77: ...in the proc sys dev raid speed_limit_min and proc sys dev raid speed_limit_max files To speed up the process echo a larger number into the speed_limit_min file 6 6 5 Configuring mdadm to Send an E Mai...

Page 78: ...either because it finished normally or was aborted syslog priority Warning Fail Yes An active component device of an array has been marked as faulty syslog priority Critical Fail Spare Yes A spare com...

Page 79: ...n is added in the command then any other md devices that appear in proc mdstat are also monitored For more information about using mdadm see the mdadm 8 and mdadm conf 5 man pages 4 To configure the e...

Page 80: ...80 SLES 10 Storage Administration Guide novdocx en 6 April 2007 6 You must now reinitialize the disks for other uses just as you would when adding a new disk to your system...

Page 81: ...lows for additional fault tolerance by using a second independent distributed parity scheme dual parity Even if one of the hard disk drives fails during the data recovery process the system continues...

Page 82: ...the RAID 6 device dev md0 6 Reboot the server The RAID 6 device is mounted to local 7 Optional Add a hot spare to service the RAID array For example at the command prompt enter mdadm dev md0 a dev sd...

Page 83: ...mber device in the RAID 0 is mirrored individually multiple disk failures can be tolerated and data remains available as long as the disks that fail are in different mirrors You can optionally configu...

Page 84: ...dit the etc mdadm conf file to add entries for the component devices and the RAID device dev md2 6 Edit the etc fstab file to add an entry for the RAID 1 0 device dev md2 7 Reboot the server The RAID...

Page 85: ...1 device At the command prompt enter the following command using the software RAID 0 devices you created in Step 2 mdadm create dev md2 run level 1 raid devices 2 dev md0 dev md1 4 Create a file syste...

Page 86: ...ed The default number of replicas is 2 but the value can be 2 to the number of devices in the array Number of Devices in the mdadm RAID10 You must use at least as many component devices as the number...

Page 87: ...r the mdadm RAID10 yields read and write performance similar to RAID 0 over half the number of drives Near layout with an even number of disks and two replicas sda1 sdb1 sdc1 sde1 0 0 1 1 2 2 3 3 4 4...

Page 88: ...on 1 In YaST create a 0xFD Linux RAID partition on the devices you want to use in the RAID such as dev sdf1 dev sdg1 dev sdh1 and dev sdi1 2 Open a terminal console then log in as the root user or equ...

Page 89: ...ready has data on it In that case you create a degraded array with other devices copy data from the in use device to the RAID that is running in degraded mode add the device into the RAID then wait wh...

Page 90: ...90 SLES 10 Storage Administration Guide novdocx en 6 April 2007...

Page 91: ...supports resizing only for software RAID levels 1 4 5 and 6 These RAID levels provide disk fault tolerance so that one component partition can be removed at a time for resizing In principle it is pos...

Page 92: ...educed disk fault tolerance Even for RAIDs that can tolerate multiple concurrent disk failures do not remove more than one component partition at a time WARNING If a RAID does not have disk fault tole...

Page 93: ...ssed by the system When the replacement device is added back into the RAID it takes much longer to synchronize the data because all of the data that was on the original device must be rebuilt 5 Re add...

Page 94: ...94 If your array was not resized as you expected you must reboot then try this procedure again 8 2 3 Increasing the Size of the File System After you increase the size of the array see Section 8 2 2 I...

Page 95: ...mount command that is specific to the JFS file system mount o remount resize mnt Using the resize option is valid only during a remount when the volume is already mounted read write The mount point is...

Page 96: ...0 Wait until the resizing is completed before continuing 3 If the file system is not mounted mount it now For example to mount an ReiserFS file system for a RAID named dev md0 at mount point raid ente...

Page 97: ...ile system by entering df h The Disk Free df command shows the total size of the disk the number of blocks used and the number of blocks available on the file system The h option print sizes in human...

Page 98: ...tions Be very careful when removing partitions and make sure that you have a backup of your data available The procedure in this section uses the device names shown in the following table Make sure to...

Page 99: ...rtitions in the RAID the RAID array configuration continues to use the original array size until you force it to be aware of the newly available space You can specify a size for the RAID or use the ma...

Page 100: ...100 SLES 10 Storage Administration Guide novdocx en 6 April 2007...

Page 101: ...the secondary device in a way that ensures that both copies of the data are always identical By default DRBD uses the TCP port 7788 for communications between DRBD nodes The open source version of DR...

Page 102: ...file on the primary node node1 in a text editor modify the following parameters in the on hostname sections then save the file hostname device disk address meta disk All of these options are explaine...

Page 103: ...rimary node by entering touch r0mount from_node1 2 Test the DRBD service on node 2 2a Open a terminal console then log in as the root user or equivalent 2b Dismount the disk on node 1 by typing the fo...

Page 104: ...5 2 Host Names on page 104 Section 9 5 3 TCP Port 7788 on page 105 Section 9 5 4 The do what i say Option on page 105 9 5 1 Configuration If the initial drbd setup does not work as expected there is...

Page 105: ...ion The following open source resources are available for DRBD The following man pages for DRBD are available in the distribution drbd 8 drbddisk 8 drbdsetup 8 drbdadm 8 drbd conf 5 DRBD org http www...

Page 106: ...106 SLES 10 Storage Administration Guide novdocx en 6 April 2007...

Page 107: ...you set up a separate boot boot partition when using multipathing This issue has been resolved in Support Pack 1 and later 10 2 Rescue System Cannot Find Devices That Are Managed by EVMS The Linux re...

Page 108: ...roper order each time your servers reboot 10 5 Device Nodes Are Not Automatically Re Created on Restart Effective in SUSE Linux Enterprise 10 the dev directory is on tmpfs and the device nodes are aut...