Symantec Veritas Storage Foundation HP-UX, Installation Manual

Page 1: ...N18486G Veritas Storage Foundation Cluster File System Installation Guide HP UX 5 0 ...

Page 2: ...c Corporation and its licensors if any THIS DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS REPRESENTATIONS AND WARRANTIES INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE OR NON INFRINGEMENT ARE DISCLAIMED EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID SYMANTEC CORPORATION SHALL NOT BE LIABLE FOR INCIDENTAL OR CONS...

Page 3: ...wlett Packard Development Company L P Licensing and registration Veritas Storage Foundation Cluster File System is a licensed product See the Veritas Storage Foundation Cluster File System Installation Guide for license installation instructions Technical support For technical assistance visit http support veritas com and select phone or email support Use the Knowledge Base search feature to acces...

Page 4: ......

Page 5: ...stalling the product 20 Configuring the Components 23 Using the log files 25 Installation log file 25 Response file 25 Summary file 25 Verifying the configuration files 26 Low Latency Transport configuration files 26 Checking Low Latency Transport operation 27 Group Membership and Atomic Broadcast configuration files 29 Checking Group Membership and Atomic Broadcast operation 29 Checking cluster o...

Page 6: ...ns 50 Chapter 3 Adding and removing a node Adding a node to a cluster 54 Configuring SFCFS and CVM agents on the new node 56 Removing a node from a cluster 57 Chapter 4 Uninstalling the product Appendix A Troubleshooting and recovery Installation issues 63 Incorrect permissions for root on remote system 63 Resource temporarily unavailable 64 Inaccessible system 64 Storage Foundation Cluster File S...

Page 7: ...ow Latency Transport LLT and Veritas Group Membership and Atomic Broadcast GAB for messaging and cluster membership the Veritas Volume Manager VxVM to create the shared volumes necessary for cluster file systems and the Veritas File System VxFS package Topics covered in this chapter include Hardware overview Software components Preinstallation Prerequisites Installing the product Using the log fil...

Page 8: ...loying Fibre Channel technology SFCFS can be used in conjunction with the latest Veritas Storage Area Network SAN applications to provide a complete data storage and retrieval solution The figure below shows the configuration of a cluster file system on a Fibre Channel fabric with a disk array Figure 1 1 Four Node SFCFS Cluster Built on Fibre Channel Fabric Public Fiber Optic Private LAN Disk Arra...

Page 9: ...age devices The primary component of the Fibre Channel fabric is the Fibre Channel switch Cluster platforms There are several hardware platforms that can function as nodes in a cluster file system cluster See the Storage Foundation Cluster File System Release Notes Install the HP UX 11i 64 bit operating system with the September 2004 HP UX 11i Version 2 0 or later on each node and install a Fibre ...

Page 10: ... The following table shows the package name and contents for each package Package Contents VRTSperl Veritas Perl 5 8 8 Redistribution VRTSvlic Veritas Licensing VRTSicsco Symantec Common Infrastructure VRTSpbx Symantec Private Branch Exchange VRTSsmf Symantec Service Management Framework VRTSat Symantec Product Authentication Service VRTSobc33 Veritas Enterprise Administrator Core Service VRTSob V...

Page 11: ...rvices Provider VRTSvmpro Veritas Volume Manager Management Services Provider VRTSdcli Veritas Distributed Command Line Interface VRTSalloc Veritas Volume Manager Intelligent Storage Provisioning VRTSvdid Veritas Device Identification API VRTSvsvc Veritas Volume Server and Client Provider VRTSddlpr Veritas Device Discovery Layer Services Provider VRTSvrpro Veritas Volume Replicator Client Extensio...

Page 12: ...Required HP UX patches on page 15 Packages Contents VRTScfsdc Veritas Cluster File System Documentation VRTScmccc Veritas Cluster Management Console Cluster Connector VRTScmcs Veritas Cluster Management Console Single Cluster Mode VRTScscm Veritas Cluster Server Cluster Manager VRTScssim Veritas Cluster Server Simulator VRTSfsdoc Veritas File System Documentation VRTSvcsdc Veritas Cluster Server D...

Page 13: ...Enhancement to quota 1 for supporting large uids PHCO_32524 Enhancement to edquota for supporting large uids PHCO_32551 Enhancement to quotaon quotaoff for supporting large uids PHCO_32552 Enhancement to repquota for supporting large uids PHCO_32596 Enables df 1M PHCO_32608 Enables bdf 1M PHCO_32609 Enables fstyp 1M PHCO_32610 Enables mount 1M PHCO_32611 Fix fs_wrapper to accept vxfs from subtype ...

Page 14: ...latest patches go to the Veritas support website to view the following TechNote http support veritas com docs 281875 Note Most of the above patches are available in the Feature11i bundle The Feature11i bundle and the EnableVxFS bundle are available from HP software download site http h20293 www2 hp com Search for Veritas 5 0 on this site and follow the instructions PHKL_32430 Changes to separate v...

Page 15: ...stallation on the number and the type of systems for which you purchased the license A key may enable the operation of more products than specified on the certificate but you are legally limited to the number of product licenses purchased The VRTSvlic package executes Veritas product licensing The following commands and their manual pages are available on the system after VRTSvlic is installed vxl...

Page 16: ...llation add opt VRTS bin to your PATH and opt VRTS man to your MANPATH environment variables Command line examples in this guide assume these environment variables are set To prevent conflicts with VxFS manual pages previously installed with JFS OnLineJFS 3 5 the VxFS 5 0 manual pages are installed in the opt VRTS vxfs5 0 man directory The opt VRTS vxfs5 0 man directory is automatically added to e...

Page 17: ... any patch information before you install VEA Requirements The following are system recommendations for the GUI Alternatively remove the file sbin init d vmsa server Prerequisites Each cluster node must be connected to the public network and each must have a unique host name by which it can be addressed on the public network The local node from which you install does not have to be part of the clu...

Page 18: ... the recommended method to license and install the product The installer also enables you to configure the product verify preinstallation requirements and view the product s description At most points during an installation you can type b back to return to a previous section of the installation procedure The back feature of the installation scripts is context sensitive so it returns to the beginni...

Page 19: ...st specify on the command line installer rsh 7 From the Installation menu choose the I option for Install and select 6 the Veritas Storage Foundation Cluster File System You are prompted to continue with the installation 8 Enter y to continue with the installation Do you wish to continue with the installation y n q y 9 Enter one or more system names to install SFCFS Enter the system names separted...

Page 20: ...to continue Press Return to continue 5 Press Return to continue You are prompted to configure I O fencing in enabled mode Will you be configuring I O Fencing in enabled mode y n q y 6 Enter y or n to configure I O fencing in enable mode You are prompted to configure VCS for SFCFS Enter the unique cluster name Enter the unique Cluster ID number between 0 65535 b Enter the NIC for the first private ...

Page 21: ... Foundation user Is this information correct y n q y 11 Enter y or n to verify if this information is correct You are prompted to configure the cluster monitor Do you want to configure the Cluster Monitor y n q n 12 Enter y or n to configure the cluster monitor You are prompted to configure SMTP notification Do you want to configure SMTP notification y n q y 13 Enter y or n to configure SMTP notif...

Page 22: ...Veritas products are working properly on your system Technical Support will need these log files for debugging purposes Installation log file The installation log file contains commands executed during the procedure the output and any errors generated by the commands This file is for debugging installation problems and can be used by Veritas Technical Services Response file The response file conta...

Page 23: ...s the LLT system ID in the first column with the LLT host name This file is identical on each system in the cluster For example the file etc llthosts contains entries that resemble 0 system01 1 system02 etc llttab The file llttab 4 contains information that is derived during installation and used by the utility lltconfig 1M After installation this file lists the network links that correspond to th...

Page 24: ...te Links 0 system01 OPEN 2 1 system02 OPEN 2 Note Each system has two links and that each system is in the OPEN state An asterisk denotes the system on which the command is typed With LLT configured correctly the output of lltstat n shows all of the systems in the cluster and two links for each system If the output shows otherwise you can use the verbose option of lltstat For example type lltstat ...

Page 25: ...ach link However in the example above the output shows that for node system02 the private network may have failed or the information in etc llttab may be incorrect To obtain information about the ports open for LLT type lltstat p on any system In the following example lltstat p is typed on one system in the cluster 4 Log into system01 lltstat p Output resembles LLT port information Port Usage Cook...

Page 26: ... least N systems are ready to form the cluster N is the number of systems in the cluster Checking Group Membership and Atomic Broadcast operation This section describes how to check GAB operation To check GAB operation Enter the following command on each node in the cluster sbin gabconfig a If GAB is operational the following output displays with GAB port membership information GAB Port Membership...

Page 27: ... group state lists the cvm group which is online on system01 and offline on system02 See the hastatus 1M manual page See the Veritas Cluster Server User s Guide 2 Enter the following command on any systems hasys display The example on the next page shows the output of system01 The list continues with similar information for system02 not shown and any other systems in the cluster On each system the...

Page 28: ...tem01 Frozen 0 system01 GUIIPAddr system01 LLTNodeId 0 system01 Limits system01 LoadTimeCounter 1890 system01 LoadTimeThreshold 600 system01 LoadWarningLevel 80 system01 MajorVersion 2 system01 MinorVersion 0 system01 NodeId 0 system01 OnGrpCnt 1 system01 ShutdownTimeout 60 system01 SourceFile main cf system01 SysName system01 system01 SysState RUNNING system01 SystemLocation system01 SystemOwner ...

Page 29: ...stem clocks on all nodes are synchronized using some external component such as the Network Time Protocol NTP daemon If the nodes are not in sync timestamps for change ctime and modification mtime may not be consistent with the sequence in which operations actually happened Configuring VCS Configuring VCS means conveying to the VCS engine the definitions of the cluster service groups resources and...

Page 30: ... VCS configuration file main cf is created during the installation procedure After installation the main cf file contains the base definitions of the cluster and its nodes Additionally the file types cf listed in the include statement defines the bundled agents for VCS resources See the Veritas Cluster Server User s Guide A typical VCS configuration file for SFCFS file resembles include types cf i...

Page 31: ...ger Web Console a service group ClusterService was created that includes IP Process and Notifier resources These resources were configured according to information you provided during the installation procedure A resource dependency was also created VCS application failover services If you installed SFCFS HA you can begin implementing the application monitoring failover services provided by the Ve...

Page 32: ...34 Installing and configuring the product VCS application failover services ...

Page 33: ...age Foundation Cluster File System you can upgrade your product using the procedures described in this chapter Topics covered in this chapter include Preparing to upgrade the product Upgrade Overview Upgrading from 3 5 to 5 0 Upgrading from 4 1 to 5 0 Upgrading the disk layout versions ...

Page 34: ...that the prerequisite patches required for SFCFS 5 0 are accessible Upgrade paths The upgrade paths for Veritas Storage Foundation Cluster File System are From Upgrade to Tasks Storage Foundation Cluster File System 3 5 formerly known as SANPoint Foundation Suite 3 5 Storage Foundation Cluster File System 5 0 Proceed to Upgrading from 3 5 to 5 0 on page 39 Storage Foundation Cluster File System 3 ...

Page 35: ...Foundation Cluster File System 5 0 Proceed to Upgrading from 3 5 to 5 0 on page 39 Storage Foundation Cluster File System 4 1 Storage Foundation Cluster File System 5 0 Proceed to Upgrading from 4 1 to 5 0 on page 45 Storage Foundation Cluster File System 4 1 MP1 Storage Foundation Cluster File System 5 0 Proceed to Upgrading from 4 1 to 5 0 on page 45 From Upgrade to Tasks ...

Page 36: ...tep 5 below for a phased upgrade Note A phased upgrade should not be performed from one of the nodes in the cluster The stages of the phased upgrade procedure are 1 Select two or more nodes to upgrade 2 Install the new version 3 Shut down VCS on remaining non upgraded nodes 4 Modify the configuration information in the main cf file 5 Install the new version on each remaining node and reboot them F...

Page 37: ...DVD drive ioscan fnC disk Make a note of the device file as it applies to your system 5 Create a directory in which to mount the software disc and mount the disc using the appropriate drive name For example mkdir p dvdrom usr sbin mount F cdfs dev dsk c3t2d0 dvdrom 6 Change to the top level directory on the disc cd dvdrom 7 Offline all SFCFS resources on nodes selected in step 2 by running the fol...

Page 38: ...Required HP UX patches on page 15 12 Install SFCFS 5 0 on system01 and system02 and reboot these nodes See Installing the product on page 19 Note Do not configure SFCFS after reboot 13 Shutdown VCS on remaining non upgraded nodes a Run the following commands from one of the non upgraded nodes haconf makerw hagrp unfreeze cvm persistent hagrp unfreeze service_group persistent haconf dump makero whe...

Page 39: ... Process VRTSweb PathName opt VRTSvcs bin haweb Arguments 10 129 96 64 8181 b Replace it with VRTSWebApp VCSweb Critical 0 AppName vcs InstallDir opt VRTSweb VERITAS TimeForOnline 5 c Add the NIC resource in the ClusterService group For example where the name of the NIC resource is named csgnic and the public NIC device is hme0 add NIC csgnic Device hme0 d Add new dependencies for the new resource...

Page 40: ...re disc into your system s DVD drive 3 Determine the block device file for the DVD drive ioscan fnC disk Make a note of the device file as it applies to your system 4 Create a directory in which to mount the software disc and mount the disc using the appropriate drive name For example mkdir p dvdrom usr sbin mount F cdfs dev dsk c3t2d0 dvdrom 5 Change to the top level directory on the disc cd dvdr...

Page 41: ... the nodes See the HP UX Operating System documentation 10 Install all the prerequisite patches on all the nodes See section Required HP UX patches on page 15 11 Install SFCFS 5 0 and reboot all the nodes See Installing the product on page 19 Note Do not configure SFCFS after reboot 12 Start vxfen on all the nodes vxfen can be started either in disable or enable mode For starting vxfen in disabled...

Page 42: ... VRTSvcs bin haweb Arguments 10 129 96 64 8181 b Replace it with VRTSWebApp VCSweb Critical 0 AppName vcs InstallDir opt VRTSweb VERITAS TimeForOnline 5 c Add the NIC resource in the ClusterService group For example where the name of the NIC resource is named csgnic and the public NIC device is hme0 add NIC csgnic Device hme0 d Add new dependencies for the new resources in the ClusterService group...

Page 43: ...rom 4 1 to 5 0 1 Log in as superuser 2 Select one or more nodes to upgrade say system01 and system02 3 Insert the appropriate software disc into your system s DVD drive 4 Determine the block device file for the DVD drive ioscan fnC disk Make a note of the device file as it applies to your system 5 Create a directory in which to mount the software disc and mount the disc using the appropriate drive...

Page 44: ...se nodes See Installing the product on page 19 Note Do not configure SFCFS after reboot 10 Shutdown VCS on remaining non upgraded nodes a Run the following commands from one of the non upgraded nodes haconf makerw hagrp unfreeze cvm persistent hagrp unfreeze service_group persistent haconf dump makero where service_group is the command that displays the service groups by hagrp dep cvm b Run the fo...

Page 45: ...ame opt VRTSvcs bin haweb Arguments 10 129 96 64 8181 b Replace it with VRTSWebApp VCSweb Critical 0 AppName vcs InstallDir opt VRTSweb VERITAS TimeForOnline 5 c Add the NIC resource in the ClusterService group For example where the name of the NIC resource is named csgnic and the public NIC device is hme0 add NIC csgnic Device hme0 d Add new dependencies for the new resources in the ClusterServic...

Page 46: ...priate software disc into your system s DVD drive 3 Determine the block device file for the DVD drive ioscan fnC disk Make a note of the device file as it applies to your system 4 Create a directory in which to mount the software disc and mount the disc using the appropriate drive name For example mkdir p dvdrom usr sbin mount F cdfs dev dsk c3t2d0 dvdrom 5 Change to the top level directory on the...

Page 47: ... hares delete qlogckd opt VRTS bin haconf dump makero opt VRTS bin hastop all force 11 If you have configured the VCS Cluster Manager Web Console complete step a through step d to modify the etc VRTSvcs conf config main cf file a Remove VRTSweb Process VRTSweb PathName opt VRTSvcs bin haweb Arguments 10 129 96 64 8181 b Replace it with VRTSWebApp VCSweb Critical 0 AppName vcs InstallDir opt VRTSwe...

Page 48: ... 7 for cluster mounted file systems If you have cluster mounted file systems with disk layout versions lower than 6 then after upgrading to SFCFS 5 0 perform the following additional steps to prepare the file system for being mounted on all nodes of the cluster 1 Select one of the nodes of the cluster and mount the file system locally on this node For example mount it without the ocluster option E...

Page 49: ...rading the disk layout versions 4 On the node selected in step 1 after the disk layout has been successfully upgraded unmount the file system umount mnt1 5 This file system can be mounted on all nodes of the cluster using cfsmount ...

Page 50: ...52 Upgrading the product Upgrading the disk layout versions ...

Page 51: ...e This chapter provides information on how to add a node to an existing cluster and removing a node from a cluster Topics include Adding a node to a cluster Configuring SFCFS and CVM agents on the new node Removing a node from a cluster ...

Page 52: ... File System nohup pfs_mountd nohup pfsd 4 Create a directory in which to mount the software disc and mount the disc using the appropriate drive name For example mkdir p dvdrom usr sbin mount F cdfs dev dsk c3t2d0 dvdrom 5 Add opt VRTS bin to your PATH and opt VRTS man to your MANPATH environment variables See Setting PATH and MANPATH environment variables on page 18 6 Change to the SFCFS director...

Page 53: ...te networks are configured the same as on the other cluster nodes set node system03 set cluster 100 link lan2 dev lan 2 ether link lan0 dev lan 0 ether b Copy etc llthosts from one other node of the cluster and add a new line to it with the name of this node c Copy the same llthosts file to all nodes of the cluster See etc llttab on page 25 15 Create etc gabtab file the same as it is on another no...

Page 54: ...ck that there are no service groups dependent on CVM such as SFCFS that are still online hagrp dep cvm 3 If there are any dependencies take them offline then take the CVM service group offline hagrp offline cvm sys system01 hagrp offline cvm sys system02 4 Open the VCS configuration for writing haconf makerw 5 Add the new node to the CVM system list and specify a failover priority of zero hagrp mo...

Page 55: ...oups to the cluster configuration cfsdgadm add cfsdg system03 sw 12 Create a mnt on system03 and run the following commands cfsmntadm modify mnt add system03 rw Refer to cfsmntadm man page for more details 13 Use cfsmount command to cluster mount mnt back on all the nodes cfsmount mnt Removing a node from a cluster This section describes how to remove a node from a cluster As in previous examples ...

Page 56: ...being removed 7 Edit etc gabtab on the remaining nodes of the cluster and edit the gabconfig command to reflect the correct and new number of nodes in the cluster 8 Login to system03 and remove the following files rm etc vxfenmode rm etc llthosts rm etc llttab rm etc gabtab 9 If fencing was enabled on the cluster run the following commands rm etc vxfentab rm etc vxfendg 10 If necessary modify the ...

Page 57: ...the scripts directory run the uninstallsfcfs script and remove SFCFS on system03 uninstallsfcfs If you do not want to remove the Veritas Cluster Server software enter n when prompted to uninstall VCS See the Veritas Cluster Server Installation Guide on the software disc ...

Page 58: ...60 Adding and removing a node Removing a node from a cluster ...

Page 59: ...mand to uninstall SFCFS The uninstallsfcfs script uses ssh to communicate with remote nodes as default uninstallsfcfs If you want to use rsh you must specify on the command line uninstallsfcfs rsh 4 Enter the system names to uninstall SFCFS Enter the system names separated by spaces on which to uninstall SFCFS system01 system02 5 Enter y to uninstall SFCFS Are you sure you want to uninstall SFCFS ...

Page 60: ...62 Uninstalling the product ...

Page 61: ...te root access permission on each system to which you are installing Checking communication with system01 FAILED Remote remsh rcp permissions not available on system01 Correct permissions and continue Continue Y N Suggested solution You need to set up the systems to allow remote access using ssh or rsh See the Veritas Storage Foundation and High Availability Solutions Getting Started Guide Note Re...

Page 62: ... is not accessible This could be for a variety of reasons such as the system name was entered incorrectly or the system is not available over the network Checking communication with system01 FAILED System not accessible system01 Suggested solution Verify that you entered the system name correctly use the ping 1M command to verify the accessibility of the host If a system cannot access the software...

Page 63: ...ed on some nodes and fail on others due to different mount options To avoid this situation ensure that Quick I O licensing is uniformly applied or be careful to mount the cluster file system with the qio noqio option appropriately specified on each node of the cluster See the mount 1M manual page A shared CVM volume was not specified The device is still mounted as a local file system somewhere on ...

Page 64: ...ables on page 18 The mount fsck and mkfs utilities reserve a shared volume They fail on volumes that are in use Be careful when accessing shared volumes with other utilities such as dd it is possible for these commands to destroy data on the disk Running some commands such as vxupgrade n 7 vol02 can generate the following error message vxfs vxupgrade ERROR not primary in a cluster file system This...

Page 65: ... of two possible states Either the last network link is broken called a network partition condition or the last network link is okay but the node crashed in which case it is not a network partition problem It is not possible to identify whether it is the first or second state so a kernel message is issued to indicate that a network partition may exist and there is a possibility of data corruption ...

Page 66: ...eat communication links may not be able to allocate kernel memory If this occurs a node halts to avoid any chance of network partitioning Reduce the load on the node if this happens frequently A similar situation may occur if the values in the etc llttab files on all cluster nodes are not correct or identical ...