TruCluster 380578-B21, Supplementary Manual

Page 1: ...ovided in the TruCluster Software Products Hardware Configuration manual Chapter 6 This update describes how to set up and maintain the shared Fibre Channel mass storage devices in a Compaq Tru64 UNIX TruCluster configuration including new information on the use of the Fibre Channel arbitrated loop topology It also includes support for the use of the emx manager emxmgr utility which is part of New...

Page 2: ...errors or omissions contained herein The information in this publication is subject to change without notice and is provided as is without warranty of any kind The entire risk arising out of the use of this information remains with recipient In no event shall Compaq be liable for any direct consequential incidental special punitive or other damages whatsoever including without limitation damages f...

Page 3: ...nel Hardware Supported for Cluster Storage 2 1 2 2 Fibre Channel Requirements and Restrictions 2 1 3 Using the emx Manager emxmgr Utility 3 1 Using the emxmgr Utility to Display Fibre Channel Adapter Information 3 1 3 1 1 Using the emxmgr Utility in an Arbitrated Loop Topology 3 4 3 2 Using the emxmgr Utility to Modify Worldwide Name to Target ID Mapping 3 5 3 2 1 Using the emxmgr Utility Interact...

Page 4: ... Channel Adapter Module 4 17 4 5 3 Configuring the KGPSA CA Adapter for Loop Mode 4 17 4 5 4 Configuring the KGPSA Adapter 4 19 4 6 Configuring the emx Driver for Fibre Channel Operation 4 19 4 6 1 Initial Worldwide Name to Target ID Mapping 4 20 4 6 2 Synchronizing the Initial Assignment of Target IDs in a Cluster 4 20 4 6 3 Remapping Target IDs After Adding or Replacing a Fibre Channel Device 4 ...

Page 5: ...ndex Figures 1 1 Point to Point Topology 1 4 1 2 Fabric Topology 1 5 1 3 Arbitrated Loop Topology 1 6 1 4 Fibre Channel Single Switch Transparent Failover Configuration 1 8 1 5 Mirroring Storage Units with LSM in a Cluster 1 9 1 6 Single Switch Configuration with Full Complement 1 10 1 7 Arbitrated Loop Configuration with One Storage Array 1 11 1 8 Arbitrated Loop Maximum Configuration 1 12 1 9 A ...

Page 6: ......

Page 7: ...cts patch kit 4 duv40fas0004 20000613 tar in addition to new hardware support NHD 2 or later You may obtain the patch kit from http www support compaq com patches index html The Version 1 6 TruCluster Software Products Hardware Configuration manual provides a method of using the sys data emx_data c file and a kernel rebuild to synchronize worldwide name to target ID mapping in a fabric topology Th...

Page 8: ...nance procedures to support the Compaq Tru64 UNIX TruCluster equipment including troubleshooting guidelines and instructions for recovery from various outages that may involve the Fibre Channel subsystems Outages of other subsystems such as network or parallel SCSI devices are not considered Related Documents Consult the following documentation to assist you in Fibre Channel storage configuration ...

Page 9: ...ller configuration information Release Notes Compaq StorageWorks HSG80 Array Controller ACS Version 8 4 for Compaq Tru64 UNIX HSG80 Array Controller ACS Version 8 4 Configuration and CLI Reference Guide HSG80 Array Controller ACS Version 8 4 Maintenance and Service Guide Release Notes Compaq StorageWorks HSG80 Array Controller ACS Version 8 5 for Compaq Tru64 UNIX HSG80 Array Controller ACS Versio...

Page 10: ......

Page 11: ... high transmit clock frequency to achieve the high data rate Using optical fibre transmission lines allows the high frequency information to be sent up to 70 km which is the maximum distance between transmitter and receiver Copper transmission lines may be used for shorter distances Tru64 UNIX does not support copper transmission lines ______________________ Note _______________________ The TruClu...

Page 12: ...anufactured An N_Port is connected directly to another N_Port in a point to point topology An N_Port is connected to an F_Port in a fabric topology NL_Port In an arbitrated loop topology information is routed around a loop A node port that can operate on the loop is called an NL_Port node loop port The information is repeated by each NL_Port until it reaches its destination Each port has a 64 bit ...

Page 13: ...ction 1 2 2 Arbitrated loop Section 1 2 3 ______________________ Note _______________________ You can interconnect an arbitrated loop with fabric creating a hybrid configuration The fabric must have ports with loop functionality FL_Ports to attach an arbitrated loop to the fabric Tru64 UNIX does not support hybrid configurations and they are not discussed in this document The following sections di...

Page 14: ...rconnected by the fabric switching function which routes the frame from one F_Port to another F_Port within the fabric Communication between two switches is routed between two expansion ports E_Port When an N_Port is connected to a fabric F_Port the fabric is responsible for the assignment of the Fibre Channel address to the N_Port that is attached to the fabric The fabric is also responsible for ...

Page 15: ...rbitrated loop addresses so it is not addressable by a Fibre Channel arbitrated loop port The nodes arbitrate to gain control become master of the loop After a node becomes master the nodes select by way of setting bits in a bitmask their own Arbitrated Loop Physical Address AL_PA The AL_PA is used to address nodes on the loop The AL_PA is dynamic and can change each time the loop is initialized a...

Page 16: ... those of a fabric Arbitrated loop supports a theoretical limit of 127 nodes in a loop Arbitrated loop nodes are self configuring and do not require Fibre Channel address switches Arbitrated loop provides reduced cost at the expense of bandwidth all nodes in a loop share the bandwidth 100 MB sec per loop and bandwidth degrades slightly as nodes and cables are added Nodes on the loop see all traffi...

Page 17: ...When to Use Arbitrated Loop When to Use Fabric In clusters of up to two members In clusters of more than two members In applications where low total solution cost and simplicity are key requirements In multinode cluster configurations when possible temporary traffic disruption due to reconfiguration or repair is a concern In applications where the shared bandwidth of an arbitrated loop configurati...

Page 18: ...arent failover units D00 through D99 are accessed through port 1 of both controllers Units D100 through D199 are accessed through port 2 of both HSG80 controllers with the limit of a total of 128 storage units You cannot achieve no single point of failure NSPOF with the configuration shown in Figure 1 4 because the switch represents a single point of failure You can however add the hardware for a ...

Page 19: ...s with LSM in a Cluster KGPSA Member System 2 KGPSA KGPSA Member System 1 KGPSA ZK 1533U AI Memory Channel Memory Channel Memory Channel Interface Fibre Channel Switch Fibre Channel Switch HSG 80 Controller A HSG 80 Controller B RA8000 ESA12000 Port 1 Port 2 Port 1 Port 2 HSG 80 Controller A HSG 80 Controller B RA8000 ESA12000 Port 1 Port 2 Port 1 Port 2 Fibre Channel Overview 1 9 ...

Page 20: ...on with Full Complement ZK 1532U AI KGPSA Member System 1 Memory Channel KGPSA Memory Channel Memory Channel Interface Member System 2 Fibre Channel Switch HSG 80 Controller A HSG 80 Controller B RA8000 ESA12000 Port 1 Port 1 Port 2 Port 2 HSG 80 Controller A HSG 80 Controller B RA8000 ESA12000 Port 1 Port 1 Port 2 Port 2 HSG 80 Controller A HSG 80 Controller B RA8000 ESA12000 Port 1 Port 1 Port 2...

Page 21: ...trollers and an SWXHB 07 Fibre Channel hub Figure 1 7 Arbitrated Loop Configuration with One Storage Array ZK 1697U AI SWXHB 07 HSG 80 Controller A HSG 80 Controller B RA8000 ESA12000 Port 1 Port 2 Port 1 Port 2 KGPSA CA Member System 1 Memory Channel KGPSA CA Memory Channel Memory Channel Interface Member System 1 Fibre Channel Overview 1 11 ...

Page 22: ...imum Configuration KGPSA Member System 2 KGPSA KGPSA Member System 1 KGPSA ZK 1699U AI Memory Channel Memory Channel Memory Channel Interface SWXHB 07 SWXHB 07 HSG 80 Controller A HSG 80 Controller B RA8000 ESA12000 Port 1 Port 2 Port 1 Port 2 HSG 80 Controller A HSG 80 Controller B RA8000 ESA12000 Port 1 Port 2 Port 1 Port 2 1 5 Zoning and Cascaded Switches This section provides a brief overview ...

Page 23: ... with each zone containing an independent cluster Remember with a configuration such as the one shown in Figure 1 9 the switch is a single point of failure Figure 1 9 A Simple Zoned Configuration KGPSA Memory Channel KGPSA KGPSA KGPSA Fibre Channel Switch Cluster 1 Member System 1 Cluster 1 Member System 2 Cluster 2 Member System 2 Cluster 2 Member System 1 0 2 4 6 8 10 12 14 ZK 1709U AI Memory Ch...

Page 24: ...see the SAN Switch Zoning documentation that is provided with the switch 1 5 2 Cascaded Switches Multiple switches may be connected to each other When cascading switches a maximum of three switches is supported with a maximum of two hops between switches The maximum hop length is 10 km longwave single mode or 500 meters shortwave multimode Fibre Channel cable 1 14 Fibre Channel Overview ...

Page 25: ...pter 168794 B21 DD302a1 awc DS DSGGA AA 8 port Fibre Channel switch 380591 B21 1 6b DS DSGGA AB 16 port Fibre Channel switch 380578 B21 1 6b DS DSGGB AA 8 port Fibre Channel switch 158222 B21 2 03a DS DSGGB AB 16 port Fibre Channel switch 158223 B21 2 03a DS SWXHB 07 7 port Fibre Channel Hub 234453 001 NA RA8000 with dual redundant HSG80 array controller 380560 B21 ACS V8 4 or V8 5a ESA12000 with ...

Page 26: ...age A maximum of four member systems is supported in fabric configurations A maximum of two member systems is supported in arbitrated loop configurations The only supported Fibre Channel adapters are the KGPSA BC and KGPSA CA PCI to optical Fibre Channel host bus adapters The KGPSA BC adapter is supported in fabric configurations only the KGPSA CA adapter is supported in both fabric and arbitrated...

Page 27: ...rray 8000 RA8000 midrange departmental storage subsystem and Fibre Channel Enterprise Storage Array 12000 ESA12000 house two HSG80 dual channel controllers There are provisions for six UltraSCSI channels The TruCluster Software Products support only disk devices attached to the HSG80 Fibre Channel to Six Channel UltraSCSI Array controller No tape devices are supported Tru64 UNIX Version 4 0F limit...

Page 28: ...l automatically fail over to the bottom controller and vice versa You can configure which controller presents each HSG80 storage element unit to the cluster Ordinarily the left port of either controller serves the units that are designated D0 through D99 and the right port serves those that are designated D100 through D199 However as previously mentioned because only eight units can be presented t...

Page 29: ...e cluster member system s perspective there is little difference in the configurations 3 1 Using the emxmgr Utility to Display Fibre Channel Adapter Information Although your primary use of the emxmgr utility will be to remap target IDs you often use the utility to display Fibre Channel information Use the usr sbin emxmgr d command to display the presence of KGPSA Fibre Channel adapters on the sys...

Page 30: ...id 1 2 portname 5000 1FE1 0001 8931 nodename 5000 1FE1 0001 8930 Present Logged in FCP Target FCP Logged in N_Port at FC DID 0x210213 SCSI tgt id 2 2 portname 5000 1FE1 0001 8941 nodename 5000 1FE1 0001 8940 Present Logged in FCP Target FCP Logged in N_Port at FC DID 0x210313 SCSI tgt id 4 2 portname 5000 1FE1 0001 8942 nodename 5000 1FE1 0001 8940 Present Logged in FCP Target FCP Logged in N_Port...

Page 31: ...n The adapter and remote N_Port have exchanged initialization parameters and have an open channel for communications nonprotocol specific communications FCP Target This N_Port acts as a SCSI target device it receives SCSI commands FCP Logged in The adapter and remote N_Port have exchanged FCP specific initialization parameters and have an open channel for communications Fibre Channel protocol spec...

Page 32: ...C DID 0x000002 SCSI tgt id 6 portname 1000 0000 C920 043C nodename 1000 0000 C920 043C Present Logged in FCP Initiator FCP Target FCP Logged in N_Port at FC DID 0x00006b SCSI tgt id 2 portname 2200 0020 3704 846F nodename 2000 0020 3704 846F Present Logged in FCP Target FCP Logged in N_Port at FC DID 0x00006c SCSI tgt id 3 portname 2200 0020 3704 A822 nodename 2000 0020 3704 A822 Present Logged in...

Page 33: ..._data c file is an empty file If it is not correct or empty the new kernel will have incorrect mapping data The etc emx db etc emx db bak and etc emx info files will be updated according to the contents of the sys data emx_data c file and contain mapping information that is not correct for your configuration Your configuration will not operate correctly 3 2 1 Using the emxmgr Utility Interactively...

Page 34: ... An example using the emxmgr in the interactive mode to modify worldwide name to target ID mapping follows emxmgr Now issuing commands to emx0 Select Option against emx0 1 View adapter s current Topology 2 View adapter s Target Id Mappings 3 Change Target ID Mappings d Display Attached Adapters a Change targeted adapter x Exit 3 Enter input method interactive file help i f h i Enter adapter instan...

Page 35: ...id pn0 pn1 pn2 pn3 nn0 nn1 nn2 nn3 For example 0 5 0x0050 0xe11f 0x0000 0xb20c 0x0050 0xe11f 0x0000 0xb00c 2 The form as reported by the HSG80 controller with the bytes swapped in the 16 bit values instance tgtid FC Port Name FC Node Name For example emx0 5 5000 1FE1 0000 0CB2 5000 1FE1 0000 0CB0 ______________________ Note _______________________ The emxmgr utility will verify that the syntax is ...

Page 36: ...odifications Continue with the change in id mappings y n n y Change in Target id mappings has been performed You can avoid the verification message by using the f force option with the c option 3 8 Using the emx Manager emxmgr Utility ...

Page 37: ...6 Verifying the Fibre Channel device configuration Section 4 7 Determining the device special file name for a disk Section 4 8 Ensure that the member systems the Fibre Channel switch or hub and the HSG80 array controllers are placed within the lengths of the optical cables you will be using ______________________ Note _______________________ The maximum length of the optical cable between the KGPS...

Page 38: ...bnet mask from the front panel or from a PC or terminal with the DS DSGGB AA before you can manage the switch through a Telnet session SNMP or the Web The DSGGA switch has slots to accommodate up to four DS DSGGA AA or eight DS DSGGA AB plug in interface modules Each interface module in turn supports two Giga Bit Interface Converter GBIC modules The GBIC module is the electrical to optical convert...

Page 39: ...ed see Section 4 1 2 5 The switch IP address and subnet mask must be initially set from the front panel except for the DS DSGGB AA 8 port Fibre Channel switch In this case you have to connect a PC or terminal to the switch You must use a Telnet session to set the switch name 6 Reboot the switch to enable the change in IP address and subnet mask to take effect For more information on the individual...

Page 40: ... 255 An additional increment at a count of 255 resets the count to 0 Down Downward triangle Scrolls the menu down which effectively moves up the list of commands or decreases the value that is being displayed Tab Esc Leftward triangle Allows you to tab through multiple optional functions for example the fields in an IP address You can use this button to abort an entry which takes you to the previo...

Page 41: ...ntil you have the address set correctly 3 Use the Enter button to accept the value and step to the next submenu item Ethernet Submask and then repeat step 2 to set the Ethernet subnet mask 4 Press the Enter button to accept the Ethernet subnet mask 5 Press the Tab Esc button repeatedly to get back to the configuration menu in the main menu ____________________ Note _____________________ After chan...

Page 42: ... or terminal to set the Ethernet IP address and subnet mask To set the Ethernet IP address and subnet mask for the DS DSGGB AA switch follow these steps 1 Connect the switch serial port to a terminal or PC COM port with a standard serial cable with a DB9 connector The serial port is only used for initial power on self test POST verification IP address configuration or for resetting factory default...

Page 43: ...ption other n a Allowed to execute commands that end in Show such as dateShow and portShow user user Can execute all commands that end in Show plus any commands from the help menu that do not change the state of the switch for example version and errDump Can change the passwords for all users up to and including the current user s security level admin admin Provides access to all the commands that...

Page 44: ... switch the next time the prompt will include the switch name for example fcsw1 Admin 4 2 Installing and Setting Up the SWXHB 07 Hub The SWXHB 07 hub supports up to seven 1 6025 Gb sec ports The ports can be connected to the KGPSA CA PCI to Fibre Channel host bus adapter or to an HSG80 array controller Unlike the DSGGA switch the SWXHB 07 hub does not have any controls or even a power on switch Si...

Page 45: ...le click into place The GBIC module has a built in guide key that prevents you from inserting it incorrectly Do not use excessive force 3 Connect the optical fiber cables To do this plug one end of an MMF cable into one of the GBIC modules installed in the hub Attach an MMF cable for all active port connections Unused ports or improperly seated GBIC modules remain in loop bypass and do not affect ...

Page 46: ...lation Guide 4 3 Setting Up the HSG80 Array Controller for a Fibre Channel Cluster This section covers setting up the HSG80 controller for operation with Tru64 UNIX and the TruCluster products For more information on installing the HSG80 see the HSG80 Array Controller ACS Version 8 4 Configuration and CLI Reference Guide or the HSG80 Array Controller ACS Version 8 5 CLI Reference Guide The steps d...

Page 47: ... controllers 5 Remove the program card ESD cover and insert the controller s program card Replace the ESD cover 6 Install disks into storage shelves 7 Connect a terminal to the maintenance port on one of the HSG80 controllers You need a local connection to configure the controller for the first time The maintenance port supports serial communication with the following default values 9600 bps 8 dat...

Page 48: ...et other port_1_profile plda 3 set other port_1_topology fabric 4 set other port_2_profile plda 3 set other port_2_topology fabric 4 set failover copy this_controller 5 1 Specifies the host protocol to use Also specifies how the Command Console LUN CCL is handled When set to SCSI 2 the CCL is not fixed at a particular location but floats depending on the configuration For example if D0 and D1 are ...

Page 49: ...6D9 set NEWCON05 unit_offset 0 1 set NEWCON05 operating_system TRU64_UNIX 2 restart this 3 restart other 3 1 Set the relative offset for LUN numbering to 0 2 Specify that the host environment that is connected to the Fibre Channel port is TRU64_UNIX 3 Restart both controllers to cause all changes to take effect Use the show this and show other commands to verify the changes ____________________ No...

Page 50: ...ntroller cannot come up on the loop In particular if you do not set PORT_x_AL_PA multiple ports might attempt to use the default address thus causing a conflict The valid AL_PA addresses are within the range of 0 EF hexadecimal but not all addresses within this range are valid the default value is 69 hexadecimal The list of valid AL_PA addresses is as follows 0x01 0x02 0x04 0x08 0x0F 0x10 0x17 0x1...

Page 51: ...eed to verify their target ID mappings in the operating system Each HSG80 controller s worldwide name ends in 0 for example 5000 1FE1 FF0C EE00 The worldwide name and checksum of an HSG80 controller also known as the node name appears on a sticker placed above the controllers You can also use the SHOW THIS_CONTROLLER Array Controller Software ACS command to display the worldwide name For HSG80 con...

Page 52: ...ces have both a node name and a port name worldwide name both of which are 64 bit numbers Most commands you use with Fibre Channel only show the port name There are multiple ways to obtain the KGPSA port name worldwide name You can obtain the worldwide name from a label on the KGPSA module before you install it You can use the show dev command as follows P00 show dev pga0 0 0 1 0 PGA0 WWN 1000 000...

Page 53: ...PSA CA adapter has an embedded optical interface 3 Install the KGPSA in an open 32 bit or 64 bit PCI slot 4 Insert the fiber optic cable SC connectors into the GLM of the KGPSA BC adapter or into the embedded optical interface of the KGPSA CA adapter The SC connectors are keyed to prevent their being plugged in incorrectly Do not use unnecessary force Remember to remove the transparent plastic cov...

Page 54: ...CD ROM Version 5 6 and later The steps required to set the link type are summarized here see the Wwidmgr User s Manual for complete information and additional examples Assuming that you have the required Version 5 6 console firmware use the wwidmgr utility to set the link type as follows 1 Display the adapter on the system to determine its configuration POO wwidmgr show adapter item adapter WWN Cu...

Page 55: ...connected between the switch or hub KGPSA adapters and HSG80 RAID controller Run the ase_fix_config utility to ensure that the emx KGPSA adapters are configured symmetrically For more information on the ase_fix_config utility see the TruCluster Software Products Software Installation manual If this is an new installation and the software has not been installed install the Fibre Channel hardware Wh...

Page 56: ...ID mapping is written to the emx driver data and backup files etc emx db and etc emx db bak The database allows the mapping to persist across power cycles and reboots A text version of the database file is written to etc emx info The mapping is done on a first device encountered basis Therefore if you boot all cluster member systems each system might have a different target ID mapping and the cont...

Page 57: ... file systems set the root file system writeable and start the network by entering the following commands on each of those systems sbin lsmbstartup sbin bcheckrc sbin init d inet start ____________________ Note _____________________ You only have to execute sbin lsmbstartup if you are using the Logical Storage Manager LSM 6 Use the ftp utility to copy the etc emx db and etc emx db bak files from t...

Page 58: ... or Replacing a Fibre Channel Device When you modify the cluster Fibre Channel storage configuration by adding or replacing a Fibre Channel device you potentially disrupt the worldwide name to target ID mapping that was established when the cluster was established causing this mapping to go out of synchronization across cluster members This section describes the general procedure to reestablish a ...

Page 59: ... it is not present in the configuration In the following example a KGPSA host bus adapter has been replaced usr sbin emxmgr t emx1 emx1 state information 1 Link connection is UP Point to Point Fabric attached FC DID 0x210413 Link is SCSI bus 3 e g scsi3 SCSI target id 3 portname is 1000 0000 C921 18A4 nodename is 1000 0000 C921 18A4 N_Port at FC DID 0x210013 SCSI tgt id 5 portname 5000 1FE1 0001 8...

Page 60: ...the file that you just created for example call the file tmp KGPSA SCSI 3 remapping usr sbin emxmgr c tmp KGPSA SCSI 3 remapping Continue with the change in id mappings y n n y Change in Target id mappings has been performed ____________________ Note _____________________ You can also run the emxmgr in interactive mode and use the file or type the entries when you are prompted 7 Boot the system to...

Page 61: ... recent messages are displayed first dia R i osf 300 more Use the dd command to generate some I O activity by reading from the Fibre Channel disks Visually verify that the correct disk or set of disks is active For example dd if dev rrzb49c of dev null 4 8 Determining the Device Special File Name for a Disk The actual device number of a particular disk will be determined both by The SCSI target as...

Page 62: ...rst KGPSA adapter then its LUN 0 is dev rrz49 and its LUN 1 is dev rrzb49 These device assignments are seldom changed throughout the life of the installation Although you can change these device assignments with the emxmgr utility consider such changes carefully plan ahead of time and do so deliberately 4 26 Installing Fibre Channel Hardware and Configuring the emx Driver ...

Page 63: ... Section 5 1 4 Disks Section 5 1 5 5 1 1 Fibre Channel KGPSA Adapter Failures The failure of a KGPSA adapter has similar effects as that of a KZPBA adapter in a parallel SCSI configuration Although either adapter type is primarily a SCSI interface they differ in medium Certain failure modes and their associated problems do not exist in the Fibre Channel domain For example each Fibre Channel fabric...

Page 64: ... highlights a failing port it is most useful for quickly spotting a problem To access the GUI interface from your browser use the URL http nnn nnn nnn nnn where the http is mandatory and nnn nnn nnn nnn is the IP address of the switch The Telnet interface is useful for managing the switch by typing commands manually making it potentially available to someone dialing in from outside the facility us...

Page 65: ...e SAN Switch Fabric Operating System Management Guide 5 1 3 Fibre Channel Hub Failures The Fibre Channel hub represents a single point of failure SPOF for the SCSI bus that it serves A total failure of one hub will cause the entire shared bus to fail Depending on how the storage is configured this failure can result in the loss of access to some or all of the data If the cluster has two hubs as sh...

Page 66: ...hecksum If you must replace the PCMCIA memory cards themselves you must manually load a fresh copy of the ACS on the new cards off line You must then reboot the HSG80 using the new cards and enter the worldwide name and checksum using the appropriate ACS commands 5 1 5 Disk Failures The HSG80 Array Control Software ACS monitors all of the HSG80 subsystem including the individual disks If a problem...

Page 67: ...ibre Channel based shared bus an additional Fibre Channel switch is required Each Fibre Channel switch will require a 10Mbit sec twisted pair Ethernet connection to the network an IP address and sufficient ports to support the configuration 5 3 Adding and Replacing Fibre Channel Devices This section covers the different ways to add Fibre Channel devices 5 3 1 Adding Disks to an Existing Configurat...

Page 68: ...pter When you replace a KGPSA adapter in a Fibre Channel configuration you must perform the following tasks If this adapter is to be used in an arbitrated loop configuration configure it as described in Section 4 5 3 Update the target ID mapping for the old adapter with the new adapter s worldwide name Delete the connection to the old adapter in all HSG80 controllers to which it was attached This ...

Page 69: ...dename 2000 0000 C921 07F4 Present Logged in FCP Initiator FCP Target FCP Logged in 3 Record the SCSI ID port name and node name of the emx adapter to be replaced In this example the SCSI ID is 7 the port name is 1000 0000 C921 07C4 and the node name is 1000 0000 C921 07C4 ____________________ Note _____________________ If the KGPSA Fibre Channel adapter is faulty to the point that the emxmgr is u...

Page 70: ...d additional information 5 3 3 2 Adding a New Fibre Channel Bus to a Cluster Adding a new KGPSA adapter to a single cluster member implies that an entire new storage bus is being added to the cluster as a whole An additional KGPSA adapter will be added to each cluster member as well along with a Fibre Channel switch or hub HSG80 controllers and cabling required for the storage To add a new shared ...

Page 71: ...switch or hub is invisible to the cluster members because there is no unit specific configuration information about the switch or hub used or stored by each system The worldwide name of the switch is not explicitly known or stored by any system the hub does not have a worldwide name The only considerations to adding or replacing a Fibre Channel switch is configuring the new switch in the 10 Mb Eth...

Page 72: ... itself does not affect the Tru64 UNIX software configuration However to add a new RA8000 or ESA12000 you must follow these steps 1 Configure the disks on the new HSG80 controllers into units Performing this step first allows the various devices to be usable when the member systems are rebooted 2 If this is an arbitrated loop configuration set up the HSG80 for arbitrated loop as described in Secti...

Page 73: ... must first delete the old member as discussed in Section 2 4 of the TruCluster Software Products Administration manual and then add the new member Either operation requires the following The new member must be connected to the cluster s shared storage in the same way as the other cluster members for example its emx0 adapter must be connected to the same switch as the emx0 adapters of other cluste...

Page 74: ...t the new member system Device special files will automatically be created Use the scsimgr command to create the device special files scsimgr scan_bus bus 7 At the HSG80 controller make sure that the connection for the new member s adapter s worldwide name is TRU64_UNIX 8 Boot the system to multi user mode 9 Add the system as a cluster or ASE member 5 4 2 Replacing a Cluster Member System If you a...

Page 75: ..._ Note _____________________ You only have to execute sbin lsmbstartup if you are using the Logical Storage Manager LSM 6 On any other cluster member system execute the usr sbin emxmgr t command to determine the worldwide name of the new KGPSA host bus adapter and the target ID of the KGPSA that is no longer on this bus Repeat for all new KGPSAs 7 Create a file that contains the worldwide name to ...

Page 76: ... the etc emx info file usr sbin emxmgr c etc emx info Continue with the change in id mappings y n n y Change in Target id mappings has been performed 11 Ensure that device special files are created using one of the following methods Reboot the new member system Device special files will be automatically created Use the scsimgr command and manually create the device special files scsimgr scan_bus b...

Page 77: ...A emxmgr Reference Page Because there is no formal documentation provided with the NHD 2 release the emxmgr utility reference page is provided here emxmgr Reference Page A 1 ...

Page 78: ...instance numbers of Fibre Channel host adapters that are currently present on the system such as emx0 emx1 emx2 This instance is then used in place of the command argument emx to identify a specific host adapter on which an operation is to be performed f Forces target ID mapping changes without prompting for confirmation h Provides command help information m Displays the target ID mappings for a s...

Page 79: ...nfiguration for a cluster ______________________ Note _______________________ Any changes made in the file emx_data c and built into the running kernel will supersede any changes made using emxmgr the next time that the system is booted The emxmgr utility interacts with the emx driver to change the driver s mappings The emx driver then saves the revised mappings to the etc emx db and etc emx info ...

Page 80: ...de revised mapping data using the mapping_file option you can specify multiple mapping changes at the same time using one of two syntaxes The following rules apply to both syntaxes Each input line contains either a Fibre Channel worldwide name to CAM target ID mapping or a comment Any text not matching one of the mapping syntaxes is considered a comment The syntaxes for the mapping_file file are a...

Page 81: ...nn3 In this syntax Is the Fibre Channel host adapter instance specified as a single positive decimal value such as 0 for emx0 tgid Is either a decimal value or the string delete If the decimal value is negative it is equivalent to delete pn Are the components of the N _port s port name specified 16 bits at a time byte swapped for endian ness See emx_data c 4 for more information nn Are the compone...

Page 82: ...e node name 5000 1FE1 0000 0CB0 but uses the alternate format 0 5 0x0050 0xe11f 0x0000 0xb20c 0x0050 0xe11f 0x0000 0xb00c FILES usr sbin emxmgr The executable image sys BINARY emx mod The device driver module etc emx db The binary file that contains persistent Fibre Channel worldwide name to CAM target ID mappings sys data emx_data c The configuration file for the emx device driver The emxmgr util...

Page 83: ...emxmgr 8 SEE ALSO emx_data c 4 emx 7 The TruCluster documentation for information on symmetrical configurations emxmgr Reference Page A 7 ...

Page 84: ......

Page 85: ...ommand 5 4 D device special file name determining for Fibre Channel disk 4 25 disks adding 5 5 failure behavior 5 4 E emxmgr 3 1 5 6 displaying adapters 3 1 displaying target ID mapping 3 1 displaying topology 3 2 3 4 reference page A 3 use 3 1 3 3 using interactively 3 6 using with file 3 7 ESA12000 adding 5 10 configuring 2 4 failure behavior 5 3 port configuration 2 4 replacing controllers of 5...

Page 86: ...emx db bak 3 5 4 20 etc emx info 3 5 4 20 var adm messages 4 16 5 7 FL_Port 1 3 ftp 4 21 H host bus adapter adding 5 6 5 8 replacing 5 6 HSG80 controller configuring 2 4 4 10 configuring for loop 4 14 failover mode 4 13 failure behavior 5 3 nocommand_console_lun 4 12 obtaining the worldwide name of 4 15 PLDA addressing 4 12 port configuration 2 4 port_n_topology 4 12 4 14 replacing 5 3 scsi_versio...

Page 87: ...0 adding 5 10 configuring 2 4 failure behavior 5 3 port configuration 2 4 replacing controllers of 5 3 transparent failover mode 2 4 unit configuration 2 4 replacing host bus adapter 5 6 HSG80 controllers 5 3 KGPSA 5 6 required firmware Fibre Channel 2 1 restrictions 2 3 5 5 S SAVE_CONFIGURATION command 5 4 SC connector 4 2 4 8 4 17 SCSI LUNs number of 2 3 SCSI targets number of 2 3 scu 5 5 SHOW T...

Page 88: ... emx_data c 3 5 T target ID assigning to Fibre Channel devices 4 20 remapping 4 22 5 6 5 8 5 12 synchronizing the mapping of 4 20 telnet managing a Fibre Channel switch using 5 2 troubleshooting 5 1 U units adding 5 6 moving 5 6 V var adm messages 4 16 5 7 W worldwide name description 4 16 mapping to target ID 4 20 remapping 4 22 5 6 5 8 5 12 Index 4 ...