IBM SAN768B, Installation, Service And User Manual

Page 1: ... and User Guide Service information 2499 384 GA32 0574 05 Read Before Using This product contains software that is licensed under written license agreements Your use of such software is subject to the license agreements under which they are provided ...

Page 2: ......

Page 3: ...IBM System Storage SAN768B Installation Service and User Guide Service information 2499 384 GA32 0574 05 ...

Page 4: ... MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND EITHER EXPRESS OR IMPLIED INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE Some states or regions do not allow disclaimer of express or implied warranties in certain transactions therefore this statement may not apply to you Copyright IBM Corporation 2008 201...

Page 5: ... v New mini SFP mSFP transceivers associated with the FC8 64 blade v Introduction of 8 Gbps FICON Accelerator v Addition of the Server Application Optimization SAO optional feature v A change to the Fabric Operating System FOS version required for the FC8 64 blade v Update of the Notices section Fourth edition The following changes were made in the this edition v The addition of the CEE 10GbE 24 p...

Page 6: ...Search for the product Machine type or product name For Fabric OS Release Notes and access to Fabric OS firmware downloads go to the IBM Support Portal www ibm com supportportal Search for the product Machine type or product name and then follow links for Downloads More detailed instructions are available through the Accessing firmware updates and OS documentation updates link on the product docum...

Page 7: ...omments or suggestions for improving this document send us your comments by e mail to starpubs us ibm com Be sure to include the following v Exact publication title v Publication form number for example GC26 1234 02 v Page table or illustration numbers v A detailed description of any information that should be changed Read this first v ...

Page 8: ...vi SAN768B Installation Service and User Guide ...

Page 9: ...17 Removing the chassis door and cable management comb 18 Using the lift tool the 24 inch load plate and the bridge tool 19 Port numbering 25 Chassis slots 26 Cable organization 27 Removing a SAN768B from the cabinet 28 Chapter 3 Starting and configuring the SAN768B 31 Providing power to the SAN768B 33 Establishing a serial connection and logging on to the SAN768B 33 Logging in to the serial conso...

Page 10: ...95 Removing and installing transceivers 96 Time required 96 Items required 96 Removing and replacing an SFP SFP or XFP optical transceiver 96 Removing and replacing an mSFP optical transceiver 97 Removing and installing inter chassis link ICL cables 98 Time and items required 99 Removing an ICL cable 99 Installing an ICL cable 100 Removing the batteries 107 Removing and replacing the chassis 109 V...

Page 11: ...D Blade port numbering 153 Notices 163 Trademarks 165 Electronic emission notices 166 Federal Communications Commission FCC Class A Statement 166 Industry Canada Class A Emission Compliance Statement 166 Avis de conformité à la réglementation d Industrie Canada 166 European Union EMC Directive Conformance Statement 166 Germany Electromagnetic Compatibility Directive 167 People s Republic of China ...

Page 12: ...x SAN768B Installation Service and User Guide ...

Page 13: ...Cable management comb 71 31 Port application and encryption blade removal and installation FC8 48 port blade shown as example 74 32 Filler panel removal and installation 76 33 Removing the control processor CP8 blade 79 34 Removing the core switch blade 83 35 FR4 18i router blade and components 85 36 Power supply locations 88 37 Removing and installing a power supply 90 38 Removal and replacement ...

Page 14: ...xii SAN768B Installation Service and User Guide ...

Page 15: ... failure 66 12 WWN bezel LED descriptions 67 13 Commands identifying the WWN card status 93 14 WWN card related system log messages 93 15 ICL connector port LEDs 99 16 Critical information checklist example 111 17 Sample cable routing table for SAN768B 119 18 Cable routing table for SAN768B 64 ports shown 120 19 General specifications 135 20 System architecture 135 21 System sizes and weights 138 ...

Page 16: ...xiv SAN768B Installation Service and User Guide ...

Page 17: ...the danger caution and attention notices in the sections that follow v Note These notices provide important tips guidance or advice v Attention notices on page xxi These notices indicate potential damage to programs devices or data v Caution notices on page xviii These statements indicate situations that can be potentially hazardous to you v Danger notices These statements indicate situations that...

Page 18: ...place dangerous voltage on the metal shell If any of the conditions are not as described STOP Ensure the improper voltage or impedance conditions are corrected before proceeding D003 DANGER An electrical outlet that is not correctly wired could place hazardous voltage on metal parts of the system or the devices that attach to the system It is the responsibility of the customer to ensure that the o...

Page 19: ...ible use one hand only to connect or disconnect signal cables v Never turn on any equipment when there is evidence of fire water or structural damage v Disconnect the attached power cords telecommunications systems networks and modems before you open the device covers unless instructed otherwise in the installation and configuration procedures v Connect and disconnect cables as described below whe...

Page 20: ...nt relocation or disposal 500 lbs 227 kg a69i0333 DANGER Heavy equipment personal injury or equipment damage might result if mishandled D006 Caution notices A caution notice calls attention to a situation that is potentially hazardous to people because of some existing condition A caution notice can be accompanied by different symbols as in the examples below If the symbol is It means A hazardous ...

Page 21: ...rical outlet to avoid electrical shock C018 CAUTION This product might contain one or more of the following devices CD ROM drive DVD ROM drive DVD RAM drive or laser module which are Class 1 laser products Note the following information v Do not remove the covers Removing the covers of the laser product could result in exposure to hazardous laser radiation There are no serviceable parts inside the...

Page 22: ...ANGER Hazardous voltage current or energy levels are present inside any component that has this label attached Do not open any cover or barrier that contains this label L001 DANGER Rack mounted devices are not to be used as a shelf or work space L002 DANGER Multiple power cords The product might be equipped with multiple power cords To remove all hazardous voltages disconnect all power cords L003 ...

Page 23: ...ibre cable to a radius less than 5 cm 2 in you can damage the cable Tie wraps are not recommended for optical cables because they can be easily overtightened causing damage to the cable ESD precautions Attention Many of the field replaceable units FRUs are sensitive to electrostatic discharge ESD and can potentially be damaged by improper handling Wear a wrist grounding strap connected to chassis ...

Page 24: ... to ensure that the outlet is correctly wired and grounded to prevent an electrical shock R001 part 1 of 2 CAUTION v Do not install a unit in a rack where the internal rack ambient temperatures will exceed the manufacturer s recommended ambient temperature for all your rack mounted devices v Do not install a unit in a rack where the air flow is compromised Ensure that air flow is not blocked or re...

Page 25: ...Verify that the route that you choose can support the weight of the loaded rack cabinet Refer to the documentation that came with your rack cabinet for the weight of a loaded rack cabinet Verify that all door openings are at least 760 x 2030 mm 30 x 80 in Ensure that all devices shelves drawers doors and cables are secure Ensure that the four leveling pads are raised to their highest position Ensu...

Page 26: ...ower cable for damage 4 Check the external signal cable for damage 5 Check the cover for sharp edges damage or alterations that expose the internal parts of the device 6 Check that any unused serial ports are covered for dust and ESD protection The cover should be kept on the serial port whenever it is not being used 7 Correct any problems that you find Internal machine checks Perform the followin...

Page 27: ...r translated environmental statements and information regarding product recycling and disposal This document may be provided either in printed version or on the product documentation CD See Removing the batteries on page 107 for battery removal instructions if needed to meet environmental regulations Safety and environmental notices xxv ...

Page 28: ...xxvi SAN768B Installation Service and User Guide ...

Page 29: ...al on the documentation CD that is shipped with the product or available on the web through the IBM Support Portal or IBM Publications Center v IBM System Storage SAN768B Installation Service and User Guide GA32 0574 this document which is also available in accessible HTML format on the documentation CD v IBM System Storage and TotalStorage b type Director and Cabinet Safety Notices v IBM Systems ...

Page 30: ...000 IBM Converged Switch B32 3758 Models B32 and L32 Brocade 7800 SAN06B R 2498 Model R06 Brocade 7500E SAN04B R 2005 Model R04 Brocade 5300 SAN80B 4 2498 Model B80 Brocade 5100 SAN40B 4 2498 Models B40 and 40E Brocade 300 SAN24B 4 2498 Models B24 and 24E Accessibility features for SAN768B Accessibility features help users who have a disability such as restricted mobility or limited vision to use ...

Page 31: ...y information about its products Related accessibility information You can view the publications for the SAN768B in Adobe Portable Document Format PDF using the Adobe Acrobat Reader The PDFs are provided on a CD that is packaged with the product IBM and accessibility See the IBM Human Ability and Accessibility Center for more information about the commitment that IBM has to accessibility www ibm c...

Page 32: ...xxx SAN768B Installation Service and User Guide ...

Page 33: ...ns v Support for high performance port blades running at 1 2 4 8 or 10 Gbps enabling flexible system configuration v Up to 1024 ports in the same rack can connect with the use of inter chassis links ICLs Up 1536 ports can be connected via ICLs in adjacent racks v Supports 1 2 4 and 8 Gbps auto sensing Fibre Channel ports Trunking technology groups up to eight ports to create high performance 64 Gb...

Page 34: ...t 12 FC 10 GbE 2 10GbE FCIP extension blade enabling long distance communication over existing IP infrastructure FCOE10 24 24 port 24 10GbE CEE based FCoE blade enabling enhanced connectivity using existing Ethernet infrastructure This blade cannot be used in the same chassis as the high density port blade FC8 64 or any of the application blades FX8 24 FR4 18i or FS8 18 v Modular hot swappable enc...

Page 35: ...upply has its own connector Note Airflow in the SAN768B is from the non port non cable side to the port cable side and out the exhaust vent Figure 1 shows a sample configuration of the port side of the SAN768B with the chassis door removed 1 Exhaust vent 3 Control processor blade CP8 2 Core switch blade CR8 4 FC8 48 port blade 5 Cable management comb Figure 2 on page 4 shows a sample configuration...

Page 36: ...ailable for the SAN768B Table 2 Blades available for the SAN768B Description Feature Code Name Function Control processor blade N A CP8 The CP8 blade contains the control plane for the chassis There are two CP8 blades for redundancy b768ig014 Figure 2 Blower non port side of the SAN768B sample configuration 4 SAN768B Installation Service and User Guide ...

Page 37: ...connections for the storage server blade 16 port 8 Gbps port blade FC3816 FC8 16 A 16 port blade supporting 1 2 4 and 8 Gbps port speeds 32 port 8 Gbps port blade FC3832 FC8 32 A 32 port blade supporting 1 2 4 and 8 Gbps port speeds 48 port 8 Gbps port blade FC3848 FC8 48 A 48 port blade supporting 1 2 4 and 8 Gbps port speeds 64 port 8 Gbps port blade FC3864 FC8 64 A 64 port port blade supporting...

Page 38: ...ports and two 10 GbE ports available Fibre Channel over Ethernet blade FC3880 FCOE10 24 The FCOE10 24 blade enables FCoE functionality over existing Ethernet infrastructure utilizing CEE protocols It has twenty four 10 GbE ports available This blade cannot be used in the same chassis with an FC8 64 high density port blade or any of the other application blades FR4 18i FX8 24 or FS8 18 Encryption b...

Page 39: ...ection and correction mechanisms to ensure reliability of data v Data is protected by the Error Detection and Correction mechanism which checks for encoder errors and fault isolation such as cyclic redundancy checking CRC parity checking checksum and illegal address checking v Power on self test POST v Dual control processors that enable hot nondisruptive fast firmware upgrades v Each control proc...

Page 40: ... in a SAN768B SAN80B 4 and SAN40B 4 to be configured as an EX_port supporting Fibre Channel FC routing This eliminates the need to add a routing blade or use of the SAN18B R for FC routing FCR purposes and this also provides double the bandwidth for each FC router connection when connected to another 8 Gbps capable port v 8 Gbps FICON Accelerator an optional feature FC 7893 leverages emulation tec...

Page 41: ...t domain ID Upfront login in Web Tools E_port disable Network Manageability The SAN768B has a single domain and is managed as a single element with the Data Center Fabric Manager Enterprise Fabric Connectivity Manager EFCM or Fabric Manager The SAN768B responds to its own IP address and appears as a separate entity to the Telnet protocol and SNMP All management interfaces such as Telnet Web Tools ...

Page 42: ...10 SAN768B Installation Service and User Guide ...

Page 43: ...B One SAN768B is installed in the cabinet at the factory If the SAN768B is already installed in the cabinet continue on to Chapter 3 Starting and configuring the SAN768B on page 31 Note Some illustrations in this chapter display different models of IBM SAN directors Details such as the cable management comb and blades may differ but the procedures remain the same for the SAN768B Attention Refer to...

Page 44: ...during operation Ordering the lift tool Important The lift tool is required only when you install an additional SAN768B or when you install or remove the SAN768B from the cabinet The ordering procedures for the lift tool vary depending on your location You should direct questions about these procedures to your regional representative Note The bridge tool PN 18P5855 which is required for installati...

Page 45: ...ou are responsible for ensuring that all of the paperwork and components are packed and restored in the arbocrate shipping container of the lift tool Ensure that the lift tool is functioning properly before you release the tool to UPS Logistics for return You are accountable for the lift tool until UPS Logistics picks up the lift tool for return delivery to their parts storage facility Contact you...

Page 46: ...intake side is less than 40 degrees Celsius 104 degrees Fahrenheit during operation CAUTION Use safe lifting practices when moving the product C015 Note A fully populated SAN768B eight FC8 64 port cards 512 ports weighs approximately 159 2 kg 351 lb and requires the specified lift tool to install it Installing a SAN768B in a cabinet This section describes how to install the SAN768B in the Model C3...

Page 47: ...t shelf bracket rail brackets may differ from the illustration 1 C 10 32 x 5 8 inch 1 58 cm panhead Phillips screw washer 6 For cabinets that have vertical rails with round holes D 10 32 clip nut 6 E 1 4 20 x 1 2 in 1 27 cm panhead Phillips screw with lock washer 16 For cabinets that have vertical rails with square holes F 10 32 retainer nut 6 G 1 4 20 x 1 2 in 1 27 cm panhead Phillips screw with ...

Page 48: ...on top of the plastic shipping tray if the chassis must be transported to the installation location Note The standard packaging does not incorporate a wood pallet and pallet brackets The chassis sits on top of a plastic shipping tray 5 Use the lift tool to transport the new chassis to the installation area Doorways must be wider than 91 cm 36 in to accommodate the chassis on the shipping tray E 16...

Page 49: ...Tighten the screws to a torque of 92 cm kg 80 in lb v For rails with square holes Use eight screws G eight alignment washers H for each bracket four at each end Tighten the screws to a torque of 92 cm kg 80 in lb Note Standard EIA rails have holes in sets of three spaces between the holes are 1 58 cm 1 58 cm 1 27 cm 5 8 in 5 8 in and 1 2 in If cables are to be routed down through the cable managem...

Page 50: ... be installed into the cabinet The door is not hinged It is held to the chassis by mounting hardware on the door that snaps onto eight ball studs on the chassis To remove the door grip both sides of the door and gently pull the door straight out until it releases from the ball studs on the chassis Set the door aside for later reinstallation Note Reinstall the chassis door once the chassis is insta...

Page 51: ...et for the lower SAN768B or the upper SAN768B This procedure uses parts from the 14U chassis mount kit See Items required on page 110 for a list of these parts svc00169 55 kg 121 2 lbs CAUTION A fully populated SAN768B weighs approximately 159 2 kg 351 lb and requires a minimum of two people and a lift tool to install it C011 Attention Before you install it verify that the additional weight of the...

Page 52: ...e load platform to the same height b Slide the SAN768B from the pallet onto the load platform c Center the switch on the load platform 5 Attach the retention straps around the drawer 6 If the load platform is not at its lowest position lower the load platform to its lowest position 7 Move the lift tool to a position near the rear of the cabinet See Figure 7 Note If you are passing through a narrow...

Page 53: ...box and adjust the two supports 45 b Hold the bridge tool between the two vertical supports near EIA unit 3 and move the two supports another 45 Make sure that the following conditions are met v The two pin section of the support is on the back of the vertical support v The one pin section of the support is on the front of the vertical support c Align the two pin section of the support on the back...

Page 54: ...ottom hole and the middle hole in EIA unit 20 d Align the one pin section of the support on the front of the vertical rail with the middle hole in EIA unit 18 13 Turn the winch crank counterclockwise 1 4 turn to set the winch brake 14 Set the wheel brake and place the wheel chocks around the wheels 15 Remove the retention straps 16 Slide the SAN768B into the cabinet until it rests securely on the ...

Page 55: ...or 21 Reinstall the cable management comb if it was removed see Removing and installing a cable management comb on page 70 22 If ICL cables will not be used ensure that the ICL sockets in the core switch blades have EMI plugs inserted 23 Reinstall the chassis door Align the door with the chassis and push it into place as shown in Figure 10 on page 24 and Figure 11 on page 25 b768ig031 Figure 9 Att...

Page 56: ...b768ig032 Figure 10 Aligning the chassis door with the ball studs on the chassis 24 SAN768B Installation Service and User Guide ...

Page 57: ...FX8 24 FCIP extension blade 3890 FCoE10 24 Fibre Channel over Ethernet blade 3880 FS8 18 16 port encryption blade 3895 The SAN768B uses the following port numbering scheme for each of the supported blades see also Appendix D Blade port numbering on page 153 for sample configurations v FC8 16 port blade Ports are numbered from 0 through 15 from bottom to top v FC8 32 port blade Ports are numbered f...

Page 58: ...ernet 10GbE or XGE ports are located in the middle of the blade on the left side and are numbered 0 and 1 from bottom to top Ten GbE ports are located towards the top of the blade with eight clustered at the top and the remaining two located to the right of the two 10GbE ports The GbE ports 0 through 3 numbered from bottom to top are located on the left side at the top of the blade Ports 4 and 5 a...

Page 59: ...offered by several major manufacturers Contact your IBM representative for options regarding different cable and patch panel configurations to simplify cable management with higher density FC8 64 port blades 1 mini SFP connector 3 1 6 mm cable 2 Duplex clip black 4 SFP connector Note that the duplex clip on the mSFP end of the cable is black for easier recognition See Table 25 on page 141 for a li...

Page 60: ...See Removing the chassis door and cable management comb on page 18 5 Remove the three 10 32 x 5 8 in Phillips panhead screws with washers on each side See Figure 9 on page 23 for the location of the screws 6 If you are replacing the lower SAN768B go to step 7 If you are replacing the upper SAN768B go to step 8 7 Install the bridge tool by completing the following steps a Remove the bridge tool fro...

Page 61: ...bracket d Secure the anti tip bars with the anti tip bracket and bolt 10 Move the lift tool to a position near the rear of the cabinet 11 Turn the winch crank clockwise to raise the switch Move the lift tool into position in front of the cabinet while you align the 24 inch load plate with the line on the bridge tool 12 Turn the winch crank counterclockwise 1 4 turn to set the winch brake 13 Set th...

Page 62: ...30 SAN768B Installation Service and User Guide ...

Page 63: ...d gateway addresses for both CP blades v Switch name v Domain ID for the SAN768B optional v WWN for the SAN768B Note The SAN768B WWN is initially set by the factory to match the license ID which is based on the chassis serial number The WWN can be changed but the license ID cannot be modified The configuration information is mirrored to the standby CP blade which allows the most current configurat...

Page 64: ...D mode and connect the SAN768B to the fabric See Verifying the PID mode and connecting to the fabric on page 39 5 Enable software licenses See Enabling software licenses on page 39 6 Configure the optional FS8 18 encryption blade See Configuring for encryption optional on page 42 7 Back up the configuration See Verifying correct operation and backing up the configuration on page 43 Configuration T...

Page 65: ...onnection and logging on to the SAN768B Note The preferred method of establishing a connection to the SAN768B is via the serial port However if a serial connection is not possible for example if a serial cable is not available then an Ethernet connection can be established using an Ethernet crossover cable To establish an Ethernet connection you must know the IP address of the control processor To...

Page 66: ...al Modify passwords To skip modifying the password press Ctrl C Passwords can be 8 to 40 characters long They must begin with an alphabetic character They can include numeric characters the dot and the underscore _ Passwords are case sensitive and they are not displayed when you enter them on the command line For more information on passwords refer to the Fabric OS Administrator s Guide Configurin...

Page 67: ...cp 1 Enter the requested information at the prompts The following is a sample IP configuration for the SAN768B chassis and the two CP blades cp0 and cp1 swDir admin ipaddrset chassis Ethernet IP Address 0 0 0 0 123 123 123 120 Ethernet Subnetmask 0 0 0 0 123 123 123 123 Fibre Channel IP Address 0 0 0 0 Fibre Channel Subnetmask 0 0 0 0 Issuing gratuitous ARP Done Committing configuration Done swDir...

Page 68: ...d by remote connection using any of the available management tools such as such as Telnet Web Tools or DCFM 4 To complete any additional SAN768B configuration procedures through a Telnet session log in to the SAN768B by Telnet using the admin login The default password is password Customize a switch name The switch name of the SAN768B can be up to 30 characters long as of Fabric OS release 6 3 0 c...

Page 69: ...and time settings are used for logging events Switch operation does not depend on the date and time a switch with an incorrect date and time value still functions properly However because the date and time are used for logging error detection and troubleshooting they should be set correctly Setting the date To set the date follow these steps 1 If necessary log on to the SAN768B by Telnet using the...

Page 70: ...To synchronize the local time of the principal or primary switch with that of an external NTP server follow these steps 1 If necessary log on to the switch by Telnet using the admin account 2 switch admin tsclockserver ntp1 ntp2 where ntp1 is the IP address or DNS name of the first NTP server which the switch must be able to access The variable ntp2 is the second NTP server and is optional The ope...

Page 71: ...ation about the fabric Enabling software licenses To determine which licenses are currently enabled enter the licenseShow command swDir admin licenseshow AAbbccDDeeFFeeGG Web license Zoning license Extended Fabric license Fabric Watch license Performance Monitor license Trunking license Security license In the example above the license key is AAbbccDDeeFFeeGG Make sure you keep a copy of your lice...

Page 72: ...erted with the correct orientation If a cable does not slide in easily ensure that it is correctly oriented 3 Repeat steps 1 and 2 for the remaining ports 4 Organize the cables as required For recommendations regarding cable management refer to Managing cables 5 Verify the SAN768B and port status using the switchShow command 6 Verify fabric connectivity using the fabricShow command Note The chassi...

Page 73: ...ential to enabling a secure encryption functionality The following guidelines should be followed when planning for encryption with the 16 port encryption blade FS8 18 which is available for the SAN768B and SAN384B products or the SAN32B E4 v Redundancy of hardware is essential because if the encryption path is disrupted access to the encrypted data will be lost with a single encryption device You ...

Page 74: ...ld be registered before exporting MK or encrypting LUNs If the secondary key vault is registered midway after encryption is done for some of the LUNs then the key database should be backed up and restored on the secondary TKLM from the already registered primary TKLM before registering the secondary TKLM The following is a suggested order for the initial steps needed to create a secure connection ...

Page 75: ...rformance and does not add security Verifying correct operation and backing up the configuration To verify correct operation and to back up the system follow these steps 1 Check the LEDs to verify that all components are functional For information about LED patterns see Chapter 4 Monitoring system components on page 47 2 If necessary log on to the switch by Telnet using the admin account Encryptio...

Page 76: ...able for downloading to a replacement device Fabric OS firmware updates To ensure optimum functioning of your system IBM recommends operating your system with the latest Fabric OS version and firmware updates Refer to the latest Fabric OS Release Notes and Interoperability Matrix for information regarding Fabric OS compatibility This is particularly important for fabrics with 1 Gbps switches using...

Page 77: ...abric OS Administrator s Guide as needed 3 Enter firmwareDownload s on your to download the firmware to one of the CP blades from your server Powering off the SAN768B If you need to power off the system follow these steps to avoid loss of data 1 Shut down the system using the sysShutdown command switch admin sysshutdown This command will shutdown the operating systems on your switch You are requir...

Page 78: ...46 SAN768B Installation Service and User Guide ...

Page 79: ...he LEDs on the blade v Figure 15 on page 48 illustrates the FC8 16 port blade v Figure 16 on page 49 illustrates the FC8 32 port blade v Figure 17 on page 50 illustrates the FC8 48 port blade v Figure 18 on page 51 illustrates the FC8 64 port blade v Figure 19 on page 52 illustrates the FC10 6 port blade v Figure 20 on page 53 illustrates the FR4 18i routing blade v Figure 21 on page 54 illustrate...

Page 80: ... 1 Blade power LED 3 Fibre Channel port 2 Blade status LED 4 Port status LED 3 4 1 2 b768ig026 Figure 15 FC8 16 port blade 48 SAN768B Installation Service and User Guide ...

Page 81: ... 1 Blade power LED 3 Fibre Channel port 2 Blade status LED 4 Port status LED 1 3 4 2 b768ig025 Figure 16 FC8 32 port blade Chapter 4 Monitoring system components 49 ...

Page 82: ...rt status LED 32 8 33 9 34 10 35 11 24 0 25 1 26 2 27 3 28 4 29 5 30 6 31 7 32 8 33 9 34 10 35 11 36 12 37 13 38 14 39 15 40 16 41 17 42 18 43 19 44 20 45 21 46 22 47 23 3 4 47 1 2 b768ig024 Figure 17 FC8 48 port blade 50 SAN768B Installation Service and User Guide ...

Page 83: ...atus LED 3 Fibre Channel port 2 Blade power LED 4 Port status LED Note The FC8 64 port blade requires narrower OM 3 LC cables 3 4 1 2 B768ig046 Figure 18 FC8 64 port blade Chapter 4 Monitoring system components 51 ...

Page 84: ... 1 Blade power LED 3 Fibre Channel port 2 Blade status LED 4 Port status LED 4 3 1 2 b768ig041 Figure 19 FC10 6 port blade 52 SAN768B Installation Service and User Guide ...

Page 85: ... LED 3 Fibre Channel port 2 Blade status LED 4 Port status LED 7 6 5 4 3 2 1 0 15 14 13 12 11 10 9 8 FR4 18i GE1 GE0 1 2 7 6 5 4 3 4 B768ig049 Figure 20 FR4 18i routing blade Chapter 4 Monitoring system components 53 ...

Page 86: ... 1 Blade power LED 3 Fibre Channel port 2 Blade status LED 4 Port status LED 3 4 b768ig060 1 2 Figure 21 FS8 18 encryption blade 54 SAN768B Installation Service and User Guide ...

Page 87: ...de power LED 4 GbE port 6 LED 7 Port map 2 Blade status LED 5 10 GbE port 0 3 GbE port 6 6 10 GbE port 0 LED 4 3 5 6 7 2 1 b768ig047 Figure 22 FX8 24 extension blade Chapter 4 Monitoring system components 55 ...

Page 88: ...runking groups on this blade 1 Blade power LED 4 10 GbE FCoE port 9 LED 2 Blade status LED 5 10 GbE FCoE port 19 3 10 GbE FCoE port 9 6 10 GbE FCoE port 19 LED 2 1 4 3 5 6 b768ig048 Figure 23 FCOE10 24 blade 56 SAN768B Installation Service and User Guide ...

Page 89: ...ght LED is off Blade is either healthy or does not have power Verify that the power LED is on Steady amber Blade is faulty Ensure blade is firmly seated and check status by entering the slotShow command Blade may not be compatible with other blades in the chassis See FCOE10 24 blade on page 145 for limitations If LED remains amber contact IBM Slow flashing amber on 2 seconds then off 2 seconds Bla...

Page 90: ... diagnostic tests or portDisable or portCfgPersistentEnable command Reset the port from the workstation using the portEnable or portCfgPersistentEnable command Fast flashing amber on 1 2 second then off 1 2 second Transceiver or port is faulty Change the transceiver or reset the switch from the workstation Alternating green amber Port is bypassed Reset port from workstation using the portEnable or...

Page 91: ... command from a cmsh session Fast flashing amber on 1 4 second then off 1 4 second Transceiver or port is faulty Change the transceiver or reset the port by doing a shutdown and no shutdown sequence from a cmsh session For more information on the CEE Management Shell cmsh see the Fabric OS Command Reference Manual or the Converged Enhanced Ethernet Command Reference Control processor blade CP8 sta...

Page 92: ...ED Table 7 CP8 blade LED descriptions LED purpose Color Status Recommended Action Blade power Steady green CP blade is on No action required No light LED is off CP blade is not on Ensure blade is firmly seated and has power 3 4 5 6 7 8 1 2 b768ig023 Figure 24 Control processor blade CP8 components 60 SAN768B Installation Service and User Guide ...

Page 93: ...ower Ethernet cable is firmly seated and connected device is functioning Flickering green amber Ethernet link is healthy and traffic is flowing through port No action required Ethernet link speed No light LED is off Ethernet link speed is 10 Mbps or CP blade does not have incoming power Ensure CP has power Note To force a persistent Ethernet link speed type the ifModeSet command Steady green Ether...

Page 94: ... Status Recommended Action Blade power Steady green CP blade is on No action required No light LED is off CP blade is not on Ensure that the blade is firmly seated and has power 1 2 ICL 0 LINK ATTN ICL 0 LINK ATTN ICL 1 LINK ATTN 3 4 5 b768ig022 Figure 25 Core switch blade CR8 LEDs 62 SAN768B Installation Service and User Guide ...

Page 95: ...d Environmental range exceeded Check for out of bounds environmental condition and correct it LINK LED No light LED is off No ICL or ICL is cable present but no connection Ensure that the ICL cable is correctly installed Steady green ICL connected No action required ATTN LED No light LED is off No ICL No action required Blinking amber Attention required Ensure that the ICL cable is correctly insta...

Page 96: ...y is about to fail Replace power supply Blower assembly status To determine the status of a blower assembly 1 Check the LED indicators on the blower assembly see Figure 27 on page 65 The LED patterns may temporarily change during POST and other diagnostic tests for information about how to interpret the LED patterns see Table 10 on page 65 2 Check the blower assembly status using the fanShow comma...

Page 97: ...lor Status Recommended action Power No Light LED is off Blower assembly does not have power Ensure that the blower assembly is firmly seated and has power Steady green Blower assembly has power No action required 1 2 b768ig042 Figure 27 Blower assembly LEDs Chapter 4 Monitoring system components 65 ...

Page 98: ...termine the status of the WWN card Note The WWN bezel covers the WWN cards The LEDs on the WWN cards are not visible unless the bezel is removed 1 Enter the chassisShow command to display information about the WWN card WWN units correspond to information specific to the WWN card see Figure 39 on page 95 Error messages that may indicate problems with a WWN card are summarized in Table 11 Table 11 M...

Page 99: ...ezel LED descriptions LED purpose Color Status Recommended action Port blade CP blade CR blade Power Steady green Power is OK No action required Port blade CP blade CR blade Status Steady amber Blade is faulty Check blade No light LED is OFF Blade is OK No action required Note If a blade slot or power supply bay has a filler panel installed the corresponding LEDs on the WWN blade do not light up 2...

Page 100: ...68 SAN768B Installation Service and User Guide ...

Page 101: ...ge 82 v Removing and installing an FR4 18i router blade on page 84 v Removing and installing a power supply on page 88 v Removing and installing a blower assembly on page 91 v Removing and installing a WWN bezel and WWN card on page 92 v Removing and installing transceivers on page 96 v Removing and installing inter chassis link ICL cables on page 98 v Removing the batteries on page 107 Removing a...

Page 102: ...g a cable management comb To remove the cable management comb do the following 1 Remove the chassis door Removing and installing the chassis door on page 69 2 Remove the cables from the cable management comb and rearrange around the comb 3 Unscrew and save the four screws holding the comb to the chassis see Figure 30 on page 71 Support the comb to prevent it from falling 4 Remove the cable managem...

Page 103: ... encryption blades This section describes how to remove and install port application router and encryption See Removing and installing an FR4 18i router blade on page 84 for instructions specific to that application blade Attention A filler panel should be removed only when being replaced with another blade or a filler panel Any slot that is not occupied by a blade should be occupied by a filler p...

Page 104: ...ng the chassis door on page 69 2 A failed port or application blade can be identified by inspecting the LEDs on the front panel of each blade Check the power LED status LED and port status LED to identify any possible problems See Figure 15 on page 48 to Figure 23 on page 56 for LED locations 3 Establish a Telnet or console session Before installing a blade establish a Telnet or console connection...

Page 105: ... the top ejector down to the off position This initiates a hot swap request 9 Wait for the power LED to turn off in response to the hot swap request before removing the blade 10 Perform the appropriate following action based on the type of blade v For FC8 16 FC8 32 FC8 48 FC8 64 port blades FX8 24 and FCOE10 24 application blades and FS8 18 encryption blades Open the ejectors by pulling them towar...

Page 106: ...rform the appropriate following action based on the type of blade v For FC8 16 FC8 32 FC8 48 FC8 64 port blades FX8 24 and FCOE10 24 application blades and FS8 18 encryption blades Adjust the ejectors to the open position align the flat side of the blade inside the upper and lower rail guides in the slot and slide the blade into the slot until it is firmly seated v For FC10 6 port blade and the FR...

Page 107: ... 6 v For FC10 6 port blade and the FR4 18i application blades Turn the blade on by sliding the slider switch in the top ejector up covering the thumbscrew 6 Verify that the power LED on the blade displays a steady green light it might require a few seconds to turn on If it does not turn on ensure that the blade is firmly seated 7 Install the SFP SFP or XFP FC10 6 only transceivers and cables in th...

Page 108: ...r Removing and installing a control processor blade CP8 This section describes how to remove and install a control processor CP8 blade Each SAN768B has two CP8 blades located in slot 6 and slot 7 Note If the new CP blade does not have the same firmware as the active CP blade you must upgrade the new blade to the same firmware version b768ig020 Figure 32 Filler panel removal and installation 76 SAN...

Page 109: ...rror v The clock is inaccurate or the CP blade does not boot up or shut down normally v Any of the following messages display in the error log Slot unknown message relating to a CP slot CP blade errors or I2 C timeouts FRU FRU_FAULTY messages for a CP blade Configuration loader messages or Sys PCI config messages Generic system driver messages FABSYS Platform system driver messages Platform EM mes...

Page 110: ...te the firmware version of the active CP blade 8 EnterhaDisable from the active CP blade to prevent failover or communication between the CP blades during the replacement 9 Logged into the active CP run the following two steps to upload the system configuration to a specified FTP server Enter information at the prompts a Enter the configupload vf command This command uploads the system virtual fab...

Page 111: ...de the lower and upper blade guides in the slot Slide the CP blade into the slot until it is firmly seated 3 Tighten the thumb screw inside each handle using a Phillips screwdriver 4 Turn the CP blade on by sliding the ON OFF switch in the top handle up to cover the thumb screw 5 Verify that the power LED is green If not ensure that the CP blade has power and is firmly seated and that the ejectors...

Page 112: ...oad s to download the firmware to the standby CP blade The s option also disables the autoreboot so you will have to manually issue a reboot after the download finishes to initiate firmwareCommit Enter all requested information use default values 3 To download firmware a Go to the IBM Support Portal www ibm com supportportal b Search for your product machine type 2499 or product name c On the disp...

Page 113: ...r firmwareDownload s U vX X X where X X X is the firmware version you are downloading to download the firmware to the standby CP blade The s option also disables the autoreboot so you must manually issue a reboot after the download finishes to initiate firmwarecommit Enter all requested information use default values The U option indicates that the download should come from the USB device If you d...

Page 114: ...s LED on the core switch blade is lit amber or the power LED is not lit v The slotShow command does not show that the core switch blade is enabled v The haShow command indicates an error v Any of the following messages display in the error log Slot unknown message relating to a core switch blade slot core switch blade errors or I2 C timeouts FRU FRU_FAULTY messages for a core switch blade Configur...

Page 115: ...he ejector handles to approximately 45 degrees Orient the CR blade so that the handles are toward you and the flat metal side is on your left 2 Align the flat metal side of the core switch blade inside the lower and upper blade guides in the slot Slide the core switch blade into the slot until it is firmly seated 3 Close the ejector handles and then tighten the thumbscrews inside each handle using...

Page 116: ...ation or replacement procedure v Do not force the router blade into the chassis If the blade does not slide in easily ensure that it is correctly aligned inside the rail guides before continuing Installing a router blade with incorrect alignment can damage both the chassis and the router blade v Hold the router blade by the edges of the metal pan Do not use the ejectors to hold a router blade v Di...

Page 117: ...tatus of blades on the SAN768B 3 Check for adequate cable slack Make sure there is plenty of cable slack to remove a router blade without optical power or Ethernet cable obstruction Refer to Managing cables on page 40 for cabling guidelines 4 Ensure that you have the proper spare part filler panel or replacement router blade 15 14 13 12 56 0000590 01 Re v A 7 6 5 4 3 2 1 0 GE1 GE0 15 14 13 12 11 1...

Page 118: ...o FR4 18i router blade in slots 1 4 or 9 12 If you are installing two router blades install one in one of the slots 1 4 and the other in one of the slots 9 12 to maintain proper cooling The control processor CP blades occupy slots 6 and 7 The core switch CR blades occupy slots 5 and 8 Any slots that are not occupied by a port blade router blade or CP blade must have a filler panel installed to ens...

Page 119: ...e product documentation CD 11 Log into the SAN768B as the admin user Configuring FCIP and Fibre Channel routing services and enabling the ports Refer to the Fabric OS Administrator s Guide or the Web Tools Administrator s Guide for detailed information on configuring the router blade The following steps provide only a brief overview of the procedure 1 Verify that the FR4 18i router blade is in an ...

Page 120: ... verify that all components are functional For information about LED patterns see Table 6 on page 57 9 Log in to the SAN768B as admin 10 Verify the correct operation of the FR4 18i router blade ports by typing the switchShow command from the SAN768B command prompt This command provides information about switch and port status Once the FR4 18i router blade is installed and fully configured in a SAN...

Page 121: ...ng the replacement check the power LEDs to verify that the minimum number of power supplies is functioning A fully populated SAN768B requires a minimum of one power supply slot at all times to ensure power to the SAN768B 2 Turn off the power switch 3 Remove the power cord 4 Loosen the thumbscrew 5 Grasp the handle and pull sliding the power supply from the chassis supporting the power supply from ...

Page 122: ... the power supply is seated by gently pulling on the handle The power supply should resist movement as you pull 3 Tighten the thumbscrew 4 Replace the power cord 5 Turn on the power switch 6 Verify that the power LED on the power supply displays a steady green light b768ig003 Figure 37 Removing and installing a power supply 90 SAN768B Installation Service and User Guide ...

Page 123: ...er assembly takes less than 5 minutes The following items are required for the blower assembly installation v Replacement blower assembly v Phillips 2 screwdriver Removing a blower assembly To remove a blower assembly do the following 1 Before removing a blower assembly verify that the other blower assemblies are functioning correctly The power LEDs should be steady green 2 Use the Phillips screwd...

Page 124: ... a WWN bezel and WWN card Note The World Wide Name WWN cards contain fully redundant circuits and normally do not require installation Two WWN cards are located beneath the WWN bezel Time and items required Allow approximately 20 minutes to replace a WWN card The following items are needed to install a WWN card v ESD grounding strap v 2 Phillips screwdriver required only for some versions of the W...

Page 125: ...ck the WWN card status using the commands listed in Table 13 Difficulty retrieving or modifying this data could indicate a WWN card failure Table 13 Commands identifying the WWN card status Data Related commands WWN values wwn chassisShow Data about the chassis and WWN card chassisShow Ethernet and Fibre Channel IP address information for the CP blades ipAddrShow ipAddrSet History log information ...

Page 126: ... please follow the recovery procedure in Core Switch WWN Card Removal and Replacement document Do you wish to continue y n Backing up WWN card data please wait about 25 seconds for further instruction Please install the new FRU now If this session lost for any reason please re enter the frureplace command and follow the instructions to complete the operation Please enter the word continue after th...

Page 127: ... the four screws Note If a serial console session is active several removal detected and insertion detected messages display on the console because of the installation 4 In the CLI session enter continue to indicate that the replacement has been completed Please enter the word continue after the new WWN card has been installed continue Restoring the information to the replacement FRU now please wa...

Page 128: ... and SFP s auto negotiate at 2 4 and 8 Gbps The 4 Gbps SFPs auto negotiate at 1 2 and 4 Gbps The mSFPs are compatible only with the FC8 64 high density port blade Note The XFP transceivers are used only with the FC10 6 port card They should not be used in any 1 GE port Time required The replacement procedure for a transceiver takes less than five minutes Items required v Replacement mSFP SFP SFP o...

Page 129: ...licks Transceivers are keyed so that they can only be inserted with the correct orientation If a transceiver does not slide in easily ensure that it is correctly oriented 2 Position a cable so that the key the ridge on one side of the cable connector is aligned with the slot in the transceiver Insert the cable into the transceiver until the latching mechanism clicks Cables are keyed so that they c...

Page 130: ...d so that they can only be inserted with the correct orientation If a transceiver does not slide in easily ensure that it is correctly oriented Removing and installing inter chassis link ICL cables Note ICL ports can be used only with an ICL license After the addition or removal of a license the license enforcement is performed on the ICL ports only when the portDisable and portEnable commands are...

Page 131: ... and labeled for ease of installation Figure 44 on page 101 through Figure 49 on page 106 show the acceptable cabling configurations for the inter chassis link ICL feature Connect the ICL cables in one of the configurations shown The cables can connect ICL connector ports on the SAN768B core switch blades CR8 or SAN384B core switch blades CR4S 8 Figure 48 on page 105 and Figure 49 on page 106 show...

Page 132: ...ention Connect the cables from the top connectors ICL 1 of the core blades in one chassis to the bottom connectors ICL 0 of the core blades in another chassis Similarly connect the cables from the bottom connectors ICL 0 of the core blades in one chassis to the top connectors ICL 1 of the core blades in another chassis Pay special attention to this configuration when creating a 3 way connection be...

Page 133: ...ntrol processor blades CP8 4 Port blades 5 ICL connector ICL 1 6 ICL connector ICL 0 7 ICL cables 8 Chassis 2 3 4 5 6 7 2 8 1 b768ig036 Figure 44 ICL cable connections between two SAN768B chassis Chapter 5 Removing and installing components 101 ...

Page 134: ...ntrol processor blades CP8 4 Port blades 5 ICL connector ICL 1 6 ICL connector ICL 0 7 ICL cables 8 Chassis 2 3 4 5 6 7 2 8 1 b768ig037 Figure 45 ICL cable connections between two SAN768B chassis 102 SAN768B Installation Service and User Guide ...

Page 135: ...ntrol processor blades CP8 4 Port blades 5 ICL connector ICL 1 6 ICL connector ICL 0 7 ICL cables 8 Chassis 2 3 4 5 6 7 2 8 1 b768ig038 Figure 46 ICL cable connections between two SAN768B chassis Chapter 5 Removing and installing components 103 ...

Page 136: ...ntrol processor blades CP8 4 Port blades 5 ICL connector ICL 1 6 ICL connector ICL 0 7 ICL cables 8 Chassis 2 3 4 5 6 7 2 8 1 b768ig039 Figure 47 ICL cable connections between two SAN768B chassis 104 SAN768B Installation Service and User Guide ...

Page 137: ... 8 3 Chassis 2 SAN768B 4 Chassis 3 SAN768B 5 ICL connector ICL 1 6 ICL connector ICL 0 2 5 6 2 2 6 3 5 6 5 4 1 384b054 Figure 48 3 way ICL cable connections between a SAN384B and two SAN768B chassis Chapter 5 Removing and installing components 105 ...

Page 138: ...S 8 3 Chassis 2 SAN768B 4 Chassis 3 SAN384B 5 ICL connector ICL 1 6 ICL connector ICL 0 5 6 3 5 2 5 6 1 2 6 2 4 384b055 Figure 49 3 way ICL cable connections between a SAN768B and two SAN384B chassis 106 SAN768B Installation Service and User Guide ...

Page 139: ... slot 3 in the SAN384B and slot 5 in the SAN768B and the other set is connected to the blades in the high order slots slot 5 in the SAN384B and slot 8 in the SAN768B Keeping the sets apart in this manner minimizes the cable confusion Removing the batteries Local regulations may require removal of the battery before recycling or disposing of the product Batteries are located in the control processo...

Page 140: ...C68 C66 C69 R158 R377 R381 R376 R378 R380 R375 R170 R373 R374 R365 R372 R171 R172 R366 R368 R173 R367 R175 R174 R176 R178 R177 C26 C27 R383 R382 R379 C83 C81 C82 C80 R623 R624 R516 C12 C15 C16 C19 C7 C10 C11 C25 C13 C14 C17 C18 C8 C9 C23 C24 R387 R388 R63 R61 R59 R57 R55 R53 R25 R51 R24 R49 R23 R22 R21 R47 R45 R43 R41 R20 R19 R39 R18 R37 R17 R35 R33 R628 R646 R645 R639 R641 R642 R635 R79 R74 R75 R...

Page 141: ... Refer to the following sections of this chapter for specific information v Verifying necessity of replacement on page 110 v Preparing for a chassis replacement on page 110 Time required on page 110 Items required on page 110 Recording critical SAN768B and SAN information on page 110 v Disconnecting the SAN768B from the network and fabric on page 113 v Removing components from the chassis on page ...

Page 142: ...en replaced Preparing for a chassis replacement Use the following information to help prepare for a chassis replacement Attention Refer to Safety notices and labels on page xv before performing any removal or installation procedures Time required The chassis replacement procedure takes approximately 3 hours to complete Items required The following items are required for the chassis replacement pro...

Page 143: ...the SAN768B as admin The default password is password Refer to your records for current passwords Enable the logging function on your Telnet or serial console connection 2 Back up the current configuration Note If you are using the virtual fabric feature you should run configupload vf before running the configupload command in order to save the logical switch configuration Enter configUpload all t...

Page 144: ...nd Serial Num are listed under Chassis WWN Unit 1 If the current WWN cards are the original cards the factory serial number listed is the same as the chassis serial number If a Serial Num value displays contact IBM to determine whether this serial number should be modified after the new WWN card is installed switch admin chassisshow Chassis Backplane Revision 1F SW BLADE Slot 1 Header Version 2 Po...

Page 145: ...ce and ports See Cable routing tables on page 119 Disconnecting the SAN768B from the network and fabric To disconnect the SAN768B from the network and fabric complete these steps 1 Shut down the system by entering theswitchShutdown command switch admin sysshutdown This command will shutdown the operating systems on your switch You are required to power cycle the switch in order to restore operatio...

Page 146: ...ribed in Removing and installing a core switch blade CR8 on page 82 5 Remove the control processor blades CP8 as described in Removing and installing a control processor blade CP8 on page 76 6 Remove the power supply units or filler panels as described in Removing and installing a power supply on page 88 7 Remove the blower assembly as described in Removing and installing a blower assembly on page...

Page 147: ...cribed in Removing and installing a power supply on page 88 4 Install the control processor CP8 blades in the new chassis as described in Removing and installing a control processor blade CP8 on page 76 5 Replace the core switch blades CR8 as described in Removing and installing a core switch blade CR8 on page 82 6 If ICL cables are not used insert EMI plugs in the ICL sockets in the core switch b...

Page 148: ...ame config txt config switch txt Password xxxxxxxx download complete switch admin 3 Reboot the system Verifying the correct operation of the system Follow these steps to verify that the hardware is operating correctly 1 Log in to the SAN768B as admin switch admin login login admin password xxxxxxxx switch admin 2 Enter the slotShow m command and verify that all the installed cards are detected and...

Page 149: ...3 4 Fibre Channel Subnetmask 255 255 255 0 CP0 Ethernet IP Address 10 32 50 10 Ethernet Subnetmask 255 55 0 0 HostName cp0 Gateway Address 10 32 40 1 CP1 Ethernet IP Address 10 32 50 11 Ethernet Subnetmask 255 55 0 0 HostName cp1 Gateway Address 10 32 40 1 Backplane IP address of CP0 10 0 0 5 Backplane IP address of CP1 10 0 0 6 switch admin switch admin Reconnecting the system to the network and ...

Page 150: ...The minimum bend radius should be no smaller than 10x the cable radius Tie wraps are not recommended for optical cables because they are easily overtightened and can unknowingly break the cable causing connectivity problems d Repeat steps a through c for each of the remaining port and application blades 3 Organize the cables as required see Managing cables on page 40 Verifying the correct configur...

Page 151: ...N768B or fabric v If there are any mechanical problems try reseating the associated component v If the configuration information is not correct for the SAN768B modify as required v If other issues exist contact technical support Cable routing tables Table 17 provides a sample of cable routing information Create your own table or use copies of Table 18 on page 120 to record this information if it i...

Page 152: ...shown Slot Port Cable labels Connected device Slot Port of device Slot Port Switch end Device end 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 120 SAN768B Installation Service and User Guide ...

Page 153: ...64 ports shown continued Slot Port Cable labels Connected device Slot Port of device Slot Port Switch end Device end 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 Chapter 5 Removing and installing components 121 ...

Page 154: ...122 SAN768B Installation Service and User Guide ...

Page 155: ...3816 FC3832 FC3848 Installing an 8 Gb port blade 16 32 or 48 ports These features can be installed while the SAN768B is operational Attention Follow these precautions for all procedures in this section to avoid damaging the port blades or chassis v Wear a grounded ESD strap when handling a port blade v Carefully remove the protective strip if any covering the blade connectors before installing any...

Page 156: ... when being replaced with a port blade or new filler panel Any slot that is not occupied by a port blade should be occupied by a filler panel to ensure correct cooling of the chassis and protection from dust 5 Remove the filler panel or port blade Refer to Removing a filler panel on page 75 or Removing a blade on page 72 6 Follow the instructions for Installing a blade on page 74 7 Follow the inst...

Page 157: ...er installing a new FC3864 in the SAN768B chassis Time required 20 minutes or more per port blade due to port and cable density Items required v ESD electrostatic discharge grounding strap v Workstation computer v Port blade v Phillips screwdriver v mSFP transceivers as needed v OM 3 LC optical cables as needed Procedures Follow the links in the steps below to complete this installation 1 Remove t...

Page 158: ...rower OM 3 LC cables are used to connect the FC8 64 These cables are offered by several major manufacturers Contact your IBM representative for options regarding different cable and patch panel configurations to simplify cable management with higher density FC8 64 port blades 8 Insert an OM 3 narrow cable into an mSFP and then insert the mSFP cable unit into a port See Removing and replacing an mS...

Page 159: ...able SFP transceivers as needed v Optical cables as needed Procedures Follow the links in the steps below to complete this installation 1 Remove the chassis door See Removing and installing the chassis door on page 69 2 Verify that the router blade is the correct part 23R1646 3 Verify which slot to install the blade Slots are numbered from one through ten from left to right when facing the port si...

Page 160: ... section to avoid damaging the port blades or chassis v Wear a grounded ESD strap when handling a port blade v Carefully remove the protective strip if any covering the blade connectors before installing any blades v Do not force the port blade installation If the port blade does not slide in easily ensure that it is correctly aligned inside the rail guides before continuing Installing a port blad...

Page 161: ... FC3880 Installing an FCOE10 24 blade This feature can be installed while the SAN768B is operational Attention This blade cannot be used in the same chassis with an FC8 64 high density port blade or the FR4 18i FX8 24 or FS8 18 application blades Attention Follow these precautions for all procedures in this section to avoid damaging the port blades or chassis v Wear a grounded ESD strap when handl...

Page 162: ...ed only when being replaced with a port blade or new filler panel Any slot that is not occupied by a port blade should be occupied by a filler panel to ensure correct cooling of the chassis and protection from dust 6 Install the blade Installing a blade on page 74 Remove the protective strip from the blade contacts if any before installing the blade 7 Repeat steps 1 5 for additional blades if any ...

Page 163: ...equired v ESD electrostatic discharge grounding strap v Workstation computer v Port blade v Phillips screwdriver v Transceivers as needed v Optical cables as needed Procedures Follow the links in the steps below to complete this installation 1 Remove the chassis door See Removing and installing the chassis door on page 69 2 Verify that the blade is the correct part 3 Verify which slot to install t...

Page 164: ...tion Thoroughly review Planning for encryption optional on page 41 prior to installing the FS8 18 encryption blade You must also refer to the Fabric OS Encryption Administrator s Guide Supporting Tivoli Key Lifecycle Manager TKLM Environments to successfully complete installation configuration and management of the encryption functionality This feature can be installed while the SAN768B is operati...

Page 165: ...de filler panels on page 75 Attention A filler panel should be removed only when being replaced with a blade or new filler panel Any slot that is not occupied by a blade should be occupied by a filler panel to ensure correct cooling of the chassis and protection from dust 5 Install the encryption blade Installing a blade on page 74 Remove the protective strip from the blade contacts if any before ...

Page 166: ...eps below to complete this installation 1 Remove the chassis door See Removing and installing the chassis door on page 69 2 Verify that the inter chassis link kit is the correct part 21R9976 The cable PN is 21R9959 3 Verify with the customer that the ICL license is installed on each chassis to be connected Enter the licenseIdShow command for each chassis If required install the ICL license followi...

Page 167: ...nel Physical and Signalling Interface standard Modes of operation Fibre Channel Class 2 Class 3 and Class F Fabric initialization Complies with FC SW 5 0 FC IP IP over fibre channel Complies with FC IP 2 3 of the FCA profile Aggregate I O bandwidth 4 09 Tbps per chassis fully populated with eight FC8 64 blades Port to port latency Less than 2 microseconds with no contention destination port is fre...

Page 168: ... in a trunk group to form a 64 Gbps trunk Aggregate bandwidth 4 09 Tbps per chassis Switch latency FC8 16 FC8 32 FC8 48 FC8 64 blades 2 1 msec any port to any port at 8 Gbps cut through routing For the FR4 18i blade 3 6 msec any port to any port at 4 Gbps cut through routing For the FX8 24 blade 2 1 msec any port to any port at 8 Gbps cut through routing For the FCOE10 24 blade CEE to CEE and FCOE...

Page 169: ... product v Go to the IBM SAN web page www ibm com systems storage san v Select the SAN768B product link v On the SAN768B product page click the Interoperability Matrix link v In the Interoperability matrix click the Transceivers link Fabric services Access Control Lists ACLs Advanced Performance Monitoring Adaptive Networking Ingress Rate Limiting Traffic Isolation QoS auditing BB credit recovery ...

Page 170: ...ts the weights of the compatible blades and FRUs Table 22 System FRU weights FRU Weight CP blade CP8 3 0 kg 6 6 lb CR blade CR8 3 27 kg 7 2 lb FC8 16 port blade 3 0 kg 6 6 lb without media FC8 32 port blade 3 27 kg 7 2 lb without media FC8 48 port blade 3 36 kg 7 4 lb without media FC8 64 port blade 3 36 kg 7 4 lb without media FC10 6 port blade 3 1 kg 6 8 lb without media FR4 18i router blade 3 3...

Page 171: ...oduct might be equipped with multiple power cords To remove all hazardous voltages disconnect all power cords L003 The power subsystem is a redundant 48 V dc power distribution system with a provision for up to four 2000 watt 48V dc bulk power supplies Bulk power supplies produce the intermediate distribution voltage in the distributed power system The power specifications listed in Table 23 are c...

Page 172: ...ne wave 33G 11ms duration half sine wave Vibration 1G p p 5 500 Hz at 1 0 octave minute 2 4G p p 5 500 Hz at 1 0 octave minute Air flow 595 cu m hr 350 cu ft min None required Heat dissipation 1337 Watts or 4564 BTU hr 384 port configuration Not applicable Note The 10 to 40 Celsius range applies to the ambient air temperature at the air intake vents on the nonport side of the SAN768B The temperatu...

Page 173: ...t N A mSFP SFP SFP 2 Gbps 150 m 492 ft 300 m 984 ft 500 m 1640 ft N A mSFP SFP SFP 4 Gbps 70 m 229 ft 150 m 492 ft 380 m 1264 ft N A mSFP SFP 8 Gbps 21 m 68 ft 50 m 164 ft 150 m 492 ft N A XFP 10 Gbps N A N A 300 m 984 ft N A LWL SFP 2 Gbps N A N A N A 30 km 18 6 mi SFP 4 Gbps N A N A N A 30 km 18 6 mi SFP 8 Gbps N A N A N A N A XFP 10 Gbps N A N A N A 10 km 6 2 mi ELWL XFP 10 Gbps N A N A N A 40 ...

Page 174: ...P Fibre Channel links over each physical GbE connection share this bandwidth Power cords Two power cords for connecting between the SAN768B and the power distribution units on the 2109 C36 cabinet are shipped with the product Two additional power cords are provided if the optional upgrade power supply feature is ordered Feature Code 7880 See the IBM TotalStorage SAN Cabinet 2109 Model C36 Installa...

Page 175: ...ng Services can be used over the FCIP link Fabrics connected through FCIP merge if the ports are configured as VE_Ports and do not merge if they are configured as VEX_Ports If VE_Ports are used in a Fibre Channel Routing Services backbone fabric configuration then the backbone fabric merges but the EX_Port attached edge fabrics do not merge For more information refer to the Fabric OS Administrator...

Page 176: ...ding VE_ports in 10GbE mode The FX8 24 blade is intended as a platform for FCIP and Fibre Channel Routing Services Refer to the Fabric OS Administrator s Guide for information on configuring these features The FX8 24 blade provides the following hardware features v 12 autosensing FC ports with link speeds of 1 2 4 or 8 Gbps v Ten GbE ports supporting FCIP with fixed link speed at 1 Gbps v Two 10Gb...

Page 177: ...hanced Ethernet CEE standards a low latency lossless Ethernet standard This does not require dedicated Ethernet lines but rather can make use of existing Ethernet infrastructure to reduce costs Attention This blade cannot be used in the same chassis with any of the other application blades FR4 18i FX8 24 FS8 18 There are no licensing requirements for functionality on this blade Refer to the Fabric...

Page 178: ...146 SAN768B Installation Service and User Guide ...

Page 179: ... logs refer to the Fabric OS System Error Messages Reference Manual and the Fabric OS Administrator s Guide for more information v Review RASlog entries v Enter the sensorShow command to determine the status of the hardware components v Run diagnostic tests see Diagnostics on page 149 v Reboot the SAN768B or power the entire chassis off Powering off the SAN768B on page 45 and then back on Providin...

Page 180: ... the standby CP blade POST The system automatically performs POST each time it is powered on or reset To verify that POST has completed without error v Verify that all LEDs return to a normal state after POST completes see Chapter 4 Monitoring system components on page 47 If one or more LEDs do not return to a normal state and this is not due to the SAN768B being set to beacon refer to the relevan...

Page 181: ...ires maintenance Each diagnostic test can be implemented by entering the related command through a Telnet or serial session For a list of diagnostic tests and commands refer to the Fabric OS Administrator s Guide All diagnostic tests are run at link speeds of 1 2 4 and 8 Gb sec They might temporarily lock the transmit and receive speeds to a specific speed Some diagnostic tests require interconnec...

Page 182: ... the CP Ethernet link speed negotiation set up by the network Specify the CP Ethernet link speed by entering the ifModeSet command Configuration data is inaccurate or cannot be accessed Chassis was powered off on while the WWN card was uninstalled or failed Install an operational WWN card and power the system off on again The SAN768B was rebooted while the WWN card was uninstalled or failed Initia...

Page 183: ...g shut down the port blades as necessary to return the temperature readings to the operating ranges Blades might be faulty Enter the slotShow command to determine status For more information enter the diagDisablePost command then the slotPowerOff slot number and slotPowerOn slot number commands Resolve the source of the problem or replace the blade as required Pins on the blade or on the backplane...

Page 184: ...152 SAN768B Installation Service and User Guide ...

Page 185: ...gure 54 on page 155 shows an FC8 32 port blade v Figure 55 on page 156 shows an FC8 48 port blade v Figure 56 on page 157 shows an FC8 64 port blade v Figure 57 on page 158 shows an FC10 6 port blade v Figure 58 on page 159 shows an FR4 18i routing blade v Figure 59 on page 160 show an FS8 18 encryption blade v Figure 60 on page 161 shows the FX8 24 extension blade v Figure 61 on page 162 shows th...

Page 186: ... 1 Blade power LED 3 FC ports 0 15 bottom to top 2 Blade status LED 1 2 3 B768ig054 Figure 53 FC8 16 port blade 154 SAN768B Installation Service and User Guide ...

Page 187: ... 1 Blade power LED 3 FC ports 16 31 bottom to top 2 Blade status LED 4 FC ports 0 15 bottom to top 2 1 3 4 b768ig055 Figure 54 FC8 32 port blade Appendix D Blade port numbering 155 ...

Page 188: ... 1 Blade power LED 3 FC ports 24 47 bottom to top 2 Blade status LED 4 FC ports 0 23 bottom to top 3 2 1 4 B768ig056 Figure 55 FC8 48 port blade 156 SAN768B Installation Service and User Guide ...

Page 189: ...D 5 port 61 LED 2 Blade status LED 6 port 29 LED 3 Port 63 7 FC ports 32 63 bottom to top 4 Port 30 8 FC ports 0 31 bottom to top 8 7 3 2 1 5 6 4 b768ig057 Figure 56 FC8 64 port blade Appendix D Blade port numbering 157 ...

Page 190: ...FC 10 6 2 3 4 5 1 0 b768ig058 Figure 57 FC10 6 port blade ports 0 5 bottom to top 158 SAN768B Installation Service and User Guide ...

Page 191: ...rts 0 15 bottom to top 2 GbE ports GE0 bottom GE1 top 7 6 5 4 3 2 1 0 GE1 GE0 15 14 13 12 11 10 9 8 56 0000590 01 Re v A FR4 18i b768ig059 1 2 Figure 58 FR4 18i routing blade Appendix D Blade port numbering 159 ...

Page 192: ...ey management 1000Base media interfaces 3 FC ports 0 15 bottom to top 2 Key management smart card interface b768ig061 3 2 1 Figure 59 FS8 18 encryption blade 160 SAN768B Installation Service and User Guide ...

Page 193: ... LED 6 FC ports 6 11 numbered bottom to top 3 1 GbE ports 0 3 numbered bottom to top 7 1 GbE ports 4 9 numbered bottom to top 4 10 GbE ports 0 1 numbered bottom to top 3 4 5 6 7 1 2 b768ig034 Figure 60 FX8 24 extension blade Appendix D Blade port numbering 161 ...

Page 194: ...tus LED 3 10 GbE FCoE ports 0 11 numbered bottom to top 2 Power LED 4 10 GbE FCoE ports 12 23 numbered bottom to top b768ig035 3 4 1 2 Figure 61 FCOE10 24 blade 162 SAN768B Installation Service and User Guide ...

Page 195: ...al law INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND EITHER EXPRESS OR IMPLIED INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF NON INFRINGEMENT MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE Some states do not allow disclaimer of express or implied warranties in certain transactions therefore this statement may not apply to you...

Page 196: ... products was obtained from the suppliers of those products their published announcements or other publicly available sources IBM has not tested those products and cannot confirm the accuracy of performance compatibility or any other claims related to non IBM products Questions on the capabilities of non IBM products should be addressed to the suppliers of those products If you are viewing this in...

Page 197: ...ks of Adobe Systems Incorporated in the United States and or other countries Java and all Java based trademarks and logos are trademarks of Sun Microsystems Inc in the United States other countries or both Linux is a registered trademark of Linus Torvalds in the United States other countries or both Microsoft and Windows are trademarks of Microsoft Corporation in the United States other countries ...

Page 198: ...oid the user s authority to operate the equipment This device complies with Part 15 of the FCC Rules Operation is subject to the following two conditions 1 this device may not cause harmful interference and 2 this device must accept any interference received including interference that may cause undesired operation Industry Canada Class A Emission Compliance Statement This Class A digital apparatu...

Page 199: ...en werden Warnung Dieses ist eine Einrichtung der Klasse A Diese Einrichtung kann im Wohnbereich Funk Störungen verursachen in diesem Fall kann vom Betreiber verlangt werden angemessene Maßnahmen zu ergreifen und dafür aufzukommen Deutschland Einhaltung des Gesetzes über die elektromagnetische Verträglichkeit von Geräten Dieses Produkt entspricht dem Gesetz über die elektromagnetische Verträglichk...

Page 200: ...e user may be required to take corrective actions Japan Electronics and Information Technology Industries Association JEITA Statement Japanese Electronics and Information Technology Industries Association JEITA Confirmed Harmonics Guideline products greater than 20 A per phase Korea Communications Commission KCC Class A Statement Please note that this equipment has obtained EMC registration for co...

Page 201: ...A Statement Australia and New Zealand Class A Statement Attention This is a Class A product In a domestic environment this product might cause radio interference in which case the user might be required to take adequate measures rusemi Notices 169 ...

Page 202: ...170 SAN768B Installation Service and User Guide ...

Page 203: ... v components installing 69 removing 69 configuration backing up 43 encryption blade 31 parameters 33 configuration file backup 44 uploading 44 configuration parameters initial 34 configuring encryption 42 IP addresses 34 SAN768B 31 configuring FCIP FR4 18i 87 router blade 87 configuring Fibre channel routing services FR4 18i 87 router blade 87 connecting to the fabric 39 connection serial 33 conn...

Page 204: ...g Fibre channel routing services 87 enabling ports 87 installing 86 removing 84 FR4 18i blade 143 installing 74 removing 72 status 47 FR4 18i router blade installing 84 removing 84 FR4 18i routing blade 53 127 components 53 LEDs 57 status 57 FRU weights 138 FRUs Smart card 42 FRX8 24 blade components 55 FS8 18 description 143 encryption blade iii 47 143 features 143 FS8 18 blade installing 74 remo...

Page 205: ...s 163 PID mode verifying 39 port blade FC10 6 47 FC8 16 47 FC8 32 47 FC8 48 47 FC8 64 47 FCOE10 24 47 FX8 24 47 installing 71 74 123 125 128 monitoring 47 removing 71 status 47 port blades 3870 128 FC10 6 71 FC8 16 71 123 FC8 32 71 123 FC8 48 71 FC8 64 71 port blades 3470 128 port blades FC10 6 71 port blades FC8 16 71 123 port blades FC8 32 71 123 port blades FC8 48 71 port blades FC8 64 71 125 p...

Page 206: ... software features 8 software licenses enabling 39 specifications power 139 specifications product 135 standard features software Advanced Zoning 8 Fabric Watch 8 ISL Trunking 8 Performance Monitoring 8 Web Tools 8 standard software features 8 starting SAN768B 31 status encryption blades 57 FC10 6 blade 57 FC8 16 blade 57 FC8 32 blade 57 FC8 48 blade 57 FC8 64 blade 57 FR4 18i routing blade 57 FS8...

Page 207: ......

Page 208: ... Part Number 45W8666 Printed in USA GA32 0574 05 1P P N 45W8666 ...