Cisco Nexus 9000 Series, Hardware Installation Manual

Page 1: ...dware Installation Guide First Published 2017 04 11 Last Modified 2019 03 28 Americas Headquarters Cisco Systems Inc 170 West Tasman Drive San Jose CA 95134 1706 USA http www cisco com Tel 408 526 4000 800 553 NETS 6387 Fax 408 527 0883 ...

Page 2: ...isco will have no liability for delay in the delivery or failure to deliver any products or feature roadmap items that may be set forth in this document Any Internet Protocol IP addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers Any examples command display output network topology diagrams and other figures included in the document are shown...

Page 3: ...A P T E R 2 Temperature Requirements 5 Humidity Requirements 5 Altitude Requirements 5 Dust and Particulate Requirements 6 Minimizing Electromagnetic and Radio Frequency Interference 6 Shock and Vibration Requirements 7 Grounding Requirements 7 Planning for Power Requirements 7 Airflow Requirements 9 Rack and Cabinet Requirements 9 Clearance Requirements 10 Installing the Switch Chassis 13 C H A P...

Page 4: ...a Gigabit Ethernet module GEM 34 Virtual Port Channel Migration Migration of Nodes from a First Generation Switch to a Second Generation Switch 34 Setting Up an Optional Console Interface 35 Setting Up an Optional Management Connection 36 Optic Transceiver Removal Using the Optics Extraction Tool 37 Maintaining Transceivers and Optical Cables 38 Replacing Components 39 C H A P T E R 5 Replacing a ...

Page 5: ... 930 W DC Power Supply Specifications 50 Power Cable Specifications 51 Power Cable Specifications for AC Power Supplies 51 HVAC HVDC Power Cables Supported by ACI Mode and NX OS Mode Switches 52 DC Power Cable Specifications 53 Regulatory Standards Compliance Specifications 53 LEDs 55 A P P E N D I X C Switch Chassis LEDs 55 Fan Module LEDs 56 Power Supply LEDs 56 Additional Kits 59 A P P E N D I ...

Page 6: ...Cisco Nexus 93108TC FX ACI Mode Hardware Installation Guide vi Contents ...

Page 7: ...c text indicates arguments for which the user supplies the values Italic Square brackets enclose an optional element keyword or argument x Square brackets enclosing keywords or arguments separated by a vertical bar indicate an optional choice x y Braces enclosing keywords or arguments separated by a vertical bar indicate a required choice x y Nested set of square brackets or braces indicate option...

Page 8: ...available on Cisco com at the following URL https www cisco com c en us support cloud systems management application policy infrastructure controller apic tsd products support series home html Web Based Documentation Cisco APIC Management Information Mode Reference Cisco APIC Online Help Reference Cisco APIC Python SDK Reference Cisco ACI Compatibility Tool Cisco ACI MIB Support List Downloadable ...

Page 9: ...rdware Installation Guide Cisco Nexus 93120TX ACI Mode Switch Hardware Installation Guide Cisco Nexus 93128TX ACI Mode Switch Hardware Installation Guide Cisco Nexus 93180LC EX ACI Mode Switch Hardware Installation Guide Cisco Nexus 93180YC EX ACI Mode Switch Hardware Installation Guide Cisco Nexus 93180YC FX ACI Mode Switch Hardware Installation Guide Cisco Nexus 9332PQ ACI Mode Switch Hardware I...

Page 10: ...ttps www cisco com c en us support switches application virtual switch tsd products support series home html Documentation Feedback To provide technical feedback on this document or to report an error or omission please send your comments to apic docfeedback cisco com We appreciate your feedback Obtaining Documentation and Submitting a Service Request For information on obtaining documentation usi...

Page 11: ...facing ports Autonegotiation is supported on 100M 1G 10G ports Note The chassis for this switch includes the following user replaceable components You can order this switch in a 24 port configuration N9K C93108TC FX 24 Note Fan modules four with the following airflow choices Port side intake airflow with burgundy coloring NXA FAN 30CFM B Port side exhaust airflow with blue coloring NXA FAN 30CFM F...

Page 12: ...hite coloring N9K PUV 1200W If you are using the 1200 W HVAC HVDC power supply the power supply automatically uses the same airflow direction as the fan modules installed switch Note The following figure shows the hardware features seen from the port side of the chassis Screw holes 6 for attaching rack mounting brackets 4 Beacon BCN Status STS and Environment ENV LEDs 1 Grounding pad 5 48 10 1 Gig...

Page 13: ...take Airflow Coloring Replaceable Modules Blue Burgundy Fans Blue Burgundy AC power supplies White HVAC HVDC power supplies Blue Burgundy DC power supplies The fan and power supply modules are field replaceable and you can replace one fan module or one power supply module during operations so long as the other modules are operating If you have only one power supply that is installed you can instal...

Page 14: ...Cisco Nexus 93108TC FX ACI Mode Hardware Installation Guide 4 Overview Overview ...

Page 15: ...it 40 to 70 degrees Celsius Humidity Requirements High humidity can cause moisture to enter the switch Moisture can cause corrosion of internal components and degradation of properties such as electrical resistance thermal conductivity physical strength and size The switch is rated to withstand from 5 to 95 percent noncondensing relative humidity Buildings in which the climate is controlled by air...

Page 16: ...to other devices through the power cable and power source or through the air as transmitted radio waves The Federal Communications Commission FCC publishes specific regulations to limit the amount of EMI and RFI that are emitted by computing equipment Each switch meets these FCC regulations To reduce the possibility of EMI and RFI follow these guidelines Cover all open expansion slots with a blank...

Page 17: ...natively you can ground the chassis by using a customer supplied grounding cable that meets your local and national installation requirements For U S installations we recommend 6 AWG wire Connect your grounding cable to the chassis with a grounding lug provided in the switch accessory kit and to the facility ground You automatically ground AC power supplies when you connect them to AC power source...

Page 18: ...d to the switch For AC input application please refer to the following statement Note Statement 1005 Circuit Breaker This product relies on the building s installation for short circuit overcurrent protection Ensure that the protective devices are rated not greater than 20A North America 16A Europe and 13A UK Warning For DC input application please refer to the following statement Note Statement 1...

Page 19: ...ough the port end in the cold aisle and exhausts through the fan and power supply modules in the hot aisle Dual direction airflow The direction of the installed fan modules determines the airflow You can identify the airflow direction of each fan and power supply module by its coloring as follows Blue coloring indicates port side exhaust airflow Burgundy coloring indicates port side intake airflow...

Page 20: ...et required for cabling 3 0 inches 7 6 cm between the rear of the chassis and the interior of the cabinet required for airflow in the cabinet if used No clearance is required between the chassis and the sides of the rack or cabinet no side airflow Also you must have power receptacles that are located within reach of the power cords that are used with the switch Statement 1048 Rack Stabilization Th...

Page 21: ... depth of the chassis 7 Chassis width 17 3 in 43 9 cm 3 Width of the front clearance area equal to the width of the chassis with two rack mount brackets that are attached to it 19 0 in 43 3 cm 4 Both the front and rear of the chassis must be open to both aisles for airflow Note Cisco Nexus 93108TC FX ACI Mode Hardware Installation Guide 11 Preparing the Site Clearance Requirements ...

Page 22: ...Cisco Nexus 93108TC FX ACI Mode Hardware Installation Guide 12 Preparing the Site Clearance Requirements ...

Page 23: ...witch see the Regulatory Compliance and Safety Information for the Cisco Nexus 3000 and 9000 Series for important Safety Information Statement 1071 Warning Definition IMPORTANT SAFETY INSTRUCTIONS Before you work on any equipment be aware of the hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents Read the installation instructions before usin...

Page 24: ... The rack or cabinet that you use must meet the requirements listed the in General Requirements for Cabinets and Racks on page 45 section You are responsible for verifying that your rack and rack mount hardware comply with the guidelines that are described in this doc Note Airflow Considerations The switch comes with fan and power supply modules that have either port side intake or port side exhau...

Page 25: ...ology These UPS types can become unstable with the switch which can have substantial current draw fluctuations because of fluctuating data traffic patterns Caution Ensure that circuits are sized according to local and national codes Typically this often requires one or both of the following AC power supplies typically require at least a 15 A or 20 A AC circuit 100 to 240 VAC and a frequency of 50 ...

Page 26: ...d rack or grounded chassis there must be a metal to metal connection to the earth ground Step 1 Compare the shipment to the equipment list provided by your customer service representative and verify that you have received all items including the following Accessory Kit Step 2 Check for damage and report any discrepancies or damage to your customer service representative Have the following informat...

Page 27: ...Step 1 Install two front rack mount brackets and the two rear rack mount brackets to the switch as follows a Determine which end of the chassis is to be located in the cold aisle as follows If the switch has port side intake modules fan modules with burgundy coloring position the switch so that its ports will be in the cold aisle If the switch has port side exhaust modules fan modules with blue co...

Page 28: ...re installing the chassis into a grounded rack you can skip this step Step 3 Install the slider rails on the rack or cabinet as follows a Determine which two posts of the rack or cabinet you should use for the slider rails Of the four vertical posts in the rack or cabinet two will be used for the front mount brackets attached to the easiest accessed end of the chassis and the other two posts will ...

Page 29: ...n the two rear rack mount guides on either side of the switch with the slider rails installed in the rack Slide the rack mount guides onto the slider rails and then gently slide the switch all the way into the rack until the front rack mount brackets come in contact with two rack or cabinet posts c Holding the chassis level insert screws 12 24 or 10 32 depending on the rack type in each of the two...

Page 30: ...the rack Typically the front of the rack is the side easiest to access for maintenance You must supply the eight 10 32 or 12 24 screws required to mount the slider rails and switch to the rack Note Before you begin You have inspected the switch shipment to ensure that you have everything ordered Make sure that the switch rack mount kit includes the following parts Front rack mount brackets 2 Rear ...

Page 31: ...f the six screw holes on the side of the chassis see the two ways to mount these brackets on a typical chassis in following figure The holes that you use depend on the requirements of your rack and the amount of clearance required for interface cables 3 inches 7 6 mm minimum and module handles 1 inch 2 5 mm minimum Note Front rack mount bracket aligned to the module end of the chassis 5 Front rack...

Page 32: ...is into a grounded rack you must attach a customer supplied grounding wire to the chassis as explained in the Grounding the Chassis on page 24 section If you are installing the chassis into a grounded rack you can skip this step Step 4 Install the slider rails on the rack or cabinet as follows a Determine which two posts of the rack or cabinet you should use for the slider rails Of the four vertic...

Page 33: ...lider rails until the front rack mount brackets come in contact with the front rack mount rails 2 b Align the two rear rack mount guides on either side of the switch with the slider rails installed in the rack Slide the rack mount guides onto the slider rails and then gently slide the switch all the way into the rack until the front rack mount brackets come in contact with two rack or cabinet post...

Page 34: ...e product chassis and the metal surface of the enclosure or rack in which it is mounted or to a grounding conductor Electrical continuity shall be provided by using thread forming type mounting screws that remove any paint or non conductive coatings and establish a metal to metal contact Any paint or other non conductive coatings shall be removed on the surfaces between the mounting hardware and t...

Page 35: ...imately 0 75 inch 19 mm of the covering from the end of the grounding wire We recommend 6 AWG wire for the U S installations Step 2 Insert the stripped end of the grounding wire into the open end of the grounding lug Use a crimping tool to crimp the lug to the wire see the following figure Verify that the ground wire is securely attached to the grounding lug by attempting to pull the wire out of t...

Page 36: ... might need an optional jumper power cord to connect the switch to your outlet receptacle Note The switch is not connected to the network this includes any management or interface connections The fan and power supply modules are fully secured in their chassis slots Step 1 For each AC power supply do the following a Using the recommended AC power cable for your country or region connect one end to ...

Page 37: ...tacle on the power supply Make sure that the connector block clicks when fully inserted in the receptacle and does not pull out d If there is a safety cover for the terminals place and secure it over the terminals to avoid an electrical shock hazard e Turn on the power at the circuit breaker for the DC power source Step 4 Verify that the power supply LED is on and green Step 5 Listen for the fans ...

Page 38: ...Cisco Nexus 93108TC FX ACI Mode Hardware Installation Guide 28 Installing the Switch Chassis Starting the Switch ...

Page 39: ... Transceivers and Optical Cables on page 38 ACI Fabric Topology The ACI fabric topology includes the following major components Application Centric Infrastructure Controller APIC appliance cluster of APICs Leaf switches for switch compatibility please see the data sheets Spine switches for switch compatibility please see the data sheets For additional information please see the Cisco APIC Installa...

Page 40: ...ou shield any potential noise sources by housing them in a grounded metallic conduit Note The optical transceivers that are not already assembled to their cables come separate from their cables To prevent these transceivers and their cables from being damaged we recommend that you keep the transceivers disconnected from their cables when installing them in ports and then insert the optical cable i...

Page 41: ...commended for redundancy Cisco Nexus 9300 platform ACI mode leaf switches to each Application Policy Infrastructure Controller APIC in your ACI fabric The type of virtual interface card VIC installed on the APIC determines the types of interface cables that you can use to connect the leaf switches to the APICs The VIC 1225T module supports copper connectors copper cables and switches with copper d...

Page 42: ...Gigabit or 25 Gigabit Port 1 and port 2 is one pair corresponding to eth2 1 on APIC and port 3 and port 4 is another pair corresponding to eth2 2 on APIC Only one connection is allowed for each pair For example you can connect one cable to either port 1 or port 2 and connect another cable to either port 3 or port 4 please do not connect two cables on any pair To determine which transceivers adapte...

Page 43: ...le or connector Do not view directly with optical instruments Viewing the laser output with certain optical instruments for example eye loupes magnifiers and microscopes within a distance of 100 mm may pose an eye hazard Warning Before you begin The leaf and spine switches in the fabric such as N9k C9364C N9K C9332C N9K C9316D GX and N9K C9332D GX2B must be fully installed in their racks and groun...

Page 44: ...links Installing a Gigabit Ethernet module GEM Step 1 Clear the switch s current configuration by using the setup clean config command Step 2 Power off the switch by disconnecting the power Step 3 Replace the current GEM card with the new GEM card Step 4 Power on the switch Virtual Port Channel Migration Migration of Nodes from a First Generation Switch to a Second Generation Switch Initially the ...

Page 45: ...e new switch and the vPC legs are kept down since there is a mismatch of generation switches At this point vPC primary continues to send the data traffic Step 6 From the APIC GUI perform the Remove From Controller operation for vPC primary This switch is clean rebooted by the APIC Wait for about 10 minutes for this operation to finish The vPC leg on the second generation switch which was kept down...

Page 46: ...nection for monitoring and troubleshooting purposes To do this depending on your switch you connect either the RJ 45 management port or the SFP management port on the switch to an external hub switch or router Before you begin To prevent an IP address conflict you must complete the initial configuration for the switch and establish an IP address before you create the management connection Step 1 C...

Page 47: ...ic discharge ESD strap To ground the strap attach it directly to earth ground or to a grounded rack or chassis There must be a metal to metal connection to earth ground Step 1 Remove the fiber optic cables from the transceiver module before removing the transceiver module Step 2 Use the wide side of the optics extraction tool to release the bale latch see the following image Cisco Nexus 93108TC FX...

Page 48: ...s necessary Repeated removals and insertions can shorten its useful life Keep all optical connections covered when not in use Clean them before using to prevent dust from scratching the fiber optic cable ends Do not touch the ends of connectors Touching the ends can leave fingerprints and cause other contamination Clean the connectors regularly the required frequency of cleaning depends upon the e...

Page 49: ...m a cold aisle and exhausts to a hot aisle Otherwise the switch can overheat and shutdown If you are changing the airflow direction of all the modules in the chassis you must shutdown the switch before replacing all the fan and power supply modules with modules using the other airflow direction During operations all of the modules must have the same direction of airflow Caution Removing a Fan Modu...

Page 50: ...power supply providing the necessary power for operations you can replace the other power supply during operations so long as the new power supply has the same airflow direction as the other modules in the chassis You can replace a power supply with another supported power supply that has the same power source type and the same wattage rating as the other installed power supply Additionally the ai...

Page 51: ...neath the module and the other hand holding the handle turn the power supply so that its release latch is on the right side and align the back end of the power supply the end with the electrical connections to the open power supply slot before carefully sliding the power supply all the way into the slot until it clicks into place If the power supply does not fit into the open slot turn the module ...

Page 52: ...essing the tab on the top of the Anderson Power SAF D Grid connector and pull the cable and connector out of the power supply c Grasp the power supply handle while pressing the colored release latch towards the power supply handle d Place your other hand under the power supply to support it while you slide it out of the chassis Do not touch the electrical connections on the back side of the module...

Page 53: ...direction Note An DC power source must be within reach of the power cables that will be used with the replacement power supply If you are using n n power redundancy there must be a separate power source for each power supply installed in the chassis There must be an earth ground connection to the chassis in which you are installing the replacement power supply DC power supplies connected to DC pow...

Page 54: ...n power supply slot before carefully sliding the power supply all the way into the slot until it clicks into place If the power supply does not fit into the open slot turn the module over before sliding it carefully into the open slot Note b Test the installation by trying to pull the power supply out of the slot without using the release latch If the power supply does not move out of place it is ...

Page 55: ...en racks If you are selecting an enclosed cabinet we recommend one of the thermally validated types either standard perforated or solid walled with a fan tray Note We do not recommend that you use racks that have obstructions such as power strips because the obstructions could impair access to field replaceable units FRUs Note General Requirements for Cabinets and Racks The cabinet or rack must al...

Page 56: ...is in an open rack no side panels or doors ensure that the rack meets the following requirements The minimum vertical rack space per chassis must be equal to the rack unit RU of the chassis One rack unit is equal to 1 75 inches 4 4 cm The distance between the chassis air vents and any walls should be 2 5 inches 6 4 cm Requirements Specific to Perforated Cabinets A perforated cabinet has perforatio...

Page 57: ...page 51 Regulatory Standards Compliance Specifications on page 53 Environmental Specifications Specification Environment 32 to 104 F 0 to 40 C Ambient operating temperature Temperature 40 to 158 F 40 to 70 C Ambient nonoperating 5 to 85 Nonoperating Relative humidity 0 to 13 123 feet 0 to 4 000 meters Operating Altitude Switch Dimensions Height Depth Width Switch 1 72 inches 4 4 cm 1 RU 22 5 inche...

Page 58: ...eiver modules products device support tables list html To see the transceiver specifications and installation information see https www cisco com c en us support interfaces modules transceiver modules products device support tables list html Switch Power Input Requirements The following table lists the typical amount of power that the switch consumes It also lists the maximum amount of power that ...

Page 59: ...peres 500 W Maximum output power per power supply 33 A sub cycle duration Maximum inrush current 12 ms at 500 W Maximum hold up time 12 VDC Power supply output voltage 12 VDC Power supply standby voltage Climate Savers Platinum Efficiency 80Plus Platinum certified Efficiency rating RSP1 Form factor 1200 W HVAC HVDC Power Supply Specifications These specifications apply to the 1200 W HVAC HVDC N9K ...

Page 60: ... factor 930 W DC Power Supply Specifications These specifications apply to the following power supplies NXA PDC 930W PE NXA PDC 930W PI Specification Characteristic Nominal range 48 to 60 VDC nominal Range 40 to 60 VDC DC input voltage range 23 A at 48 VDC Maximum DC input current 930 W Maximum output power per power supply 35 A sub cycle duration Maximum inrush current 8 ms at 930 W Maximum hold ...

Page 61: ...n 10 A 8 2 feet 2 5 m CAB IND 10A India Power Cord Jumper C13 C14 Connectors 6 6 feet 2 0 m CAB C13 C14 2M IN India Power Cord Jumper C13 C14 Connectors 9 8 feet 3 0 m CAB C13 C14 3M IN India 250 V 10 A 8 2 feet 2 5 m CAB 250V 10A IS Israel 250 VAC 10 A CEI 23 16 VII plug 8 2 feet 2 5 m CAB 9K10A IT Italy Power Cord Jumper C13 C14 Connectors 6 6 feet 2 0 m CAB C13 C14 2M JP Japan 125 VAC 13 A NEMA...

Page 62: ... D Grid and SD connectors 277V AC CAB HVAC SD 0 6M HVAC 6 6 foot 2 0 m cable with Saf D Grid and C14 use for up to 240 V connector 250V AC CAB HVAC C14 2M HVAC 2 foot 0 6 m cable with Saf D Grid and RT connector 277V AC CAB HVAC RT 0 6M HVDC 6 6 foot 2 0 m cable with Saf D Grid and three terminal connectors 300V AC 400V DC 200 200 V DC CAB HVDC 3T 2M Not applicable All except Argentina Brazil and ...

Page 63: ...wing table lists the regulatory standards compliance for the switch Table 3 Regulatory Standards Compliance Safety and EMC Description Specification Products should comply with CE Markings according to directives 2004 108 EC and 2006 95 EC Regulatory compliance CAN CSA C22 2 No 60950 1 Second Edition EN 60950 1 Second Edition IEC 60950 1 Second Edition IEC 623681 AS NZS 60950 1 GB4943 Safety Cisco...

Page 64: ...000 3 2 EN61000 3 3 KN22 Class A CNS13438 Class A EMC Emissions EN55024 CISPR24 EN300386 KN 61000 4 series EMC Immunity The product is RoH 6 compliant with exceptions for leaded ball grid array BGA balls and lead press fit connectors RoHS Cisco Nexus 93108TC FX ACI Mode Hardware Installation Guide 54 System Specifications System Specifications ...

Page 65: ...has activated this LED to identify this switch in the chassis Flashing blue BCN This switch is not being identified Off The switch is operational Green STS The switch is booting up Flashing amber Temperature exceeds the minor alarm threshold Amber Temperature exceeds the major alarm threshold Red The switch is not receiving power Off Fans and power supply modules are operational Green ENV At least...

Page 66: ...onal fan is probably not functional Red Fan module is not receiving power Off Power Supply LEDs The power supply LEDs are located on the left front portion of the power supply Combinations of states indicated by the Okay and Fault LEDs indicate the status for the module as shown in the following table Status LED LED Power supply is on and outputting power to the switch Off Green Power supply is co...

Page 67: ... of the following conditions High voltage High power Low voltage Power supply installed in chassis but not connected to a power source Slow power supply fan Flashing amber Green Cisco Nexus 93108TC FX ACI Mode Hardware Installation Guide 57 LEDs LEDs ...

Page 68: ...Cisco Nexus 93108TC FX ACI Mode Hardware Installation Guide 58 LEDs LEDs ...

Page 69: ...ty Description Illustration 1 Rack mount kit Front brackets 2 Rear brackets 2 Slider rails 2 M4 Phillips pan head screws 6 1 Ground lug kit Two hole lug 1 M4 x 8 mm Phillips pan head screws 2 1 EAC Compliance document Not applicable 1 Hazardous substances list for customers in China Not applicable The following table lists and illustrates the console cable CAB CONSOLE RJ45 that can be ordered Cisc...

Page 70: ...acket 2 Slider rails 2 M4 x 7 mm mounting screws 16 1 Ground lug kit Two hole lug 1 M4 x 8 mm Phillips pan head screws 2 1 EAC Compliance document Not applicable 1 Hazardous substances list for customers in China Not applicable The following table lists and illustrates the console cable CAB CONSOLE RJ45 that can be ordered Quantity Description Illustration 1 Console cable with DB 9F and RJ 45F con...

Page 71: ...tion and accessibility The following table lists the site planning tasks that we recommend that you complete before you install the switch Your completion of each task ensures a successful switch installation Verification Time and Date Planning Activity Space evaluation Space and layout Floor covering Impact and vibration Lighting Physical access Maintenance access Environmental evaluation Ambient...

Page 72: ...center ground Cable and interface equipment evaluation Cable type Connector type Cable distance limitations Interface equipment transceivers EMI evaluation Distance limitations for signaling Site wiring RFI levels Contact and Site Information Use the following worksheet to record contact and site information for the installation Contact person Contact phone Cisco Nexus 93108TC FX ACI Mode Hardware...

Page 73: ...ial number Product number Use the following worksheet to record network related information Switch IP address Switch IP netmask Hostname Domain name IP broadcast address Gateway router address DNS address Use the following worksheet to record information about the modules in the switch Notes Module Serial Number Module Type Module Slot Fan module 1 Cisco Nexus 93108TC FX ACI Mode Hardware Installa...

Page 74: ...le Type Module Slot Fan module 2 Fan module 3 Fan module 4 Power Supply 1 Power Supply 2 Cisco Nexus 93108TC FX ACI Mode Hardware Installation Guide 64 Site Preparation and Maintenance Records Site Preparation and Maintenance Records ...