IBM 8677 - BladeCenter Rack-mountable - Power..., Planning And Installation Manual

Page 1: ...BladeCenter Type 8677 Planning and Installation Guide GA27 4327 01 ERserver...

Page 2: ......

Page 3: ...BladeCenter Type 8677 Planning and Installation Guide GA27 4327 01 ERserver...

Page 4: ...al information in Appendix C Notices on page 107 Second Edition August 2003 Copyright International Business Machines Corporation 2003 All rights reserved US Government Users Restricted Rights Use dup...

Page 5: ...ut network topology considerations and deployment considerations v Use Chapter 4 Installation considerations on page 41 and Appendix A Planning worksheets on page 85 to plan for the physical environme...

Page 6: ...iv BladeCenter Type 8677 Planning and Installation Guide...

Page 7: ...4 Power modules 15 Blowers 15 Acoustic attenuation option 16 Switch modules 16 Ethernet switch module 17 Standards 19 Fibre Channel switch module 20 Chapter 3 Deployment considerations 21 Network topo...

Page 8: ...t output 51 Power connections 51 Power layout example 1 single BladeCenter units 52 Power layout sample 2 two BladeCenter units in NetBAY 25 or 42U racks 54 Power layout sample 3 three BladeCenter uni...

Page 9: ...90 Cabling worksheet 92 Power worksheet 94 Appendix B Configuration Worksheets 95 Management module configuration worksheet 95 Ethernet switch module configuration worksheet 98 Blade server configura...

Page 10: ...viii BladeCenter Type 8677 Planning and Installation Guide...

Page 11: ...ng the design These features make it possible for you to customize the hardware to meet your needs today while providing flexible expansion capabilities for the future What the BladeCenter unit offers...

Page 12: ...ant network connections v System management capabilities The BladeCenter unit comes with a system management processor in the management module This system management processor in conjunction with the...

Page 13: ...r firmware Security features v Login password for remote connection v Secure socket layer SSL security for remote management access Predictive Failure Analysis PFA alerts v Blowers v Blade dependent f...

Page 14: ...chitecture include v Modular scalability Unlike traditional 8 way or 16 way servers blade servers are designed to scale out rather than up Adding a new server typically involves simply sliding a new s...

Page 15: ...attach a KVM cable power cable Ethernet cable and systems management cable per server you may only need to attach one of each cable per BladeCenter unit which contains multiple servers Even if you dec...

Page 16: ...e Manual and Troubleshooting Guide This document contains the information to help you solve BladeCenter problems yourself and information for service technicians v IBM Eserver BladeCenter HS20 Type 86...

Page 17: ...ions are available from the IBM Redbook Web site at http www ibm com redbooks From this Web site search for BladeCenter v The Cutting Edge IBM ERserver BladeCenter REDP3581 This document contains an i...

Page 18: ...nter unit v Information about IBM Director is available at this Web site http www 1 ibm com servers eserver xseries systems_management director_3 html v Information about Remote Deployment Manager RDM...

Page 19: ...nt module USB port CD ROM drive Diskette drive Attention To maintain proper system cooling each module bay must contain either a module or a filler module and each blade bay must contain either a blad...

Page 20: ...OM activity LED System service card Rear view This section identifies the components on the rear of the BladeCenter unit AC AC DC DC Switch module bay 3 Switch module 1 Switch module bay 4 Switch modu...

Page 21: ...s are configured automatically Use a USB cable to connect a device to the USB connector If you need to connect more than one USB device use a USB hub One 10 100 Mbps Ethernet for remote console and ma...

Page 22: ...s ideally suited for networking environments that require superior microprocessor performance efficient memory management flexibility and reliable data storage Control panel Release levers 2 IBM Enter...

Page 23: ...l SCSI storage expansion unit Size v Height 24 5 cm 9 7 inches v Depth 44 6 cm 17 6 inches v Width 2 9 cm 1 14 inches v Maximum weight 5 4 kg 12 lb Integrated functions v Two Gigabit Ethernet controll...

Page 24: ...twork management tasks This provides a greater level of security and a different network path to the BladeCenter unit if you experience an outage on the network path to the Ethernet switch modules The...

Page 25: ...e present in power bays 1 and 2 When any blade server or option is in blade bay 7 through 14 power modules must be present in power bays 1 and 2 and power modules must be present in power bays 3 and 4...

Page 26: ...ch module 1 is designated as the first connection or Local Area Connection by Microsoft Windows 2000 but is designated as the second connection or eth1 by Red Hat Linux Version 7 3 The Ethernet contro...

Page 27: ...nter unit 4 Network connection 4 from all blade server I O expansion options in the BladeCenter unit Ethernet switch module The following illustration shows the major components of your Ethernet switc...

Page 28: ...k aggregation with 802 3ad support on four external ports for up to two static trunk groups or two link aggregation control protocol LACP 802 3ad link aggregation groups Management v Spanning Tree Alg...

Page 29: ...e 100 meters maximum UTP Category 5 100 meters maximum EIA TIA 568B 100 ohm STP 100 meters maximum Standards The following standards apply to the Ethernet switch module v IEEE 802 3 10BASE T Ethernet...

Page 30: ...nel Classes 2 and 3 connectionless v External Port Type GL_Port v Port characteristics External ports are auto discovering and self configuring v Scalability Maximum 239 switches depending on configur...

Page 31: ...v Small server farm in a central location with one zone v Large server farm in a central location with one zone v Small server farm in a remote location with one zone Integrated infrastructure The Bla...

Page 32: ...blades v Blades 7 9 Collaboration solution consisting of Windows 2000 Domino blades v Blades 10 11 Terminal Serving solution consisting of Windows 2000 Citrix MetaFrame blades v Blades 12 13 File ser...

Page 33: ...tions are as follows in order of preference v Multi port Link Aggregation Group or Trunk 1 Gbps 1000 Mbps per port v Single uplink port 1 Gbps v Multi port Link Aggregation Group or Trunk 100 Mbps per...

Page 34: ...oops the spanning tree protocol on the switch module external ports must remain enabled Linux high performance cluster You can also deploy a series of BladeCenter units in a Linux High Performance Clu...

Page 35: ...ces in each rack can communicate directly with devices in another rack through the Ethernet Gb switches in each rack This solution can support up to 210 processor nodes It could also be expanded throu...

Page 36: ...ervice outage Seven blades provide Citrix Metaframe services to the infrastructure One is designated as the Data Collector and an additional blade is used to install a Web server and Citrix NFuse to p...

Page 37: ...ne domain controller per zone has been deployed In zone 1 two clustered file print servers relying on external Fibre Channel storage solutions not pictured have been installed to support all of the us...

Page 38: ...has a domain controller print server and RDM server which is dedicated to hosting only operating system images The RDM server communicates with the main RDM server in a central location through standa...

Page 39: ...wser to the IP address of the switch modules or the blade server to configure those components directly v Through a Telnet interface The Telnet interface provides a command line interface through the...

Page 40: ...Deployment infrastructure Although the deployment of the BladeCenter unit does not require you to do so you should consider setting up a separate management network to configure and manage your Blade...

Page 41: ...nnect directly to a management station from the Ethernet port of the management module Complete the following steps to connect the Ethernet cable to the management module 1 Connect one end of a Catego...

Page 42: ...nd configured dynamic host configuration protocol DHCP server on the network the host name IP address gateway address subnet mask and DNS server IP address are set automatically v If the DHCP server d...

Page 43: ...ce IBM Director and DHCP This scenario uses the following assumptions v The IBM Director server and console are on the same subnet as the external management port of the management module v A DHCP ser...

Page 44: ...se you are managing the BladeCenter unit from IBM Director you should disable local power control local keyboard video mouse KVM switching and local USB media switching 13 Configure each switch module...

Page 45: ...agement module a Enter a static IP address for NIC 1 if desired b Enter a static IP address for NIC 2 in same subnet as NIC 1 When prompted to reboot the management module click Yes 10 After the manag...

Page 46: ...nst the actual chassis components Note Turn on the BladeCenter BlueLight to identify the BladeCenter unit you are managing 5 Configure the management module Selecting the appropriate links from the We...

Page 47: ...face is described in the IBM BladeCenter Installation and User s Guide on the IBM BladeCenter Documentation CD BIOS When the BIOS code page jumper J12 is moved from pins 1 and 2 the default position t...

Page 48: ...can use either Remote Deployment Manager RDM 3 1 01 If you use RDM 3 1 01 make sure that you install Patch 2 and that you have a DHCP server in your management infrastructure Windows Remote Installat...

Page 49: ...tion of the BladeCenter management module These problems can be resolved after installation During installation you may have to double click to make selections that normally only require a single clic...

Page 50: ...pieces and redpapers available at http www ibm com redbooks v Deploying Lotus Domino on IBM Eserver BladeCenter v Deploying Citrix MetalFrame on IBM Eserver BladeCenter v Deploying Microsoft Exchange...

Page 51: ...site http www pc ibm com us eserver xseries library configtools Physical planning This section describes dimensions weight floor space and noise considerations Dimensions A BladeCenter unit has the f...

Page 52: ...can be installed in a 42U rack Table 3 BladeCenter noise emissions Number of BladeCenter units per rack Maximum sound power idle Maximum sound power operating Without Acoustics Module 1 7 4 bels 7 4 b...

Page 53: ...exceed regulatory limits Rack considerations IBM NetBAY When planning for IBM NetBAY racks the following factors must be considered v floor space that provides adequate access to the front and rear o...

Page 54: ...g 75 lb 185 kg 407 lb 1 1 1 When determining your floor load limits use the combined weight of the rack populated BladeCenter units power distribution units and other rack mounting hardware to ensure...

Page 55: ...e advantage of the power redundancy provided by the BladeCenter unit Table 5 shows how power is provided to the BladeCenter components Table 5 BladeCenter power modules Component Power module bays Bla...

Page 56: ...PN 32P17361 Not supported2 5 10 PDUs PN 32P1736 20 PDUs PN 32P17361 Not supported2 6 12 PDUs PN 32P1736 Not supported1 Not supported2 Notes 1 A maximum of 12 PDUs can be mounted in the zero U space o...

Page 57: ...2 PDUs PN 32P1767 or 2 PDUs PN 73P5855 2 PDUs PN 32P1767 or 2 PDUs PN 73P5855 2 2 PDUs PN 32P1767 and 4 PDUs PN 32P1736 2 PDUs PN 73P5855 2 PDUs PN 73P5855 3 2 PDUs PN 32P1767 and 6 PDUs PN 32P1736 4...

Page 58: ...ways of providing power sources to BladeCenter racks see Power connections on page 51 This section provides diagrams that show different ways to provide redundant power sources to one or more BladeCe...

Page 59: ...ttached to a specific BladeCenter unit when needed or plan for rack space for installing a keyboard mouse and monitor Electrical input The BladeCenter unit requires the following electrical input v Si...

Page 60: ...t per minute CFM of air circulation v Each BladeCenter unit has two blowers each one provides 300 CFM of air circulation v Air flow direction is from front to back v All BladeCenter chassis bays must...

Page 61: ...sufficient airflow volume and cooling capacity This may cause equipment at the top of the rack to run hot resulting in reduced reliability component failure data loss or server shut down Heat output...

Page 62: ...to 1200 W power supplies using power cable part number 36L8886 Figure 16 represents PDU part numbers North America 32P1766 24 Amps and 32P1767 48 Amps You can use either PDU in this postion When usin...

Page 63: ...ower supplies using power cable part number 00N7701 A B 48A 3 P HD 48A 3 P HD Figure 17 Power layout of a single BladeCenter installed in a NetBAY 25 or 42U rack with 1400 or 1800 W power supplies 3 p...

Page 64: ...ents PDU part numbers 32P1766 and 32P1767 North America You can use either PDU in this position The PDU is cabled to the 1400 W power supplies using power cable part number 74P4430 and to the 1800 W p...

Page 65: ...upplies using power cable part number 00N7701 A B 48A 1 P HD 48A 1 P HD Figure 20 Power layout of two BladeCenter units installed in NetBAY 25 to 42U racks with 1400 or 1800 W power supplies A B 48A 3...

Page 66: ...art number North America 32P1766 24 Amps Figure 23 represents rack PDU part number North America 32P1736 15 Amps and front end PDU part number North America 32P1767 48 Amps Figure 24 on page 57 front...

Page 67: ...Figure 24 and Figure 25 show configurations to support 1400 and 1800 W power supplies 4 3 2 1 4 3 2 1 4 3 2 1 48A 48A 48A 48A A B Figure 24 Three BladeCenter units in a NetBAY 25 or 42 rack 1400 or 18...

Page 68: ...merica 32P1736 15 Amps and 321767 48 Amps Figure 27 represents PDU part number North America 32L1767 48 Amps 4 3 2 1 A B 15A 15A 15A 15A 4 3 2 1 48A 48A 4 3 2 1 15A 15A 15A 15A 4 3 2 1 48A 48A 48A Fig...

Page 69: ...5847 48 Amps 3 phase A B 48A 1 P HD 48A 1 P HD 48A 1 P HD 48A 1 P HD Figure 28 Power layout for four BladeCenter units in a NetBAY 42U rack with 1400 or 1800 W power supplies Figure 29 Power layout fo...

Page 70: ...27 on page 58 Figure 28 on page 59 and Figure 30 show configurations to support 1400 and 1800 W power supplies Figure 29 on page 59 is valid only with 1400 W power supplies A B 48A 3 P HD 48A 3 P HD...

Page 71: ...k Figure 31 represents PDU part number North America 32P1736 15 Amps and front end PDU part number North America 32P1767 48 Amps Figure 32 on page 62 represents front end PDU part number North America...

Page 72: ...North America 73P5855 48 Amps 4 3 2 1 4 3 2 1 4 3 2 1 4 3 2 1 4 3 2 1 48A 48A 48A 48A 48A 48A 48A 48A A B Figure 32 Five BladeCenter units in a NetBAY 42 rack 1400 and 1800 W power supplies 62 BladeC...

Page 73: ...part number North America 73P5847 48 Amps 3 phase A B 48A 1 P HD 48A 1 P HD 48A 1 P HD 48A 1 P HD Figure 33 Power layout for five BladeCenter units in a NetBAY 42 rack with 1400 W power supplies Chapt...

Page 74: ...pplies Figure 33 on page 63 shows an example configuration to support only 1400 W power supplies Figure 34 shows an example configuration to support 1400 or 1800 W power supplies A B 48A 3 P HD 48A 3...

Page 75: ...represents PDU part number North America 32P1736 15 Amps and front end PDU part number North America 32P1767 48 Amps Figure 36 on page 66 represents front end PDU part number North America 32P1767 48...

Page 76: ...merica 73P5847 48 Amps 3 phase 4 3 2 1 4 3 2 1 4 3 2 1 48A 48A 48A 48A 4 3 2 1 4 3 2 1 4 3 2 1 48A 48A 48A 48A A B Figure 36 Six BladeCenter units in a NetBAY 42U rack 1400 or 1800 W power supplies 66...

Page 77: ...es Figure 36 on page 66 and Figure 37 show example configurations to support 1400 or 1800 W power supplies A B 48A 3 P HD 48A 3 P HD 48A 3 P HD 48A 3 P HD Figure 37 Power layout for six BladeCenter un...

Page 78: ...s to the rack Note To meet the de rated amperage requirements shown in this figure IBM provides the following PDUs v 15A NetBAY rack 15 amp power distribution unit part number 37L6866 v 48A NetBAY sin...

Page 79: ...er distribution for three BladeCenter units installed in a NetBAY 42 standard rack Power sources A and B represent separate power feeds to the rack 1 4 B B A A 3 2 I I I I 48A 1 4 3 2 1 1 1 4 4 4 3 3...

Page 80: ...ote To meet the de rated amperage requirements shown in this figure IBM provides the following PDUs v 15A NetBAY rack 15 amp power distribution unit part number 37L6866 v 48A NetBAY single phase 60 am...

Page 81: ...ck Note To meet the de rated amperage requirements shown in this figure IBM provides the following PDUs v 15A NetBAY rack 15 amp power distribution unit part number 37L6866 v 48A NetBAY single phase 6...

Page 82: ...38 seconds Blade server SCSI disk drive 7 seconds Blade server I O expansion option 2 minutes Blade server into BladeCenter unit 9 seconds BladeCenter rack mounting kit and BladeCenter unit into rack...

Page 83: ...formation provided in this section along with the worksheet provided see Management module configuration worksheet on page 95 to plan for the configuration of the management module In addition complet...

Page 84: ...on planning One Ethernet switch module is required in switch module bay 1 of your BladeCenter unit The switch module provides four external Ethernet ports which you can use to communicate with an exte...

Page 85: ...re the management module will override any settings you specify here v If the switch is going to obtain an IP address manually you need to enter an IP address subnet mask and default gateway The switc...

Page 86: ...the best transmission speed duplex mode and flow control settings based on the capabilities of the device at the other end v You can enable or disable Flow Control in conjunction with all of the Spee...

Page 87: ...h switches to connect to the same external network v Ethernet switch ports used to span multiple BladeCenter units must be configured as Tagged with a specific default VLAN ID for any untagged packets...

Page 88: ...U E 2 14 2 VLAN 2 U E 2 Ext 1 1 Default T E 1 2 3 Ext 2 1 Default T E 1 2 3 Ext 3 1 Default T E 1 2 3 Ext 4 1 Default T E 1 2 3 Untagged packets are tagged as 1 Default Tagged packets are not changed...

Page 89: ...able Port mirroring The switch module provides the capability of copying frames that were transmitted and received on a source port and redirecting the copies to another target port The source port ca...

Page 90: ...state Instead it jumps from the listening state directly to the forwarding state 4 You can choose whether to enable or disable the STP state for a port Class of service The switch module supports 802...

Page 91: ...through your network The method you choose is used as a hashing function to determine which packets go to which port 802 3ad link aggregation Notes 1 The Mode determines whether or not a port is enabl...

Page 92: ...s not established unless both stations are configured correctly and the port to port cables are installed on a subset of configured ports or all of the configured ports For the purpose of this discuss...

Page 93: ...em will not detect the device this is the equivalent of unplugging the device v If you disable the on board IDE controller and no other controller and mass storage device are installed operating syste...

Page 94: ...84 BladeCenter Type 8677 Planning and Installation Guide...

Page 95: ...orksheet on page 90 helps you aggregate total BladeCenter weight PDU weight and mounting kit weight to ensure that v The weight of the BladeCenter units and other hardware does not exceed the rack loa...

Page 96: ...ion for the operating system 4 Record the applications for this server 5 Chose which internal Ethernet switch module this server uses circle 1 or 2 6 Select DHCP or record static IP address informatio...

Page 97: ...0 11 12 13 14 BladeCenter chassis number 1 2 3 4 5 6 Rack number and location ________________________________________ Component Quantity Details Weight Base or optional feature 8678 blade server or 8...

Page 98: ...Note Four power modules are required if you install a blade server or SCSI Storage Expansion option in any one of bay locations 7 through 14 on the BladeCenter unit 4 If any are to be installed circl...

Page 99: ...ty 0 91 kg or 2 lb ___ Optional Required if this BladeCenter unit provides network connectivity for an external fibre channel storage device or a SAN Blowers 2 5 67 kg 12 6 lb Base Media tray 1 N A Ba...

Page 100: ...o the total rack load and record it on the worksheet Verify that this weight does not exceed the Floor load limits of the location where the rack is being installed Table 16 Rack worksheet Component L...

Page 101: ...Table 16 Rack worksheet continued Component Location Quantity Weight Total floor load Rack weight___ Total rack load ___ Appendix A Planning worksheets 91...

Page 102: ...6 The rack number and location The From port and To connections See Table 18 on page 93 for details on cable media types and distances Table 17 Cabling worksheet BladeCenter number ___ Location in ra...

Page 103: ...EIA TIA 568 100 Ohm STP 100 meters 328 1 ft RJ45 Ethernet 10BASE T 10 Mbps Category 3 4 or 5 UTP 100 Ohm STP 100 meters 328 1 ft RJ45 Fibre Channel 1 062 or 2 124 Gigabits 50 micron multimode Fibre C...

Page 104: ...nput power Number of PDU outlets Number of power cords 1 and 2 1 to 6 Power modules 1 and 2 2 kVA 2 outlets 2 1 per module 1 2 3 and 4 7 to 14 Power modules 1 2 3 and 4 4 kVA 4 outlets 4 1 per module...

Page 105: ...nagement module For information on filling out the worksheet see Management module configuration planning on page 73 and the BladeCenter Type 8677 Installation and User s Guide General Settings Manage...

Page 106: ...N Host name or IP Address e mail address Global Remote Alert Settings Remote alerts retry limit Delay between retries Include event log with e mail alerts ___ Yes ___ No Alerts to be Monitored Critica...

Page 107: ...P address DNS DNS __ Enable __ Disable DNS Server IP address 1 _____ _____ _____ _____ DNS Server IP address 2 _____ _____ _____ _____ DNS Server IP address 3 _____ _____ _____ _____ Security Secure S...

Page 108: ...l contact Get IP from manual BOOTP DHCP IP address ____ ____ ____ ____ Subnet mask ____ ____ ____ ____ Default gateway ____ ____ ____ ____ Switch GVRP enabled or disabled MAC address aging timer enabl...

Page 109: ...____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____...

Page 110: ...onfiguration ____ ____ ____ ____ History log ____ ____ ____ ____ 802 1Q VLANs VLAN ID VID ____________________ VLAN name _______________________ Port Tag None Egress Forbidden 1 2 3 4 5 6 7 8 9 10 11...

Page 111: ...disabled __ Enabled __ Disabled Querier State __ Non Querier __ V1 querier __ V2 querier Query Interval 1 to 65 500 seconds Maximum response 1 to 25 seconds Robustness variable 1 to 255 Multicast for...

Page 112: ...s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 EX1 EX2 EX3 EX4 Spanning tree settings Status enabled or disabled Maximum age 6 to 40 seconds Hello time 1 to 10 seconds Forward delay 4 to 30 seconds Priority 0 to...

Page 113: ...e __ Enabled __ Disabled __ Weighted Round Robin Class Weight 0 to 16 Max latency High priority Med H priority Med L priority Low priority 802 1p priority mapping 802 1p priority mapping class __ Low...

Page 114: ...bled Class __ Low priority __ Med low priority __ Med high priority __ High priority Code point Class Code point Class Low M L M H High Low M L M H High 0 32 1 33 2 34 3 35 4 36 5 37 6 38 7 39 8 40 9...

Page 115: ...P Src Dest IP Non IP N A N A N A IP Port trunking Group ID Port Method EX1 EX2 EX3 EX4 Trunk Disabled 802 3ad link aggregation Link Aggregation Control Protocol enabled or disabled __________ System P...

Page 116: ...eCenter HS20 Installation and User s Guide Bay Num Blade server name IDE Controller Ethernet Controller Startup boot sequence E D E D IDE drive 1 IDE drive 2 CD ROM Diskette Network 1 2 3 4 5 6 7 8 9...

Page 117: ...IMPLIED INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF NON INFRINGEMENT MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE Some jurisdictions do not allow disclaimer of express or implied war...

Page 118: ...Pentium and ProShare are trademarks of Intel Corporation in the United States other countries or both Microsoft Windows and Windows NT are trademarks of Microsoft Corporation in the United States oth...

Page 119: ...Commission FCC statement Note This equipment has been tested and found to comply with the limits for a Class A digital device pursuant to Part 15 of the FCC Rules These limits are designed to provide...

Page 120: ...ation of the laws of the Member States relating to electromagnetic compatibility IBM cannot accept responsibility for any failure to satisfy the protection requirements resulting from a nonrecommended...

Page 121: ...rtified by the Canadian Standards Association CSA For units intended to be operated at 230 volts U S use Use a UL listed and CSA certified cord set consisting of a minimum 18 AWG Type SVT or SJT three...

Page 122: ...112 BladeCenter Type 8677 Planning and Installation Guide...

Page 123: ...drive specifications 3 E electrical input 3 13 49 electronic emission Class A notice 109 environment 3 13 Ethernet cable specifications 17 configuring settings 32 port cabling 32 Ethernet continued st...

Page 124: ...er planning worksheet 94 power supply specifications 3 product data 12 R rack planning worksheet 90 racks 43 remote management port 11 S size 3 specifications 3 data transmission rates 17 network cabl...

Page 125: ......

Page 126: ...Printed in USA GA27 4327 01...