IBM REDPAPER 520Q, Technical Overview And Introduction

Page 1: ...ion Giuliano Anselmi Charlie Cler Carlo Costantini Bernard Filhol SahngShin Kim Gregor Linzmeier Ondrej Plachy Finer system granulation using Micro Partitioning technology to help lower TCO Support fo...

Page 2: ......

Page 3: ...International Technical Support Organization IBM System p5 520 and 520Q Technical Overview and Introduction September 2006...

Page 4: ...ication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp Second Edition September 2006 This edition applies to IBM System p5 520 product number 9131 52A Linux and IBM AIX 5L Version...

Page 5: ...for Express Product Offerings 11 1 5 System racks 12 1 5 1 IBM 7014 Model T00 rack 13 1 5 2 IBM 7014 Model T42 rack 13 1 5 3 IBM 7014 Model S11 rack 14 1 5 4 IBM 7014 Model S25 rack 14 1 5 5 S11 rack...

Page 6: ...rage DS6000 and DS8000 Series 47 2 10 Logical partitioning 48 2 10 1 Dynamic logical partitioning 48 2 11 Virtualization 48 2 11 1 POWER Hypervisor 49 2 12 Advanced POWER Virtualization feature 51 2 1...

Page 7: ...ocation 81 3 1 8 Serviceability 82 3 2 Manageability 83 3 2 1 Service processor 83 3 2 2 Partition diagnostics 84 3 2 3 Service Agent 85 3 2 4 IBM System p5 firmware maintenance 87 3 3 Cluster solutio...

Page 8: ...vi IBM System p5 520 and 520Q Technical Overview and Introduction...

Page 9: ...aterials for this IBM product and use of those Web sites is at your own risk IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obliga...

Page 10: ...POWER5 PTX Redbooks RS 6000 Service Director System p System p5 System Storage TotalStorage Virtualization Engine 1350 The following terms are trademarks of other companies Internet Explorer Microsoft...

Page 11: ...based in Rome Italy For seven years he was an IBM Sserver pSeries Systems Product Engineer supporting the Web Server Sales Organization in EMEA IBM Sales IBM Business Partners Technical Support Organ...

Page 12: ...from Czech Technical University CVUT Prague He has worked at Supercomputing Centre of Czech Technical University for four years and currently works for IBM seven years in the AIX 5L support team The p...

Page 13: ...edbook form found at ibm com redbooks Send your comments in an e mail to redbook us ibm com Mail your comments to IBM Corporation International Technical Support Organization Dept HYTD Mail Station P0...

Page 14: ...xii IBM System p5 520 and 520Q Technical Overview and Introduction...

Page 15: ...r p5 520Q Express Product Offering that is only available on an initial order request you might qualify for processor activation at no extra charge The number of processors total memory quantity and s...

Page 16: ...feature supports up to 40 micro partitions using Micro Partitioning technology The Integrated Virtualization Manager provides partition management in settings where an HMC is unavailable or not desir...

Page 17: ...on page 4 shows the system Table 1 2 Physical attributes of the deskside model Description Range Operating temperature 5 to 35 degrees Celsius 41 to 95 F Relative humidity 8 to 80 Operating voltage 1...

Page 18: ...office environments The system is designed to be set up by the client and in most cases does not require the use of any tools The system includes full setup instructions The GXT135P 2D graphics accele...

Page 19: ...mounted version FC 7190 must be selected In addition to the rack mounted version the server can be installed in either IBM or OEM racks Therefore you are required to select one of the following featur...

Page 20: ...optional tape device The p5 520 and p5 520Q including the service processor that is described in 3 2 1 Service processor on page 83 support the following native ports Two 10 100 1000 Ethernet ports o...

Page 21: ...ng the disk drive features available at the time of writing The p5 520 and p5 520Q feature up to eight disk drive bays two slim line media device bays and one half height media bay The minimum configu...

Page 22: ...and lightweight USB V2 attached diskette drive takes its power requirements from the USB port The drive can be attached to the integrated USB ports or to a USB adapter FC 2738 A maximum of one USB dis...

Page 23: ...embly Seven PCI X slots 3 3 volt keyed 133 MHz hot pluggable Two 6 pack hot swappable SCSI bays optional Redundant hot swap power optional Two RIO 2 ports and two SPCN ports The I O drawer has the fol...

Page 24: ...a 1U 19 inch rack mountable drawer that has two native Ethernet ports and can be extended with one additional two port 10 100 1000 Gb adapter In HMC managed installations with very high demand for hi...

Page 25: ...ber on AIX 5L and either Red Hat or SUSE Linux You can choose either the lower priced AIX 5L or Linux subscription but not both If you choose AIX 5L for your lower priced operating system you can also...

Page 26: ...improved floor space utilization Feature code description Rack mounted feature codes Deskside feature code System bezel and hardware 7190 7916 x 1 Rack mount rail kit 7160 x 1 n a 850 Watt power suppl...

Page 27: ...ion panel is fixed to the top of the rack Up to four power distribution units PDUs can be mounted in the PDU bays see Figure 1 4 on page 17 additional PDUs can fit inside the rack See 1 5 6 The ac pow...

Page 28: ...nternal threads 1 5 3 IBM 7014 Model S11 rack The Model S11 rack satisfies many light duty requirements for organizing smaller rack mount servers and expansion drawers The 0 6 meter high rack has a pe...

Page 29: ...and therefore reduces the space available for mounting servers and other components FC 0469 Customer Specified Rack Placement provides the ability to specify the physical location of the system module...

Page 30: ...tilation in the rack Machine type model Name Supported in 7014 S11 rack 7014 S25 rack 7037 A50 IBM System p5 185 Y Y 7031 D24 T24 EXP24 Disk Enclosure Y Y 7311 D20 I O Expansion Drawer Y Y 9110 510 IB...

Page 31: ...ent countries and applications by varying the PDU to wall power cord which must be ordered separately Each power cord provides the unique design characteristics for the specific power requirements To...

Page 32: ...tional options for the rack This section highlights some solutions available to provide a single point of management for environments composed of multiple p5 520 or p5 520Q servers or other IBM System...

Page 33: ...that occupies only 1U 1 75 inches in a 19 inch standard rack Ability to mount the IBM Travel Keyboard in the 7316 TF3 rack keyboard tray Support for the new 1x8 LCM switch FC 4280 the Netbay LCM2 FC 4...

Page 34: ...ximum video resolution of 1280x1024 The Console Switch allows for two levels of tiering and supports up to 64 servers at a single user location through switch tiering The previous VGA switch FC 4200 t...

Page 35: ...ry of this standard is available in the publication IBM System p5 IBM Eserver p5 and i5 and OpenPower Planning SA38 0508 The key points mentioned in this documentation are as follows The front rack op...

Page 36: ...e holes spaced from bottom to top 15 9 mm 0 625 in 15 9 mm 0 625 in and 12 67 mm 0 5 in on center making each three hole set of vertical hole spacing 44 45 mm 1 75 in apart on center Rail mounting hol...

Page 37: ...must be compatible with drawer mounting rails including a secure and snug fit of the rail mounting pins and screws into the rack or cabinet rail support hole Note The OEM rack must only support ac po...

Page 38: ...24 IBM System p5 520 and 520Q Technical Overview and Introduction...

Page 39: ...bridge 3 IDE controller 133 MHz 64 bit Dual SCSI Ultra320 64 bit RAID enablement card 4 pack disk drive backplane P2 T15 L15 L0 4 pack disk drive backplane P3 T14 L15 L0 Tape drive P4 D1 Optional medi...

Page 40: ...due to cooling restrictions The POWER5 design provides the following additional enhancements New page sizes in ERAT and TLB Two new pages sizes 64 KB and 16 GB which were recently added in PowerPC ar...

Page 41: ...e The 1 core p5 520 POWER5 system planar contains a single core module SCM and the local memory storage subsystem for that SCM The POWER5 single core processor is packaged in the SCM The 1 core 1 65 G...

Page 42: ...DCM and associated memory Figure 2 5 The p5 520 POWER5 2 1 GHz DCM with DDR2 memory socket layout view The storage structure for the POWER5 processor is a distributed memory architecture that provide...

Page 43: ...essor has access to the same memory slots through the Vertical Fabric Bus I O connects to the p5 520Q QCM using the GX bus The QCM provides a single GX bus One POWER5 processor has direct access to th...

Page 44: ...k speed in MHz The following output is from a 4 core p5 520Q system running at 1 5 GHz with simultaneous multithreading enabled Cpu 0 runs at 1498 MHz Cpu 1 runs at 1498 MHz Cpu 2 runs at 1498 MHz Cpu...

Page 45: ...mory DIMMs slot location information SMI II SMI II DIMM CX JXX Ax DIMM CX JXX Ax DIMM CX JXX Ax DIMM CX JXX Ax DIMM CX JXX Ax DIMM CX JXX Ax DIMM CX JXX Ax DIMM CX JXX Ax 2x8 B 528 MHz J2A J2B J2C J2D...

Page 46: ...ory and the processor There are two Synchronous Memory Interface SMI II chips each with a single 8 byte read and 2 byte write high speed Elastic Interface II bus to the memory controller of the proces...

Page 47: ...s other GX bus hubs to be connected into the system Each Enhanced I O Controller can provide four separate PCI X 2 0 buses Each PCI X 2 0 bus is 64 bits in width and individually capable of running ei...

Page 48: ...I X to PCI X bridges to the Enhanced I O Controller The connections of the PCI X slots and integrated I O devices to the PCI X to PCI X bridges are properly distributed to maximize the system performa...

Page 49: ...IM using Ethernet and token ring adapters CHRP1 is the platform type Token ring is not allowed as the initial order Table 2 4 Available LAN adapters 2 6 2 SCSI adapters To connect to external SCSI dev...

Page 50: ...iSCSI is an open standards based approach by which SCSI information is encapsulated using the TCP IP protocol to allow its transport over IP networks It allows transfer of data between storage and ser...

Page 51: ...ia iSCSI to Fibre Channel storage routers and switches iSCSI solutions always involve the following software and hardware components Initiators These are the device drivers and adapters that reside on...

Page 52: ...Small Computer System Interface iSCSI functionality Support for Network File System NFS Common Internet File System CIFS and iSCSI protocols Data ONTAP software at no charge with plenty of additional...

Page 53: ...d fabric An IBM System p5 server proposal might also include the Topspin Server Switch model 120 and 270 in an initial system order The Topspin Server Switch models 120 and 270 are programmable switch...

Page 54: ...T administered standard and certification program for cryptographic modules The firmware for the FC 4764 is available on a separately ordered and distributed CD This firmware is an LPO product 5733 CY...

Page 55: ...system planar The virtual SCSI Enclosure Services VSES hot plug control functions are provided by the Ultra320 SCSI controllers 2 7 1 Internal media devices The p5 520 and p5 520Q provide two slim lin...

Page 56: ...Volume Manager when using disk hot swap capabilities For more information see Problem Solving and Troubleshooting in AIX 5L SG24 5496 After the SCSI disk hot swap procedure you can expect to find SCS...

Page 57: ...ust have the RIO 2 loop adapter FC 6417 to be connected to the p5 520 or p5 520Q systems The PCI X host bridge inside the I O drawer provides two primary 64 bit PCI X buses running at 133 MHz Therefor...

Page 58: ...is feature provides two Remote I O ports for attaching up to four 7311 Model D20 I O drawers to the system in a single loop Figure shows how you could connect four I O drawers to one system Note Any 6...

Page 59: ...monitor the status of power and cooling within the I O drawer The SPCN is a loop the cabling starts from SPCN port 0 on the p5 520 or p5 520Q system to SPCN port 0 on the first I O drawer The loop is...

Page 60: ...ility storage solutions or simply high capacity storage for a single server installation the unit provides a cost effective solution It provides 24 hot swappable disk bays 12 accessible from the front...

Page 61: ...up to 16 bootable partitions or 64 bootable partitions if the turbo option is selected that are attached with a Fibre Channel Adapter Model DS4500 provides up to 64 bootable partitions Model DS4800 pr...

Page 62: ...nhancements to enable dynamic LPAR was introduced in 2002 The ability to reconfigure dynamic LPARs encourages system administrators to dynamically redefine all available system resources to reach the...

Page 63: ...emory to support the logical partitions on the server The amount of memory required by the POWER Hypervisor firmware varies according to several factors Factors influencing the POWER Hypervisor memory...

Page 64: ...ng IEEE 802 1Q VLAN standard A partition supports 256 virtual Ethernet adapters Besides a default port VLAN ID the number of additional VLAN ID values that can be assigned per virtual Ethernet adapter...

Page 65: ...lization in general see http www ibm com servers eserver pseries ondemand ve resources html 2 12 1 Micro Partitioning technology The concept of Micro Partitioning technology allows you to allocate fra...

Page 66: ...logical processors was introduced The operating system AIX 5L or Linux sees one physical processor as two logical processors if the simultaneous multithreading feature is on It can be turned off whil...

Page 67: ...imum usable processor units is two You are allowed to define more virtual processors than physical processors In that case the virtual processor is waiting for dispatch more often and you should consi...

Page 68: ...f physical adapters in the system The Virtual I O Server eliminates the requirement that every partition own a dedicated network adapter disk adapter and disk drive Figure 2 15 on page 55 shows an org...

Page 69: ...Network Install Manager NIM Two major functions are provided with the Virtual I O Server a shared Ethernet adapter and Virtual SCSI Note To increase the performance of I O intensive applications use d...

Page 70: ...rtual I O Server partition is limited only by the resource availability because there are no configuration limits For a more detailed discussion about virtual networking see http www ibm com servers a...

Page 71: ...thernet and shared ethernet adapter support including TCP IP Acceleration Large Block Send 2 12 4 Partition Load Manager Partition Load Manager PLM provides automated processor and memory distribution...

Page 72: ...requirements for an IVM managed server are as follows A server managed by IVM cannot be simultaneously managed by an HMC IVM with Virtual I O Server must be installed as the first operating system An...

Page 73: ...nage and provide virtual I O for client partitions But the dispatch of partitions on physical processors is done by the POWER Hypervisor as in HMC managed servers The rules for mapping the physical pr...

Page 74: ...c and private networks if you need both The HMC 7310 Model C05 is a deskside model with one integrated 10 100 1000 Mbps Ethernet port and two additional PCI slots The 7310 Model CR3 is a 1U 19 inch ra...

Page 75: ...ff Acting as a service focal point The HMC provides both graphical and command line interface for all management tasks Remote connection to the HMC using Web based System Manager or SSH is possible Fo...

Page 76: ...em Planning Tool SPT is the next generation of the IBM LPAR Validation Tool LVT It contains all of the function from the LVT and is integrated with the IBM Systems Workload Estimator WLE System plans...

Page 77: ...o create a system that is based upon performance and capacity data from an existing system or that is based on new workloads that you specify You can use the SPT before you order a system to determine...

Page 78: ...AIX 5L V5 2 and V5 3 for POWER5 systems was made available This is a way for clients to receive software and associated publications online as opposed to waiting for a physical shipment to arrive Clie...

Page 79: ...user can specify page sizes to use for three regions process address space with an environment variable or with settings in an application s XCOFF binary using the ldedit or ld commands These three r...

Page 80: ...ither through IBM Global Services or through the distributor to receive updates and technical assistance as needed Support is not included in the price of the subscription The new lower priced Linux s...

Page 81: ...2 15 2 Securing a rack mounted system into a rack The optional rack mount drawer rail kit is a unique kit designed for use with the rack mounted model No tools are required to install the server or d...

Page 82: ...ack When the rails are fully extended the rail safety latches lock into place This action prevents the system or expansion unit from being pulled out too far The IBM Systems Hardware Information Cente...

Page 83: ...ride and system operating mode Function 03 Start IPL Function 04 Lamp test Function 05 Reserved Function 06 Reserved Function 07 SPCN functions Function 08 Fast power off Functions 09 to 10 Reserved F...

Page 84: ...or maintains two copies of the server firmware One copy is considered the permanent or backup copy and is stored on the permanent side sometimes referred to as the p side The other copy is considered...

Page 85: ...current level of firmware you copy the firmware level that is currently installed on the permanent side from the permanent side to the temporary side System firmware download Web site For the system f...

Page 86: ...Change Internal Code wizard on the HMC provides a step by step process for you to perform the procedure to install the fix Perform these steps 1 Ensure that you have a connection to the service provi...

Page 87: ...Processor see 3 2 1 Service processor on page 83 2 15 8 Hardware management user interfaces This section provides a brief overview of the different hardware management user interfaces available Advanc...

Page 88: ...m resources are in use during that phase Accessing the ASMI using an ASCII console The Advanced System Management Interface on an ASCII console supports a subset of the functions provided by the Web i...

Page 89: ...n HMC installed in a rack that is used to access another HMC remotely A remote HMC can be present in an open network Redundant HMC A redundant HMC manages a server that is already managed by another H...

Page 90: ...memory and is replaced when AIX 5L or Linux takes control Partition firmware is needed only for the time that is necessary to load AIX 5L or Linux into the partition server memory The global Open Firm...

Page 91: ...n features that help lower the total cost of ownership TCO IBM reliability availability and service RAS technology allow you to improve your TCO architecture by reducing unplanned down time This chapt...

Page 92: ...escribe the concepts that form the basis of leadership RAS features of IBM System p5 systems in more detail 3 1 1 Fault avoidance IBM System p5 servers are built on a quality based design that is inte...

Page 93: ...allocation on page 81 3 1 3 Permanent monitoring The service processor SP included in the p5 520 or p5 520Q provides a way to monitor the system even when the main processor is inoperable Mutual surve...

Page 94: ...gle bit error correction using ECC without reaching error thresholds for main L2 and L3 cache memory L3 cache line deletes extended from 2 to 10 for additional self healing ECC extended to inter chip...

Page 95: ...Partial L2 cache deallocation PCI X bus and slots For dynamic processor deallocation the service processor performs a predictive failure analysis based on any recoverable processor errors that have b...

Page 96: ...overy of intermittent PCI bus parity errors and graceful transition to the I O device available state in the case of a permanent parity error in the PCI bus EEH enabled adapters respond to a special d...

Page 97: ...ed for service Concurrent maintenance Concurrent Maintenance provides replacement of the following parts while the system remains running Disk drives Cooling fans Power subsystems PCI X adapter cards...

Page 98: ...n AIX 5L Online diagnostics when installed are resident with AIX 5L on the disk or server They can be booted in single user mode service mode run in maintenance mode or run concurrently concurrent mod...

Page 99: ...tomatic problem reporting where service calls are placed to IBM without intervention Automatic client notification In addition there are Commonly viewed hardware errors You can view hardware event log...

Page 100: ...ted information and error messages about the entire system and partitions If software level information such as performance data is also required you can also install Service Agent on any of the parti...

Page 101: ...om an HMC or from a running partition For update details see 2 15 6 System firmware on page 70 If you use an HMC to manage your server you can use the HMC interface to view the levels of server firmwa...

Page 102: ...for distributed and clustered server management Simplified use of IT resources Designed for 24x7 access to data applications Business continuity in the event of a disaster The POWER processor based A...

Page 103: ...luster Systems Management for AIX 5L HMC control cluster building block servers and cluster software available visit the following links Cluster 1600 http www ibm com servers eserver clusters hardware...

Page 104: ...90 IBM System p5 520 and 520Q Technical Overview and Introduction...

Page 105: ...Technical Overview and Introduction REDP 4024 IBM Eserver p5 510 Technical Overview and Introduction REDP 4001 IBM Eserver p5 520 Technical Overview and Introduction REDP 9111 IBM Eserver p5 550 Tech...

Page 106: ...dapters that you can use in this system System Unit Safety Information SA23 2652 contains translations of safety information used throughout the system documentation Online resources These Web sites a...

Page 107: ...view or download Redbooks Redpapers Hints and Tips draft publications and Additional materials as well as order hardcopy Redbooks or CD ROMs at this Web site ibm com redbooks Help from IBM IBM Support...

Page 108: ...94 IBM System p5 520 and 520Q Technical Overview and Introduction...

Page 109: ......

Page 110: ...ted cluster solutions This IBM Redpaper is a comprehensive guide that covers the IBM System p5 520 and 520Q UNIX servers It introduces major hardware offerings and discusses their prominent functions...