IBM 9123710 - eServer OpenPower 710, Introduction Manual

Page 1: ...r 710 Technical Overview and Introduction Dexter Charles Peter Jürß Finer system granulation using Micro Partitioning technology to help lower TCO Outstanding performance based on IBM POWER5 processor technology From Web servers to integrated cluster solutions ...

Page 2: ......

Page 3: ...International Technical Support Organization IBM Sserver OpenPower 710 Technical Overview and Introduction June 2005 ...

Page 4: ...sers Restricted Rights Use duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp First Edition June 2005 This edition applies to the IBM Sserver OpenPower 710 server and Linux Note Before using this information and the product it supports read the information in Notices on page v ...

Page 5: ...6000 7014 Model T42 Enterprise Rack 9 1 6 3 AC Power Distribution Unit and rack content 9 1 6 4 Rack mounting rules for OpenPower 710 server 10 1 6 5 Additional options for rack 11 1 6 6 OEM rack 12 Chapter 2 Architecture and technical overview 15 2 1 The POWER5 chip 16 2 1 1 Simultaneous multi threading 17 2 1 2 Dynamic power management 18 2 2 Processor and cache 19 2 2 1 Available processor spee...

Page 6: ...tem firmware 42 2 9 5 Service processor 44 2 9 6 Hardware management user interfaces 45 Chapter 3 Reliability availability and serviceability 49 3 1 Reliability fault tolerance and data integrity 50 3 1 1 Memory error correction extensions 50 3 1 2 Redundancy for array self healing 50 3 1 3 Service processor 51 3 1 4 Fault monitoring functions 51 3 1 5 Mutual surveillance 51 3 1 6 First Failure Da...

Page 7: ...in this publication at any time without notice Any references in this information to non IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk IBM may use or distribute any of the information you supply in a...

Page 8: ...erPC POWER POWER4 POWER4 POWER5 Redbooks RS 6000 Tivoli Trackpoint TotalStorage WebSphere The following terms are trademarks of other companies Java and all Java based trademarks and logos are trademarks or registered trademarks of Sun Microsystems Inc in the United States other countries or both Microsoft Windows Windows NT and the Windows logo are trademarks of Microsoft Corporation in the Unite...

Page 9: ...ing at the International Technical Support Organization Austin Center Dexter Charles is an IBM Accredited IT Specialist with seven years in the IT industry Dexter has worked four years as a pSeries Specialist with Americas Techline providing pre sales technical support for IBM Eserver p5 pSeries Linux on POWER and WebSphere sizing on pSeries platform Dexter holds a number of IBM certifications inc...

Page 10: ...a bonus you ll develop a network of contacts in IBM development labs and increase your productivity and marketability Find out more about the residency program browse the residency index and apply online at ibm com redbooks residencies html Comments welcome Your comments are important to us We want our papers to be as helpful as possible Send us your comments about this Redpaper or other Redbooks ...

Page 11: ...applications The OpenPower 710 server contains five device bays The five device bays are front accessible four bays are for hot swap capable disk drives and can accommodate up to 1 2 TB of disk storage The fifth bay is used for a slimline DVD ROM or DVD RAM Other integrated features include three 64 bit PCI X slots integrated service processor integrated 10 100 1000 Mbps two port Ethernet two serv...

Page 12: ...oved from a rack Figure 1 1 The OpenPower 710 rack mount server The OpenPower 710 server is a 2U high rack mounted server designed to be installed in a 19 inch rack There is no deskside model available Description Range Operating temperature 5 to 35 degrees Celsius 41 to 95 degrees Farenheit Relative humidity 8 to 80 percent Operating voltage 100 127 or 200 240 volts AC auto ranging Operating freq...

Page 13: ...5 inches x 1147 mm 45 2 inches if one 42U high rack is used The 14 ft length of the jumper cords between the drawer and the PDU will require additional cable management The extra length of a cord not used in its routing between its two ends will have to be accounted for inside rack space Included with the OpenPower 710 rack mounted server packaging are all of the components and instructions necess...

Page 14: ...r supports the Linux operating system OS and requires the following specific levels SUSE LINUX Enterprise Server 9 SLES 9 for POWER or later Red Hat Enterprise Linux AS for POWER Version 3 RHEL AS 3 or later 1 4 1 Processor features The OpenPower 710 server features one POWER5 chip with one or two active processor cores running at 1 65 GHz The OpenPower 710 server is available as a one way or two ...

Page 15: ...t swappable disk drive options A DVD ROM or DVD RAM drive can be installed in the slimline bay DVD ROM drive FC 1994 default DVD RAM drive FC 1993 A logical partition running a supported release of the Linux operating system requires a DVD drive to provide a method to run the hardware diagnostics from the CD Feature code Description 1965 POWER Hypervisor and Virtual I O Server Feature code Descrip...

Page 16: ...determine an appropriate service strategy See Hardware Management Console on page 33 for detailed information on the HMC Table 1 7 lists the HMC options for POWER5 processor based systems available at the time this paper was written Table 1 7 Supported HMC 1 5 Express Product Offerings New specially priced Express Product Offerings are now available for the OpenPower 710 servers These Express Prod...

Page 17: ...ifier will be printed on the packing list and on a label readable and barcode on the outside of the box Also it will appear on invoices and billing statements The SKU Identifier helps improve IBM distributors inventory management The SKU Identifier numbers for OpenPower Express configurations are listed in Table 1 8 above Express Offering Identifier Processors Memory MB Disk Config Number SKU a a ...

Page 18: ...nd bolt in steel front filler panels Support for both AC and DC configurations DC rack height is increased to 1926 mm 75 8 inches if a power distribution panel is fixed to the top of the rack Up to four Power Distribution Units PDUs can be mounted in the PDU bays see Figure 1 3 on page 10 but others can fit inside the rack See AC Power Distribution Unit and rack content on page 9 An optional rack ...

Page 19: ... can be installed horizontally in the T00 and three in the T42 rack The four vertical mounting locations will be filled first Mounting PDUs horizontally consumes 1U per PDU and reduces the space available for other racked components When mounting PDUs horizontally it is recommended that fillers be used in the EIA units occupied by these PDUs to facilitate proper air flow and ventilation in the rac...

Page 20: ... bottom Any remaining space in the rack can be used to install other systems or peripherals provided that the maximum permissible weight of the rack is not exceeded and the installation rules for these devices are followed Before placing a OpenPower 710 server into the service position it is essential that the rack manufacturer s safety instructions are followed regarding rack stability A maximum ...

Page 21: ...D ROM media only and program distribution DVD RAM drive with up to 2 7 MB sec throughput Using 3 1 compression a single disk can store up to 28 GB of data Supported DVD disk native capacities on a single DVD RAM disk are as follows Up to 2 6 GB 4 7 GB 5 2 GB and 9 4 GB Flat panel display options The IBM 7316 TF3 Flat Panel Console Kit can be installed in the system rack This 1U console uses a 15 i...

Page 22: ...racks This standard is published by the Electrical Industries Alliance and a summary of this standard is available in the publication IBM Eserver Planning SA38 0508 The key points mentioned in this documentation are as follows The front rack opening must be 451 mm wide 0 75 mm 17 75 inches 0 03 inches and the rail mounting holes must be 465 mm 0 8 mm 18 3 inches 0 03 inches apart on center horizon...

Page 23: ...n IBM rack specifications dimensions Figure 1 5 Rack specifications dimensions top front view Figure 1 6 Rack specifications dimensions bottom front view It might be necessary to supply additional hardware such as fasteners for use in some manufacturer s racks ...

Page 24: ...nd snug fit of the rail mounting pins and screws into the rack or cabinet rail support hole Note The OEM rack must only support AC powered drawers It is strongly recommended to use a power distribution unit PDU that meets the same specifications as IBM PDUs to supply rack power Rack or cabinet power distribution devices s must meet the drawer power requirements as well as that of any additional pr...

Page 25: ...5 P1 T1 SP Comm 1 P1 T6 HMC P1 T7 HMC P1 T8 Rack Indicator Light cable port P1 T5 RJ45 P1 T2 RIO 2 Bus 2x1GHz 2B GX Bus 550MHz ratio 3 1 SCSI BUS 0 P1 T11 SCSI BUS 1 P1 T10 PCI X to PCI X bridge Port 1 Port 2 Port 1 Port 2 SP Comm 2 P1 T7 Port 0 Port 1 USB 2 Port 10 100 1000 Mbps Ethernet P1 C1 P1 C2 P1 C3 P1 T3 P1 T4 P1 D1 P1 T11 L8 L0 P1 D2 P1 T11 L5 L0 P1 D3 P1 T11 L4 L0 P1 D4 P1 T11 L3 L0 Core...

Page 26: ...urces to each thread Software controlled thread prioritization Dynamic power management to reduce power consumption without affecting performance Micro Partitioning technology Virtual storage virtual Ethernet Enhanced scalability parallelism Enhanced memory subsystem Figure 2 2 on page 17 shows the high level structures of POWER5 processor based systems POWER5 processor supports a 1 9 MB on chip L...

Page 27: ...ruction level parallelism to support multiple pipelines to the processor The simultaneous multi threading mode maximizes the usage of the execution units In the POWER5 chip more rename registers have been introduced for floating point operation rename registers increased to 120 which are essential for out of order execution and then vital for simultaneous multi threading If simultaneous multi thre...

Page 28: ...5 processor gives all the physical resources to the active thread allowing it to achieve higher performance than a POWER4 processor based system at equivalent frequencies Highly optimized scientific codes are one example where a single threading operation may provide more throughput 2 1 2 Dynamic power management In current Complementary Metal Oxide Semiconductor CMOS technologies chip power is on...

Page 29: ...e L3 cache is 16 byte read 16 byte write at a bus frequency of 825 MHz which equates to 26400 MB second or 25 78 GB s The processor core contains a single processor module and the local memory storage subsystem for that processor module Figure 2 3 shows a POWER5 processor core layout view Figure 2 3 POWER5 processor core with DDR1 memory socket layout view 2 2 1 Available processor speeds At the t...

Page 30: ...oss the DIMM slots The Service Information label located on the top cover of the system provides memory DIMMs slot location information Figure 2 4 Memory placement for the OpenPower 710 server 2 3 2 Memory restriction OEM memory is not supported by IBM on the OpenPower 710 server OEM memory is not certified by IBM for the use in OpenPower servers If the OpenPower 710 server is populated with OEM m...

Page 31: ...frequency is 550 MHz with a CPU to GX ratio of 3 1 The GX bus is connected to an internal Remote I O 2 bus on the system board through a RIO 2 hub Two RIO 2 buses are available inside the system Each RIO 2 bus provides 1 byte at 1 GHz in each direction which leads to a theoretical maximum bandwidth of 4 GB s 2 5 Internal I O subsystem The internal I O subsystem and the service processor reside dir...

Page 32: ... IBM Sserver p5 or OpenPower models the OpenPower 710 server uses only one of the available SCSI busses for internal connection to the four disk drives The second SCSI bus has an external VHDCI LVD connector on the rear of the system Table 2 3 shows a list of additional SCSI adapters available at the time of writing All listed adapters can be used as boot adapters Restriction The integrated PCI X ...

Page 33: ...available serial adapters available at the time of writing Table 2 4 Available serial adapter 2 6 Internal storage There is one integrated dual channel Ultra320 SCSI controller managed by an EADS X chip that is used to drive the internal disk drives The OpenPower 710 server supports four bays that are designed for hot swappable disk drives The disk drive backplane docks directly to the system plan...

Page 34: ...y operating system actions must be undertaken to ensure that the disk is capable of being deconfigured After the disk drive has been deconfigured the SCSI enclosure device will power off the bay enabling safe removal of the disk You should ensure that the appropriate planning has been given to any operating system related disk layout Feature code Description 1994 IDE Slimline DVD ROM Drive 1993 IB...

Page 35: ...h the Gigabit Fibre Channel adapter FC 1977 Model DS4400 provides up to 64 bootable partitions In most cases both the TotalStorage DS family and the OpenPower 710 server are connected to a storage area network SAN If only space for the rootvg is needed the Model DS4100 is a good solution For support of additional features and for further information about the IBM TotalStorage DS4000 Storage Server...

Page 36: ...al partitioning and dynamic logical partitioning The POWER Hypervisor technology offers the capability to divide the OpenPower 710 server into logical partitions where each LPAR runs an operating environment on dedicated attached devices such as processors memory and I O components The HMC is required when partitioning the OpenPower 710 server The HMC is required to create and modify the partition...

Page 37: ...g two paired adapters A virtual SCSI server adapter and a virtual SCSI client adapter Virtual SCSI requires FC 1965 the appropriate level of Linux and an HMC to define the virtual SCSI devices as follows Virtual Ethernet The POWER Hypervisor provides a virtual Ethernet switch function that allows partitions on the same server a means for fast and secure communication Virtual Ethernet working on LA...

Page 38: ...ftware is not required for static or dynamic LPAR Micro Partitioning virtual Ethernet or virtual TTY The Virtual I O Server is a special purpose partition that provides virtual I O resources to client partitions The Virtual I O Server will own the real resources that will be shared with the other LPARs The Virtual I O technology allows a physical adapter assigned to a partition to be shared by one...

Page 39: ...I O Server partition The virtual SCSI client adapter allows the client partition to access standard SCSI devices and LUNs that are assigned to the client partition All current storage device types such as SAN SCSI and RAID are supported iSCSI and SSA are not supported Note To increase the performances of I O intensive applications dedicated physical adapters should be preferred in dedicated partit...

Page 40: ...th of a processor when using Micro Partitioning technology At least 128 MB of physical memory per additional partition At least one disk either physical or virtual to store the operating system At least one disk adapter either physical or virtual or integrated adapter to access the disk At least one Ethernet adapter either physical or virtual per partition to provide a network connection to the HM...

Page 41: ...alled memory The actual amount required will generally be less than 8 percent However there are some server models that require an absolute minimum amount of memory for server firmware regardless of the previously mentioned considerations The minimum amount of physical memory for each partition is 128 MB but in most cases the actual requirements and recommendations are between 256 MB and 512 MB fo...

Page 42: ...e same LPAR Therefore for additional LPARs without using virtualization external disk space is necessary which can be accomplished by using external disk subsystems The external disk space must be attached with a separate adapter for each LPAR by using SCSI or Fibre Channel adapters depending on the subsystem For additional LPARs using virtualization the required disk drives for each partition are...

Page 43: ...ts The 7310 Model CR3 is a 1U 19 inch rack mountable drawer that has two native 10 100 1000 Mbps Ethernet ports and two additional PCI slots The HMC connects to the OpenPower 710 server using a standard 10 100 Ethernet connection and uses HMC Port 1 A second HMC can be attached to HMC Port 2 of the OpenPower 710 server for redundancy For more details of the possible HMC network connections refer t...

Page 44: ...artition design by selecting PCI slots for given adapters specific drives to selected bays and much more The LVT provides a useful report that can complement the organization and validation of features required for configuration of a complex partition solution The LVT supports IBM Eserver p5 and IBM Eserver i5 servers iSeries and OpenPower systems A proficient knowledge of LPAR design requirements...

Page 45: ...ent and eConfig The LVT also provides the output report that is used for the Customer Specified Placement CSP offering The LVT output is uploaded on the CSP site for submission to manufacturing The CSP offering enables the placement of adapters and disks for an exact built to order system based on the client s specifications Manufacturing uses the LVT output to custom build the server The server w...

Page 46: ...ower 710 server requires the following version of Linux distributions SUSE LINUX Enterprise Server 9 for POWER or later Red Hat Enterprise Linux AS for POWER Version 3 or later Information on features and external devices supported by Linux on the OpenPower 710 server can be found at http www ibm com servers eserver pseries linux Note The order submitted must include the CSP feature code 8453 on t...

Page 47: ...le server when compared to the subscription price for each server individually These offerings are available only for p5 510 and p5 510 Express p5 520 and p5 520 Express and OpenPower 710 server SLES 9 clusters require a head node which must be one of the supported servers for the offering for administration Cluster software RHEL AS 3 options for clustered servers The RHEL AS Version 3 and RHEL AS...

Page 48: ...Power 710 server Blue and Terra cotta When both colors are present terra cotta takes precedence and the rules for a terra cotta only touch point apply 2 9 2 Securing system into racks The rack mount drawer rail kit is a unique kit designed for use with the OpenPower 710 server No tools are required to install the OpenPower 710 server or drawer rails into the rack The kit has a modular design that ...

Page 49: ... processor provides an interface to the control panel which is used to display system status and diagnostic information The OpenPower 710 server has a control panel that is a re packaging of existing control panels so that it fits into a smaller space In its normal position the control panel is seated inside the chassis on the left of the DVD optical device if you stand in front of the server The ...

Page 50: ...ntrol panel functions The primary control panel functions are defined as functions 01 to 20 including options to view and manipulate IPL modes system operating modes IPL speed and IPL type Here is a list of all available primary functions Function 01 Display selected IPL type system operating mode and IPL speed Function 02 Select IPL type IPL speed override and system operating mode Function 03 St...

Page 51: ...urrent OS IPL type is displayed with a pointer The current system operating mode is displayed The current firmware mode is displayed 3 Use the Increment or Decrement buttons to scroll through the OS IPL types until the firmware mode is displayed with a pointer 0 2 _ _ A _ _ N _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ P _ _ 4 Press Enter to select the system operating mode 0 2 _ _ A _ _ M _ _ _ _ _ _ _ _...

Page 52: ...s correctly you can permanently install the server firmware fix When you permanently install a server firmware fix you copy the temporary firmware level from the temporary side to the permanent side Conversely if you decide that you do not want to keep the new level of server firmware you can remove the current level of firmware When you remove the current level of firmware you copy the firmware l...

Page 53: ...s you can use the interface on the HMC to install the firmware fix from one of the repository locations or from the optical media The Change Internal Code wizard on the HMC provides a step by step process for you to perform the required steps to install the fix 1 Ensure that you have a connection to the service provider if you have an Internet connection from the HMC or server 2 Determine the avai...

Page 54: ...processor The service processor is an embedded controller based on a PowerPC 405GP processor PPC405 implementation running the service processor internal operating system The service processor operating system contains specific programs and device drivers for the service processor hardware The key components include a flexible service processor base FSP B and an extender chipset FSP E Figure 2 12 ...

Page 55: ...e server such as the error log and vital product data This interface is accessible using a Web browser on a client system that is connected to the service processor on an Ethernet network It can also be accessed using a terminal attached to a serial port on the server The service processor and the ASMI are standard on all IBM Sserver i5 Sserver p5 and OpenPower servers You may be able to use the s...

Page 56: ...Focal Point 4 In the content area click Service Utilities 5 From the Service Utilities window select the managed system you want to work with 6 From the Selected menu on the Service Utilities window select Launch ASM menu For more detailed information about usage of ASMI please refer to the IBM Sserver Hardware Information Center http publib boulder ibm com infocenter eserver v1r2s en_US index htm...

Page 57: ...st management tasks using the Web based System Manager Remote Client The remote HMC and the Web based System Manager Remote Client allow you the flexibility to access your managed systems including HMCs from multiple locations using multiple HMCs For more detailed information about usage of the HMC please refer to the IBM Sserver Hardware Information Center http publib boulder ibm com infocenter e...

Page 58: ...fined in partition profiles The partition manager manages the operational partitioning transactions It responds to commands from the service processor external command interface that originate in the application that is running on the HMC The ASMI can be accessed during boot time Use the ASMI to boot to an Open Firmware prompt For more information on Open Firmware refer to Partitioning Implementat...

Page 59: ... All rights reserved 49 Chapter 3 Reliability availability and serviceability The following chapter provides more detailed information on IBM Sserver OpenPower 710 server reliability availability and serviceability features 3 ...

Page 60: ...y for memory reliability fault tolerance and integrity Memory has single error correct and double error detect ECC circuitry designed to correct single bit memory failures The double bit detection is designed to help maintain data integrity by detecting and reporting multiple errors beyond what the ECC circuitry can correct The memory chips are organized such that the failure of any specific memor...

Page 61: ...etailed information on the service processor refer to 2 9 5 Service processor on page 44 3 1 4 Fault monitoring functions The following are a few of the fault monitoring systems included with an OpenPower 710 server BIST and power on self test POST check the processor L3 cache memory and associated hardware required for proper booting of the operating system every time the system is powered on If ...

Page 62: ...monitoring increases the fan speed rotation when ambient temperature is above the normal operating range Temperature monitoring warns the system administrator of potential environmental related problems for example air conditioning and air circulation around the system so that appropriate corrective actions can be taken before a critical failure threshold is reached It also performs an orderly sys...

Page 63: ...e The System Management Services SMS error log is accessible from the SMS menu for tests performed through SMS programs For results of service processor tests access the error log from the service processor menu 3 2 1 Service Agent The Service Agent is available at no additional charge When installed on an IBM Sserver system the Service Agent can enhance IBM s ability to provide the system with ma...

Page 64: ...llowing URL http www beowulf org Myricom has the Myrinet switch available for Linux on IBM Eserver p5 pSeries and OpenPower systems The Linux distributions that support the respective technologies also support the Myrinet switch specifically SLES 8 systems SLES 9 systems and RHEL AS 3 systems It can be used as a high speed interconnect to cluster systems of IBM Eserver p5 pSeries and OpenPower mac...

Page 65: ...pSeries Clusters SG24 7014 Understanding IBM Sserver pSeries Performance and Sizing SG24 4810 Advance POWER Virtualization on IBM Sserver p5 Servers SG24 7940 IBM Sserver OpenPower 720 Technical Overview and Introduction REDP 1965 01 Online resources These Web sites and URLs are also relevant as further information sources IBM Sserver Hardware Information Center http publib16 boulder ibm com infoc...

Page 66: ...test in IBM Linux news subscribe to the Linux Line See https www14 software ibm com webapp iwm web preLogin do source linuxline Many of the features described in this document are operating system dependant and may not be available on Linux For more information see http www ibm com servers eserver linux power whitepapers linux_overview LPAR validation tool http www ibm com servers eserver iseries ...

Page 67: ...Related publications 57 Help from IBM IBM Support and downloads ibm com support IBM Global Services ibm com services ...

Page 68: ...58 IBM eServer OpenPower 710 Technical Overview and Introduction ...

Page 69: ......

Page 70: ...s to integrated cluster solutions This document is a comprehensive guide covering the IBM Sserver OpenPower 710 server We introduce major hardware offerings and discuss their prominent functions Professionals wishing to acquire a better understanding of IBM Sserver OpenPower products should consider reading this document The intended audience includes Clients Sales and marketing professionals Tech...