Fujitsu SPARC Enterprise T5440 Server, Service Manual

Page 1: ......

Page 2: ......

Page 3: ...SPARC EnterpriseTM T5440 Server Service Manual Part No 875 4392 11 July 2009 Revision A Manual Code C120 E512 02EN ...

Page 4: ...demarks of Fujitsu Limited All SPARC trademarks are used under license and are registered trademarks of SPARC International Inc in the U S and other countries Products bearing SPARC trademarks are based upon architecture developed by Sun Microsystems Inc SPARC64 is a trademark of SPARC International Inc used under license by Fujitsu Microelectronics Inc and Fujitsu Limited SSH is a registered trad...

Page 5: ...ce et sont des marques de fabrique ou des marques déposées de SPARC International Inc aux Etats Unis et dans d autres pays Les produits portant les marques SPARC sont basés sur une architecture développée par Sun Microsystems Inc SPARC64 est une marques déposée de SPARC International Inc utilisée sous le permis par Fujitsu Microelectronics Inc et Fujitsu Limited SSH est une marque déposée registre...

Page 6: ......

Page 7: ...ts 9 Understanding Fault Handling Options 9 Server Diagnostics Overview 9 Diagnostic Flowchart 11 Options for Accessing the Service Processor 14 ILOM Overview 15 ALOM CMT Compatibility Shell Overview 17 Solaris Predictive Self Healing Overview 17 SunVTS Overview 18 POST Fault Management Overview 19 POST Fault Management Flowchart 20 Memory Fault Handling Overview 21 Connecting to the Service Proce...

Page 8: ...g How POST Runs 26 Change POST Parameters 27 Run POST in Maximum Mode 28 Detecting Faults 30 Detecting Faults Using LEDs 30 Detecting Faults Using ILOM show faulty Command 32 Detect Faults Using the ILOM show faulty Command 33 Detecting Faults Using Solaris OS Files and Commands 35 Check the Message Buffer 36 View System Message Log Files 36 Detecting Faults Using the ILOM Event Log 37 View ILOM E...

Page 9: ...o Service the System 59 Safety Information 59 Safety Symbols 60 Electrostatic Discharge Safety Measures 61 Antistatic Wrist Strap 61 Antistatic Mat 61 Required Tools 62 Obtain the Chassis Serial Number 62 Obtain the Chassis Serial Number Remotely 62 Powering Off the System 63 Power Off From the Command Line 64 Power Off Graceful Shutdown 64 Power Off Emergency Shutdown 65 Disconnect Power Cords Fr...

Page 10: ... Hard Drive Hot Plug 73 Install a Hard Drive Hot Plug 75 Remove a Hard Drive 77 Install a Hard Drive 78 Hard Drive Device Identifiers 79 Hard Drive LEDs 80 Servicing Fan Trays 81 Remove a Fan Tray Hot Swap 81 Install a Fan Tray Hot Swap 82 Remove a Fan Tray 83 Install a Fan Tray 84 Fan Tray Device Identifiers 84 Fan Tray Fault LED 84 Servicing Power Supplies 85 Remove a Power Supply Hot Swap 86 In...

Page 11: ...ory Module Configurations 104 Servicing FB DIMMs 104 Remove FB DIMMs 105 Install FB DIMMs 105 Verify FB DIMM Replacement 106 Add FB DIMMs 109 Supported FB DIMM Configurations 110 FB DIMM Device Identifiers 112 FB DIMM Fault Button Locations 113 Servicing Field Replaceable Units 115 Servicing the Front Bezel 115 Remove the Front Bezel 116 Install the Front Bezel 117 Servicing the DVD ROM Drive 118 ...

Page 12: ...ge 129 Remove the Fan Tray Carriage 129 Install the Fan Tray Carriage 131 Servicing the Hard Drive Backplane 132 Remove the Hard Drive Backplane 132 Install the Hard Drive Backplane 133 Servicing the Motherboard 135 Remove the Motherboard 135 Install the Motherboard 138 Motherboard Fastener Locations 139 Servicing the Flex Cable Assembly 140 Remove the Flex Cable Assembly 141 Install the Flex Cabl...

Page 13: ... Module 157 Reconfiguring I O Device Nodes 158 Reconfigure the I O and PCIe Fabric 158 Temporarily Disable All Memory Modules 160 Re Enable All Memory Modules 161 Reset the LDoms Guest Configuration 162 System Bus Topology 162 I O Fabric in 2P Configuration 164 I O Fabric in 4P Configuration 165 Connector Pinouts 167 Serial Management Port Connector Pinouts 167 Network Management Port Connector Pi...

Page 14: ...xii SPARC Enterprise T5440 Server Service Manual July 2009 Field Replaceable Units 176 Index 179 ...

Page 15: ...ation regarding the use and handling of this product Read this manual thoroughly Pay special attention to the section Notes on Safety on page xix Use the product according to the instructions and information available in this manual Keep this manual handy for further reference Keep this manual handy for further reference Fujitsu makes every effort to prevent users and bystanders from being injured...

Page 16: ...9 Describes the steps necessary to prepare the server for service Servicing Customer Replaceable Units on page 71 Describes how to service customer replaceable units CRUs Servicing Field Replaceable Units on page 115 Describes how to service field replaceable units FRUs Returning the Server to Operation on page 149 Describes how to bring the server back to operation after performing service proced...

Page 17: ...SPARC Enterprise PRIMEQUEST Common Installation Planning Manual Requirements and concepts of installation and facility planning for the setup of SPARC Enterprise and PRIMEQUEST C120 H007 SPARC Enterprise T5440 Server Site Planning Guide Server specifications for site planning C120 H029 SPARC Enterprise T5440 Server Installation and Setup Guide Detailed rackmounting cabling power on and configuring...

Page 18: ...es Guide Information and procedures for accessing ILOM 3 0 functions using the ILOM CLI C120 E575 Integrated Lights Out Manager 3 0 SNMP and IPMI Procedure Guide Information and procedures for accessing ILOM 3 0 functions using SNMP or IPMI management hosts C120 E579 Integrated Lights Out Manager 3 x Feature Updates and Release Notes Enhancements that have been made to ILOM firmware since the ILOM...

Page 19: ...com Text Conventions The settings on your browser might differ from these settings Typeface Meaning Examples AaBbCc123 The names of commands files and directories on screen computer output Edit your login file Use ls a to list all files You have mail AaBbCc123 What you type when contrasted with on screen computer output su Password AaBbCc123 Book titles new words or terms words to be emphasized Re...

Page 20: ...ential hazard if the user does not perform the procedure correctly Caution This indicates a hazardous situation that could result in minor or moderate personal injury if the user does not perform the procedure correctly This signal also indicates that damage to the product or other property may occur if the user does not perform the procedure correctly Caution This indicates that surfaces are hot ...

Page 21: ...garding this product and the optional products provided from Fujitsu should only be performed by a certified service engineer Users must not perform these tasks Incorrect operation of these tasks may cause malfunction Also important alert messages are shown in Important Alert Messages on page xix Notes on Safety Important Alert Messages This manual provides the following important alert signals Ca...

Page 22: ...de rails can be enough to overturn an equipment rack Before you begin deploy the antitilt feature on your cabinet The server weighs approximately 88 lb 40 kg Two people are required to lift and mount the server into a rack enclosure when using the procedures in this chapter Task Warning Maintenance Electric shock Never attempt to run the server with the covers removed Hazardous voltage present Bec...

Page 23: ...d inspections repairing and regular diagnosis and maintenance Caution The following tasks regarding this product and the optional products provided from Fujitsu should only be performed by a certified service engineer Users must not perform these tasks Incorrect operation of these tasks may cause malfunction Unpacking optional adapters and such packages delivered to the users Plugging or unpluggin...

Page 24: ...ARC Enterprise T5440 Server Service Manual July 2009 Alert Label The following is a label attached to this product Never peel off the label The following label provides information to the users of this product ...

Page 25: ...ument or if you find any unclear statements in the document please state your points specifically on the form at the following URL For Users in U S A Canada and Mexico https download computers us fujitsu com For Users in Other Countries http www fujitsu com global contact computing sparce_index ht ml ...

Page 26: ...xxiv SPARC Enterprise T5440 Server Service Manual July 2009 ...

Page 27: ...odules memory control subsystem up to eight PCIe expansion slots and a service processor slot The motherboard also contains a top cover safety interlock kill switch Note 10 Gbit Ethernet XAUI cards are shared in Slots 4 and 5 CMP module Each CMP module contains an UltraSPARC T2 Plus chip slots for four FB DIMMs and associated DC DC converters Description Links Overview of the infrastructure boards...

Page 28: ...rboard and the disk drive backplane via a flex cable High voltage power is provided to the motherboard via a bus bar assembly Hard drive backplane This board includes the connectors for up to four hard drives It is connected to the motherboard via a flex cable assembly Each drive has its own Power Activity Fault and Ready to Remove LEDs Front control panel This board connects directly to the mothe...

Page 29: ...ipped and the two front USB ports FIGURE Front Panel Features on page 3 shows front panel features on the SPARC Enterprise T5440 server For a detailed description of front panel controls and LEDs see Front Panel LEDs on page 4 FIGURE Front Panel Features Related Information Front Panel LEDs on page 4 Figure Legend 1 Locator Button LED 5 Component Fault LEDs 2 Service Required LED 6 DVD ROM Drive 3...

Page 30: ...ired LED amber If on indicates that service is required POST and ILOM are two diagnostics tools that can detect a fault or failure resulting in this indication The ILOM show faulty command provides details about any faults that cause this indicator to light Under some fault conditions individual component fault LEDs are lit in addition to the system Service Required LED Power OK LED green Provides...

Page 31: ...d hold for 4 seconds to initiate an emergency shutdown For more information about powering on and powering off the system see the SPARC Enterprise T5440 Server Administration Guide Fan Fault LED amber TOP FAN Provides the following operational fan indications Off Indicates a steady state no service action is required Steady on Indicates that a fan failure event has been acknowledged and a service ...

Page 32: ...d Information Front Panel Diagram on page 3 Rear Panel LEDs on page 7 Ethernet Port LEDs on page 8 Detecting Faults Using LEDs on page 30 Figure Legend 1 Power supplies 2 Serial port 3 Serial management port 4 System status LEDs 5 USB ports 6 Network management port 7 Gigabit Ethernet ports ...

Page 33: ... amber If on indicates that service is required POST and ILOM are two diagnostics tools that can detect a fault or failure resulting in this indication The ILOM show faulty command provides details about any faults that cause this indicator to light Under some fault conditions individual component fault LEDs are lit in addition to the system Service Required LED Power OK LED green Provides the fol...

Page 34: ...sing LEDs on page 30 TABLE Ethernet Port LEDs LED Color Description Left LED Amber or green Speed indicator Amber on The link is operating as a Gigabit connection 1000 Mbps Green on The link is operating as a 100 Mbps connection Off The link is operating as a 10 Mbps connection The NET MGT port only operates in 100 Mbps or 10 Mbps so the speed indicator LED will be green or off never amber Right L...

Page 35: ...on methodology Understanding Fault Handling Options on page 9 Configuring and using the service processor Connecting to the Service Processor on page 22 Displaying system configuration information with the service processor Displaying FRU Information with ILOM on page 24 Configuring POST for diagnostic purposes Controlling How POST Runs on page 26 Detecting system faults Detecting Faults on page 3...

Page 36: ...ris OS log files and ILOM system event log can be accessed and displayed on the device of your choice SunVTS software The SunVTS software exercises the system provides hardware validation and discloses possible faulty components with recommendations for repair The LEDs ILOM Solaris OS PSH and many of the log files and console messages are integrated For example a fault detected by the Solaris soft...

Page 37: ...gnostic Flowchart on page 11 is a flowchart of the diagnostics available to troubleshoot faulty hardware TABLE ILOM Parameters Used for POST Configuration on page 26 has more information about each diagnostic in this chapter FIGURE Diagnostic Flowchart ...

Page 38: ...faulty output includes an error string such as Ext sensor or Ext FRU it indicates a fault in the External I O Expansion Unit Detect Faults Using the ILOM show faulty Command on page 33 3 Check the Solaris log files and ILOM system event log for fault information The Solaris log files and the ILOM system event log record system events and provide information about faults Browse the ILOM system even...

Page 39: ... the fault will automatically clear If the fault indicates that a fan or power supply is bad you can perform a hot swap of the FRU You can also use the fault LEDs on the server to identify the faulty FRU fans and power supplies If the FRU displayed by the show faulty command is SYS the fault is a configuration problem SYS indicates no faulty FRU has been diagnosed but there is a problem with the s...

Page 40: ...ID where message ID is the value of the sunw msg id property displayed by the show faulty command After the FRU is replaced perform the procedure to clear PSH detected faults Identifying Faults Detected by PSH on page 44 Clear Faults Detected by PSH on page 49 9 Determine if the fault was detected by POST POST performs basic tests of the server components and reports faulty FRUs When POST detects ...

Page 41: ...t would otherwise require physical proximity to the server s serial port You can also configure ILOM to send email alerts of hardware failures hardware warnings and other events related to the server or to ILOM The service processor runs independently of the server using the server s standby power Therefore ILOM firmware and software continue to function when the server OS goes offline or when the...

Page 42: ...while the service processor is powered off for example if the system power cables are unplugged during service procedures This function enables ILOM to know that a fault diagnosed to a specific FRU has been repaired Note ILOM does not automatically detect hard drive replacement Many environmental faults can automatically recover A temperature that is exceeding a threshold might return to normal li...

Page 43: ... sends the alert through email to a configured email address and writes the event to the ILOM event log The ILOM event log is also available using the ALOM CMT compatibility shell See the Integrated Lights Out Manager 3 0 Supplement for SPARC Enterprise T5440 Server for comparisons between the ILOM CLI and the ALOM CMT compatibility CLI and for instructions for adding an ALOM CMT account Related I...

Page 44: ...can use the message ID to get additional information about the problem from the knowledge article database The Predictive Self Healing technology covers the following server components UltraSPARC T2 Plus multicore processor Memory I O subsystem The PSH console message provides the following information about each detected fault Type Severity Description Automated response Impact Suggested action f...

Page 45: ...the system boots and accesses software If POST detects a faulty component the component is disabled automatically preventing faulty hardware from potentially harming any software If the system is capable of running without the disabled component the system will boot when POST is complete For example if one of the processor cores is deemed faulty by POST the core will be disabled The system will bo...

Page 46: ...20 SPARC Enterprise T5440 Server Service Manual July 2009 POST Fault Management Flowchart FIGURE Flowchart of Variables for POST Configuration Related Information Diagnostic Flowchart on page 11 ...

Page 47: ...rrectable memory fault is detected POST displays the fault with the device name of the faulty FB DIMMs and logs the fault POST then disables the faulty FB DIMMs Depending on the memory configuration and the location of the faulty FB DIMM POST disables half of physical memory in the system or half the physical memory and half the processor threads When this offlining process occurs in normal operat...

Page 48: ...cessor Before you can run ILOM commands you must connect to the service processor There are several ways to connect to the service processor Related Information Diagnostic Flowchart on page 11 Topic Links Connect an ASCII terminal directly to the serial management port SPARC Enterprise T5440 Server Installation and Setup Guide Use the ssh command to connect to service processor through an Ethernet...

Page 49: ...rver Installation and Setup Guide SPARC Enterprise T5440 Server Administration Guide Switch From the System Console to the Service Processor ILOM or ALOM CMT Compatibility Shell To switch from the system console to the service processor prompt type Hash Period Switch From ILOM to the System Console From the ILOM prompt type start SP console Switch From the ALOM CMT Compatibility Shell to the Syste...

Page 50: ...bled CODE EXAMPLE Output of the show components Command With No Disabled Components show components Target Property Value SYS MB PCIE0 component_state Enabled SYS MB PCIE3 component_state Enabled SYS MB PCIE1 component_state Enabled SYS MB PCIE4 component_state Enabled SYS MB PCIE2 component_state Enabled SYS MB PCIE5 component_state Enabled SYS MB NET0 component_state Enabled SYS MB NET1 componen...

Page 51: ...MB NET1 component_state Enabled SYS MB NET2 component_state Enabled SYS MB NET3 component_state Enabled SYS MB PCIE component_state Enabled CODE EXAMPLE show Command Output show SYS MB CPU0 CMP0 BR1 CH0 D0 SYS MB CPU0 CMP0 BR1 CH0 D0 Targets R0 R1 SEEPROM SERVICE PRSNT T_AMB Properties type DIMM component_state Enabled fru_name 1024MB DDR2 SDRAM FB DIMM 333 PC2 5300 fru_description FBDIMM 1024 Mby...

Page 52: ...nds cd show TABLE ILOM Parameters Used for POST Configuration Parameter Values Description keyswitch_mode normal The system can power on and run POST based on the other parameter settings For details see FIGURE Flowchart of Variables for POST Configuration on page 20 This parameter overrides all other commands diag The system runs POST based on predetermined settings stby The system cannot power o...

Page 53: ...cessor on page 22 diag_trigger none Does not run POST on reset user_reset Runs POST upon user initiated resets power_on_reset Only runs POST for the first power on This option is the default error_reset Runs POST if fatal errors are detected all_resets Runs POST after any reset diag_verbosity none No POST output is displayed min POST output displays functional tests with a banner and pinwheel norm...

Page 54: ...r repair 1 Access the ILOM prompt See Connecting to the Service Processor on page 22 2 Set the virtual keyswitch to diag so that POST will run in service mode 3 Reset the system so that POST runs There are several ways to initiate a reset CODE EXAMPLE Initiating POST With a Power Cycle on page 29 shows a reset using a power cycle command sequence For other methods refer to the SPARC Enterprise T54...

Page 55: ...eptune 1G Loopback Test Port 2 2007 12 19 22 01 22 553 0 0 0 2007 12 19 22 01 22 542 0 0 0 Begin Neptune 1G Loopback Test Port 3 2007 12 19 22 01 22 556 0 0 0 INFO STATUS Running BMAC level Loopback Test 2007 12 19 22 01 32 004 0 0 0 End Neptune 1G Loopback Test Port 3 2007 12 19 22 01 27 271 0 0 0 T5440 No Keyboard Enter to return to ALOM 2007 12 19 22 01 32 012 0 0 0 INFO 2007 12 19 22 01 27 274...

Page 56: ...memory module LEDs See Servicing CMP Memory Modules on page 98 FB DIMM Fault LEDs See FB DIMM Fault Button Locations on page 113 These LEDs provide a quick visual check of the state of the system Task Topic Use front panel and back panel LEDs to identify system faults Detecting Faults Using LEDs on page 30 Use the ILOM show faulty command to detect faults Detecting Faults Using ILOM show faulty Co...

Page 57: ...ply Fault LED Individual power supply Fault LED Front Panel LEDs on page 4 Rear Panel LEDs on page 7 Power Supply LED on page 91 Servicing Power Supplies on page 85 Fan tray Service Required LED front and rear panel Front panel Fan Fault LED Individual fan tray Fault LED Overtemp LED if overtemp condition exists Front Panel LEDs on page 4 Rear Panel LEDs on page 7 Fan Tray Fault LED on page 84 Ser...

Page 58: ...module or a problem with the CMP module itself See these sections Front Panel LEDs on page 4 Rear Panel LEDs on page 7 Servicing CMP Memory Modules on page 98 Servicing FB DIMMs on page 104 FB DIMM Service Required LED front and rear panel CMP Module Fault LED or Memory Module Fault LED FB DIMM Fault LED CMP and memory modules when FB DIMM Locate button is pressed See these sections Front Panel LE...

Page 59: ...ny faults have been diagnosed in the system To verify that the replacement of a FRU has cleared the fault and not generated any additional faults Related Information Diagnostic Flowchart on page 11 Detecting Faults Using LEDs on page 30 ILOM to ALOM CMT Command Reference on page 53 SPARC Enterprise T5440 Server Installation and Setup Guide SPARC Enterprise T5440 Server Administration Guide Integra...

Page 60: ...Property Value SP faultmgmt 0 fru SYS MB FT1 SP faultmgmt 0 timestamp Dec 14 23 01 32 SP faultmgmt 0 timestamp Dec 14 23 01 32 faults 0 SP faultmgmt 0 sp_detected_fault TACH at SYS MB FT1 has faults 0 exceeded low non recoverable threshold show faulty Target Property Value SP faultmgmt 0 fru SYS SP faultmgmt 0 timestamp Mar 17 08 17 45 SP faultmgmt 0 timestamp Mar 17 08 17 45 faults 0 SP faultmgmt...

Page 61: ...plement of Solaris OS files and commands available for collecting information and for troubleshooting If POST ILOM or the Solaris PSH features do not indicate the source of a fault check the message buffer and log files for notifications for faults Hard drive faults are usually captured by the Solaris message files Use the dmesg command to view the most recent system message To view the system mes...

Page 62: ... Files The error logging daemon syslogd automatically records various system warnings errors and faults in message files These messages can alert you to system problems such as a device that is about to fail The var adm directory contains several message files The most recent messages are in the var adm messages file After a period of time usually every week a new messages file is automatically cr...

Page 63: ...Enterprise T5440 Server View ILOM Event Log Type the following command Note The ILOM event log can also be viewed through the ILOM BUI or the ALOM CMT CLI If a major or critical event is found that was not expected and not included under ILOM show faulty than it may indicate a system fault The following is an example of unexpected major events in the log show SP logs event list show sp logs event ...

Page 64: ...owchart on page 11 Verify Installation of SunVTS Software on page 38 Start the SunVTS Browser Environment on page 39 SunVTS Software Packages on page 41 Useful SunVTS Tests on page 42 SPARC Enterprise T5440 Server Administration Guide SunVTS 7 0 User s Guide Verify Installation of SunVTS Software To perform this procedure the Solaris OS must be running on the server and you must have access to the...

Page 65: ...r a list of required SunVTS software packages 2 If the SunVTS software is not installed you can obtain the installation packages from the following places Solaris Operating System DVDs Download from the web Refer to the Preface for information on how to access the web site Start the SunVTS Browser Environment For information about test options and prerequisites refer to the SunVTS 7 0 User s Guide...

Page 66: ...t Group Screen 4 Optional Select the test categories you want to run Certain test categories are enabled by default You can choose to accept these Note TABLE Useful SunVTS Tests on page 42 lists test categories that are especially useful to run on this server 5 Optional Customize individual tests Click on the name of the test to select and customize individual tests Tip Use the System Excerciser H...

Page 67: ... test messages area Solaris OS Messages var adm messages A file containing messages generated by the operating system and various applications Test Messages var sunvts logs sunvts info A directory containing the SunVTS log files SunVTS Software Packages TABLE SunVTS Software Packages on page 41 lists SunVTS packages Related Information Diagnostic Flowchart on page 11 Useful SunVTS Tests on page 42...

Page 68: ...ode on page 28 POST error messages use the following syntax c s ERROR TEST failing test c s H W under test FRU c s Repair Instructions Replace items in order listed by H W under test above c s MSG test error message c s END_ERROR In this syntax c the core number s the strand number Warning and informational messages use the following syntax TABLE Useful SunVTS Tests SunVTS Tests FRUs Exercised by ...

Page 69: ...information In this example SYS MB CPU0 CMP0 BR1 CH0 D0 is disabled The system can boot using memory that was not disabled until the faulty component is replaced Note You can use ASR commands to display and control disabled components See Disabling Faulty Components on page 50 Related Information Diagnostic Flowchart on page 11 CODE EXAMPLE POST Error Message 7 2 7 2 ERROR TEST Data Bitwalk 7 2 H ...

Page 70: ...ge Showing Fault Detected by PSH on page 44 The ILOM show faulty command provides summary information about the fault See Detect Faults Using the ILOM show faulty Command on page 33 for more information about the show faulty command CODE EXAMPLE Console Message Showing Fault Detected by PSH SUNW MSG ID SUN4V 8000 DX TYPE Fault VER 1 SEVERITY Minor EVENT TIME Wed Sep 14 10 09 46 EDT 2005 PLATFORM S...

Page 71: ...put of fmdump is the same after the FRU has been replaced Use the fmadm faulty command to verify that the fault has cleared See Clear Faults Detected by PSH on page 49 1 Check the event log using the fmdump command with v for verbose output In CODE EXAMPLE Output from the fmdump v Command on page 46 a fault is displayed indicating the following details Date and time of the fault Jul 31 12 47 42 20...

Page 72: ...action 3 Follow the suggested actions to repair the fault CODE EXAMPLE Output from the fmdump v Command fmdump v u fd940ac2 d21e c94a f258 f8a9bb69d05b TIME UUID SUNW MSG ID Jul 31 12 47 42 2007 fd940ac2 d21e c94a f258 f8a9bb69d05b SUN4V 8000 JA 100 fault cpu ultraSPARC T2 misc_regs Problem in cpu cpuid 16 serial 5D67334847 Affects cpu cpuid 16 serial 5D67334847 FRU hc serial 101083 part 541215101...

Page 73: ...PARC Enterprise T5120 T5140 T5220 T5240 T5440 Servers Schedule a repair procedure to replace the affected CPU the identity of which can be determined using fmdump v u EVENT_ID Details The Message ID SUN4V 8000 JA indicates diagnosis has determined that a CPU is faulty The Solaris fault manager arranged an automated attempt to disable this CPU Task Topic Clear faults detected during POST Clear Faul...

Page 74: ...inguished from other kinds of faults by the text Forced fail No UUID number is reported Refer to CODE EXAMPLE Fault Detected by POST on page 48 If no fault is reported you do not need to do anything else Do not perform the subsequent steps 2 Use the component_state property of the component to clear the fault and remove the component from the ASR blacklist Use the FRU name that was reported in the...

Page 75: ...ted perform Step 3 and Step 4 3 Use the clear_fault_action property of the FRU to clear the fault from the service processor For example 4 Clear the fault from all persistent fault records In some cases even though the fault is cleared some persistent fault information remains and results in erroneous fault messages at boot time To ensure that these messages are not displayed perform the following...

Page 76: ...tion command to clear a fault in the External I O Expansion Unit Disabling Faulty Components You can use the Automatic System Recovery ASR feature to configure the server to automatically disable failed components until they can be replaced The following components are managed by the ASR feature UltraSPARC T2 Plus processor strands Memory FB DIMMs show faulty Target Property Value SP faultmgmt 0 f...

Page 77: ...rompt Note The asrkeys vary from system to system depending on how many cores and memory are present Use the show components command to see the asrkeys on a given system Note A reset or power cycle is required after disabling or enabling a component If the status of a component is changed there is no effect to the system until the next reset or power cycle Related Information Diagnostic Flowchart ...

Page 78: ...o determine if the host has powered off Re Enable System Components The component_state property enables a component by removing it from the ASR blacklist 1 At the prompt set the component_state property to Enabled 2 Reset the server so that the ASR command takes effect set SYS MB CPU0 CMP0 BR1 CH0 D0 component_state Disabled stop SYS Are you sure you want to stop SYS y n y Stopping SYS start SYS ...

Page 79: ...mands ILOM Command ALOM CMT Command Description help command help command Displays a list of all available commands with syntax and descriptions Specifying a command name as an option displays help for that command set HOST send_break_action true break y c D y skips the confirmation question c executes a console command after the break command completes D forces a core dump of the Solaris OS Takes...

Page 80: ... value normal reset_nvram bootscript string Enables control of the firmware during system initialization with the following options normal is the default boot mode reset_nvram resets OpenBoot PROM parameters to their default values bootscript string enables the passing of a string to the boot command stop SYS start SYS powercycle f The f option forces an immediate poweroff Otherwise the command at...

Page 81: ... set SYS keyswitch_state value normal stby diag locked setkeyswitch y value normal stby diag locked y enables you to skip the confirmation question when setting the keyswitch to stby Sets the virtual keyswitch set SUS LOCATE value value Fast_blink Off setlocator value on off Turns the Locator LED on the server on or off No ILOM equivalent showenvironment Displays the environmental status of the ho...

Page 82: ...ate showkeyswitch Displays the status of the virtual keyswitch show SYS LOCATE showlocator Displays the current state of the Locator LED as either on or off show SP logs event list showlogs b lines e lines v g lines p logtype r p Displays the history of all events logged in the service processor event buffers in RAM or the persistent buffers show SYS showplatform v Displays information about the o...

Page 83: ... max max Description of POST execution This is the default POST configuration This configuration tests the system thoroughly and suppresses some of the detailed POST output POST does not run resulting in quick system initialization This is not a suggested configuration POST runs the full spectrum of tests with the maximum output displayed POST runs the full spectrum of tests with the maximum outpu...

Page 84: ...58 SPARC Enterprise T5440 Server Service Manual July 2009 ...

Page 85: ...installing parts in the SPARC Enterprise T5440 server Topic Links Observe proper safety practices Safety Information on page 59 Gather the tools needed to perform service procedures Required Tools on page 62 Obtain the chassis serial number Obtain the Chassis Serial Number on page 62 Power off the system Powering Off the System on page 63 Slide the server out of the equipment rack Extending the Se...

Page 86: ...llow the electrostatic discharge safety practices as described in this section Related Information Safety Symbols on page 60 Antistatic Wrist Strap on page 61 Antistatic Mat on page 61 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Safety Symbols Note the meanings of the following symbols that might appear in this document Caution There is a risk of personal injury or eq...

Page 87: ...s documented in this chapter Related Information Safety Information on page 59 Antistatic Wrist Strap on page 61 Antistatic Mat on page 61 Antistatic Wrist Strap Wear an antistatic wrist strap and use an antistatic mat when handling components such as hard drive assemblies circuit boards or PCI cards When servicing or removing server components attach an antistatic strap to your wrist and then to ...

Page 88: ...blade screwdriver battery removal Pen or pencil power on server Obtain the Chassis Serial Number To obtain support for your system you need your chassis serial number The chassis serial number is located on a sticker that is on the front of the server and another sticker on the side of the server Obtain the Chassis Serial Number Remotely Use the ILOM show SYS command to obtain the chassis serial n...

Page 89: ...Administration Guide Related Information Power Off From the Command Line on page 64 Power Off Graceful Shutdown on page 64 Power Off Emergency Shutdown on page 65 FAN_FAULT Properties type Host System keyswitch_state Normal product_name T5440 product_serial_number 0723BBC006 fault_state OK clear_fault_action none power_state On Commands cd reset set show start stop ...

Page 90: ...command Ensure that all data is saved before entering this command Power Off Graceful Shutdown Press and release the Power button If necessary use a pen or pencil to press the Power button shutdown g0 i0 y svc startd The system is coming down Please wait svc startd 91 system services are now being stopped Jun 12 19 46 57 wgs41 58 syslogd going down on signal 15 svc stard The system is down syncing...

Page 91: ...use 3 3v standby power is always present in the system you must unplug the power cords before accessing any cold serviceable components Extending the Server to the Maintenance Position The following components can be serviced with the server in the maintenance position Fan trays CMP memory modules FB DIMMs PCIe XAUI cards Service processor Power supply backplane Hard drive backplane Related Inform...

Page 92: ... the server is extended Although the cable management arm CMA that is supplied with the server is hinged to accommodate extending the server you should ensure that all cables and cords are capable of extending 3 From the front of the server release the two slide release latches FIGURE Extending the Server Into the Maintenance Position on page 66 Squeeze the slide rail locks to release the slide ra...

Page 93: ... Motherboard Caution Two people must dismount and carry the chassis FIGURE Lift Warning Related Information Front Panel Diagram on page 3 Rear Panel Diagram on page 5 Extend the Server to the Maintenance Position on page 66 Remove the Server From the Rack on page 67 Remove the Server From the Rack 1 Disconnect all the cables and power cords from the server 2 Extend the server to the maintenance po...

Page 94: ...CMA is still attached to the cabinet but the server is now disconnected from the CMA FIGURE Removing the Server From the Rack Caution Use two people to dismount and carry the chassis 4 From the front of the server press inner rail release buttons and pull the server forward until it is free of the rack rails 5 Set the server on a sturdy work surface Figure Legend 1 Disconnect system cables and CMA...

Page 95: ... removal installation or replacement process Place ESD sensitive components such as the printed circuit boards on an antistatic mat The following items can be used as an antistatic mat Antistatic bag used to wrap a replacement part ESD mat A disposable ESD mat shipped with some replacement parts or optional system components 2 Attach an antistatic wrist strap When servicing or removing server comp...

Page 96: ...Measures on page 69 1 Loosen the two captive No 2 Phillips screws at the rear edge of the top panel 2 Slide the top cover to the rear about 0 5 inch 12 7 mm 3 Remove the top cover Lift up and remove the cover Caution If the top cover is removed before the server is powered off the server will immediately disable the front panel Power button and shut down After such an event you must replace the to...

Page 97: ...uggable and Hot Swappable Devices on page 72 Remove install and add hard drives Servicing Hard Drives on page 72 Remove and install fan trays Servicing Fan Trays on page 81 Remove and install power supplies Servicing Power Supplies on page 85 Remove install and add PCIe cards Servicing PCIe Cards on page 92 Remove install and add CMP or memory modules Servicing CMP Memory Modules on page 98 Remove...

Page 98: ...cting the rest of the server s capabilities In the SPARC Enterprise T5440 server the following devices are hot swappable Fan trays Power supplies Note The chassis mounted hard drives can be hot swappable depending on how they are configured Related Information Servicing Hard Drives on page 72 Servicing Fan Trays on page 81 Servicing Power Supplies on page 85 Server Components on page 173 Servicing...

Page 99: ...ese conditions you must power off the server before you replace the hard drive Related Information Identifying Server Components on page 1 Managing Faults on page 9 Powering Off the System on page 63 Hot Pluggable and Hot Swappable Devices on page 72 Hard Drive Device Identifiers on page 79 Hard Drive LEDs on page 80 Server Components on page 173 Remove a Hard Drive Hot Plug Removing a hard drive ...

Page 100: ...nfigure command to unconfigure the disk For example type where c0 dsk c0t1d1 is the disk that you are trying to unconfigure 3 Wait until the blue Ready to Remove LED lights This LED will help you identify which drive is unconfigured and can be removed 4 On the drive you plan to remove push the hard drive release button to open the latch FIGURE Removing a Hard Drive on page 74 FIGURE Removing a Har...

Page 101: ... a slot in the server you must install the replacement drive in the same slot as the drive that was removed 3 Slide the drive into the drive slot until it is fully seated CODE EXAMPLE Sample Ap_id Output Ap_id Type Receptacle Occupant Condition c0 scsi bus connected configured unknown c0 dsk d1t0d0 disk connected configured unknown c0 dsk d1t1d0 disk connected configured unknown usb0 1 unknown emp...

Page 102: ... to CODE EXAMPLE Sample Ap_id Output on page 77 6 Type the cfgadm c configure command to configure the disk For example type where c0 sd1 is the disk that you are trying to configure 7 Wait until the blue Ready to Remove LED is no longer lit on the drive that you installed 8 At the Solaris prompt type the cfgadm al command to list all drives in the device tree including any drives that are not con...

Page 103: ...on Measures on page 69 Do the following 1 Note the location of each hard drive iostat E CODE EXAMPLE Sample Ap_id Output Ap_id Type Receptacle Occupant Condition c0 scsi bus connected configured unknown c0 dsk d1t0d0 disk connected configured unknown c0 sd1 disk connected unconfigured unknown usb0 1 unknown empty unconfigured ok usb0 2 unknown empty unconfigured ok usb0 3 unknown empty unconfigure...

Page 104: ... Hard Drive If you are installing a hard drive after servicing another component in the system do the following 1 Align the replacement drive to the drive slot Hard drives are physically addressed according to the slot in which they are installed If you removed an existing hard drive from a slot in the server you must install the replacement drive in the same slot as the drive that was removed 2 S...

Page 105: ...ames on page 79 lists physical drive locations and their corresponding default path names in OpenBoot PROM and Solaris for the SPARC Enterprise T5440 server Note Hard drive names in ILOM messages are displayed with the full FRU name such as SYS HDD0 TABLE Physical Drive Locations FRU Names and Default Drive Path Names Device Device Identifier OpenBoot PROM Solaris Default Drive Path Name HDD0 SYS ...

Page 106: ...d drive fault Related Information Hard Drive Device Identifiers on page 79 TABLE Hard Drive Status LEDs No LED Color Notes 1 Ready to Remove Blue This LED is lit to indicate that a hard drive can be removed safely during a hot plug operation 2 Service Required Amber This LED is lit when the system is running and the hard drive is faulty 3 OK Activity Green This LED lights when data is being read f...

Page 107: ...onents on page 1 Managing Faults on page 9 Powering Off the System on page 63 Hot Pluggable and Hot Swappable Devices on page 72 Fan Tray Device Identifiers on page 84 Fan Tray Fault LED on page 84 Server Components on page 173 Remove a Fan Tray Hot Swap Before you begin complete these tasks Read the section Safety Information on page 59 Perform the task Extend the Server to the Maintenance Positi...

Page 108: ...ay is oriented correctly Airflow in the system is from front to back 2 Verify proper fan tray operation See Fan Tray Fault LED on page 84 Next Steps If you are replacing a faulty fan tray due to an overtemperature condition monitor the system to ensure proper cooling Slide the Server Into the Rack on page 151 If you performed any additional service procedures see Power On the Server on page 153 ...

Page 109: ...ormation on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Perform the task Extend the Server to the Maintenance Position on page 66 Perform the task Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Do the following Press the fan tray latches toward the center of the fan tray and pull the fan tray up and ou...

Page 110: ... Server Into the Rack on page 151 Power On the Server on page 153 Fan Tray Device Identifiers TABLE Fan Tray Device Identifiers on page 84 describes the FRU device names for the fan trays in the server Related Information Managing Faults on page 9 Hot Pluggable and Hot Swappable Devices on page 72 Fan Tray Fault LED on page 84 Fan Tray Fault LED Each fan tray contains a Fault LED that is located o...

Page 111: ...ot Swappable Devices on page 72 Fan Tray Fault LED on page 84 Servicing Power Supplies The server is equipped with redundant hot swappable power supplies Redundant power supplies enable you to remove and replace a power supply without shutting the server down provided that at least two other power supplies are online and working Note If a power supply fails and you do not have a replacement availa...

Page 112: ...st disconnect the cable management arm support strut 1 Identify which power supply requires replacement An amber LED on a power supply indicates that a failure was detected In addition the show faulty command indicates which power supply is faulty See Detecting Faults on page 30 2 Gain access to the rear of the server where the faulty power supply is located If necessary slide the system partially...

Page 113: ...7 FIGURE Removing a Power Supply 5 Pull the power supply out of the chassis Install a Power Supply Hot Swap 1 Align the replacement power supply with the empty power supply bay 2 Slide the power supply into the bay until it is fully seated ...

Page 114: ...ply LED is green or blinking green 4 Verify that the system Power Supply Fault LED and the front and rear Service Required LEDs are not lit Note See Front Panel LEDs on page 4 and Rear Panel LEDs on page 7 for more information about identifying and interpreting system LEDs 5 At the ILOM prompt use the show faulty command to verify the status of the power supplies ...

Page 115: ...e these tasks Read the section Safety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Disconnect Power Cords From the Server on page 65 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Note If you are servicing Power Supply 0 you must disconnect the cable management arm support strut 1 Grasp t...

Page 116: ...pplies following another service tasks complete these steps 1 Align the replacement power supply with the empty power supply bay FIGURE Installing a Power Supply 2 Slide the power supply into the bay until it is fully seated Next Steps Connect the Power Cords to the Server on page 153 Power On the Server on page 153 ...

Page 117: ... page 91 Power Supply LED Each power supply contains a dual color LED that is visible when looking at the back panel of the system See TABLE Power Supply Status LEDs on page 91 for a description of power supply LED modes and their function listed from top to bottom TABLE Power Supply FRU Names Device Device Identifier PS0 SYS PS0 PS1 SYS PS1 PS2 SYS PS2 PS3 SYS PS3 TABLE Power Supply Status LEDs L...

Page 118: ...mation Managing Faults on page 9 Hot Pluggable and Hot Swappable Devices on page 72 Front Panel LEDs on page 4 Rear Panel LEDs on page 7 Servicing PCIe Cards Up to eight low profile PCIe cards may be installed in the system All slots are wired to x8 PCIe lanes Slot 1 and Slot 7 support graphics cards with x16 connectors Slot 4 and Slot 5 also support 10 Gbyte Ethernet cards XAUI cards When a XAUI ...

Page 119: ...on Powering Off the System on page 63 Extend the Server to the Maintenance Position on page 66 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Remove the Top Cover on page 69 Do the following 1 Identify the PCIe card you want to remove 2 Open the PCIe card latch FIGURE Removing a PCIe Card 3 Remove the PCIe card the system 4 Place the PCIe card on an antistatic mat 5 If y...

Page 120: ... latch Next Steps Install the Top Cover on page 150 Slide the Server Into the Rack on page 151 Power On the Server on page 153 Add a PCIe Card Before you begin complete these tasks Read the section Safety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Disconnect Power Cords From the Server on page 65 Extend the Serve...

Page 121: ...or installation See PCIe Device Identifiers on page 96 and PCIe Slot Configuration Guidelines on page 97 2 Open the PCIe card latch 3 Remove the PCIe filler panel 4 Insert the PCIe card into its slot FIGURE Installing a PCIe Card 5 Close the PCIe card latch Next Steps Install the Top Cover on page 150 Slide the Server Into the Rack on page 151 Power On the Server on page 153 ...

Page 122: ...f a CMP module is brought offline For more information see the SPARC Enterprise T5440 Server Product Notes Related Information Managing Faults on page 9 PCIe Slot Configuration Guidelines on page 97 System Bus Topology on page 162 Performing Node Reconfiguration on page 155 TABLE PCIe Device Identifiers Device Device Identifier Notes PCIe0 SYS MB PCIE0 x8 slot PCIe1 SYS MB PCIE1 x16 slot operating...

Page 123: ... Memory pair 3 Related Information PCIe Device Identifiers on page 96 System Bus Topology on page 162 I O Fabric in 2P Configuration on page 164 I O Fabric in 4P Configuration on page 165 TABLE PCIe Slot Configuration Guidelines PCIe XAUI Card Type Number of CMP Memory Modules Installation Order Notes 10 GBit Ethernet XAUI card 1 2 3 or 4 Slot 4 5 Install XAUI cards first External I O Expansion Un...

Page 124: ...ystem Each CMP module is paired with a memory module CMP modules and memory modules are keyed uniquely to prevent incorrect insertion into the wrong type of slot A faulty CMP or memory module is indicated with an alluminated fault LED An alluminated module LED also might indicate a faulty FB DIMM on that module FIGURE CMP Memory Module Pairs ...

Page 125: ...ion on page 164 I O Fabric in 4P Configuration on page 165 Remove a CMP Memory Module Before you begin complete these tasks Read the section Safety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Extend the Server to the Maintenance Position on page 66 Perform Electrostatic Discharge Antistatic Prevention Measures on ...

Page 126: ...antistatic mat Install a CMP Memory Module Note If you are replacing a faulty CMP or memory module you must transfer the FB DIMMs on the faulty module to the replacement module Replacement CMP memory modules do not include FB DIMMs For more information about installing FB DIMMs see Servicing FB DIMMs on page 104 1 Identify the correct slot for installation ...

Page 127: ...Power On the Server on page 153 Add a CMP Memory Module Before you begin complete these tasks Read the section Safety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Extend the Server to the Maintenance Position on page 66 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Remove the Top Cover o...

Page 128: ...he chassis 3 If you are installing the module into a previously empty slot remove the plastic connector cover on the motherboard 4 Slide the module down into its slot FIGURE Installing a CMP Module 5 Rotate the ejector levers down to secure the module into place Next Steps Install the Top Cover on page 150 Slide the Server Into the Rack on page 151 Power On the Server on page 153 ...

Page 129: ... and memory module names in ILOM messages are displayed with the full FRU name such as SYS MB CPU0 Related Information Managing Faults on page 9 Supported FB DIMM Configurations on page 110 Performing Node Reconfiguration on page 155 TABLE CMP Memory Module Device Identifier Device Device Identifier CMP0 SYS MB CPU0 CMP0 MEM0 SYS MB MEM0 CMP0 CMP1 SYS MB CPU1 CMP1 MEM1 SYS MB MEM1 CMP1 CMP2 SYS MB...

Page 130: ...ch CMP memory module pair Related Information Managing Faults on page 9 Remove FB DIMMs on page 105 Install FB DIMMs on page 105 Verify FB DIMM Replacement on page 106 Add FB DIMMs on page 109 Supported FB DIMM Configurations on page 110 FB DIMM Device Identifiers on page 112 FB DIMM Fault Button Locations on page 113 Servicing CMP Memory Modules on page 98 Performing Node Reconfiguration on page ...

Page 131: ...want to remove a Press the FB DIMM fault button See FB DIMM Fault Button Locations on page 113 b Note which FB DIMM fault LED is illuminated 2 Push down on the ejector tabs on each side of the FB DIMM until the FB DIMM is released Caution FB DIMMs might be hot Use caution when servicing FB DIMMs 3 Grasp the top corners of the faulty FB DIMM and remove it from the CMP memory module 4 Place the FB D...

Page 132: ...l the Top Cover on page 150 Slide the Server Into the Rack on page 151 Power On the Server on page 153 Verify FB DIMM Replacement 1 Access the ILOM prompt Refer to the Integrated Lights Out Manager 3 0 Supplement for SPARC Enterprise T5440 Server for instructions 2 Run the show faulty command to determine how to clear the fault The method you use to clear a fault depends on how the fault is identi...

Page 133: ... diag so that POST will run in Service mode b Power cycle the system Note The server takes about one minute to power off Use the show HOST command to determine when the host has been powered off The console will display status Powered Off show faulty Target Property Value SP faultmgmt 0 fru SYS MB CPU0 CMP0 BR1 CH0 D0 SP faultmgmt 0 timestamp Dec 21 16 40 56 SP faultmgmt 0 timestamp Dec 21 16 40 5...

Page 134: ...sole and issue the Solaris OS fmadm faulty command No memory faults should be displayed If faults are reported refer to the diagnostics flowchart in FIGURE Diagnostic Flowchart on page 11 for an approach to diagnose the fault 4 Switch to the ILOM command shell 5 Run the show faulty command If the fault was detected by the host and the fault information persists the output will be similar to the fo...

Page 135: ...rver to the Maintenance Position on page 66 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Remove the Top Cover on page 69 Remove a CMP Memory Module on page 99 1 Unpackage the FB DIMMs and place them on an antistatic mat 2 Ensure that the ejector tabs are in the open position 3 Line up the FB DIMM with the connector Align the FB DIMM notch with the key in the connector ...

Page 136: ... industry standard FB DIMMs 4 FB DIMM slots are located on the CMP module 12 FB DIMM slots are located on the memory module All FB DIMMs in the system must be the same density same capacity At minimum Channel 0 FB DIMM Slot 0 in all branches must be populated In branches populated with more than one FB DIMM for example in 8 and 16 FB DIMM configurations FB DIMMs are addressed in pairs Each pair mu...

Page 137: ...onding slots on the CMP memory modules Related Information Managing Faults on page 9 FB DIMM Device Identifiers on page 112 FB DIMM Fault Button Locations on page 113 Performing Node Reconfiguration on page 155 Figure Legend 1 Configuration 1 4 FB DIMMs 4 on CMP Module Only 2 Configuration 2 8 FB DIMMs 4 on CMP Module 4 on Memory Module 3 Configuration 3 16 FB DIMMs 4 on CMP Module 12 on Memory Mo...

Page 138: ... Node Reconfiguration on page 155 TABLE FB DIMM Configurations and Device Identifiers Location FB DIMM Device Identifiers Connector Number FB DIMM Group CMP module SYS MB CPUx CMPx BR1 CH0 D0 SYS MB CPUx CMPx BR1 CH1 D0 SYS MB CPUx CMPx BR0 CH0 D0 SYS MB CPUx CMPx BR0 CH1 D0 Motherboard connector J792 J896 J585 J687 Group 1 4 FB DIMMs Minimum configuration Memory module SYS MB MEMx CMPx BR1 CH1 D2...

Page 139: ...ation of the FB DIMM fault buttons on the CMP module and the memory module Press this button to illuminate the fault indicator on the module Replace the FB DIMM identified by the indicator Note You must replace a faulty FB DIMM with an identical part same part number See Supported FB DIMM Configurations on page 110 for more information ...

Page 140: ...terprise T5440 Server Service Manual July 2009 FIGURE FB DIMM Fault Button Locations Related Information Managing Faults on page 9 Supported FB DIMM Configurations on page 110 FB DIMM Device Identifiers on page 112 ...

Page 141: ...e and install field replaceable components Servicing the Front Bezel on page 115 Servicing the DVD ROM Drive on page 118 Servicing the Service Processor on page 120 Servicing the IDPROM on page 123 Servicing the Battery on page 125 Servicing the Power Distribution Board on page 126 Servicing the Fan Tray Carriage on page 129 Servicing the Hard Drive Backplane on page 132 Servicing the Motherboard ...

Page 142: ...rocedures power off the server using one of the methods described in the section Powering Off the System on page 63 Extend the Server to the Maintenance Position on page 66 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Do the following 1 Grasp the front bezel on the left and right sides 2 Pull the bezel off of the front of the chassis The bezel is secured with three sna...

Page 143: ...ually pulling it from the middle and both ends simultaneously Install the Front Bezel 1 Align the bezel with the chassis front panel 2 Press the bezel onto the front panel The bezel is oriented with four guide pins and is secured with three snap in posts Next Steps Slide the Server Into the Rack on page 151 ...

Page 144: ...tion on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Extend the Server to the Maintenance Position on page 66 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Remove the Top Cover on page 69 Remove the Front Bezel on page 116 Do the following 1 Remove the flex cable retainer Loosen the captive No 2 Philli...

Page 145: ... drive out of the chassis Install the DVD ROM Drive 1 Slide the DVD ROM drive into its bay FIGURE Installing the DVD ROM Drive 2 Connect the DVD ROM drive to the flex cable assembly 3 Install the flex cable retainer Place the retainer into position and tighten the captive No 2 Phillips screw ...

Page 146: ...n page 125 Remove the Service Processor Before you begin complete these tasks Read the section Safety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Extend the Server to the Maintenance Position on page 66 Disconnect Power Cords From the Server on page 65 Perform Electrostatic Discharge Antistatic Prevention Measures...

Page 147: ...cessor up and out of the system 4 Place the service processor on an antistatic mat Next Steps If you are replacing a faulty service processor you must install the IDPROM onto the new service processor Do the following Remove the IDPROM from the old service processor See Remove the IDPROM on page 123 ...

Page 148: ...e power cords are disconnected from the system 2 Lower the service processor into position Ensure that the service processor is oriented correctly over the motherboard connector and the two snap on standoffs FIGURE Installing the Service Processor 3 Press down evenly to plug the service processor into the motherboard 4 Secure the service processor with the two captive No 2 Phillips screws ...

Page 149: ...rvice processor to the new one Related Information Servicing the Service Processor on page 120 Servicing the Battery on page 125 Remove the IDPROM Before you begin complete these tasks Read the section Safety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Extend the Server to the Maintenance Position on page 66 Disco...

Page 150: ...ction Safety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Extend the Server to the Maintenance Position on page 66 Disconnect Power Cords From the Server on page 65 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Remove the Top Cover on page 69 Remove the Service Processor on page 120 ...

Page 151: ...the Battery Before you begin complete these tasks Read the section Safety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Extend the Server to the Maintenance Position on page 66 Disconnect Power Cords From the Server on page 65 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Remove the Top C...

Page 152: ...ough the flex cable circuit to the motherboard Related Information Safety Information on page 59 Servicing Power Supplies on page 85 Remove the Power Distribution Board Before you begin complete these tasks Read the section Safety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Disconnect Power Cords From the Server o...

Page 153: ...g the flex cable from the power distribution board 3 Unplug the auxiliary power cable from the power distribution board 4 Remove the No 2 Phillips screw 5 Remove the two 7 mm hex nuts securing the bus bars to the power distribution board FIGURE Disconnecting the Power Distribution Board From the Chassis 6 Slide the power distribution board up and out of the chassis ...

Page 154: ...m nuts securing the bus bars to the power distribution board 5 Plug in the flex cable connector Ensure that the auxilliary power cable is routed under the flex cable connector 6 Plug in the auxiliary power cable 7 Install the flex cable retainer Place the retainer into position and tighten the captive No 2 Phillips screw Next Steps Install the Top Cover on page 150 Slide the Server Into the Rack o...

Page 155: ...g the Front I O Board on page 146 Remove the Fan Tray Carriage Before you begin complete these tasks Read the section Safety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Extend the Server to the Maintenance Position on page 66 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Remove a Fan Tr...

Page 156: ...ay carriage to the top of the chassis FIGURE Removing the Fan Tray Carriage 2 Loosen the seven captive No 2 Phillips securing the bottom of the fan tray carriage to the motherboard assembly 3 Lift the fan tray carriage up and out of the system Install the Fan Tray Carriage 1 Lower the fan tray carriage into the system ...

Page 157: ...s Next Steps Install a Fan Tray on page 84 Note Install all four fan trays Install the Top Cover on page 150 Slide the Server Into the Rack on page 151 Power On the Server on page 153 Servicing the Hard Drive Backplane The hard drive backplane provides the power and data interconnect to the internal hard drives Related Information Servicing Hard Drives on page 72 ...

Page 158: ...rge Antistatic Prevention Measures on page 69 Remove the Top Cover on page 69 Remove a Hard Drive on page 77 Note You must remove all four hard drives from the server Note the location of each hard drive you remove You must re install each hard drive in the correct bay Remove a Fan Tray on page 83 Note You must remove all four fan trays Remove the Fan Tray Carriage on page 129 Do the following 1 R...

Page 159: ...ing the Hard Drive Backplane 4 Lift the backplane up and out of the system Install the Hard Drive Backplane 1 Lower the hard drive backplane into the system Align the tab on the lower edge the backplane with the corresponding slot in the chassis floor ...

Page 160: ...flex cable retainer Place the retainer into position and tighten the captive No 2 Phillips screw Next Steps Install the Fan Tray Carriage on page 131 Install a Fan Tray on page 84 Install a CMP Memory Module on page 100 Install the Top Cover on page 150 Install a Hard Drive on page 78 Note You must install the hard drives in the correct slots Slide the Server Into the Rack on page 151 Power On the...

Page 161: ...you begin complete these tasks Read the section Safety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Disconnect Power Cords From the Server on page 65 Remove the Server From the Rack on page 67 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Remove the Top Cover on page 69 Remove a PCIe Car...

Page 162: ...t is secured with six captive No 2 Phillips screws See FIGURE CMP Memory Module Bracket Captive Screw Locations on page 136 FIGURE CMP Memory Module Bracket Captive Screw Locations 2 Remove the flex cable retainer Loosen the captive No 2 Phillips screw and lift the retainer up and out of the chassis 3 Unplug the flex cable from J9801 on the motherboard 4 Unplug the auxiliary power cable from J9803...

Page 163: ...the chassis floor See FIGURE Motherboard Fastener Locations on page 140 for the fastener locations 9 Lift the motherboard up and out of the chassis Guide the flex cable connector out from under the midwall partition FIGURE Removing the Motherboard 10 Place the motherboard on an antistatic mat Next Steps If you are replacing a faulty motherboard you must program the chassis serial number and produc...

Page 164: ...oard Fastener Locations on page 140 4 Lower and secure the midwall partition 5 Install the six No 2 Phillips screws that secure the bus bar assembly to the motherboard 6 Install the CMP memory module bracket The bracket is secured with six No 2 Phillips screws 7 Plug in the auxiliary power cable to J9803 8 Plug in the flex cable connector to J9801 9 Install the flex cable retainer Place the retain...

Page 165: ...memory modules Install the Service Processor on page 122 Install a PCIe Card on page 94 Install the Top Cover on page 150 Install the Server Into the Rack on page 150 Connect the Power Cords to the Server on page 153 Power On the Server on page 153 Motherboard Fastener Locations FIGURE Motherboard Fastener Locations on page 140 shows the location of the captive screws that secure the motherboard t...

Page 166: ... Information Servicing the Motherboard on page 135 Servicing the Flex Cable Assembly The flex cable assembly provides the power and data connection between the power supply backplane hard drive backplane and motherboard Related Information Safety Information on page 59 Servicing Power Supplies on page 85 ...

Page 167: ... Safety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Extend the Server to the Maintenance Position on page 66 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Remove the Top Cover on page 69 Do the following 1 Unplug the power cords 2 Remove the flex cable retainer Loosen the captive No 2 P...

Page 168: ...plane connection 5 Unplug the flex cable to DVD ROM drive connection 6 Unplug the flex cable to motherboard connection 7 Lift the flex cable up and out of the system Install the Flex Cable Assembly 1 Ensure the power cables are unplugged 2 Plug in the motherboard connector 3 Plug in the hard drive backplane connector 4 Plug in the DVD ROM drive connector 5 Plug in the power supply backplane connec...

Page 169: ...ner Place the retainer into position and tighten the captive No 2 Phillips screw FIGURE Installing the Flex Cable Retainer 7 Plug in the power cables Next Steps Install the Top Cover on page 150 Slide the Server Into the Rack on page 151 Power On the Server on page 153 ...

Page 170: ...afety Information on page 59 Power off the server using one of the methods described in the section Powering Off the System on page 63 Disconnect Power Cords From the Server on page 65 Remove the Server From the Rack on page 67 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Remove the Top Cover on page 69 Remove a Fan Tray on page 83 Remove the Fan Tray Carriage on page ...

Page 171: ...s 145 FIGURE Removing the Front Control Panel 4 Lift the front control panel up and out of the system 5 Place the front control panel on an antistatic mat Install the Front Control Panel 1 Lower the front control panel into the system ...

Page 172: ...el connector into J9901 on the motherboard Next Steps Install the Fan Tray Carriage on page 131 Install a Fan Tray on page 84 Install the Top Cover on page 150 Install the Server Into the Rack on page 150 Connect the Power Cords to the Server on page 153 Power On the Server on page 153 Servicing the Front I O Board The front I O board contains two USB connectors You must remove the front control p...

Page 173: ...one of the methods described in the section Powering Off the System on page 63 Disconnect Power Cords From the Server on page 65 Remove the Server From the Rack on page 67 Perform Electrostatic Discharge Antistatic Prevention Measures on page 69 Remove the Top Cover on page 69 Remove a Fan Tray on page 83 Remove the Fan Tray Carriage on page 129 1 Unplug the front control panel cable from J9901 on...

Page 174: ... O board into the system 2 Install the two No 2 Phillips screws 3 Plug the front control panel connector into the front I O board 4 Plug the front control panel connector into J9901 on the motherboard Next Steps Install the Fan Tray Carriage on page 131 Install a Fan Tray on page 84 Install the Top Cover on page 150 Install the Server Into the Rack on page 150 Connect the Power Cords to the Server...

Page 175: ...rvice the System on page 59 Servicing Customer Replaceable Units on page 71 Servicing Field Replaceable Units on page 115 Topic Links Install the top cover after servicing internal components Install the Top Cover on page 150 Re attach the server to the cabinet slide rails after performing a bench procedure Install the Server Into the Rack on page 150 Slide the server back into the equipment rack ...

Page 176: ...he rear edge Install the Server Into the Rack The following procedure explains how to insert the server into the rack Caution The weight of the server on extended slide rails can be enough to overturn an equipment rack Before you begin deploy the antitilt feature on your cabinet Caution The server weighs approximately 88 lb 40 kg Two people are required to lift and mount the server into a rack enc...

Page 177: ...r the inner slide assemblies 3 Ensure that the inner rails are engaged with the ball bearing retainers on both inner slide assemblies Note If necessary support the server with the mechanical lift while aligning the inner rails parallel to the rack mounted inner slide assemblies Slide the Server Into the Rack 1 Press the inner rail release buttons FIGURE Slide Rail Release Button Location on page 1...

Page 178: ...essary re attach the CMA a Attach the CMA support strut to the inner glide b Attach the CMA to the inner glide Slide the hinge plate into the end of the outer rail until the retaining pin snaps into place 4 Reconnect the cables to the back of the server If the CMA is in the way slide the server partially out of the cabinet to access the necessary rear panel connections Figure Legend 1 Inner rail r...

Page 179: ...on sequence from the service processor prompt issue the poweron command You will see an Alert message on the system console This message indicates that the system is reset You will also see a message indicating that the VCORE has been margined up to the value specified in the default scr file that was previously configured Example To initiate the power on sequence manually use a pen or pencil to p...

Page 180: ...154 SPARC Enterprise T5440 Server Service Manual July 2009 ...

Page 181: ...in the new system configuration Related Information Managing Faults on page 9 Topic Links Learn about how CMP memory modules map to I O devices I O Connections to CMP Memory Modules on page 156 Learn how to reconfigure the server to temporarily bypass a failed CMP memory module Reconfiguring I O Device Nodes on page 158 Disable memory modules Temporarily Disable All Memory Modules on page 160 Reco...

Page 182: ...for more information If a CMP module fails the onboard devices and slots directly connected to it become unavailable Recovery of the I O services connected to the failed CMP requires I O node reconfiguration For example in a 4P system if CMP0 goes offline the following devices become unavailable PCIe0 PCIe1 Onboard hard drives In this failure scenario the system is unable to boot from internal dri...

Page 183: ...odule Note At a minimum a functioning CMP module must be installed in CMP Slot 0 If you are performing a node reconfiguration following a failure in CMP Slot 0 you must move one of the remaining CMP modules to CMP Slot 0 3 If neither option 1 nor 2 is possible you must do the following Temporarily Disable All Memory Modules on page 160 Reconfigure the I O and PCIe Fabric on page 158 Re Enable All ...

Page 184: ...tions to CMP Memory Modules on page 156 System Bus Topology on page 162 I O Fabric in 2P Configuration on page 164 I O Fabric in 4P Configuration on page 165 Temporarily Disable All Memory Modules on page 160 Reconfigure the I O and PCIe Fabric on page 158 Re Enable All Memory Modules on page 161 Reset the LDoms Guest Configuration on page 162 Reconfigure the I O and PCIe Fabric The reconf pl scri...

Page 185: ...mpStart server 3 Power off the system 4 Log into the ALOM compatibility shell Type 5 Power on the system 6 Boot from the network Type 7 Mount the system boot disk under the mnt directory Type 8 Change to the root directory of your boot disk and copy the reconf pl script to the root of the boot disk Type 9 Do one of the following If your Jumpstart server is exporting Solaris 10 8 07 or Solaris 10 5...

Page 186: ...of the memory modules in order to work around this complication If you are recovering from a failed CMP module you must temporarily disable the FB DIMMS on all memory modules when Solaris is halted and the system is powered off The FB DIMMs are re enabled after the I O and PCIe devices are reconfigured You can either physically remove the memory modules from the system or remotely disable all FB D...

Page 187: ...B MEMx CMPx BR0 CH0 D1 sc disablecomponent SYS MB MEMx CMPx BR0 CH0 D2 sc disablecomponent SYS MB MEMx CMPx BR1 CH1 D3 CODE EXAMPLE Using the disablecomponent command to disable all FB DIMMs on MEM1 sc disablecomponent SYS MB MEM1 CMP1 BR0 CH0 D1 sc disablecomponent SYS MB MEM1 CMP1 BR0 CH0 D2 sc disablecomponent SYS MB MEM1 CMP1 BR0 CH0 D3 sc disablecomponent SYS MB MEM1 CMP1 BR0 CH1 D1 sc disabl...

Page 188: ...logy FIGURE System Bus Topology on page 163 describes the system bus topology for the SPARC Enterprise T5440 server CODE EXAMPLE Using the enablecomponent command to enable all FB DIMMs on CMP1 sc enablecomponent SYS MB MEM1 CMP1 BR0 CH0 D1 sc enablecomponent SYS MB MEM1 CMP1 BR0 CH0 D2 sc enablecomponent SYS MB MEM1 CMP1 BR0 CH0 D3 sc enablecomponent SYS MB MEM1 CMP1 BR0 CH1 D1 sc enablecomponent...

Page 189: ...Performing Node Reconfiguration 163 FIGURE System Bus Topology Related Information I O Fabric in 2P Configuration on page 164 I O Fabric in 4P Configuration on page 165 ...

Page 190: ...mation System Bus Topology on page 162 I O Fabric in 4P Configuration on page 165 TABLE Devices controlled by CMPs in 2P systems CMP Number Devices Controlled CMP0 Onboard disk drives Onboard USB ports Onboard DVD drive PCIe0 PCIe1 PCIe2 PCIe3 CMP1 Onboard Gbit or 10 Gbit network PCIe4 PCIe5 PCIe6 PCIe7 ...

Page 191: ...s Topology on page 162 I O Fabric in 2P Configuration on page 164 TABLE Devices controlled by CMPs in 4P systems CMP Number Devices Controlled CMP0 Onboard disk drives Onboard USB ports Onboard DVD drive PCIe0 PCIe1 CMP1 Onboard Gbit or 10 Gbit network PCIe4 PCIe5 CMP2 PCIe2 PCIe3 CMP3 PCIe6 PCIe7 ...

Page 192: ...166 SPARC Enterprise T5440 Server Service Manual July 2009 ...

Page 193: ...ial management connector labeled SERIAL MGT is an RJ 45 connector located on the back panel This port is the default connection to the system console Topic Links Reference for system connector pinouts Serial Management Port Connector Pinouts on page 167 Network Management Port Connector Pinouts on page 168 Serial Port Connector Pinouts on page 169 USB Connector Pinouts on page 169 Gigabit Ethernet...

Page 194: ...onnector labeled NET MGT is an RJ 45 connector located on the motherboard and can be accessed from the back panel This port needs to be configured prior to use TABLE Serial Management Connector Signals Pin Signal Description Pin Signal Description 1 Request to Send 5 Ground 2 Data Terminal Ready 6 Receive Data 3 Transmit Data 7 Data Set Ready 4 Ground 8 Clear to Send ...

Page 195: ...ctor Signals Pin Signal Description Pin Signal Description 1 Transmit Data 5 Common Mode Termination 2 Transmit Data 6 Receive Data 3 Receive Data 7 Common Mode Termination 4 Common Mode Termination 8 Common Mode Termination TABLE Serial Port Connector Signals Pin Signal Description Pin Signal Description 1 Data Carrier Detect 6 Data Set Ready 2 Receive Data 7 Request to Send 3 Transmit Data 8 Cle...

Page 196: ... panel FIGURE USB Connector Diagram Gigabit Ethernet Connector Pinouts Four RJ 45 Gigabit Ethernet connectors NET0 NET1 NET2 NET3 are located on the system motherboard and can be accessed from the back panel The Ethernet interfaces operate at 10 Mbit sec 100 Mbit sec and 1000 Mbit sec TABLE USB Connector Signals Pin Signal Description Pin Signal Description A1 5 V fused B1 5 V fused A2 USB0 1 B2 U...

Page 197: ...Ethernet Connector Signals Pin Signal Description Pin Signal Description 1 Transmit Receive Data 0 5 Transmit Receive Data 2 2 Transmit Receive Data 0 6 Transmit Receive Data 1 3 Transmit Receive Data 1 7 Transmit Receive Data 3 4 Transmit Receive Data 2 8 Transmit Receive Data 3 ...

Page 198: ...172 SPARC Enterprise T5440 Server Service Manual July 2009 ...

Page 199: ...nts on page 1 Servicing Customer Replaceable Units on page 71 Servicing Field Replaceable Units on page 115 Description Links A diagram and list of customer replaceable units CRUs Customer Replaceable Units on page 174 A diagram and list of components that only field service personnel can replace Field Replaceable Units on page 176 ...

Page 200: ...0 Server Service Manual July 2009 Customer Replaceable Units FIGURE Customer Replaceable Units CRUs Figure Legend 1 CMP modules 5 Front bezel 2 Memory modules 6 Hard drives 3 Fan trays 7 Power supplies 4 Removable media drive 8 ...

Page 201: ...ble Devices on page 72 Servicing Hard Drives on page 72 Servicing Fan Trays on page 81 Servicing Power Supplies on page 85 Servicing CMP Memory Modules on page 98 Servicing FB DIMMs on page 104 Servicing the Front Bezel on page 115 Servicing the DVD ROM Drive on page 118 ...

Page 202: ...r Service Manual July 2009 Field Replaceable Units FIGURE Field Replaceable Units FRUs Figure Legend 1 CMP memory module bracket 4 Power supply backplane 2 Fan cage 5 Flex cable assembly 3 Hard drive backplane 6 Auxiliary power cable ...

Page 203: ...s Related Information Servicing the Service Processor on page 120 Servicing the IDPROM on page 123 Servicing the Battery on page 125 Servicing the Power Distribution Board on page 126 Figure Legend 1 IDPROM 4 Motherboard 2 Front Control Panel 5 Battery 3 Front I O Board 6 Service Processor ...

Page 204: ...ervicing the Fan Tray Carriage on page 129 Servicing the Hard Drive Backplane on page 132 Servicing the Motherboard on page 135 Servicing the Flex Cable Assembly on page 140 Servicing the Front Control Panel on page 144 Servicing the Front I O Board on page 146 ...

Page 205: ...mmand 53 clearing POST detected faults 48 clearing PSH detected faults 49 CMP fault recovery 160 CMP module disabling to run system in degraded state 160 failure recovery 157 fault recovery 155 I O devices connected to 156 CMP memory module 100 adding 101 device identifiers 103 installing 100 removing 99 supported configurations 104 CMP memory modules supported configurations 104 CMP0 failure mode...

Page 206: ...button 5 enablecomponent command 48 environmental faults 12 13 16 33 event log checking the PSH 45 EVENT_ID FRU 45 exercising the system with SunVTS 38 External I O Expansion Unit fault detected by show faulty command 35 faults detection in 15 F Fan Fault system LED interpreting to diagnose faults 31 fan module determining fault state 31 Fault LED 31 fan module LEDs using to identify faults 31 fan...

Page 207: ...t Ethernet ports LEDs 8 pinouts 170 H hard drive about 72 addressing 75 78 determining fault state 31 device identifiers 79 Fault LED 31 hot plugging 75 installing 75 78 Ready to Remove LED 76 removing 73 77 hard drive backplane 132 about 2 installing 133 removing 132 hard drive LEDs 80 hard drive LEDs about 80 help command 53 host ID stored on SCC module 2 hot pluggable devices 72 hot plugging ha...

Page 208: ...D 5 31 Power OK system LED 12 Power Supply Fault system LED 5 31 88 92 Ready to Remove hard drive LED 74 76 Service Required system LED 4 31 92 Top system LED 5 LEDs about 30 fan module 31 fan tray 84 front panel 4 hard drive 80 network management port 8 rear panel 7 Service Required system LED 32 using to diagnose faults 30 using to identify device state 30 Locator LED and button 3 4 5 7 log file...

Page 209: ...153 following emergency shutdown triggered by top panel removal 150 153 using Power button 153 poweron command 54 power on self test POST 19 about 19 components disabled by 51 configuration flowchart 20 controlling output 26 error messages 42 fault clearing 48 faults detected by 12 33 faulty components detected by 48 parameters changing 27 running in maximum mode 28 troubleshooting with 14 using f...

Page 210: ... using to check for faults 12 using to diagnose FB DIMMs 106 using to verify successful FB DIMM replacement 108 showcomponent command 24 51 showenvironment command 55 showfaults command syntax 55 showfru command 25 56 showkeyswitch command 56 showlocator command 56 showlogs command 56 showplatform command 56 62 shutdown triggered by top cover removal emergency shutdown 150 using Power button emerg...

Page 211: ...aris OS log files 12 CMP0 failure 156 FB DIMMs 22 Power OK LED state 12 using LEDs 30 using POST 13 14 using SunVTS 12 using the show faulty command 12 U UltraSPARC T2 multicore processor 18 Universal Unique Identifier UUID 18 45 USB ports pinouts 169 USB ports front 3 V virtual keyswitch 28 107 X XAUI card about 1 configuration guidelines see PCIe configuration guidelines installing See PCIe card...

Page 212: ...186 SPARC Enterprise T5440 Server Service Manual July 2009 ...

Page 213: ......

Page 214: ......