Oracle SPARC T5-4, Service Manual

Page 1: ...Part No E29663 11 July 2016 SPARC T5 4 Server Service Manual ...

Page 2: ......

Page 3: ...ous applications including applications that may create a risk of personal injury If you use this software or hardware in dangerous applications then you shall be responsible to take all appropriate fail safe backup redundancy and other measures to ensure its safe use Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerou...

Page 4: ...ions à risque notamment dans des applications pouvant causer des dommages corporels Si vous utilisez ce logiciel ou matériel dans le cadre d applications dangereuses il est de votre responsabilité de prendre toutes les mesures de secours de sauvegarde de redondance et autres mesures nécessaires à son utilisation dans des conditions optimales de sécurité Oracle Corporation et ses affiliés déclinent...

Page 5: ... Component Service Task Reference 20 Detecting and Managing Faults 23 Understanding Diagnostics 23 Diagnostics Process 23 Tool Availability 25 Log In to Oracle ILOM Service 26 Oracle ILOM Service Related Tools 27 Interpreting LEDs 28 Front Panel Controls and LEDs 29 Rear Panel Controls and LEDs 31 Configuring POST 32 POST Overview 33 Oracle ILOM Properties That Affect POST Behavior 33 Configure PO...

Page 6: ...ing Power From the Server 51 Related Information 51 Prepare to Power Off the Server 51 Power Off the Server Oracle ILOM 52 Power Off the Server Power Button Graceful Shutdown 52 Power Off the Server Power Button Emergency Shutdown 53 Disconnect Power Cords 53 Prevent ESD Damage 54 Servicing Processor Modules 57 Server Upgrade Process 57 Processor Module Locations 59 Processor Module LEDs 60 Determ...

Page 7: ...4 Servicing the Main Module 97 Main Module LEDs 98 Determine if the Main Module Is Faulty 99 Remove the Main Module 99 Install the Main Module 102 Verify the Main Module 105 Servicing the Storage Backplanes 107 Remove a Storage Backplane 107 Install a Storage Backplane 111 Servicing the Service Processor Card 115 Determine if the Service Processor Card Is Faulty 115 Remove the Service Processor Ca...

Page 8: ...38 Install a Power Supply 140 Verify a Power Supply 143 Servicing Fan Modules 145 Fan Module Locations 145 Fan Module LED 146 Determine Which Fan Module Is Faulty 146 Remove a Fan Module 147 Install a Fan Module 149 Verify a Fan Module 150 Servicing PCIe Cards 151 Understanding PCIe Root Complex Connections 151 PCIe Root Complex Connections Single Processor Configurations 152 PCIe Root Complex Con...

Page 9: ...odule 176 Install the Rear I O Module 178 Verify the Rear I O Module 180 Servicing the Rear Chassis Subassembly 183 Rear Chassis Subassembly Components 183 Remove the Rear Chassis Subassembly 184 Install the Rear Chassis Subassembly 187 Verify the Rear Chassis Subassembly 188 Returning the Server to Operation 191 Connect Power Cords 191 Power On the Server Oracle ILOM 192 Glossary 193 Index 199 ...

Page 10: ...10 SPARC T5 4 Server Service Manual July 2016 ...

Page 11: ...ngineers with training in servicing this server Required knowledge Advanced experience troubleshooting and replacing hardware Product Documentation Library Late breaking information and known issues for this product are included in the documentation library at http www oracle com goto T5 4 docs Feedback Provide feedback about this documentation at http www oracle com goto docfeedback ...

Page 12: ...12 SPARC T5 4 Server Service Manual July 2016 ...

Page 13: ...l Components on page 14 Rear Panel Components on page 15 Chassis Subassembly Components on page 17 Processor Module Components on page 18 Main Module Components on page 19 Supported Storage and Backup Devices on page 20 Component Service Task Reference on page 20 Related Information Detecting and Managing Faults on page 23 Preparing for Service on page 45 Returning the Server to Operation on page ...

Page 14: ...Module Components on page 18 Servicing Processor Modules on page 57 2 Control panel Detecting and Managing Faults on page 23 Preparing for Service on page 45 Returning the Server to Operation on page 191 3 Main module Main Module Components on page 19 Servicing the Main Module on page 97 4 Power supplies 2 Servicing Power Supplies on page 135 Related Information Rear Panel Components on page 15 ...

Page 15: ...n modules 5 Servicing Fan Modules on page 145 2 AC power connectors 2 Preparing for Service on page 45 3 Rear I O module Servicing the Rear I O Module on page 173 4 PCIe carriers 16 Servicing PCIe Cards on page 151 These components are accessible within the rear chassis subassembly which you can access after you have removed all the components from the rear of the server ...

Page 16: ... the Rear Chassis Subassembly on page 183 3 Rear chassis subassembly Servicing the Rear Chassis Subassembly on page 183 Related Information Servicing Fan Modules on page 145 Servicing Power Supplies on page 135 Servicing the Rear I O Module on page 173 Servicing PCIe Cards on page 151 Servicing the Rear Chassis Subassembly on page 183 ...

Page 17: ...ls and indicators Front Panel Controls and LEDs on page 29 5 Processor modules 2 Servicing Processor Modules on page 57 6 System chassis 7 Rear chassis subassembly RCSA Servicing the Rear Chassis Subassembly on page 183 8 Fan modules 5 Servicing Fan Modules on page 145 9 PCIe carriers 16 Servicing PCIe Cards on page 151 10 Rear I O module Servicing the Rear I O Module on page 173 11 Power supplies...

Page 18: ...e on page 20 Processor Module Components These components are accessible within the processor module when you remove the processor module from the front of the server No Description Link 1 DIMMs Servicing DIMMs on page 69 Related Information Servicing Processor Modules on page 57 Servicing DIMMs on page 69 ...

Page 19: ... Front I O Assembly on page 131 2 Storage backplanes Servicing the Storage Backplanes on page 107 3 Main module motherboard 4 Service processor card Servicing the Service Processor Card on page 115 5 System configuration PROM Servicing the System Configuration PROM on page 121 6 System battery Servicing the System Battery on page 125 Related Information Front Panel Components on page 14 Component ...

Page 20: ...nce This table lists the names of serviceable components It also lists the system names and task locations for the components Component Max NAC Name SDM Name Link to Service Procedure Processor module 2 SYS PMx System CPU_Modules CPU_Module_x Servicing Processor Modules on page 57 Processor filler module 1 SYS PFMx Servicing Processor Modules on page 57 DIMM 64 SYS PMx CMx CMP BOBx CHx Dx System M...

Page 21: ...r_Supply_x Servicing Power Supplies on page 135 Fan module 5 SYS RCSA FBD0 FMx System Cooling Fans Fan_x Servicing Fan Modules on page 145 PCIe card 16 SYS RCSA PCIEx CAR CAR CARD System PCI_Devices Add on Device_x Servicing PCIe Cards on page 151 Rear IO module 1 SYS RIO System Networking Ethernet_NICs Servicing the Rear I O Module on page 173 Rear chassis subassembly RCSA 1 SYS RCSA None Servici...

Page 22: ...22 SPARC T5 4 Server Service Manual July 2016 ...

Page 23: ...ormation Identifying Components on page 13 Preparing for Service on page 45 Component Service Task Reference on page 20 Returning the Server to Operation on page 191 Oracle ILOM Documentation Library Understanding Diagnostics These topics explain the diagnostic process and tools Diagnostics Process on page 23 Tool Availability on page 25 Log In to Oracle ILOM Service on page 26 Oracle ILOM Service...

Page 24: ...er Service Manual July 2016 Note The diagnostic tools you use and the order in which you use them depend on the nature of the problem you are troubleshooting However for descriptive purposes this table follows the steps given in the illustration ...

Page 25: ... replace it Interpreting Log Files and System Messages on page 41 4 Run Oracle VTS software To run Oracle VTS the server must be running the Oracle Solaris OS If Oracle VTS reports a faulty component replace it If Oracle VTS does not report a faulty component run POST Refer to the Oracle VTS software documentation Contact technical support if the problem persists Related Information Tool Availabil...

Page 26: ... environment you must change the default password changeme for the default Administrator account root after your initial login to Oracle ILOM If this default Administrator account has since been changed contact your system administrator for an Oracle ILOM user account with Administrator privileges 2 Enable the Oracle ILOM 3 0 legacy name spaces set SP cli legacy_targets enabled Note In Oracle ILOM...

Page 27: ...eak_action break Takes the host server from the OS to either kmdb or OpenBoot prompt equivalent to a Stop A depending on the mode in which the Oracle Solaris OS was booted start HOST console Connects to the host show HOST console history Displays the contents of the host s console buffer set HOST bootmode property value Controls the method of booting for the host server s firmware The value of pro...

Page 28: ...he component is faulty Use the instructions in these links to determine if the component has been diagnosed as being faulty Determine if the Main Module Is Faulty on page 99 Determine Which Processor Module Is Faulty on page 61 Determine Which DIMM Is Faulty DIMM Fault Remind Button on page 75 Determine Which Hard Drive Is Faulty on page 90 Determine Which Power Supply Is Faulty on page 138 Determ...

Page 29: ...n page 50 2 Service Required LED amber The fmadm faulty command provides details about any faults that cause this indicator to light See Check for Faults on page 38 Under some fault conditions individual component fault LEDs are lit in addition to the Service Required LED 3 Power OK LED green Indicates these conditions Off Server is not running in its normal state Server power might be off The SP ...

Page 30: ...state no service action is required Steady on Indicates that a temperature failure event has been acknowledged and a service action is required 6 Fan Module Fault LED amber Rear FM Indicates these conditions Off Indicates a steady state no service action is required Steady on Indicates that a fan module failure event has been acknowledged and a service action is required on at least one of the fan...

Page 31: ...Indicates these conditions Off The link is operating as a 10 Mbps connection On or blinking The link is operating as a 100 Mbps connection 4 Network port link LED Indicates these conditions Off No link is established Blinking A link is established 5 Network port speed LED Indicates these conditions Off The link is operating as a 10 Mbps connection or there is no link Amber on The link is operating...

Page 32: ...n be quickly returned to full function Slow blink A normal but transitory activity is taking place Slow blinking might indicate that system diagnostics are running or that the system is booting 10 SP LED SP Indicates these conditions Off AC power might have been connected to the power supplies Steady on green SP is running in its normal operating state No service actions are required Blink green S...

Page 33: ...aulty processor core the core is disabled POST completes its test sequence and the server boots using the remaining cores Related Information Oracle ILOM Properties That Affect POST Behavior on page 33 Configure POST on page 35 Run POST With Maximum Testing on page 37 Oracle ILOM Properties That Affect POST Behavior Note The value of keyswitch_state must be normal when individual POST parameters a...

Page 34: ...mum set of tests HOST diag power_on_verbosity min Default Displays the minimum level of output max Displays information for each step normal Displays a moderate amount of information including component names and test results debug Displays extensive debugging information none Disables the output HOST diag error_reset_level max Default Runs the maximum set of tests min Runs a minimum set of tests ...

Page 35: ...mmand variables Related Information POST Overview on page 33 Configure POST on page 35 Run POST With Maximum Testing on page 37 Configure POST 1 Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 26 2 Set the virtual keyswitch to the value that corresponds to the POST configuration you want to run ...

Page 36: ...ou want to define the mode level verbosity or trigger set the respective parameters Syntax set HOST diag property value See Oracle ILOM Properties That Affect POST Behavior on page 33 for a list of parameters and values Examples set HOST diag mode normal set HOST diag verbosity max 4 View the current values for settings Example show HOST diag HOST diag Targets Properties error_reset_level max erro...

Page 37: ... Service on page 26 2 Set the virtual keyswitch to diag so that POST runs in service mode Alternatively you can use the System target set HOST keyswitch_state diag Set keyswitch_state to Diag 3 Run POST Alternatively you can use the System target start System Are you sure you want to start System y n y Starting System Related Information POST Overview on page 33 Oracle ILOM Properties That Affect ...

Page 38: ...n about each detected fault Type Severity Description Automated response Impact Suggested action for system administrator If PSH detects a faulty component use the fmadm faulty command to display information about the fault See Check for Faults on page 38 Related Information Check for Faults on page 38 Clear a Fault on page 40 Check for Faults The fmadm faulty command displays the list of faults d...

Page 39: ...rer Oracle Corporation Name TLA PN NRM T5 1 2 Part_Number 7061001 Revision 01 Serial_Number 465769T 12445102WR Chassis Manufacturer Oracle Corporation Name SPARC T5 8 Part_Number 12345678 13 2 Serial_Number 1248DC140 Description A fault has been diagnosed by the Host Operation System Response The service required LED on the chassis and on the affected FRU may be illuminated Impact No SP impact Act...

Page 40: ...wledge tab 5 Follow the suggested actions to repair the fault 6 If necessary clear the fault manually See Clear a Fault on page 40 Related Information PSH Overview on page 38 Clear a Fault on page 40 Clear a Fault When PSH detects faults the faults are logged and displayed on the console In most cases after the fault is repaired the corrected state is detected by the server and the fault condition...

Page 41: ...sage for the faulty component faulted and taken out of service If this message appears in the output you must reset the server after you manually repair the fault faultmgmtsp exit reset System Are you sure you want to reset System y Resetting System Related Information PSH Overview on page 38 Check for Faults on page 38 Interpreting Log Files and System Messages With the OS running on the server y...

Page 42: ...syslogd automatically records various system warnings errors and faults in message files These messages can alert you to system problems such as a device that is about to fail The var adm directory contains several message files The most recent messages are in the var adm messages file After a period of time usually every week a new messages file is automatically created The original contents of t...

Page 43: ...mation Check the Message Buffer on page 42 View Log Files Oracle Solaris on page 42 View Log Files Oracle ILOM 1 View the event log show SP logs event list 2 View the audit log show SP logs audit list Related Information Check the Message Buffer on page 42 View Log Files Oracle Solaris on page 42 ...

Page 44: ...44 SPARC T5 4 Server Service Manual July 2016 ...

Page 45: ...rom the Server on page 51 8 Gain access to service components Chassis Subassembly Components on page 17 Safety Information For your protection observe the following safety precautions when setting up your equipment Follow all cautions and instructions marked on the equipment and described in the documentation shipped with your server Follow all cautions and instructions marked on the equipment and...

Page 46: ...mation Safety Information on page 45 ESD Precautions on page 46 Antistatic Wrist Strap on page 47 Antistatic Mat on page 47 ESD Precautions ESD sensitive devices such as the PCIe cards hard drives and DIMMs require special handling Caution Circuit boards and hard drives contain electronic components that are extremely sensitive to static electricity Ordinary amounts of static electricity from clot...

Page 47: ...lizes the electrical potentials between you and the server Related Information Safety Symbols on page 46 ESD Precautions on page 46 Antistatic Mat on page 47 Antistatic Mat Place ESD sensitive components such as motherboards memory and other PCBs on an antistatic mat Related Information Safety Symbols on page 46 ESD Precautions on page 46 Antistatic Wrist Strap on page 47 Tools Needed for Service ...

Page 48: ...e topic in this document about servicing that component Related Information Safety Information on page 45 Component Service Categories on page 48 Component Service Categories Replaceable components fall into these categories Hot serviceable by the customer Hot serviceable components can be removed while the server is running Hot swappable components do not require any preparation prior to servicin...

Page 49: ...page 131 Power supply Off or On Servicing Power Supplies on page 135 Fan module Off or On Servicing Fan Modules on page 145 PCIe card Off or On Servicing PCIe Cards on page 151 Rear I O module Off X Servicing the Rear I O Module on page 173 Rear chassis subassembly Off X Servicing the Rear Chassis Subassembly on page 183 You must disconnect the ower cords before accessing this component Related In...

Page 50: ...ands cd reset set show start stop Related Information Locate the Server on page 50 Locate the Server You can use the Locator LEDs to identify a particular server 1 At the Oracle ILOM prompt type set System Locator_indicator on The white Locator LEDs one on the front panel and one on the rear panel blink 2 After locating the server with the blinking Locator LED turn it off using one of the followin...

Page 51: ...ring Boot and Restart Behavior in SPARC and Netra SPARC T5 Series Servers Administration Guide Prepare to Power Off the Server 1 Notify affected users that the server will be shut down Refer to the Oracle Solaris system administration documentation for additional information 2 Save any open files and quit all running programs Refer to your application documentation for specific information for the...

Page 52: ...want to view server status or log files You also might want to run diagnostics before you shut down the server 2 Switch from the system console to the Oracle ILOM prompt by typing the Hash Period key sequence 3 At the Oracle ILOM prompt type stop System Stopping System 4 If you are powering off the server in order to add a second processor module return to Server Upgrade Process on page 57 Related...

Page 53: ...tton Emergency Shutdown Caution All applications and files are closed abruptly without saving changes File system corruption might occur Press and hold the Power button for four seconds Related Information Power Off the Server Oracle ILOM on page 52 Power Off the Server Power Button Graceful Shutdown on page 52 Disconnect Power Cords You must disconnect the power cords before accessing the followi...

Page 54: ...icing the Front I O Assembly on page 131 Servicing the Rear I O Module on page 173 Servicing the Rear Chassis Subassembly on page 183 Prevent ESD Damage Many components contained in the processor modules and main module can be damaged by ESD To protect these components from damage perform the following steps before opening these modules for service 1 Prepare an antistatic surface to set parts on d...

Page 55: ...in Module on page 97 Servicing the Storage Backplanes on page 107 Servicing the Service Processor Card on page 115 Servicing the System Configuration PROM on page 121 Servicing the System Battery on page 125 Servicing the Front I O Assembly on page 131 Servicing PCIe Cards on page 151 Servicing the Rear I O Module on page 173 Servicing the Rear Chassis Subassembly on page 183 ...

Page 56: ...56 SPARC T5 4 Server Service Manual July 2016 ...

Page 57: ...or module as part of another component s service operation Remove a Processor Module or Processor Filler Module on page 61 Install the processor module as part of another component s service operation Install a Processor Module or Processor Filler Module on page 64 Related Information Identifying Components on page 13 Processor Module Components on page 18 Detecting and Managing Faults on page 23 ...

Page 58: ...M or DIMM Filler Panel on page 77 4 Verify that you have the correct DIMMs for your server All of the DIMMs must be either 16 or 32 GB and they must match the size and capacity of the DIMMs already installed in the server DIMM Population Rules on page 69 5 Install the DIMMs Install a DIMM on page 79 6 Check the server for faults If any fault is present you must correct the fault and clear it from ...

Page 59: ...Install a Processor Module or Processor Filler Module on page 64 Verify the Processor Module on page 67 Understanding PCIe Root Complex Connections on page 151 PCIe Card Installation Order on page 155 Returning the Server to Operation on page 191 Processor Module Locations Processor modules are accessed from the front of the server In Oracle ILOM the processor modules are numbered PM0 and PM1 star...

Page 60: ...s available for use On The server is running and the processor module is functioning correctly Off The server is powered down and the processor module is in standby mode Related Information Processor Module Components on page 18 Server Upgrade Process on page 57 Determine Which Processor Module Is Faulty on page 61 Remove a Processor Module or Processor Filler Module on page 61 Install a Processor...

Page 61: ...essor Filler Module on page 61 Related Information Processor Module Components on page 18 Processor Module LEDs on page 60 Remove a Processor Module or Processor Filler Module on page 61 Install a Processor Module or Processor Filler Module on page 64 Verify the Processor Module on page 67 Remove a Processor Module or Processor Filler Module Processor modules and processor filler modules are cold ...

Page 62: ...paring for Service on page 45 2 Locate the processor module in the server that you want to remove If you are replacing a faulty processor module see Determine Which Processor Module Is Faulty on page 61 to locate a faulty processor module If you are adding a processor module remove the processor filler module in slot 1 3 Squeeze the release latches together on the two extraction levers and pull th...

Page 63: ...essor module or procssor filler module and place the module on an antistatic mat Caution Do not touch the connectors at the rear of the module 6 Determine your next step If you are replacing or installing DIMMs within the processor module see Servicing DIMMs on page 69 If you are replacing a faulty processor module populate and install the replacement processor module a Remove all of the DIMMs fro...

Page 64: ...Module on page 67 Install a Processor Module or Processor Filler Module Processor modules are cold service components that can be replaced only by qualified service personnel For the location of the processor modules see Front Panel Components on page 14 Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of serv...

Page 65: ...dule Servicing Processor Modules 65 Note A processor filler module can only be installed in slot 1 3 Bring the levers together toward the center of the module and press the levers firmly against the module to fully seat the module back into the server ...

Page 66: ...rning the Server to Operation on page 191 5 Verify the processor module functionality See Verify the Processor Module on page 67 6 If you are adding a second processor module to the server return to Server Upgrade Process on page 57 Related Information Processor Module Components on page 18 Server Upgrade Process on page 57 Processor Module LEDs on page 60 Determine Which Processor Module Is Fault...

Page 67: ... processor as disabled go to Detecting and Managing Faults on page 23 to clear the PSH detected fault from the server 2 Verify that the OK LED is lit on the processor module and that the Fault LED is not lit See Processor Module LEDs on page 60 3 Verify that the front and rear Service Required LEDs are not lit See Front Panel Controls and LEDs on page 29 and Rear Panel Controls and LEDs on page 31...

Page 68: ... Related Information Processor Module Components on page 18 Processor Module LEDs on page 60 Determine Which Processor Module Is Faulty on page 61 Remove a Processor Module or Processor Filler Module on page 61 Install a Processor Module or Processor Filler Module on page 64 ...

Page 69: ... DIMM is Faulty PSH on page 73 Determine Which DIMM Is Faulty DIMM Fault Remind Button on page 75 DIMM Configuration Errors on page 84 Replace a DIMM Remove a DIMM or DIMM Filler Panel on page 77 Install a DIMM on page 79 Verify a DIMM on page 82 DIMM Population Rules Consider the following population rules when installing upgrading or replacing DIMMs in a processor module Two DIMM capacities are ...

Page 70: ...dules Note If you are adding a second processor module to the server you must install DIMMs of the same type and capacity that are already in the existing processor modules See Server Upgrade Process on page 57 Related Information DIMM Addresses on page 70 DIMM Rank Classification on page 72 Remove a DIMM or DIMM Filler Panel on page 77 Install a DIMM on page 79 Verify a DIMM on page 82 DIMM Addre...

Page 71: ...ule is installed For example the full NAC name for the DIMM installed in the front left corner on a processor module installed at PM0 is SYS PM0 CM1 CMP BOB0 CH0 D0 Related Information Servicing Processor Modules on page 57 DIMM Population Rules on page 69 DIMM Rank Classification on page 72 DIMM Fault Handling on page 73 DIMM Configuration Errors on page 84 ...

Page 72: ...erver or to verify the architecture of any replacment or upgrade DIMMs you intend to install The following table identifies the corresponding rank classification label shipped with each DIMM Rank Type Bank Classifications Label Dual rank x4 DIMMs 16 Gbyte 2Rx4 Quad rank x4 DIMMs 32 Gbyte 4Rx4 Note All DIMMs related to each CMP must have identical rank classifications For more information see DIMM ...

Page 73: ... and half the processor threads When this offlining process occurs in normal operation you must replace the faulty DIMMs based on the fault message and enable the disabled DIMMs with the Oracle ILOM command set device requested_config_state Enabled where device is the name of the DIMM being enabled PSH technology The Oracle PSH feature uses the Fault Manager daemon fmd to watch for various kinds o...

Page 74: ...s SYS PM0 CM1 CMP BOB0 CH0 D0 Status faulted but still in service FRU Status faulty Location SYS PM0 CM1 CMP BOB0 CH0 D0 Manufacturer Samsung Name 8192MB DDR3 SDRAM DIMM Part_Number 07042208 M393B1K70DH0 YK0 Revision 04 Serial_Number 00CE0212153367DD4B Chassis Manufacturer Oracle Corporation Name SPARC T5 4 Part_Number 7021179 Serial_Number 1201CTHC01 Description Uncorrectable errors have occurred...

Page 75: ...mponents on page 18 Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components 1 Consider your first steps Familiarize yourself with DIMM configuration rules See DIMM Population Rules on page 69 Prepare the system for service See Preparing for Service on page 45 Remove the processor module containin...

Page 76: ...s that there is power available to illuminate the faulty DIMM LED once you have pressed the DIMM Fault Remind button 4 Press the DIMM Fault Remind button on the processor module This will cause DIMM Fault LED associated with the faulty DIMM to illuminate for a few minutes 5 Note the DIMM next to the illuminated DIMM Fault LED 6 Ensure that all other DIMMs are seated correctly in their slots Relate...

Page 77: ...must install filler panels in all empty DIMM slots 1 Consider your first steps Familiarize yourself with DIMM configuration rules See DIMM Population Rules on page 69 Prepare the system for service See Preparing for Service on page 45 Remove the processor module Place the processor module on an ESD protect work surface See Remove a Processor Module or Processor Filler Module on page 61 2 Remove th...

Page 78: ...at Step 4 through Step 6 for any other DIMMs or DIMM filler panels you intend to remove 8 Determine your next step If you are installing replacement DIMMs at this time go to Install a DIMM on page 79 If you are not installing replacement DIMMs at this time go to Step 9 9 Finish the installation procedure See Install the processor module See Install a Processor Module or Processor Filler Module on ...

Page 79: ...yourself with DIMM configuration rules See DIMM Population Rules on page 69 Prepare the system for service See Preparing for Service on page 45 Remove the processor module Place the processor module on an ESD protected work surface See Remove a Processor Module or Processor Filler Module on page 61 2 Consider your next steps If you are replacing a faulty DIMM ensure that you have removed the fault...

Page 80: ...e in the open position 5 Align the DIMM notch with the key in the connector Caution Ensure that the orientation is correct The DIMM might be damaged if the orientation is reversed 6 Push the DIMM into the connector until the ejector tabs lock the DIMM in place If the DIMM does not easily seat into the connector check the DIMM s orientation 7 Repeat Step 4 through Step 6 until all new DIMMs are ins...

Page 81: ...replacement DIMMs proceed to Step 10 10 Finish the installation procedure See Install the processor module See Install a Processor Module or Processor Filler Module on page 64 Return the server to operation See Returning the Server to Operation on page 191 Verify DIMM functionality See Verify a DIMM on page 82 Related Information DIMM Population Rules on page 69 DIMM Addresses on page 70 DIMM Rank...

Page 82: ...the fault is automatically cleared from the server If show faulty still displays the fault the set command will clear it set SYS MB CM1 CMP MR3 BOB0 CH0 D0 requested_config_state Enabled Set requested_config_state to Enabled 4 For a host detected fault perform the following steps to verify the new DIMM a Set the virtual keyswitch to diag so that POST will run in Service mode set System keyswitch_s...

Page 83: ...t e Return the virtual keyswitch to normal mode set System keyswitch_state normal Set ketswitch_state to normal f Switch to the system console and type the Oracle Solaris OS fmadm faulty command fmadm faulty If any faults are reported refer to the diagnostics instructions described in Check for Faults on page 38 5 Switch to the Oracle ILOM command shell 6 Run the show faulty command show faulty Ta...

Page 84: ... Determine Which DIMM is Faulty PSH on page 73 Remove a DIMM or DIMM Filler Panel on page 77 Install a DIMM on page 79 DIMM Fault Handling on page 73 DIMM Configuration Errors on page 84 DIMM Configuration Errors When the system boots system firmware checks the memory configuration against the rules described in DIMM Population Rules on page 69 If any violations of these rules are detected the fol...

Page 85: ...ndicating the type of configuration error detected To identify the DIMMs affected use the fmadm faulty command as described in Check for Faults on page 38 Related Information Check for Faults on page 38 Clear a Fault on page 40 DIMM Population Rules on page 69 DIMM Addresses on page 70 DIMM Rank Classification on page 72 DIMM Fault Handling on page 73 ...

Page 86: ...86 SPARC T5 4 Server Service Manual July 2016 ...

Page 87: ...use failure of server components These topics describe service procedures for the hard drives in the server Hard Drive Locations on page 87 Hard Drive Hot Service Capabilities on page 88 Hard Drive LEDs on page 89 Determine Which Hard Drive Is Faulty on page 90 Remove a Hard Drive on page 90 Install a Hard Drive on page 93 Verify the Hard Drive on page 94 Hard Drive Locations You can install a mix...

Page 88: ...nd inserted while the server is powered on Depending on the configuration of the data on a particular drive the drive might also be removable while the server is online However to hot service a drive while the server is online you must take the drive offline before you can safely remove it Taking a drive offline prevents any applications from accessing it and removes logical software links to it Y...

Page 89: ... Indicates that a drive can be removed during a hot service operation 2 Service Required amber Indicates that the drive has experienced a fault condition 3 OK Activity green Indicates the drive s availability for use On Read or write activity is in progress Off Drive is idle and available for use Related Information Hard Drive Hot Service Capabilities on page 88 Hard Drive Locations on page 87 Det...

Page 90: ...fy which drive needs to be replaced See Hard Drive LEDs on page 89 The amber Service Required LED is lit on the drive that needs to be replaced 3 Remove the faulty drive See Remove a Hard Drive on page 90 Related Information Hard Drive Locations on page 87 Hard Drive Hot Service Capabilities on page 88 Hard Drive LEDs on page 89 Remove a Hard Drive on page 90 Install a Hard Drive on page 93 Verify...

Page 91: ...type the cfgadm al command to list all drives in the device tree including drives that are not configured cfgadm al This command lists dynamically reconfigurable hardware resources and shows their operational status In this case look for the status of the drive you plan to remove This information is listed in the Occupant column Example Ap_id Type Receptacle Occupant Condition c2 scsi sas connecte...

Page 92: ...our situation c Verify that the blue Ready to Remove LED on the drive is lit 4 Press the drive release button to unlock the drive 5 Pull on the latch to remove the drive from the server Caution The latch is not an ejector Do not force the latch too far to the right Doing so can damage the latch 6 Install the replacement drive or a filler tray See Install a Hard Drive on page 93 ...

Page 93: ...are sensitive to electrostatic discharge This discharge can cause failure of server components 1 Align the replacement drive to the drive slot and slide the drive in until it is seated Drives are physically addressed according to the slot in which they are installed If you are replacing a drive install the replacement drive in the same slot as the drive that was removed See Hard Drive Locations on...

Page 94: ...ht need to perform administrative tasks to reinstall software before the server can boot Refer to the Oracle Solaris OS administration documentation for more information 3 At the Oracle Solaris prompt type the cfgadm al command to list all drives in the device tree including any drives that are not configured cfgadm al This command helps you identify the drive you installed Example Ap_id Type Rece...

Page 95: ...3 w5000cca00a772bd1 0 disk path connected configured unknown c4 scsi sas connected configured unknown c4 w5000cca00a59b0a9 0 disk path connected configured unknown 7 Perform one of the following tasks based on your verification results If the previous steps did not verify the drive see Diagnostics Process on page 23 If the previous steps indicate that the drive is functioning properly perform the ...

Page 96: ...96 SPARC T5 4 Server Service Manual July 2016 ...

Page 97: ...termine if the main module is faulty Main Module LEDs on page 98 2 Prepare the server for service Preparing for Service on page 45 3 Remove the main module Remove the Main Module on page 99 4 Service main module components Servicing the Storage Backplanes on page 107 Servicing the Service Processor Card on page 115 Servicing the System Configuration PROM on page 121 Servicing the System Battery on...

Page 98: ...ates these conditions Off System is not running in its normal state System power might be off The SP might be running Steady on System is powered on and is running in its normal operating state No service actions are required Fast blink System is running in standby mode and can be quickly returned to full function Slow blink A normal but transitory activity is taking place Slow blinking might indi...

Page 99: ...dule fault Related Information Main Module LEDs on page 98 Remove the Main Module on page 99 Install the Main Module on page 102 Remove the Main Module 1 Optional If you are replacing a faulty main module you must back up ILOM configuration settings a Configure the SER MGT port to enable the configuration parameters to be uploaded Refer to the ILOM documentation for network configuration instructi...

Page 100: ...SPARC T5 4 Server Service Manual July 2016 See Front Panel Components on page 14 5 Squeeze the release latches together on the two extraction levers and pull the extraction levers out to disengage the main module from the server ...

Page 101: ...ter of the main module This will keep the levers from being damaged when the main module is outside the server Caution Due to the weight of the main module the following step requires two people to perform Do not attempt to lift the main module alone 8 Remove the main module completely from the server 9 Remove the cover from the main module ...

Page 102: ...t inside the main module use one of the following links Servicing the Service Processor Card on page 115 Servicing the System Battery on page 125 Servicing the System Configuration PROM on page 121 Servicing the Front I O Assembly on page 131 Servicing the Storage Backplanes on page 107 Related Information Main Module Components on page 19 Main Module LEDs on page 98 Install the Main Module on pag...

Page 103: ...e 103 3 Insert the main module into its slot in the server until the levers begin to engage 4 Press the levers back together toward the center of the module then press the levers firmly against the module to fully seat the module back into the server ...

Page 104: ...the main module with a new one connect a terminal or a terminal emulator PC or workstation to the SER MGT port The following message is delivered over the serial management port Unrecognized Chassis This module is installed in an unknown or unsupported chassis You must upgrade the firmware to a newer version that supports this chassis 7 Download the system firmware a Configure the SER MGT port to ...

Page 105: ...Main Module LEDs on page 98 Remove the Main Module on page 99 Verify the Main Module 1 Verify that the main module Service Required LED is not lit See Main Module LEDs on page 98 2 Verify that the front and rear system Service Required LEDs are not lit See Front Panel Controls and LEDs on page 29 and Rear Panel Controls and LEDs on page 31 3 Perform one of the following tasks based on your verific...

Page 106: ...106 SPARC T5 4 Server Service Manual July 2016 ...

Page 107: ...This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components 1 Power off the server See Removing Power From the Server on page 51 2 Remove all the hard drives from the front of the server for the storage backplane that you want to replace Note the locations of the drives before removing them so that you can i...

Page 108: ...1 Storage backplane for drives 4 7 SAS_BP1 2 Storage backplane for drives 0 3 SAS_BP0 6 Disconnect the two storage backplane cables from the storage backplane that you want to replace a Lift up on the connectors that secure the data cable to the storage backplane and the motherboard and remove the data cable from the main module ...

Page 109: ... Backplanes 109 b Lift up on the connectors that secure the power cable to the storage backplane and the motherboard and remove the power cable from the main module 1 Data cable storage backplane connection 2 Power cable storage backplane connection ...

Page 110: ...6 7 Lift up on the plastic retaining panel for the storage backplane that you want to remove to disengage the plastic panel from the top of the hard drive assembly 8 Push the plastic panel toward the rear of the main module and remove the plastic panel from the main module ...

Page 111: ...ove it from the main module Related Information Install a Storage Backplane on page 111 Install a Storage Backplane Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components 1 Take the necessary ESD precautions See Prevent ESD Damage on page 54 2 Position the storage backplane in the main module ...

Page 112: ...Install a Storage Backplane 112 SPARC T5 4 Server Service Manual July 2016 3 Lower the storage backplane into place ...

Page 113: ...o notches in the panel slide underneath the two metal mounting studs on the hard drive assembly 5 Press on the press point on the retaining panel to secure it to the top of the hard drive assembly 6 Connect the two storage backplane cables to the storage backplane and the motherboard a Connect the data cable to the storage backplane and the motherboard ...

Page 114: ... 7 Insert the main module back into the server See Install the Main Module on page 102 8 Install the hard drives that you removed back into the main module Refer to the notes that you took when removing the hard drives to install them back into their original slots See Install a Hard Drive on page 93 9 Power on the server See Returning the Server to Operation on page 191 Related Information Remove...

Page 115: ...stall the Main Module on page 102 5 Verify the replacement service processor card Verify the Service Processor Card on page 120 Determine if the Service Processor Card Is Faulty The following LEDs are illuminated when a SP fault is detected System Service Required LEDs on the front panel and rear I O module Server SP LED on the main module or rear I O module 1 Determine if the System Service Requi...

Page 116: ...laced by a customer Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components 1 Back up the SP configuration information before removing the service processor card At the Oracle ILOM prompt type cd SP config dump destination uri target where The acceptable values for uri are tftp ftp sftp scp http ...

Page 117: ...dule from the server See Remove the Main Module on page 99 4 Locate the service processor card on the main module See Main Module Components on page 19 5 Grasp the service processor card by the two grasp points and lift up to disengage the service processor card from the connectors on the motherboard 6 Lift the service processor card up and away from the motherboard ...

Page 118: ...n page 54 2 Lower the side of the service processor card with the Align Tab sticker down on the service processor tab on the motherboard 3 Lower the other side of the service processor card down and press down on the service processor card to seat it into the connectors on the motherboard 4 Install the main module back into the server See Install the Main Module on page 102 5 Connect a terminal or...

Page 119: ...racle ILOM documentation for network configuration instructions b Download the system firmware Follow the firmware download instructions in the Oracle ILOM documentation Note You can load any supported system firmware version including the firmware revision that had been installed prior to the replacement of the service processor card However Oracle strongly recommends installing the newest versio...

Page 120: ...Verify that the SP LED on the main module or rear I O module is lit green See Main Module LEDs on page 98 or Rear I O Module LEDs on page 173 2 Verify that the front and rear Service Required LEDs are not lit See Interpreting LEDs on page 28 3 Perform one of the following tasks based on your verification results If the previous steps did not clear the fault see Diagnostics Process on page 23 If th...

Page 121: ... page 99 2 Replace the system configuration PROM Remove the System Configuration PROM on page 121 Install the System Configuration PROM on page 122 3 Install the main module Install the Main Module on page 102 4 Verify the system configuration PROM Verify the System Configuration PROM on page 123 Remove the System Configuration PROM The system configuration PROM is a cold service component that ca...

Page 122: ...asp the system configuration PROM and lift it up to remove it from the main module Related Information Install the System Configuration PROM on page 122 Install the System Configuration PROM Before beginning this procedure ensure that you are familiar with the cautions and safety instructions described in Safety Information on page 45 Caution This procedure requires that you handle components that...

Page 123: ...e on page 102 4 Return the server to operation See Returning the Server to Operation on page 191 Related Information Remove the System Configuration PROM on page 121 Verify the System Configuration PROM on page 123 Verify the System Configuration PROM 1 Verify that the banner display includes an Ethernet address and a Host ID value The Ethernet address and Host ID values are read from the system c...

Page 124: ... configuration PROM Use the Oracle ILOM show command to display the MAC address show HOST macaddress HOST Properties macaddress Use Oracle Solaris OS commands to display the hostid and Ethernet address hostid 8534299c ifconfig a lo0 flags 2001000849 UP LOOPBACK RUNNING MULTICAST IPv4 VIRTUAL mtu 8232 index 1 inet 127 0 0 1 netmask ff000000 igb0 flags 201004843 UP BROADCAST RUNNING MULTICAST IPv4 m...

Page 125: ...em Battery on page 125 Install the System Battery on page 126 Verify the System Battery on page 128 Remove the System Battery The system battery is a cold service component that can be replaced by a customer Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components 1 Take the necessary ESD precauti...

Page 126: ...ier Related Information Install the System Battery on page 126 Verify the System Battery on page 128 Install the System Battery Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components 1 Take the necessary ESD precautions See Prevent ESD Damage on page 54 ...

Page 127: ...work Otherwise proceed to the next step 5 If the service processor is not configured to use NTP you must reset the Oracle ILOM clock using the Oracle ILOM CLI or the web interface For instructions see the Oracle Integrated Lights Out Manager ILOM 3 2 1 Documentation Collection 6 If the service processor is not configured to use NTP use the Oracle ILOM clock command to set the day and time The foll...

Page 128: ...ify the System Battery 1 Run show SYS MB V_BAT to check the status of the system battery In the output the SYS MB V_BAT status should be OK as in the following example show SYS MB V_BAT Targets Properties type Voltage ipmi_name MB V_VBAT class Threshold Sensor value 3 140 Volts upper_nonrecov_threshold N A upper_critical_threshold N A upper_noncritical_threshold N A lower_noncritical_threshold 2 7...

Page 129: ...Verify the System Battery Servicing the System Battery 129 Install the System Battery on page 126 ...

Page 130: ...130 SPARC T5 4 Server Service Manual July 2016 ...

Page 131: ...ly Reference The front I O assembly consists of the following components Two circuit boards FIO and VGA boards A ribbon cable connecting the front I O assembly to the motherboard Related Information Remove the Front I O Assembly on page 131 Install the Front I O Assembly on page 133 Remove the Front I O Assembly The front I O assembly is a cold service component that can be replaced only by author...

Page 132: ...cate the front I O assembly on the main module See Main Module Components on page 19 4 Disconnect the ribbon cable that connects the assembly to the motherboard and remove the front I O assembly a Pull the cable free of the connector on the motherboard panel 1 b Push the cable connector aside to access the captive screw that secures the assembly to the motherboard panel 2 Loosen the screw to relea...

Page 133: ... the main module and then remove the front I O assembly from the main module panel 3 Related Information Front I O Assembly Reference on page 131 Install the Front I O Assembly on page 133 Install the Front I O Assembly Caution This procedure requires that you handle components that are sensitive to static discharge Static discharges can cause the components to fail 1 Take the necessary ESD precau...

Page 134: ...sition with the ports inserted into the port holes in the front of the main module panel 1 b Lower the rear of the front I O assembly so that the captive screw is aligned with the screw hole on the motherboard and tighten the screw panel 2 c Connect the assembly s ribbon cable to the connector on the motherboard panel 3 Related Information Front I O Assembly Reference on page 131 Remove the Front ...

Page 135: ...ocedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components These topics describe service procedures for the power supplies in the server Power Supply and AC Power Connectors on page 135 Power Supply and AC Power Connector LEDs on page 137 Determine Which Power Supply Is Faulty on page 138 Remove a Power Supply on ...

Page 136: ...S0 2 Power Supply 1 PS1 Power cords are accessed from the rear of the server 1 Connector for Power Supply 1 PS1 2 Connector for Power Supply 0 PS0 Related Information Power Supply and AC Power Connector LEDs on page 137 Determine Which Power Supply Is Faulty on page 138 Remove a Power Supply on page 138 ...

Page 137: ...lty Note The front and rear panel Service Required LEDs are also illuminated if the server detects a power supply fault 2 OK green Lights when the power supply DC voltage from the PSU to the server is within tolerance 3 AC Present green AC Lights when AC voltage is applied to the power supply Each AC power connector has a single LED that is located on the rear I O module See Interpreting LEDs on p...

Page 138: ...power supply needs to be replaced See Power Supply and AC Power Connector LEDs on page 137 The amber Service Required LED is lit on the power supply that needs to be replaced 3 Remove the faulty power supply See Remove a Power Supply on page 138 Related Information Power Supply and AC Power Connectors on page 135 Power Supply and AC Power Connector LEDs on page 137 Remove a Power Supply on page 13...

Page 139: ...r supply 2 Go to the rear of the server and locate the AC power connector at the rear of the server that supplies power to the faulty power supply See Power Supply and AC Power Connectors on page 135 3 Disconnect that power cord 4 Go to the front of the server and on the power supply to be removed squeeze the release latches together then pull the extraction lever toward you to disengage the power...

Page 140: ...r LEDs on page 137 Determine Which Power Supply Is Faulty on page 138 Install a Power Supply on page 140 Verify a Power Supply on page 143 Install a Power Supply The power supply is a hot service component that can be replaced by a customer Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components ...

Page 141: ...Install a Power Supply Servicing Power Supplies 141 Verify that the power supply is oriented as shown in the following figure 2 Slide the power supply into the chassis ...

Page 142: ...erver standby power initializes the service processor Depending on the server s OpenBoot PROM settings the host server might automatically boot or you might need to boot it manually 5 Verify the power supply See Verify a Power Supply on page 143 Related Information Power Supply and AC Power Connectors on page 135 Power Supply and AC Power Connector LEDs on page 137 Determine Which Power Supply Is ...

Page 143: ...rform one of the following tasks based on your verification results If the previous steps did not clear the fault see Diagnostics Process on page 23 If Step 1 and Step 2 indicate that no faults have been detected then the power supply has been replaced successfully No further action is required Related Information Power Supply and AC Power Connectors on page 135 Power Supply and AC Power Connector...

Page 144: ...144 SPARC T5 4 Server Service Manual July 2016 ...

Page 145: ...rver will power down to keep from overheating You can perform a hot service on a fan module only when four or five fan modules are operational These topics describe service procedures for the fan modules in the server Fan Module Locations on page 145 Determine Which Fan Module Is Faulty on page 146 Remove a Fan Module on page 147 Install a Fan Module on page 149 Verify a Fan Module on page 150 Fan...

Page 146: ...e 149 Verify a Fan Module on page 150 Fan Module LED Each fan module has a single Service Required LED Determine Which Fan Module Is Faulty The following LEDs are illuminated when a fan module fault is detected System Service Required LEDs on the front panel and rear I O module Server Fan Fail LED on the front panel Service Required LED on the faulty fan module 1 Determine if the System Service Re...

Page 147: ... customer Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components 1 Locate the faulty fan module that you want to remove from the server See Rear Panel Components on page 15 for the locations of the fan modules in the server See Determine Which Fan Module Is Faulty on page 146 to locate a faulty ...

Page 148: ...ess the green button to disengage the fan module from the chassis 4 Pull the fan module out of the server Related Information Fan Module Locations on page 145 Determine Which Fan Module Is Faulty on page 146 Install a Fan Module on page 149 Verify a Fan Module on page 150 ...

Page 149: ... Insert the fan module into the empty fan module slot The fan snaps into position with an audible click 2 Power on the server if necessary If you had to power off the server before removing and installing a new fan module see Returning the Server to Operation on page 191 to power on the server again 3 Verify the fan module functionality See Verify a Fan Module on page 150 Related Information Deter...

Page 150: ...rols and LEDs on page 31 If these conditions are met continue to Step 5 If these conditions are not met perform the actions described in Diagnostics Process on page 23 3 Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 26 4 Start the faultmgmt shell start SP faultmgmt shell Are you sure you want to start the faultmgmt shell y n y faultmgmtsp 5 Use the fmadm faulty command to check f...

Page 151: ...Carrier on page 168 Verify the PCIe Card on page 170 Understanding PCIe Root Complex Connections All 16 PCIe slots support PCIe cards with the following characteristics Hot plug low profile adapters x8 Gen1 x8 Gen2 and x8 Gen3 cards Note If you install a 16 lane card in any slot electrical support is provided to the card s lowest 8 lanes A root complex is the CMP circuitry that provides the base t...

Page 152: ...Complex Connections Single Processor Configurations In single processor configurations the server configures the PCIe root complex paths to this topology This diagram illustrates the root complex connections between the two CPUs in PM0 and 14 PCIe I O slots Note I O slots 15 and 16 cannot be used in single processor configurations CPUs 0 and 1 support four root complex fabrics which connect to the...

Page 153: ...0 pci 8 0 1 2 9 pci 380 pci 1 pci 0 pci a 0 1 2 10 pci 380 pci 1 pci 0 pci 4 0 1 3 11 pci 3c0 pci 1 pci 0 pci e 0 1 3 12 pci 3c0 pci 1 pci 0 pci 8 0 1 3 13 pci 3c0 pci 1 pci 0 pci a 0 1 3 14 pci 3c0 pci 1 pci 0 pci 4 N A N A N A 15 No root complex for this slot N A N A N A 16 No root complex for this slot Related Information Understanding PCIe Root Complex Connections on page 151 PCIe Root Complex...

Page 154: ...wn in the diagram correspond to the pci values reported in the showdevs command output For example PM CPU Switch I O Slot Root Complex Path 0 0 0 1 pci 300 pci 1 pci 0 pci 6 0 0 0 2 pci 300 pci 1 pci 0 pci c 0 0 1 3 pci 340 pci 1 pci 0 pci 6 0 0 1 4 pci 340 pci 1 pci 0 pci c 1 2 1 5 pci 400 pci 1 pci 0 pci e 1 2 1 6 pci 400 pci 1 pci 0 pci 8 1 2 2 7 pci 440 pci 1 pci 0 pci e 1 2 2 8 pci 440 pci 1 ...

Page 155: ...on page 155 PCIe Card Installation Order Note Some PCIe cards are restricted to specific I O slots to meet system cooling requirements Other I O cards provide better performance when installed in particular slots For more information about PCIe slot restrictions for specific devices see I O Slot Restrictions in SPARC T5 4 Server Product Notes For optimal load balancing install the PCIe cards in th...

Page 156: ...PCIe cards evenly across available root complexes If you are reviewing PCIe installation order after adding a second processor module return to Server Upgrade Process on page 57 Related Information Server Upgrade Process on page 57 Understanding PCIe Root Complex Connections on page 151 Determine Which PCIe Card Is Faulty on page 157 Remove a PCIe Card on page 161 Install a PCIe Card on page 163 V...

Page 157: ...No fault is detected On A fault has been detected Note If a PCIe card fails and you do not have a replacement available leave the failed PCIe card and carrier installed to ensure proper airflow in the server Related Information Understanding PCIe Root Complex Connections on page 151 PCIe Card Installation Order on page 155 Determine Which PCIe Card Is Faulty on page 157 Remove a PCIe Card Carrier ...

Page 158: ...ers The removal steps are the same for both carrier types This topic includes illustrations only for the single wide carrier Note If you are installing a PCIe card that requires a double wide carrier you must remove two adjacent PCIe card carriers Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server comp...

Page 159: ... including PCIe cards hotplug list cv This command lists dynamically reconfigurable hardware resources and shows their operational status In this case look for the status of the PCIe card you plan to remove This information is listed in the State column For example hotplug list cv Connection State Description ______________________________________________________________________________ PCIE1 EMPT...

Page 160: ...This command lists dynamically reconfigurable hardware resources and shows their operational status In this case look for the status of the PCIe card you plan to remove This information is listed in the Occupant column For example Ap_id Type Receptacle Occupant Condition PCI EM0 sas hp connected configured ok PCI EM1 sas hp connected configured ok b Take the PCIe card offline cfgadm c disconnect A...

Page 161: ...page 157 Remove a PCIe Card on page 161 Remove a PCIe Carrier Extension on page 165 Install a PCIe Carrier Extension on page 167 Install a PCIe Card Carrier on page 168 Remove a PCIe Card Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components 1 Ensure that you have already taken antistatic measu...

Page 162: ...Remove a PCIe Card 162 SPARC T5 4 Server Service Manual July 2016 2 Unlatch and open the PCIe card carrier top cover ...

Page 163: ...ng Related Information Install a PCIe Card on page 163 Install a PCIe Card Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components 1 Determine your first step a If you are installing a new PCIe card and need an empty PCIe card carrier see Remove a PCIe Card Carrier on page 158 ...

Page 164: ...rier Extension on page 167 2 Remove the PCIe card from its packaging 3 Insert the PCIe card into the PCIe card carrier until the bottom connector is firmly seated in the PCIe card carrier s connector Caution Do not twist or turn the PCIe card as you insert it into the PCIe card carrier Ensure that the PCIe card s connector is fully seated in the PCIe card carrier s slot and that the notch in the P...

Page 165: ...all a PCIe Carrier Extension on page 167 Install a PCIe Card Carrier on page 168 Verify the PCIe Card on page 170 Remove a PCIe Carrier Extension Normally it is not necessary to remove a carrier extension when you are replacing the PCIe card However if you are reconfiguring the server to remove a double wide carrier use the steps in this procedure 1 Remove the PCIe carrier and carrier extension fr...

Page 166: ...pen and swing the carrier extension away from the main carrier 4 Remove the PCIe card from the main carrier See Remove a PCIe Card on page 161 5 Determine your next step If you are installing a new PCIe card see Install a PCIe Card on page 163 If you are not installing a PCIe card proceed to Step 6 6 Set the carrier extension aside and install two PCIe card carriers in the slots that were occupied...

Page 167: ...rd carrier extension The carrier extension provides the additional airflow that is required for proper cooling 1 Ensure that you have removed two adjacent PCIe card carriers from the server See Remove a PCIe Card Carrier on page 158 Retain the extra card carrier in a suitable storage space in case you want to remove the carrier extension later Note To ensure proper system cooling all PCIe slots mu...

Page 168: ...nformation Install a PCIe Card on page 163 Install a PCIe Card Carrier on page 168 Install a PCIe Card Carrier Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components Note Installing PCIe card carriers while the server is at the OpenBoot prompt is not supported The server must either be powered o...

Page 169: ...CIe carrier handle Rotate the handle up until it latches into place 3 Connect the cables to the PCIe card 4 Determine your next step If you replaced or installed a PCIe card in a server that is running that is if you hot serviced the PCIe card go to Step 5 If you replaced or installed a PCIe card in a powered down server power on the server using the instructions provided in Returning the Server t...

Page 170: ...page 170 Verify the PCIe Card 1 Verify that the PCIe card carrier Fault LED is not illuminated 2 Verify that the System Service Required LEDs on the front panel and rear I O module are not illuminated See Interpreting LEDs on page 28 3 Verify that the System PCIe Fault LED on the front panel is not illuminated See Front Panel Controls and LEDs on page 29 4 Perform one of the following tasks based ...

Page 171: ...CIE1 EMPTY PCIe Native PCIE7 ENABLED PCIe Native Device Usage ___________________________________________________________________________ SUNW qlc 0 fp disk fp 0 0 SUNW qlc 0 1 fp disk fp 0 0 PCIE13 EMPTY PCIe Native PCIE15 EMPTY PCIe Native Related Information Determine Which PCIe Card Is Faulty on page 157 Remove a PCIe Card on page 161 Install a PCIe Card on page 163 ...

Page 172: ...172 SPARC T5 4 Server Service Manual July 2016 ...

Page 173: ...re servicing this component See Disconnect Power Cords on page 53 These topics describe service procedures for the rear I O module in the server Rear I O Module LEDs on page 173 Determine if the Rear I O Module Is Faulty on page 175 Remove the Rear I O Module on page 176 Install the Rear I O Module on page 178 Verify the Rear I O Module on page 180 Rear I O Module LEDs The LEDs on the rear I O mod...

Page 174: ...activity green Indicates the following conditions On A link is established Blinking Transfer activity is present on the link Off No link is established 5 NET speed amber green Indicates the following conditions Green on The link is operating as a 100 Mbps connection Off There is no link 6 Unused This LED has no function 6 AC0 connector LED Indicates the state of the AC connector Green indicates th...

Page 175: ...ormal but transitory activity is taking place Slow blinking might indicate that server diagnostics are running or the server is booting 10 Service Processor LED SP Indicates the following conditions Off The AC power might have been connected to the power supplies Steady on green The SP is running in its normal operating state No service actions are required Blink green The SP is initializing the O...

Page 176: ...is a cold service component that can be replaced by a customer 1 Take the necessary ESD precautions See Prevent ESD Damage on page 54 2 Locate the failed rear I O module See Rear Panel Components on page 15 for the location of the rear I O module in the server See Determine if the Rear I O Module Is Faulty on page 175 to verify that the rear I O module has failed 3 Power off the server See Removin...

Page 177: ...Remove the Rear I O Module Servicing the Rear I O Module 177 6 Press the green buttons on the rear I O module ejection levers and spread the levers open to eject the rear I O module ...

Page 178: ...e it Related Information Preparing for Service on page 45 Rear I O Module LEDs on page 173 Determine if the Rear I O Module Is Faulty on page 175 Install the Rear I O Module on page 178 Verify the Rear I O Module on page 180 Install the Rear I O Module 1 Take the necessary ESD precautions See Prevent ESD Damage on page 54 ...

Page 179: ...he Rear I O Module 179 2 With the levers in the extended position insert the rear I O module into the slot at the rear of the server 3 Close the extraction levers until they click into place to fully seat the rear I O module into the server ...

Page 180: ... on page 176 Verify the Rear I O Module on page 180 Returning the Server to Operation on page 191 Verify the Rear I O Module 1 Ensure that you have completed the following Applied power to the server See Connect Power Cords on page 191 Started the system See Power On the Server Oracle ILOM on page 192 2 Verify that the System Service Required LED on the rear I O module is not lit See Rear I O Modu...

Page 181: ...lts were detected then the rear I O module has been replaced successfully No further action is required Related Information Detecting and Managing Faults on page 23 Rear I O Module LEDs on page 173 Determine if the Rear I O Module Is Faulty on page 175 Remove the Rear I O Module on page 176 Install the Rear I O Module on page 178 ...

Page 182: ...182 SPARC T5 4 Server Service Manual July 2016 ...

Page 183: ...the power cords before servicing this component See Disconnect Power Cords on page 53 Rear Chassis Subassembly Components on page 183 Remove the Rear Chassis Subassembly on page 184 Install the Rear Chassis Subassembly on page 187 Verify the Rear Chassis Subassembly on page 188 Related Information Identifying Components on page 13 Detecting and Managing Faults on page 23 Preparing for Service on p...

Page 184: ... Related Information Remove the Rear Chassis Subassembly on page 184 Install the Rear Chassis Subassembly on page 187 Remove the Rear Chassis Subassembly 1 Verify that the rear chassis subassembly needs to be replaced Use the server software to determine if the rear chassis subassembly needs to be replaced See Detecting and Managing Faults on page 23 for more information ...

Page 185: ...emove the Main Module on page 99 Both power supplies see Remove a Power Supply on page 138 5 Go to the rear of the server and remove the following components All five fan modules see Remove a Fan Module on page 147 All PCIe carriers or PCIe filler panels see Remove a PCIe Card Carrier on page 158 Make note of the slots for each carrier or filler panel so that you can install them into the same slo...

Page 186: ...r green mounting screws for the rear chassis subassembly 7 Using a Phillips screwdriver loosen the five screws that secure the rear chassis subassembly to the system 8 Slide the rear chassis subassembly out and away from the server Related Information Install the Rear Chassis Subassembly on page 187 ...

Page 187: ...ighten the four green screws to secure the rear chassis subassembly in the server Tighten the screws in the following order a Lower right screw b Upper left screw c Upper right screw d Lower left screw 3 Remove the connector covers from the replacement rear chassis subassembly 4 Install the following components back into the rear of the server All five fan modules see Install a Fan Module on page ...

Page 188: ...oth power supplies see Install a Power Supply on page 140 6 Connect the power cords See Connect Power Cords on page 191 7 Power on the server See Returning the Server to Operation on page 191 8 Verify the rear chassis subassembly See Verify the Rear Chassis Subassembly on page 188 Related Information Remove the Rear Chassis Subassembly on page 184 Returning the Server to Operation on page 191 Veri...

Page 189: ... If a fault was detected see Diagnostics Process on page 23 If no faults were detected then the rear chassis subassembly has been replaced successfully No further action is required Related Information Detecting and Managing Faults on page 23 Rear I O Module LEDs on page 173 Remove the Rear Chassis Subassembly on page 184 Install the Rear Chassis Subassembly on page 187 ...

Page 190: ...190 SPARC T5 4 Server Service Manual July 2016 ...

Page 191: ...formation Identifying Components on page 13 Detecting and Managing Faults on page 23 Preparing for Service on page 45 Configuring Boot and Restart Behavior in SPARC and Netra SPARC T5 Series Servers Administration Guide Oracle ILOM Documentation Library http www oracle com goto ILOM docs Connect Power Cords Note Standby power is applied as soon as the power cords are connected Depending on how the...

Page 192: ...unning before you issue the start System command 1 Check the server power state Type show System power_state System Properties power_state Off 2 If the server is powered off power on the server Type start System Starting System 3 Optional To view server boot output start a host console stream Type start HOST console 4 If you are adding a second processor module return to Server Upgrade Process on ...

Page 193: ...ule BMC Baseboard management controller BOB Memory buffer on board C chassis For servers refers to the server enclosure For server modules refers to the modular system enclosure CMA Cable management assembly CMM Chassis monitoring module server modules only The CMM is the service processor in the modular system that contains server modules Oracle ILOM runs on the CMM providing lights out managemen...

Page 194: ...NEMs See NEM FRU Field replaceable unit H HBA Host bus adapter host The part of the server or server module with the CPU and other hardware that runs the Oracle Solaris OS and other applications The term host is used to distinguish the primary computer from the SP See SP hot pluggable Describes a component that can be replaced with power applied but the component must be prepared for removal hot s...

Page 195: ...nly The modular system provides Oracle ILOM through its CMM MSGID Message identifier N name space Top level Oracle ILOM target NEBS Network Equipment Building System Netra products only NEM Network express module server modules only NEMs provide Ethernet and SAS connectivity to storage modules NET MGT Network management port An Ethernet port on the server SP the server module SP and the CMM NIC Ne...

Page 196: ... features such as Gigabit Ethernet and Fibre Channel POST Power on self test PROM Programmable read only memory PSH Predictive self healing R REM RAID expansion module server modules only Sometimes referred to as an HBA See HBA Supports the creation of RAID volumes on drives S SAS Serial attached SCSI SCC System configuration chip SER MGT Serial management port A serial port on the server SP the s...

Page 197: ... Telecommunications Industry Association Netra products only Tma Maximum ambient temperature U U S NEC United States National Electrical Code UCP Universal connector port UI User interface UL Underwriters Laboratory Inc UTC Coordinated Universal Time UUID Universal unique identifier W WWN World wide name A unique number that identifies a SAS target ...

Page 198: ...198 SPARC T5 4 Server Service Manual July 2016 ...

Page 199: ...w POST runs 35 customer replaceable components CRUs 48 D diag_level parameter 33 diag_mode parameter 33 diag_trigger parameter 33 diag_verbosity parameter 33 DIMMs addresses 70 classification labels 72 configuration errors 84 configuration reference 69 fault handling 73 installing 79 locating 18 locating faulty using DIMM Fault Remind button 75 using Oracle ILOM 73 using PSH 73 NAC names 70 removi...

Page 200: ...ge backplanes 111 system battery 126 system configuration PROM 122 K Knowledge Base 38 Knowledge Base articles 38 L LEDs AC power connectors 137 front panel 29 hard drives 89 NET Link and Activity 31 Net Management Link and Activity 31 Net Management Speed 31 NET Speed 31 PCIe carriers 156 power supplies 137 processor modules 60 rear I O module 31 173 SP 31 System Locator 29 31 System Overtemp 29 ...

Page 201: ... 161 verifying 170 PCIe carrier extension installing 167 needed for system cooling by some PCIe cards 168 removing 165 PCIe carriers installing 168 LEDs 156 locating 15 removing 158 PCIe root complex connections 151 PCIe root complex topology PM1 failure 152 PCIe slots disabled after PM1 failure 155 PM1 failure PCIe root complex topology 152 PM1 failure PCIe slots disabled after 155 POST configura...

Page 202: ...25 system configuration PROM 121 running POST in Diag Mode 37 S safety information and symbols 45 server connecting power cords 191 locating 50 powering off emergency shutdown 53 gracefully with power button 52 using service processor command 52 powering on usingstart System command 192 service categories 48 53 service processor card installing 118 locating faulty 115 removing 116 verifying 120 SP...

Page 203: ...e 42 verifying DIMMs 82 fan modules 150 hard drives 94 main module 105 PCIe cards 170 power supplies 143 processor modules 67 rear I O module 180 188 service processor card 120 system battery 128 system configuration PROM 123 viewing system message log files 42 ...

Page 204: ...204 SPARC T5 4 Server Service Manual July 2016 ...