Oracle Netra SPARC T4-1B, Service Manual

Page 1: ...Netra SPARC T4 1B Server Module Service Manual Part No E25046 02 June 2012 ...

Page 2: ...s sont concédés sous licence et soumis à des restrictions d utilisation et de divulgation Sauf disposition de votre contrat de licence ou de la loi vous ne pouvez pas copier reproduire traduire diffuser modifier breveter transmettre distribuer exposer exécuter publier ou afficher le logiciel même partiellement sous quelque forme et par quelque procédé que ce soit Par ailleurs il est interdit de pr...

Page 3: ...Oracle ILOM 11 Oracle ILOM Troubleshooting Overview 12 Fault Management 12 Fault Clearing 13 Oracle Solaris Fault Manager Commands in Oracle ILOM 14 Drive Faults 14 Access the SP Oracle ILOM 15 Display FRU Information show Command 17 Check for Faults show faulty Command 18 Check for Faults fmadm faulty Command 20 Clear Faults clear_fault_action Property 21 Service Related Oracle ILOM Commands 22 I...

Page 4: ...T Overview 29 Oracle ILOM Properties That Affect POST Behavior 30 Configure POST 33 Run POST With Maximum Testing 35 Interpret POST Fault Messages 37 Clear POST Detected Faults 37 POST Output Reference 39 Managing Faults PSH 41 PSH Overview 41 Check for PSH Detected Faults 42 Clear PSH Detected Faults 44 Managing Components ASR 45 ASR Overview 46 Display System Components 47 Disable System Compone...

Page 5: ...ut Down the OS and Host Power Button Graceful 59 Shut Down the OS and Host Emergency Shutdown 59 Set the Server Module to a Ready to Remove State 60 Remove the Server Module From the Modular System 61 Remove the Cover 63 Servicing Drives 65 Drive Configuration 66 Drive LEDs 67 Drive Hot Plugging Guidelines 68 Locate a Faulty Drive 68 Remove a Drive 69 Remove a Drive Filler 70 Install a Drive 71 In...

Page 6: ... REM 89 Remove a REM 89 Install a REM 90 Servicing the FEM 93 Remove a FEM 93 Install a FEM 94 Servicing the SP Card 97 Remove the SP Card 97 Install the SP Card 98 Servicing the ID PROM 101 Remove the ID PROM 101 Install the ID PROM 102 Verify the ID PROM 103 Servicing a USB Flash Drive 105 Remove a USB Flash Drive 105 Install a USB Flash Drive 106 Servicing the Battery 109 Replace the Battery 10...

Page 7: ...ther Enclosure Assembly 113 Returning the Server Module to Operation 117 Replace the Cover 117 Install the Server Module Into the Modular System 118 Power On the Host Oracle ILOM 120 Power On the Host Power Button 120 Glossary 123 Index 129 ...

Page 8: ...viii Netra SPARC T4 1B Server Module Service Manual June 2012 ...

Page 9: ...ssibility on page x Related Documentation Documentation Links All Oracle products http www oracle com documentation Netra SPARC T4 1B server module http www oracle com pls topic lookup ctx Netra_SPARCT4 1B Sun Netra 6000 modular system http www oracle com pls topic lookup ctx Netra6000 FEMs Network Interface Cards http www oracle com technetwork documentation oracle net sec hw 190016 html REMs Hos...

Page 10: ... king 190072 html Oracle Solaris OS and other system software http www oracle com technetwork indexes documentation sys_sw Oracle VTS software http www oracle com pls topic lookup ctx OracleVTS7 0 Description Links Access electronic support through My Oracle Support http support oracle com For hearing impaired http www oracle com accessibility support html Learn about Oracle s commitment to access...

Page 11: ...r service Illustrated Parts Breakdown on page 1 Front and Rear Panel Components on page 3 Related Information Detecting and Managing Faults on page 5 Replacing the Server Module Enclosure Assembly Motherboard on page 113 Illustrated Parts Breakdown This topic identifies components in the server module that you can install or remove and replace ...

Page 12: ...OM on page 101 4 USB flash drive Servicing a USB Flash Drive on page 105 5 Clock battery SYS MB BAT Servicing the Battery on page 109 6 Drive HD or SSD SYS HDDn Servicing Drives on page 65 7 Drive filler Servicing Drives on page 65 8 Enclosure assembly SYS MB Replacing the Server Module Enclosure Assembly Motherboard on page 113 9 SP SYS MB SP Servicing the SP Card on page 97 10 REM SYS MB REM Ser...

Page 13: ...cing the Server Module Enclosure Assembly Motherboard on page 113 Front and Rear Panel Components See Diagnostics LEDs on page 10 for more information No Description 1 RFID tag provides the serial number of the server module 2 UCP 3 Drive slots 4 White LED Locator functions as the physical presence switch 5 Blue LED Ready to Remove ...

Page 14: ...Parts Breakdown on page 1 6 Amber LED Fault Service Action Required 7 Green LED OK 8 Power button 9 Reset button NMI for service use only 10 Green LED Drive OK 11 Amber LED Drive Fault Service Action Required 12 Blue LED Drive Ready to Remove 13 Rear chassis power connector 14 Rear chassis data connection No Description ...

Page 15: ...on Preparing for Service on page 51 Diagnostics Overview You can use a variety of diagnostic tools commands and indicators to monitor and troubleshoot a server module LEDs Provide a quick visual notification of the status of the server module and of some of the FRUs Oracle ILOM This firmware runs on the SP In addition to providing the interface between the hardware and OS Oracle ILOM also tracks a...

Page 16: ...s the server module provides hardware validation and discloses possible faulty components with recommendations for repair The LEDs Oracle ILOM PSH and many of the log files and console messages are integrated For example when the Oracle Solaris OS detects a fault it displays the fault logs it and passes information to Oracle ILOM where it is logged Depending on the fault one or more LEDs might als...

Page 17: ...Detecting and Managing Faults 7 Diagnostics Process Use the flowchart to understand how to use the server module s diagnostic tools to manage faults Also see the table that follows this flowchart ...

Page 18: ... messages indicate a faulty device replace the FRU For more diagnostic information review the Oracle VTS report See number 4 Interpreting Log Files and System Messages on page 23 4 Run the Oracle VTS software If Oracle VTS reports a faulty device replace it If Oracle VTS does not report a faulty device run POST See number 5 Checking if Oracle VTS Software Is Installed on page 27 5 Run POST POST pe...

Page 19: ...d Faults on page 44 8 Determine if the fault was detected by POST POST performs basic tests of the server module components and reports faulty FRUs When POST detects a faulty FRU POST logs the fault and if possible takes the FRU offline POST detected FRUs display the following text in the fault message Forced fail reason where reason is the name of the power on routine that detected the failure Ma...

Page 20: ...odule and put the blade into ready to remove state before this LED is on Service Action Required LED Amber Indicates that service is required POST and Oracle ILOM are two diagnostics tools that can detect a fault or failure resulting in this indication Also faults detected by PSH can result in Oracle ILOM lighting this LED The Oracle ILOM show faulty command provides details about any faults that ...

Page 21: ...ooting Overview on page 12 Access the SP Oracle ILOM on page 15 Display FRU Information show Command on page 17 On Standby button n a The recessed Power button toggles the host on or off Press once to turn the host on Press once to shut the host down to a standby state Press and hold for 4 seconds to perform an emergency shutdown Drive Ready to Remove LED Blue Indicates that the drive can be remov...

Page 22: ...Software Is Installed on page 27 POST Overview on page 29 Oracle ILOM Properties That Affect POST Behavior on page 30 Oracle ILOM Troubleshooting Overview Oracle ILOM enables you to remotely run diagnostics such as POST that would otherwise require physical proximity to the server module You can also configure Oracle ILOM to send email alerts of hardware failures hardware warnings and other events...

Page 23: ...Service Action Required LEDs to be turned on the FRUID PROMs updated and a fault message logged If the FRU has status LEDs the Service Action Required LED for that FRU will also be turned on You must replace a FRU identified as having a fault condition In the event of a system fault Oracle ILOM ensures that the Service Action Required LED is turned on FRUID PROMs are updated the fault is logged an...

Page 24: ...his function enables Oracle ILOM to sense that a fault diagnosed to a specific FRU has been repaired Note Oracle ILOM does not automatically detect drive replacement Oracle ILOM does not automatically clear voltage sensor faults Oracle Solaris Fault Manager Commands in Oracle ILOM The Oracle ILOM CLI includes a feature that enables you to access Oracle Solaris fault manager commands such as fmadm ...

Page 25: ...9600 baud 8 bit no parity 1 stop bit and no handshaking and use a null modem configuration transmit and receive signals crossed over to enable DTE to DTE communication NET MGT port Connect this CMM port to your Ethernet network On the CMM this connector is labeled NET MGT This port requires an IP address By default this port uses DHCP to obtain and IP address or you can assign a static IP address ...

Page 26: ...nformation you need These commands are commonly used for fault management show command Displays information about individual FRUs See Display FRU Information show Command on page 17 show faulty command Displays environmental POST detected and PSH detected faults See Check for Faults show faulty Command on page 18 Note You can use fmadm faulty in the Oracle ILOM faultmgmt shell as an alternative to...

Page 27: ... Commands on page 22 Oracle ILOM Properties That Affect POST Behavior on page 30 Display FRU Information show Command Use the Oracle ILOM show command to display information about individual FRUs At the Oracle ILOM prompt type the show command In the following example the show command displays information about a memory module show SYS MB CMP0 BOB0 CH0 D0 SYS MB CMP0 BOB0 CH0 D0 Targets T_AMB SERV...

Page 28: ... faulty fan or power input Environmental faults can also be caused by room temperature or blocked air flow POST detected faults Faults on devices detected by the POST diagnostics PSH detected faults Faults detected by PSH 1 At the Oracle ILOM prompt type the show faulty command 2 If a fault is displayed check the output to determine the nature of the fault The following examples show the different...

Page 29: ...imestamp 2011 10 14 20 14 13 faults 0 SP faultmgmt 0 detector SYS PS0 S1 V_IN_ERR faults 0 SP faultmgmt 0 product_serial_number 1030NND0D2 faults 0 SP faultmgmt 0 chassis_serial_number 0000000 0000000000 faults 0 show faulty Target Property Value SP faultmgmt 0 fru SYS MB CMP0 BOB1 CH0 D0 SP faultmgmt 0 timestamp Oct 12 16 40 56 faults 0 SP faultmgmt 0 sp_detected_fault SYS MB CMP0 BOB1 CH0 D0 fau...

Page 30: ...ich is an alternative to the show faulty command You must run the Oracle Solaris fmadm faulty command from within the Oracle ILOM faultmgmt shell Note The characters SPT at the beginning of a message ID indicate that Oracle ILOM detected the fault 1 At the Oracle ILOM prompt access the Oracle ILOM faultmgmt shell 2 At the faultmgmtsp prompt type the fmadm faulty command SP faultmgmt 0 fru_serial_n...

Page 31: ...e fault For PSH diagnosed faults if the replacement of the FRU is detected by the SP or the fault is manually cleared on the host the fault will also be cleared from Oracle ILOM In such cases you typically do not have to clear the fault manually Note This procedure clears the fault from the SP but not from the host If the fault persists in the host clear it manually as described in Clear PSH Detec...

Page 32: ...ted faults The component is the unique ID of the device with a fault to be cleared start HOST console Connects to the host show HOST console history Displays the contents of the host s console buffer set HOST bootmode property value Controls the host server module OBP firmware method of booting property is state config or script stop SYS start SYS Powers off the host server module and then powers ...

Page 33: ...ubleshooting If POST or the PSH features do not indicate the source of a fault check the message buffer and log files for notifications for faults Drive faults are usually captured by the Oracle Solaris message files Check the Message Buffer dmesg Command on page 24 View System Message Log Files on page 24 List FRU Status prtdiag Command on page 25 show SYS LOCATE Displays the current state of the...

Page 34: ...w System Message Log Files on page 24 List FRU Status prtdiag Command on page 25 View System Message Log Files The error logging daemon syslogd automatically records various system warnings errors and faults in message files These messages can alert you to system problems such as a device that is about to fail The var adm directory contains several message files The most recent messages are in the...

Page 35: ...ag System Configuration Oracle Corporation sun4v Netra SPARC T4 1B Memory size 32256 Megabytes Virtual CPUs CPU ID Frequency Implementation Status 0 2548 MHz SPARC T4 on line 1 2548 MHz SPARC T4 on line 2 2548 MHz SPARC T4 on line 61 2548 MHz SPARC T4 on line 62 2548 MHz SPARC T4 on line 63 2548 MHz SPARC T4 on line Physical Memory Configuration Segment Table Base Segment Interleave Bank Contains ...

Page 36: ...lass 0c0310 pci 400 pci 1 pci 0 pci 0 pci 0 usb 0 SYS MB USB PCIE usb pciclass 0c0310 pci 400 pci 1 pci 0 pci 0 pci 0 usb 0 1 SYS MB USB PCIE usb pciclass 0c0320 pci 400 pci 1 pci 0 pci 0 pci 0 usb 0 2 SYS MB VIDEO PCIX display pci1a03 2000 pci 400 pci 2 pci 0 pci d pci 0 display 0 Environmental Status Fan sensors All fan sensors are OK Fan indicators All fan indicators are OK Temperature sensors ...

Page 37: ...cle VTS Overview on page 27 Check if Oracle VTS Software Is Installed on page 28 Related Information Diagnostics Overview on page 5 Diagnostics Process on page 7 Managing Faults Oracle ILOM on page 11 Interpreting Log Files and System Messages on page 23 Managing Faults PSH on page 41 Managing Faults POST on page 29 Managing Components ASR on page 45 Oracle VTS Overview Oracle VTS is a validation ...

Page 38: ...ce of security mechanisms Oracle VTS software is provided in the preinstalled Oracle Solaris OS that shipped with the server module Related Information Oracle VTS documentation Check if Oracle VTS Software Is Installed on page 28 Check if Oracle VTS Software Is Installed 1 Log in as superuser 2 Check for the presence of Oracle VTS packages If information about the packages is displayed then Oracle...

Page 39: ...ion Diagnostics Overview on page 5 Diagnostics Process on page 7 Managing Faults Oracle ILOM on page 11 Interpreting Log Files and System Messages on page 23 Managing Faults PSH on page 41 Managing Components ASR on page 45 Checking if Oracle VTS Software Is Installed on page 27 POST Overview POST is a group of PROM based tests that run when the server module is powered on or when it is reset POST...

Page 40: ...sequence the server module boots and uses the remaining cores Related Information Diagnostics Overview on page 5 Oracle ILOM Properties That Affect POST Behavior on page 30 Configure POST on page 33 Run POST With Maximum Testing on page 35 Interpret POST Fault Messages on page 37 Clear POST Detected Faults on page 37 POST Output Reference on page 39 Oracle ILOM Properties That Affect POST Behavior...

Page 41: ...t hw change Default Runs POST following an AC power cycle and when the top cover is removed power on reset Runs POST only for the first power on error reset Default Runs POST if fatal errors are detected all resets Runs POST after any reset HOST diag verbosity normal POST output displays all test and informational messages min POST output displays functional tests with a banner and pinwheel max PO...

Page 42: ...table shows combinations of Oracle ILOM parameters and associated POST modes Oracle ILOM Parameter Normal Diagnostic Mode Default Settings No POST Execution Service Mode Using the Keyswitch_state keyswitch_state normal normal diag HOST diag mode normal Off HOST diag level max ...

Page 43: ...ble values for the keyswitch_state parameter see Oracle ILOM Properties That Affect POST Behavior on page 30 HOST diag trigger hw change error reset none HOST diag verbosity normal Description of POST execution This is the default POST configuration This configuration tests the server module thoroughly and suppresses some of the detailed POST output POST does not run resulting in quick initializat...

Page 44: ...les or 4 To see the current values for settings use the show command For example showing default values Related Information POST Overview on page 29 Oracle ILOM Properties That Affect POST Behavior on page 30 set HOST diag mode normal set HOST diag verbosity max show HOST diag HOST diag Targets Properties error_reset_level max error_reset_verbosity normal hw_change_level max hw_change_verbosity no...

Page 45: ...kes about one minute to power off Type the show HOST command to determine when the host has been powered off The console will display status Powered Off 4 Switch to the host console to view the POST output The following example shows abridged POST output set SYS keyswitch_state diag Set keyswitch_state to Diag stop SYS Are you sure you want to stop SYS y n y Stopping SYS start SYS Are you sure you...

Page 46: ...4 Gbps CL 1466 MHz 8 8 Gbps CPU 0 0 0 NOTICE Initializing TSR Hoovers CPU 0 0 0 NOTICE Initializing FSR Hoovers CPU 0 0 0 NOTICE Initializing MCU 0 serdes CPU 0 0 0 NOTICE Initializing MCU 1 serdes CPU 0 0 0 NOTICE Updating Config Information for Guest Manager 2011 08 30 00 47 29 301 0 0 0 NODE PORT 0 1 AST2200 Addr f850 01000000 BDF 16 0 0 VID 1a03 DID 1150 Width 01 G1 2011 08 30 00 47 29 351 0 0...

Page 47: ...a fault was not automatically cleared This procedure describes how to identify a POST detected fault and if necessary manually clear the fault In most cases when POST detects a faulty component POST logs the fault and automatically takes the failed component out of operation by placing the component in the ASR blacklist See Managing Components ASR on page 45 Usually when a faulty component is repl...

Page 48: ...ally the front panel Fault Service Action Required LED is no longer on 5 Reset the server module You must reboot the server module for the component_state property to take effect 6 At the Oracle ILOM prompt type the show faulty command to verify that no faults are reported For example Related Information POST Overview on page 29 Oracle ILOM Properties That Affect POST Behavior on page 30 Configure...

Page 49: ...00000 00000000 2011 07 03 18 44 13 517 0 7 2 1 DESR_SOCSRE SOC non local sw_recoverable_error 2011 07 03 18 44 13 638 0 7 2 1 DESR_SOCHCCE SOC non local hw_corrected_and_cleared_error 2011 07 03 18 44 13 773 0 7 2 2011 07 03 18 44 13 836 0 7 2 Decode of NCU Error Status Reg bits 00000000 22000000 2011 07 03 18 44 13 958 0 7 2 1 NESR_MCU1SRE MCU1 issued a Software Recoverable Error Request 2011 07 ...

Page 50: ...nch 1 00000000 00000800 2011 07 03 18 44 15 842 0 7 2 DRAM Error Syndrome Reg for Branch 1 dd1676ac 8c18c045 2011 07 03 18 44 15 967 0 7 2 DRAM Error Retry Reg for Branch 1 00000000 00000004 2011 07 03 18 44 16 086 0 7 2 DRAM Error RetrySyndrome 1 Reg for Branch 1 a8a5f81e f6411b5a 2011 07 03 18 44 16 218 0 7 2 DRAM Error Retry Syndrome 2 Reg for Branch 1 a8a5f81e f6411b5a 2011 07 03 18 44 16 351 ...

Page 51: ...y negatively affect operations The Oracle Solaris OS uses the fault manager daemon fmd 1M which starts at boot time and runs in the background to monitor the server module If a component generates an error the daemon correlates the error with data from previous errors and other relevant information to diagnose the problem Once diagnosed the fault manager daemon assigns a UUID to the error This val...

Page 52: ...ay information about the fault Alternatively you can use the Oracle ILOM command show faulty for the same purpose Related Information Check for Faults show faulty Command on page 18 Check for PSH Detected Faults on page 42 Clear PSH Detected Faults on page 44 Check for PSH Detected Faults The fmadm faulty command displays the list of faults detected by PSH You can run this command either from the ...

Page 53: ...tput or from the Oracle ILOM show faulty command b Sign into the Oracle support site http support oracle com fmadm faulty TIME EVENT ID MSG ID SEVERITY Aug 13 11 48 33 21a8b59e 89ff 692a c4bc f4c5cccca8c8 SUN4V 8002 6E Major Platform sun4v Chassis_id Product_sn Fault class fault cpu generic sparc strand Affects cpu cpuid serial faulted and taken out of service FRU SYS MB hc product id product sn s...

Page 54: ...ally cleared you must clear the fault manually 1 After replacing a faulty FRU power on the server module 2 At the host prompt determine if the replaced FRU still shows a faulty state fmadm faulty TIME EVENT ID MSG ID SEVERITY Aug 13 11 48 33 21a8b59e 89ff 692a c4bc f4c5cccca8c8 SUN4V 8002 6E Major Platform sun4v Chassis_id Product_sn Fault class fault cpu generic sparc strand Affects cpu cpuid ser...

Page 55: ...the Oracle ILOM clear_fault_action property of the FRU to clear the fault Related Information PSH Overview on page 41 Clear PSH Detected Faults on page 44 Managing Components ASR These topics explain the role played by ASR and how to manage the components that ASR controls ASR Overview on page 46 Display System Components on page 47 Disable System Components on page 48 Enable System Components on ...

Page 56: ...m The database that contains the list of disabled components is the ASR blacklist asr db In most cases POST automatically disables a faulty component After the cause of the fault is repaired FRU replacement loose connector reseated and so on you might need to remove the component from the ASR blacklist The following ASR commands enable you to view add or remove components asrkeys from the ASR blac...

Page 57: ...ystem Components The show components command displays the system components asrkeys and reports their status At the Oracle ILOM prompt type show components In the following example one of the DIMMs BOB1 CH0 D0 is shown as disabled show components Target Property Value SYS MB REM component_state Enabled SYS MB FEM0 component_state Enabled SYS MB CMP0 L2T0 component_state Enabled SYS MB CMP0 L2T1 co...

Page 58: ...ponent_state Enabled NIU_CORE SYS MB CMP0 PEX component_state Enabled SYS MB CMP0 PEU0 component_state Enabled SYS MB CMP0 PEU1 component_state Enabled SYS MB CMP0 BOB0 component_state Enabled CH0 D0 SYS MB CMP0 BOB0 component_state Enabled CH1 D0 SYS MB CMP0 BOB1 component_state Disabled CH0 D0 SYS MB CMP0 BOB1 component_state Enabled CH1 D0 SYS MB CMP0 BOB2 component_state Enabled CH0 D0 SYS MB ...

Page 59: ...stem Components on page 49 Enable System Components You enable a component by setting its component_state property to Enabled This action removes the component from the ASR blacklist 1 At the Oracle ILOM prompt set the component_state property to Enabled 2 Reset the server module so that the ASR command takes effect set SYS MB CMP0 BOB1 CH0 D0 component_state Disabled stop SYS Are you sure you wan...

Page 60: ...e is no notification when the system is actually powered off Powering off takes about a minute Use the show HOST command to determine if the host has powered off Related Information View System Message Log Files on page 24 Display System Components on page 47 Disable System Components on page 48 ...

Page 61: ... tools for service Tools Needed for Service on page 54 3 find serial numbers for the modular system and the server module Find the Modular System Chassis Serial Number on page 54 Find the Server Module Serial Number on page 55 4 Identify the server module that you want to service Locate the Server Module on page 56 5 Shut down the OS and host and place the server module in a ready to remove state ...

Page 62: ...tions provided next to each symbol Caution There is a risk of personal injury or equipment damage To avoid personal injury and equipment damage follow the instructions Caution Components inside the server module might be hot Use caution when servicing components inside the server module Caution Hazardous voltages are present To reduce the risk of electric shock and danger to personal health follow...

Page 63: ...SD sensitive components such as cards and DIMMs on an antistatic mat Related Information Handling Precautions on page 53 Tools Needed for Service on page 54 Handling Precautions Review the following cautions Caution A server module can weigh as much as 20 pounds 9 0 kg During removal hold the server module firmly with both hands Caution Do not stack server modules higher than five units tall Cauti...

Page 64: ...Serial Number on page 54 Find the Modular System Chassis Serial Number To obtain support for your server module you need the serial number of the Sun Netra 6000 modular system in which the server module is located not the serial number of the server module The serial number of the modular system is provided on a label on the upper left edge of the front bezel Use the following procedure to obtain ...

Page 65: ... sticker on the RFID tag that is mounted in the center of the front panel However this label is not present on a server module that has been moved into a new enclosure assembly You also can type the Oracle ILOM show SYS command to display the number Access the Oracle ILOM CLI and type show SYS SYS Targets MB MB_ENV HDD0 Properties type Host System ipmi_name SYS keyswitch_state Normal product_name ...

Page 66: ...plan to locate 2 Type The Locator LED on the server module blinks 3 Identify the server module with a blinking white LED 4 Once you locate the server module press the Locator LED to turn it off Note Alternatively you can turn off the Locator LED by typing the Oracle ILOM set SYS LOCATE value off command Related Information Remove the Server Module From the Modular System on page 61 Preparing the S...

Page 67: ... additional information 3 Save any open files and quit all running programs Refer to the application documentation for specific information on these processes 4 If applicable Shut down all logical domains Refer to the Oracle Solaris system administration and Oracle VM Manager for SPARC documentation for additional information Description Links Perform a graceful shutdown using commands Shut Down t...

Page 68: ... Host Emergency Shutdown on page 59 Set the Server Module to a Ready to Remove State on page 60 shutdown g0 i0 y Shutdown started Tue Jun 28 13 06 20 PDT 2011 Changing to init state 0 please wait Broadcast Message from root console on server1 Tue Jun 28 13 06 20 THE SYSTEM server1 IS BEING SHUT DOWN NOW Log off now or risk your files being damaged svc startd The system is coming down Please wait s...

Page 69: ...is button Related Information Shut Down the OS and Host Commands on page 57 Shut Down the OS and Host Emergency Shutdown on page 59 Set the Server Module to a Ready to Remove State on page 60 Shut Down the OS and Host Emergency Shutdown Caution All applications and files will be closed abruptly without saving changes File system corruption might occur Press and hold the Power button for four secon...

Page 70: ...standby mode by viewing the blue Ready to Remove LED on the front of the server module See Front and Rear Panel Components on page 3 to locate this LED If the Ready to Remove LED is on the server module is ready for removal from the modular system chassis 5 Remove the server module from the chassis See Remove the Server Module From the Modular System on page 61 Related Information Remove the Serve...

Page 71: ...recautions See Safety Information on page 51 and Handling Precautions on page 53 2 If a cable is connected to the front of the server module disconnect it Press the buttons on either side of the UCP to release the connector 3 Open both ejector arms panel 1 Squeeze both latches on each of the two ejector arms ...

Page 72: ...h two hands 7 Place the server module on an antistatic mat or surface 8 Insert a filler panel into the empty chassis slot Note When the modular system is operating you must fill every slot with a filler panel or a server module within 60 seconds 9 Remove the server module cover See Remove the Cover on page 63 Related Information Remove the Cover on page 63 Install the Server Module Into the Modula...

Page 73: ...strap to your wrist and then to a metal area on the server module 3 While pressing the cover release button slide the cover toward the rear of the server module about half an inch 1 cm 4 Lift the cover off the server module chassis 5 Service the faulty component See Illustrated Parts Breakdown on page 1 Related Information Illustrated Parts Breakdown on page 1 Replace the Cover on page 117 ...

Page 74: ...64 Netra SPARC T4 1B Server Module Service Manual June 2012 ...

Page 75: ...lty drive Drive Hot Plugging Guidelines on page 68 Drive Configuration on page 66 Locate a Faulty Drive on page 68 Remove a Drive on page 69 Install a Drive on page 71 Verify Drive Functionality on page 74 Add an additional drive Drive Configuration on page 66 Remove a Drive Filler on page 70 Install a Drive on page 71 Verify Drive Functionality on page 74 Remove a drive without replacing it Drive...

Page 76: ... to the drives installed when the drive is installed into a particular slot Note The Oracle Solaris OS now uses the WWN syntax in place of the unique tn target ID field in logical device names This change affects how a target storage device is identified Refer to the Server Module Product Notes for details No Description 1 Drive slot 0 2 Drive slot 1 ...

Page 77: ...es the following drive status On Drive is idle and available for use Off Read or write activity is in progress 3 Drive Service Action Required LED Amber Indicates that the drive has experienced a fault condition 2 Drive Ready to Remove LED Blue Indicates that a drive can be removed during a hot plug operation ...

Page 78: ...you replace the drive See Shut Down the OS and Host Commands on page 57 Related Information Remove a Drive on page 69 Install a Drive on page 71 Locate a Faulty Drive This procedure describes how to identify a faulty drive using the fault LEDs on the drive You can also use the diskinfo 1M command to identify the slot in which a particular drive is installed Refer to the Administration Guide and to...

Page 79: ...to take a drive offline is the cfgadm command For more information refer to the Oracle Solaris cfgadm man page Shut down the Oracle Solaris OS If the drive cannot be taken offline shut down the Oracle Solaris OS on the server module See Shut Down the OS and Host Commands on page 57 3 Verify whether the blue Drive Ready to Remove LED is illuminated on the front of the drive See Drive LEDs on page 6...

Page 80: ...ou are replacing the drive see Install a Drive on page 71 If you are not replacing the drive install a drive filler See Install a Drive Filler on page 73 Related Information Install a Drive Filler on page 73 Install a Drive on page 71 Remove a Drive Filler All drive bays must be populated by either a drive or a filler 1 Open the filler lever panels 1 and 2 ...

Page 81: ...71 Related Information Install a Drive on page 71 Install a Drive Filler on page 73 Install a Drive The physical address of a drive is based the slot in which it is installed See Drive Configuration on page 66 1 If needed Remove a drive See Remove a Drive on page 69 2 Identify the slot in which to install the drive ...

Page 82: ...onal drive install the drive in the next available drive slot 3 If needed Remove the drive filler from this slot See Remove a Drive Filler on page 70 4 Slide the drive into the bay until it is fully seated panel 1 5 Close the latch to lock the drive in place panels 2 and 3 6 Verify the functionality of the new drive See Verify Drive Functionality on page 74 Related Information Remove a Drive on pa...

Page 83: ...populated by either a drive or a filler 1 Extend the filler handle then align the filler to the empty drive bay panel 1 2 Push the filler into place 3 Close the filler lever panels 2 and 3 Related Information Remove a Drive on page 69 Remove a Drive Filler on page 70 ...

Page 84: ... Go to Step 3 If the fault LED is lit see Detecting and Managing Faults on page 5 3 Perform administrative tasks to reconfigure the drive The procedures that you perform at this point depend on how your data is configured You might need to partition the drive create file systems load data from backups or have data updated from a RAID configuration The following commands might apply to your circums...

Page 85: ...repair memory problems This topic describes how the server module deals with memory faults The following server module features independently manage memory faults Description Links Understand memory faults Memory Faults on page 75 Replace a faulty DIMM DIMM Handling Precautions on page 79 Locate a Faulty DIMM on page 79 Remove a DIMM on page 80 Locate a Faulty DIMM on page 79 Install a DIMM on pag...

Page 86: ...ault and Verify the Functionality of the Replacement DIMM on page 82 PSH A feature of the Oracle Solaris OS PSH uses the fault manager daemon fmd to watch for various kinds of faults When a fault occurs the fault is assigned a UUID and logged PSH reports the fault and suggests a replacement for the DIMMs associated with the fault If you suspect that the server module has a memory problem follow th...

Page 87: ...uration No Description or Partial FRU Name full names start with SYS MB CMP0 1 Fault Remind button 2 Fault Remind Power LED 3 DIMMs controlled by BOB3 CH0 D1 CH0 D0 CH1 D1 CH1 D0 4 DIMMs controlled by BOB4 CH0 D1 CH0 D0 CH1 D1 CH1 D0 ...

Page 88: ...ts 4 DIMMs CH1 D0 slots white sockets 8 DIMMs CH1 D0 and CH0 D0 slots 16 DIMMs All slots Ensure that all DIMMs have the same part number Related Information Memory Faults on page 75 Locate a Faulty DIMM on page 79 Remove a DIMM on page 80 Install a DIMM on page 81 Clear the Fault and Verify the Functionality of the Replacement DIMM on page 82 5 DIMMs controlled by BOB0 CH0 D1 CH0 D0 CH1 D1 CH1 D0 ...

Page 89: ... motherboard to pinpoint the physical location of a faulty DIMM Note You can also obtain the location of the faulty DIMM using the Oracle ILOM show faulty command This command displays the FRU name such as SYS MB CMP0 BOB0 CH0 Use the FRU name and information to locate the faulty DIMM See DIMM Configuration on page 77 1 Check the front panel Fault LED See Diagnostics LEDs on page 10 When a faulty ...

Page 90: ...fficult to identify when they are not illuminated If you do not see any illuminated LEDs in the area of the DIMM LEDs assume that the DIMMs are not faulty 4 Remove the faulty DIMM See Remove a DIMM on page 80 Related Information DIMM Configuration on page 77 Remove a DIMM on page 80 Remove a DIMM 1 If needed Prepare for service See Preparing for Service on page 51 2 If needed Locate the faulty DIM...

Page 91: ... Prepare the server module for service and remove the faulty DIMM See Preparing for Service on page 51 and Remove a DIMM on page 80 2 Unpack the replacement DIMM and set it on an antistatic mat See DIMM Handling Precautions on page 79 3 Ensure that the DIMM ejector tabs are in the open position panel 1 4 Line up the replacement DIMM with the connector Align the DIMM notch with the key in the conne...

Page 92: ...82 Verify additional memory See Verify DIMM Functionality on page 86 Related Information Remove a DIMM on page 80 DIMM Configuration on page 77 Clear the Fault and Verify the Functionality of the Replacement DIMM 1 Ensure that the following conditions are met The server module is in Standby mode installed in a powered modular system but the server module s host is not started See Set the Server Mo...

Page 93: ...ir a Set the virtual keyswitch to diag so that POST will run in Service mode show faulty Target Property Value SP faultmgmt 0 fru SYS MB CMP0 BOB0 CH0 D0 SP faultmgmt 0 timestamp Dec 14 22 43 59 SP faultmgmt 0 sunw msg id SUN4V 8000 DX faults 0 SP faultmgmt 0 uuid 3aa7c854 9667 e176 efe5 e487e520 faults 0 7a8a SP faultmgmt 0 timestamp Apr 24 22 43 59 faults 0 show faulty Target Property Value SP f...

Page 94: ...faults Note Depending on the configuration of Oracle ILOM variables that affect POST and whether POST detected faults or not the server module might boot or the server module might remain at the ok prompt If the server module is at the ok prompt type boot d Return the virtual keyswitch to Normal mode stop SYS Are you sure you want to stop SYS y n y Stopping SYS start SYS Are you sure you want to s...

Page 95: ...id not clear the fault Type the set command 8 Only if previous steps did not clear the fault Switch to the host console and type the fmadm repair command with the UUID Use the same UUID that was displayed from the output of the Oracle ILOM show faulty command Related Information Install a DIMM on page 81 fmadm faulty show faulty Target Property Value SP faultmgmt 0 fru SYS MB CMP0 BOB0 CH0 D0 SP f...

Page 96: ...is detected when the SP is power cycled In those cases the fault is automatically cleared from the server module If show faulty still displays the fault the set command will clear it 4 For a host detected fault verify the new DIMM a Set the virtual keyswitch to diag so that POST will run in Service mode b Power cycle the server module host Note Use the show HOST command to determine when the host ...

Page 97: ... module remains at the ok prompt type boot e Return the virtual keyswitch to Normal mode f Switch to the host console and type the Oracle Solaris OS fmadm faulty command If any faults are reported see the diagnostics instructions in Oracle ILOM Troubleshooting Overview on page 12 5 Switch to the Oracle ILOM command shell start HOST console 0 7 2 INFO 0 7 2 POST Passed all devices 0 7 2 POST Return...

Page 98: ...UUID Use the same UUID that was displayed from the output of the Oracle ILOM show faulty command Related Information Remove a DIMM on page 80 Install a DIMM on page 81 DIMM Configuration on page 77 show faulty Target Property Value SP faultmgmt 0 fru SYS MB CMP0 BOB0 CH1 D0 SP faultmgmt 0 timestamp Dec 14 22 43 59 SP faultmgmt 0 sunw msg id SUN4V 8000 DX faults 0 SP faultmgmt 0 uuid 3aa7c854 9667 ...

Page 99: ...formation Detecting and Managing Faults on page 5 Preparing for Service on page 51 Remove a REM 1 Prepare for service See Preparing for Service on page 51 2 Lift the REM ejector arm panel 1 Description Links Troubleshoot a REM problem Refer to the documentation for the REM Replace a REM Remove a REM on page 89 Install a REM on page 90 Install a REM Install a REM on page 90 ...

Page 100: ...and 3 4 Set the card on an antistatic surface 5 Install a REM See Install a REM on page 90 Related Information Install a REM on page 90 Install a REM For information about specific configuration tasks for your REM refer to the REM documentation 1 If needed Prepare for service See Preparing for Service on page 51 ...

Page 101: ...y seated on the motherboard panel 3 If there is a rubber bumper on the REM you can press down on it directly to seat the connector 6 Return the server module to operation See Returning the Server Module to Operation on page 117 7 Configure or verify the RAID after installing the REM Refer to the SPARC and Netra SPARC T4 Series Servers Administration Guide for information about RAID configuration o...

Page 102: ...92 Netra SPARC T4 1B Server Module Service Manual June 2012 ...

Page 103: ...g Faults on page 5 Preparing for Service on page 51 Remove a FEM FEMs are available in single and double widths Figures in this procedure depict a single width FEM but the procedure applies to both types of FEMs 1 Prepare for service See Preparing for Service on page 51 2 Lift the lever to eject the FEM panel 1 Description Links Replace a FEM Remove a FEM on page 93 Install a FEM on page 94 Instal...

Page 104: ... 2 4 Remove the FEM panel 3 and place the FEM on an antistatic mat 5 If needed Install a FEM See Install a FEM on page 94 Related Information Install a FEM on page 94 Install a FEM This procedure applies to any of the form factors of FEM cards that are supported by this server module ...

Page 105: ...ge 93 3 Determine the correct set of motherboard FEM connectors for your FEM A double width FEM card 1 uses connectors FEM 0 and FEM 1 A single width FEM card 2 uses connector FEM 0 4 Insert the FEM edge into the bracket and carefully align the FEM so that the card connects with the correct motherboard connectors panels 1 and 2 ...

Page 106: ... and press the card into place Panel 3 If the card has rubber bumpers you can press directly on them to seat the card into the connectors 6 Return the server module to operation See Returning the Server Module to Operation on page 117 Related Information Remove a FEM on page 93 ...

Page 107: ...aring for Service on page 51 Remove the SP Card 1 If possible save the configuration information for the SP Refer to the related procedures using Oracle ILOM in the SPARC and Netra SPARC T4 Series Servers Administration Guide 2 Prepare for service See Preparing for Service on page 51 3 If a REM is installed in the server module remove the REM See Remove a REM on page 89 4 Push down on the tab to e...

Page 108: ...e card up and off the retainer panels 2 and 3 Set the card on an antistatic mat 6 Install the new card See Install the SP Card on page 98 Related Information Install the SP Card on page 98 Install the SP Card 1 If needed Remove the SP card See Remove the SP Card on page 97 ...

Page 109: ...th the key panel 2 3 Seat the SP card into the connector by pressing the card toward the tabs while pressing down panel 3 When the SP card is in place the lever will close 4 Return the server module to operation See Returning the Server Module to Operation on page 117 Related Information Remove the SP Card on page 97 ...

Page 110: ...100 Netra SPARC T4 1B Server Module Service Manual June 2012 ...

Page 111: ...replace the enclosure assembly swap the ID PROM from the original enclosure assembly to the replacement enclosure assembly This action ensures that your server module will maintain the same host ID and MAC address Remove the ID PROM on page 101 Install the ID PROM on page 102 Verify the ID PROM on page 103 Related Information Detecting and Managing Faults on page 5 Preparing for Service on page 51...

Page 112: ... 2 Place the ID PROM on an antistatic mat 4 Install the ID PROM See Install the ID PROM on page 102 Related Information Install the ID PROM on page 102 Verify the ID PROM on page 103 Install the ID PROM 1 If needed Remove the ID PROM See Remove the ID PROM on page 101 2 Locate the ID PROM socket on the motherboard panel 1 ...

Page 113: ...ify the ID PROM See Verify the ID PROM on page 103 Related Information Remove the ID PROM on page 101 Verify the ID PROM on page 103 Verify the ID PROM The host MAC address and the host ID values are stored in the ID PROM This task describes ways to display these values 1 Display the MAC address that is stored in the ID PROM Example using the Oracle ILOM show command show HOST macaddress HOST Prop...

Page 114: ...s ifconfig command Related Information Remove the ID PROM on page 101 Install the ID PROM on page 102 hostid 85c1bd7c ifconfig a lo0 flags 2001000849 UP LOOPBACK RUNNING MULTICAST IPv4 VIRTUAL mtu 8232 index 1 inet 127 0 0 1 netmask ff000000 igb0 flags 1004843 UP BROADCAST RUNNING MULTICAST DHCP IPv4 mtu 1500 index inet 10 6 91 117 netmask fffffe00 broadcast 10 6 91 255 ether 0 21 28 7f 68 44 ...

Page 115: ...ng for Service on page 51 Remove a USB Flash Drive 1 Prepare for service See Preparing for Service on page 51 2 Locate the USB flash drive at the rear of the server module panel 1 Description Links Replace a USB flash drive Remove a USB Flash Drive on page 105 Install a USB Flash Drive on page 106 Add a USB flash drive Install a USB Flash Drive on page 106 ...

Page 116: ...ormation Install a USB Flash Drive on page 106 Install a USB Flash Drive The server module has a USB port on the motherboard The USB port accepts USB flash drives that do not exceed a length of 39 mm 1 Prepare for service See Preparing for Service on page 51 2 If needed Remove a USB flash drive See Remove a USB Flash Drive on page 105 3 Locate the USB connector on the motherboard ...

Page 117: ...ive into the upper port of the USB connector panels 1 and 2 Do not use the lower port of this connector 5 Return the server module to operation See Returning the Server Module to Operation on page 117 Related Information Remove a USB Flash Drive on page 105 ...

Page 118: ...108 Netra SPARC T4 1B Server Module Service Manual June 2012 ...

Page 119: ...hen the server module is powered off If the server module fails to maintain the proper time when it is powered off replace the battery Use a CR2032 replacement battery 1 Prepare for service See Preparing for Service on page 51 2 Push the top of the battery forward then lift the battery from the holder panel 1 and 2 If you need more clearance remove the DIMM in slot CMP0 BOB3 CH1 D0 nearest the bat...

Page 120: ...BOB3 CH1 D0 4 If removed Replace the DIMM in CMP0 BOB3 CH1 D0 See Install a DIMM on page 81 5 Return the server module to operation See Returning the Server Module to Operation on page 117 6 Access the Oracle ILOM prompt See Access the SP Oracle ILOM on page 15 7 Set the clock s day and time For example set SP clock datetime 061716192011 show SP clock ...

Page 121: ... Battery 111 Related Information Servicing the FEM on page 93 Returning the Server Module to Operation on page 117 SP clock Targets Properties datetime Fri JUN 17 16 19 56 2011 timezone GMT GMT usentpserver disabled ...

Page 122: ...112 Netra SPARC T4 1B Server Module Service Manual June 2012 ...

Page 123: ...s not one of the replaceable FRUs described in this document the enclosure assembly must be replaced Note This procedure must be performed by an Oracle field service representative Transfer Components to Another Enclosure Assembly on page 113 Related Information Identifying Components on page 1 Detecting and Managing Faults on page 5 Preparing for Service on page 51 Transfer Components to Another ...

Page 124: ...Filler on page 73 5 Transfer the FEM if present from the original server module to the enclosure assembly Install the FEM in the same connectors in the enclosure assembly See Servicing the FEM on page 93 6 Remove the REM if present from the original server module See Servicing the REM on page 89 Before installing the REM in the enclosure assembly move the SP card to the enclosure assembly See Step...

Page 125: ...nto the Modular System on page 118 15 Start the server module host See Power On the Host Oracle ILOM on page 120 16 Perform diagnostics to verify the proper operation of the server module See Detecting and Managing Faults on page 5 17 Transfer the serial number and product number to the FRUID of the new enclosure assembly This must be done in a special service mode by trained service personnel Not...

Page 126: ...116 Netra SPARC T4 1B Server Module Service Manual June 2012 ...

Page 127: ...f components inside the server module 1 Set the cover on the server module panel 1 The cover edge hangs over the rear of the server module by about half an inch 1 cm Step Description Links 1 Replace the server module cover Replace the Cover on page 117 2 Install the server module into the modular system Install the Server Module Into the Modular System on page 118 3 Power on the server module host...

Page 128: ...ll the Server Module Into the Modular System on page 118 Remove the Cover on page 63 Install the Server Module Into the Modular System Caution Insert a filler panel into an empty modular system slot within 60 seconds of server module removal to ensure proper chassis cooling Caution Hold the server module firmly with both hands so that you do not drop it The server module can weighs as much as 20 p...

Page 129: ...osition so that both ejector levers are on the right panel 1 5 Slide the server module into the chassis panel 2 6 Close both latches simultaneously locking the server module in the modular system chassis panel 3 Once installed the following server module activities take place Standby power is applied The front panel LEDs blink three times then the green OK LED on the front panel blinks for a few m...

Page 130: ...SP Oracle ILOM on page 15 Note The server module power on process can take several minutes to complete depending on the amount of installed memory and the configured diagnostic level By default the server module boots the Oracle Solaris OS 3 Perform any diagnostics that verify the results of servicing the server module Related Information Detecting and Managing Faults on page 5 Power On the Host P...

Page 131: ...e depending on the amount of installed memory and the configured diagnostic level By default the server module boots the Oracle Solaris OS 2 Perform any diagnostics that verify the results of servicing the server module Related Information Detecting and Managing Faults on page 5 Power On the Host Oracle ILOM on page 120 ...

Page 132: ...122 Netra SPARC T4 1B Server Module Service Manual June 2012 ...

Page 133: ... AWG American wire gauge B blade Generic term for server modules and storage modules See server module and storage module blade server Server module See server module BMC Baseboard management controller BOB Memory buffer on board C chassis For servers refers to the server enclosure For server modules refers to the modular system enclosure CMA Cable management arm ...

Page 134: ...nfiguration Protocol disk module or disk blade Interchangeable terms for storage module See storage module DTE Data terminal equipment E EIA Electronics Industries Alliance ESD Electrostatic discharge F FEM Fabric expansion module FEMs enable server modules to use the 10GbE connections provided by certain NEMs See NEM FRU Field replaceable unit H HBA Host bus adapter host The part of the server or...

Page 135: ...one mouse with more than one computer L LwA Sound power level M MAC Machine access code MAC address Media access controller address Modular system The rackmountable chassis that holds server modules storage modules NEMs and PCI EMs The modular system provides Oracle ILOM through its CMM MSGID Message identifier N name space Top level Oracle ILOM CMM target NEBS Network Equipment Building System Ne...

Page 136: ...Integrated Lights Out Manager Oracle ILOM firmware is preinstalled on a variety of Oracle systems Oracle ILOM enables you to remotely manage your Oracle servers regardless of the state of the host system Oracle Solaris OS Oracle Solaris operating system P PCI Peripheral component interconnect PCI EM PCIe ExpressModule Modular components that are based on the PCI Express industry standard form fact...

Page 137: ...U and memory in a modular system Server modules might also have onboard storage and connectors that hold REMs and FEMs SP Service processor In the server or server module the SP is a card with its own OS The SP processes Oracle ILOM commands providing lights out management control of the host See host SSD Solid state drive SSH Secure shell storage module Modular component that provides computing s...

Page 138: ...e Manual June 2012 UI User interface UL Underwriters Laboratory Inc US NEC United States National Electrical Code UTC Coordinated Universal Time UUID Universal unique identifier W WWN World wide name A unique number that identifies a SAS target ...

Page 139: ...cted faults 44 clock battery 109 completing service 117 components disabled automatically by POST 46 front and rear panel 3 identifying 1 location 1 managing with ASR 45 configuring how POST runs 33 cover installing 117 removing 63 D default Oracle ILOM password 15 detecting faults 5 diag_level parameter 31 diag_mode parameter 31 diag_trigger parameter 31 diag_verbosity parameter 31 diagnostics ov...

Page 140: ...g 106 removing 105 servicing 105 fmadm command 44 82 fmadm faulty command 20 fmdump command 42 FRU ID PROMs 13 FRU information displaying 17 FRU names components 1 DIMMs 77 FRUs displaying status of 25 location of 1 H handling precautions DIMMs 79 server module 53 hot plugging drives 68 I ID PROM 13 installing 102 removing 101 servicing 101 verifying 103 identifying components 1 illustrated parts ...

Page 141: ...ault Oracle ILOM 15 POST clearing faults 37 components disabled by 46 configuration examples 33 configuring 33 faults detected by 8 30 interpreting POST fault messages 37 and memory faults 75 modes and Oracle ILOM parameters 30 output 39 running 29 running in Diag mode 35 troubleshooting with 9 using for fault diagnosis 8 POST detected faults 18 power button 59 powering on 120 preparing for servic...

Page 142: ...ow faulty command 18 22 37 44 82 showcomponent command 47 shutdown command 57 slot assignments DIMM 77 SP accessing 15 installing 98 removing 97 servicing 97 standby mode 59 60 system message log files 24 T time setting 109 tools for service 54 troubleshooting by checking Oracle Solaris OS log files 8 using Oracle VTS 8 using POST 8 9 U USB flash drive 105 UUID 42 V var adm messages file 24 verify...