NetApp AFF A400, Manual

Page 1: ...400 systems ONTAP Systems NetApp November 23 2021 This PDF was generated from https docs netapp com us en ontap systems a400 install setup html on November 23 2021 Always check docs netapp com for the latest ...

Page 2: ...Table of Contents AFF A400 System Documentation 1 Install and setup 1 Maintain 13 ...

Page 3: ...tallation of your system from racking and cabling through initial system bring up Use this guide if you are familiar with installing NetApp systems Access the Installation and Setup Instructions PDF poster AFF A400 Installation and Setup Instructions Videos AFF A400 There are two videos one showing how to rack and cable your system and one showing an example of using the System Manager Guided Setu...

Page 4: ... as additional information on your configured system You might also want to have access to the Release Notes for your version of ONTAP for more information about this system NetApp Hardware Universe Find the Release Notes for your version of ONTAP 9 You need to provide the following at your site Rack space for the storage system Phillips 2 screwdriver Additional networking cables to connect your s...

Page 5: ...2A 112 00437 2m X66033A 112 00438 3m mini SAS HD to mini SAS HD cables order dependent Optical cables X66250 2 N C 112 00342 16 Gb FC or 25GbE cables for mezzanine cards order dependent RJ 45 order dependent X6585 R6 112 00291 3m X6562 R6 112 00196 5m Management network Micro USB console cable Not applicable Console connection used during software setup if laptop or console does not support networ...

Page 6: ...node switchless cluster method or by using the cluster interconnect network Option 1 Cable a two node switchless cluster The optional data ports optional NIC cards and management ports on the controller modules are connected to switches The cluster interconnect and HA ports are cabled on both controller modules You must have contacted your network administrator for information about connecting the...

Page 7: ...abling instructions Option 2 Cable a switched cluster The optional data ports optional NIC cards mezzanine cards and management ports on the controller modules are connected to switches The cluster interconnect and HA ports are cabled on to the cluster HA switch You must have contacted your network administrator for information about connecting the system to the switches Be sure to check the direc...

Page 8: ... cabling 2 Go to Step 4 Cable controllers to drive shelves for drive shelf cabling instructions Step 4 Cable controllers to drive shelves You can cable either NSS224 or SAS shelves to you system Option 1 Cable the controllers to a single drive shelf You must cable each controller to the NSM modules on the NS224 drive shelf Be sure to check the illustration arrow for the proper cable connector pull...

Page 9: ...drive shelf Cabling the controllers to one NS224 drive shelf 2 Go to Step 5 Complete system setup and configuration to complete system setup and configuration Option 2 Cable the controllers to two drive shelves You must cable each controller to the NSM modules on both NS224 drive shelves Be sure to check the illustration arrow for the proper cable connector pull tab orientation The cable pull tab ...

Page 10: ...ve shelves Cabling controllers to two NS224 drive shelves 2 Go to Step 5 Complete system setup and configuration to complete system setup and configuration Option 3 Cable the controllers to SAS drive shelves You must cable each controller to the IOM modules on both SAS drive shelves Be sure to check the illustration arrow for the proper cable connector pull tab orientation The cable pull tab for t...

Page 11: ... should feel it click into place if you do not feel it click remove it turn it around and try again Steps 1 Use the following illustration to cable your controllers to two drive shelves Cabling the controllers to SAS drive shelves 9 ...

Page 12: ... discovery 1 Use the following animation to set one or more drive shelf IDs If your system has NS224 drive shelves the shelves are pre set to shelf ID 00 and 01 If you want to change the shelf IDs you must create a tool to insert into the hole where button is located Setting drive shelf IDs 2 Plug the power cords into the controller power supplies and then connect them to power sources on differen...

Page 13: ...l features in ONTAP Option 2 Completing system setup and configuration if network discovery is not enabled If network discovery is not enabled on your laptop you must complete the configuration and setup using this task 1 Cable and configure your laptop or console a Set the console port on the laptop or console to 115 200 baud with N 8 1 See your laptop or console s online help for how to configur...

Page 14: ... prompted by the script 3 Using System Manager on your laptop or console configure your cluster a Point your browser to the node management IP address The format for the address is https x x x x b Configure the system using the data you collected in the NetApp ONTAP Configuration guide ONTAP Configuration Guide 4 Set up your account and download Active IQ Config Advisor a Log in to your existing a...

Page 15: ...se steps on the correct node The impaired node is the node on which you are performing maintenance The healthy node is the HA partner of the impaired node Check onboard encryption keys as needed AFF A400 Prior to shutting down the impaired node and checking the status of the onboard encryption keys you must check the status of the impaired node disable automatic giveback and check what version of ...

Page 16: ...If any volumes are listed in the output NVE is configured and you need to verify the NVE configuration If no volumes are listed check whether NSE is configured 2 Verify whether NSE is configured storage encryption disk show If the command output list the drive details with Mode Key ID information NSE is configured and you need to verify the NSE configuration If no disks are shown NSE is not config...

Page 17: ... the customer s onboard key management passphrase at the prompt If the passphrase cannot be provided contact NetApp Support mysupport netapp com b Verify the Restored column shows yes for all authentication keys security key manager key query c Verify that the Key Manager type shows onboard manually backup the OKM information d Go to advanced privilege mode and enter y when prompted to continue se...

Page 18: ...ty key manager external sync If the command fails contact NetApp Support mysupport netapp com b Verify that the Restored column equals yes for all authentication keys security key manager key query c You can safely shutdown the node 4 If the Key Manager type displays onboard and the Restored column displays anything other than yes a Enter the onboard security key manager sync command security key ...

Page 19: ...ete the shutdown of the impaired node Do not use this procedure if your system is in a two node MetroCluster configuration To shut down the impaired node you must determine the status of the node and if necessary take over the node so that the healthy node continues to serve data from the impaired node storage If you have a cluster with more than two nodes it must be in quorum If the cluster is no...

Page 20: ...ut down the impaired node you must determine the status of the node and if necessary switch over the node so that the healthy node continues to serve data from the impaired node storage About this task If you are using NetApp Storage Encryption you must have reset the MSID using the instructions in the Returning SEDs to unprotected mode section of Administration overview with the CLI You must leav...

Page 21: ...override vetoes parameter If you use this optional parameter the system overrides any soft vetoes that prevent the healing operation 4 Verify that the operation has been completed by using the metrocluster operation show command controller_A_1 metrocluster operation show Operation heal aggregates State successful Start Time 7 25 2016 18 45 55 End Time 7 25 2016 18 45 56 Errors 5 Check the state of...

Page 22: ...ive Step 1 Remove the controller module To access components inside the controller module you must remove the controller module from the chassis You can use the following animation illustration or the written steps to remove the controller module from the chassis Removing the controller module Steps 1 If you are not already grounded properly ground yourself 2 Release the power cable retainers and ...

Page 23: ...e controller module and then follow the directions to replace it Before you begin Although the contents of the boot media is encrypted it is a best practice to erase the contents of the boot media before replacing it For more information see the Statement of Volatility for your system on the NetApp Support Site You must log into the NetApp Support Site to display the Statement of Volatility for yo...

Page 24: ...ss the locking tabs on the sides of the air duct in toward the middle of the controller module b Slide the air duct toward the back of the controller module and then rotate it upward to its completely open position 2 Locate and remove the boot media from the controller module 22 ...

Page 25: ...n gently push it into the socket 4 Check the boot media to make sure that it is seated squarely and completely in the socket If necessary remove the boot media and reseat it into the socket 5 Lock the boot media in place a Rotate the boot media down toward the motherboard b Placing a finger at the end of the boot media by the blue button push down on the boot media end to engage the blue locking b...

Page 26: ...flash drive a Download the service image to your work space on your laptop b Unzip the service image If you are extracting the contents using Windows do not use WinZip to extract the netboot image Use another extraction tool such as 7 Zip or WinRAR There are two folders in the unzipped service image file boot efi c Copy the efi folder to the top directory on the USB flash drive The USB flash drive...

Page 27: ...ss Ctrl C select the option to boot to Maintenance mode and then halt the node to boot to LOADER 9 Although the environment variables and bootargs are retained you should check that all required boot environment variables and bootargs are properly set for your system type and configuration using the printenv bootarg name command and correct any errors using the setenv variable name value command a...

Page 28: ...rom the USB drive restore the file system and verify the environmental variables This procedure applies to systems that are not in a two node MetroCluster configuration Steps 1 From the LOADER prompt boot the recovery image from the USB flash drive boot_recovery The image is downloaded from the USB flash drive 2 When prompted either enter the name of the image or accept the default image displayed...

Page 29: ...Restore OKM NSE and NVE as needed If your system does not have onboard keymanager NSE or NVE configured complete the steps in this section 6 From the LOADER prompt enter the boot_ontap command If you see Then The login prompt Go to the next Step Waiting for giveback a Log into the partner node b Confirm the target node is ready for giveback with the storage failover show command 7 Connect the cons...

Page 30: ...e Press Ctrl C for Boot Menu message and when the Boot Menu is displayed select option 6 5 Verify that the environmental variables are set as expected a Take the node to the LOADER prompt b Check the environment variable settings with the printenv command c If an environment variable is not set as expected modify it with the setenv environment variable name changed value command d Save your change...

Page 31: ...surviving cluster 5 Verify that the switchback operation has completed metrocluster show The switchback operation is still running when a cluster is in the waiting for switchback state cluster_B metrocluster show Cluster Configuration State Mode Local cluster_B configured switchover Remote cluster_A configured waiting for switchback The switchback operation is complete when the clusters are in the...

Page 32: ...e target node 2 Use the boot_ontap command at the LOADER prompt to boot the node 3 Check the console output If the console displays Then The LOADER prompt Boot the node to the boot menu boot_ontap menu Waiting for giveback a Enter Ctrl C at the prompt b At the message Do you wish to halt this node rather than wait y n enter y c At the LOADER prompt enter the boot_ontap menu command 4 At the Boot M...

Page 33: ...ck with the storage failover show command 10 Giveback only the CFO aggregates with the storage failover giveback fromnode local only cfo aggregates true command If the command fails because of a failed disk physically dis engage the failed disk but leave the disk in the slot until a replacement is received If the command fails because of an open CIFS sessions check with customer how to close out C...

Page 34: ...ng the net int revert command 17 Move the console cable to the target node and run the version v command to check the ONTAP versions 18 Restore automatic giveback if you disabled it by using the storage failover modify node local auto giveback true command Restore NSE NVE on systems running ONTAP 9 6 and later Steps 1 Connect the console cable to the target node 2 Use the boot_ontap command at the...

Page 35: ...the key management servers If the Restored column yes true you are done and can proceed to complete the replacement process If the Key Manager type external and the Restored column anything other than yes true use the security key manager external restore command to restore the key IDs of the authentication keys If the command fails contact Customer Support If the Key Manager type onboard and the ...

Page 36: ...e cluster is not in quorum or a healthy node shows false for eligibility and health you must correct the issue before shutting down the impaired node see the Administration overview with the CLI If AutoSupport is enabled suppress automatic case creation by invoking an AutoSupport message system node autosupport invoke node type all message MAINT number_of_hours_downh The following AutoSupport mess...

Page 37: ...r the node so that the healthy node continues to serve data from the impaired node storage About this task If you are using NetApp Storage Encryption you must have reset the MSID using the instructions in the Returning SEDs to unprotected mode section of Administration overview with the CLI You must leave the power supplies turned on at the end of this procedure to provide power to the healthy nod...

Page 38: ... 45 56 Errors 5 Check the state of the aggregates by using the storage aggregate show command controller_A_1 storage aggregate show Aggregate Size Available Used State Vols Nodes RAID Status aggr_b2 227 1GB 227 1GB 0 online 0 mcc1 a2 raid_dp mirrored normal 6 Heal the root aggregates by using the metrocluster heal phase root aggregates command mcc1A metrocluster heal phase root aggregates Job 137 ...

Page 39: ... right sides of the controller module 5 Press down on both of the locking latches and then rotate both latches downward at the same time The controller module moves slightly out of the chassis 6 Slide the controller module out of the chassis Make sure that you support the bottom of the controller module as you slide it out of the chassis 7 Set the controller module aside in a safe place and repeat...

Page 40: ...screws from the chassis mount points 2 With two people slide the old chassis off the rack rails in a system cabinet or equipment rack and then set it aside 3 If you are not already grounded properly ground yourself 4 Using two people install the replacement chassis into the equipment rack or system cabinet by guiding the chassis onto the rack rails in a system cabinet or equipment rack 5 Slide the...

Page 41: ... cable management device e Interrupt the normal boot process and boot to LOADER by pressing Ctrl C If your system stops at the boot menu select the option to boot to LOADER f At the LOADER prompt enter bye to reinitialize the PCIe cards and other components g Interrupt the boot process and boot to the LOADER prompt by pressing Ctrl C If your system stops at the boot menu select the option to boot ...

Page 42: ...operly boot_diags 3 Select Scan System from the displayed menu to enable running the diagnostics tests 4 Select Test system from the displayed menu to run diagnostics tests 5 Select the test or series of tests from the various sub menus 6 Proceed based on the result of the preceding step If the test failed correct the failure and then rerun the test If the test reported no failures select Reboot f...

Page 43: ...surviving cluster 5 Verify that the switchback operation has completed metrocluster show The switchback operation is still running when a cluster is in the waiting for switchback state cluster_B metrocluster show Cluster Configuration State Mode Local cluster_B configured switchover Remote cluster_A configured waiting for switchback The switchback operation is complete when the clusters are in the...

Page 44: ...ndisruptive operation during the replacement You must replace the failed component with a replacement FRU component you received from your provider You must be replacing a controller module with a controller module of the same model type You cannot upgrade your system by just replacing the controller module You cannot change any drives or drive shelves as part of this procedure In this procedure t...

Page 45: ...t or password prompt enter system password Take over or halt the impaired node storage failover takeover ofnode impaired_node_name When the impaired node shows Waiting for giveback press Ctrl C and then respond y Option 2 Controller is in a MetroCluster Do not use this procedure if your system is in a two node MetroCluster configuration To shut down the impaired node you must determine the status ...

Page 46: ...aired node you must determine the status of the node and if necessary switch over the node so that the healthy node continues to serve data from the impaired node storage About this task If you are using NetApp Storage Encryption you must have reset the MSID using the instructions in the Returning SEDs to unprotected mode section of Administration overview with the CLI You must leave the power sup...

Page 47: ...override vetoes parameter If you use this optional parameter the system overrides any soft vetoes that prevent the healing operation 4 Verify that the operation has been completed by using the metrocluster operation show command controller_A_1 metrocluster operation show Operation heal aggregates State successful Start Time 7 25 2016 18 45 55 End Time 7 25 2016 18 45 56 Errors 5 Check the state of...

Page 48: ...to Maintenance mode Step 1 Remove the controller module To access components inside the controller module you must remove the controller module from the chassis You can use the following illustration or the written steps to remove the controller module from the chassis Removing the controller module 1 If you are not already grounded properly ground yourself 2 Release the power cable retainers and ...

Page 49: ...u slide it out of the chassis 7 Place the controller module on a stable flat surface 8 On the replacement controller module open the air duct and remove the empty risers from the controller module using the animation illustration or the written steps Removing the empty risers from the replacement controller module 47 ...

Page 50: ...r 1 up and toward air duct lift the riser up and then set it aside d Repeat the previous step for the remaining risers Step 2 Move the power supplies You must move the power supply from the impaired controller module to the replacement controller module when you replace a controller module You can use the following animation illustration or the written steps to move the power supplies to the repla...

Page 51: ...le until the locking tab clicks into place The power supplies will only properly engage with the internal connector and lock in place one way To avoid damaging the internal connector do not use excessive force when sliding the power supply into the system 4 Repeat the preceding steps for any remaining power supplies Step 3 Move the NVDIMM battery To move the NVDIMM battery from the impaired contro...

Page 52: ... the battery cable from the socket 4 Grasp the battery and press the blue locking tab marked PUSH and then lift the battery out of the holder and controller module 5 Move the battery to the replacement controller module 6 Align the battery module with the opening for the battery and then gently push the battery into slot until it locks into place Do not plug the battery cable back into the motherb...

Page 53: ...ut of the socket 2 Move the boot media to the new controller module align the edges of the boot media with the socket housing and then gently push it into the socket 3 Check the boot media to make sure that it is seated squarely and completely in the socket If necessary remove the boot media and reseat it into the socket 4 Lock the boot media in place a Rotate the boot media down toward the mother...

Page 54: ...riser 3 right riser Moving the mezzanine card and riser 3 1 Move PCIe risers one and two from the impaired controller module to the replacement controller module a Remove any SFP or QSFP modules that might be in the PCIe cards b Rotate the riser locking latch on the left side of the riser up and toward air duct The riser raises up slightly from the controller module c Lift the riser up and then mo...

Page 55: ...to the replacement controller module e Install the mezzanine in the replacement controller and secure it with the thumbscrews f Install the third riser in the replacement controller module Step 6 Move the DIMMs You need to locate the DIMMs and then move them from the impaired controller module to the replacement controller module You must have the new controller module ready so that you can move t...

Page 56: ...roller module 4 Move the DIMMs from the impaired controller module to the replacement controller module Make sure that you install the each DIMM into the same slot it occupied in the impaired controller module a Eject the DIMM from its slot by slowly pushing apart the DIMM ejector tabs on either side of the DIMM and then slide the DIMM out of the slot Carefully hold the DIMM by the edges to avoid ...

Page 57: ...ler module After all of the components have been moved from the impaired controller module to the replacement controller module you must install the replacement controller module into the chassis and then boot it to Maintenance mode You can use the following animation illustration or the written steps to install the replacement controller module in the chassis Installing the controller module 1 If...

Page 58: ...f your system stops at the boot menu select the option to boot to LOADER f At the LOADER prompt enter bye to reinitialize the PCIe cards and other components g Interrupt the boot process and boot to the LOADER prompt by pressing Ctrl C If your system stops at the boot menu select the option to boot to LOADER Restore and verify the system configuration AFF A400 After completing the hardware replace...

Page 59: ... 1 In Maintenance mode from the new controller module verify that all components display the same HA state ha config show The HA state should be the same for all components 2 If the displayed system state of the controller module does not match your system configuration set the HA state for the controller module ha config modify controller ha state The value for HA state can be one of the followin...

Page 60: ... must ensure that the healthy node remains down You can safely respond y to these prompts Recable the system and reassign disks AFF A400 Continue the replacement procedure by recabling the storage and confirming disk reassignment Step 1 Recable the system After running diagnostics you must recable the controller module s storage and network connections Steps 1 Recable the system 2 Verify that the ...

Page 61: ...ode Partner Possible State Description node1 node2 false System ID changed on partner Old 151759755 New 151759706 In takeover node2 node1 Waiting for giveback HA mailboxes 4 From the healthy node verify that any coredumps are saved a Change to the advanced privilege level set privilege advanced You can respond Y when prompted to continue into advanced mode The advanced mode prompt appears b Save a...

Page 62: ...a MetroCluster configuration monitor the status of the node metrocluster node show The MetroCluster configuration takes a few minutes after the replacement to return to a normal state at which time each node will show a configured state with DR Mirroring enabled and a mode of normal The metrocluster node show fields node systemid command output displays the old system ID until the MetroCluster con...

Page 63: ...e available to the replacement node However if the impaired node was the only node in the cluster with a license for the feature no configuration changes to the feature are allowed Also using unlicensed features on the node might put you out of compliance with your license agreement so you should install the replacement license key or keys on the replacement node as soon as possible The licenses k...

Page 64: ...hould verify that the LIFs are on their home ports and register the serial number of the replacement node if AutoSupport is enabled and reset automatic giveback 1 Verify that the logical interfaces are reporting to their home server and ports network interface show is home false If any LIFs are listed as false revert them to their home ports network interface revert 2 Register the system serial nu...

Page 65: ...chback command from any node in the surviving cluster 5 Verify that the switchback operation has completed metrocluster show The switchback operation is still running when a cluster is in the waiting for switchback state cluster_B metrocluster show Cluster Configuration State Mode Local cluster_B configured switchover Remote cluster_A configured waiting for switchback The switchback operation is c...

Page 66: ...different procedures depending on the storage system hardware configuration Option 1 Most configurations To shut down the impaired node you must determine the status of the node and if necessary take over the node so that the healthy node continues to serve data from the impaired node storage About this task If you have a cluster with more than two nodes it must be in quorum If the cluster is not ...

Page 67: ...is not in quorum or a healthy node shows false for eligibility and health you must correct the issue before shutting down the impaired node see the Administration overview with the CLI If you have a MetroCluster configuration you must have confirmed that the MetroCluster Configuration State is configured and that the nodes are in an enabled and normal state metrocluster node show Steps 1 If AutoSu...

Page 68: ... the CLI You must leave the power supplies turned on at the end of this procedure to provide power to the healthy node Steps 1 Check the MetroCluster status to determine whether the impaired node has automatically switched over to the healthy node metrocluster show 2 Depending on whether an automatic switchover has occurred proceed according to the following table If the impaired node Then Has aut...

Page 69: ...eck the state of the aggregates by using the storage aggregate show command controller_A_1 storage aggregate show Aggregate Size Available Used State Vols Nodes RAID Status aggr_b2 227 1GB 227 1GB 0 online 0 mcc1 a2 raid_dp mirrored normal 6 Heal the root aggregates by using the metrocluster heal phase root aggregates command mcc1A metrocluster heal phase root aggregates Job 137 Job succeeded Heal...

Page 70: ...tration or the written steps to remove the controller module from the chassis Removing the controller module 1 If you are not already grounded properly ground yourself 2 Release the power cable retainers and then unplug the cables from the power supplies 3 Loosen the hook and loop strap binding the cables to the cable management device and then unplug the system cables and SFPs if needed from the ...

Page 71: ...t out of the chassis 7 Place the controller module on a stable flat surface Step 3 Replace system DIMMs Replacing a system DIMM involves identifying the target DIMM through the associated error message locating the target DIMM using the FRU map on the air duct or the lit LED on the motherboard and then replacing the DIMM You can use the following animation illustration or the written steps to repl...

Page 72: ...en rotate it upward to its completely open position 2 Locate the DIMMs on your controller module 3 Note the orientation of the DIMM in the socket so that you can insert the replacement DIMM in the proper orientation 4 Eject the DIMM from its socket by slowly pushing apart the two DIMM ejector tabs on either side of the DIMM and then slide the DIMM out of the socket Carefully hold the DIMM by the e...

Page 73: ...snap into place over the notches at the ends of the DIMM 8 Close the air duct Step 4 Install the controller module After you have replaced the component in the controller module you must re install the controller module into the chassis and then boot it to Maintenance mode You can use the following animation illustration or the written steps to install the controller module in the chassis Installi...

Page 74: ...management device e Interrupt the normal boot process and boot to LOADER by pressing Ctrl C If your system stops at the boot menu select the option to boot to LOADER f At the LOADER prompt enter bye to reinitialize the PCIe cards and other components g Interrupt the boot process and boot to the LOADER prompt by pressing Ctrl C If your system stops at the boot menu select the option to boot to LOAD...

Page 75: ..._name 3 If automatic giveback was disabled reenable it storage failover modify node local auto giveback true Step 7 Switch back aggregates in a two node MetroCluster configuration After you have completed the FRU replacement in a two node MetroCluster configuration you can perform the MetroCluster switchback operation This returns the configuration to its normal operating state with the sync sourc...

Page 76: ...l cluster_B configured normal Remote cluster_A configured normal If a switchback is taking a long time to finish you can check on the status of in progress baselines by using the metrocluster config replication resync status show command 6 Reestablish any SnapMirror or SnapVault configurations Step 8 Return the failed part to NetApp After you replace the part you can return the failed part to NetA...

Page 77: ...ing at the Attention LED on each fan module 4 Press down the release latch on the fan module cam handle and then rotate the cam handle downward The fan module moves a little bit away from the chassis 5 Pull the fan module straight out from the chassis making sure that you support it with your free hand so that it does not swing out of the chassis The fan modules are short Always support the bottom...

Page 78: ...ioning properly if not you must contact technical support Step 1 Shut down the impaired controller You can shut down or take over the impaired controller using different procedures depending on the storage system hardware configuration Option 1 Most configurations To shut down the impaired node you must determine the status of the node and if necessary take over the node so that the healthy node c...

Page 79: ...is not in quorum or a healthy node shows false for eligibility and health you must correct the issue before shutting down the impaired node see the Administration overview with the CLI If you have a MetroCluster configuration you must have confirmed that the MetroCluster Configuration State is configured and that the nodes are in an enabled and normal state metrocluster node show Steps 1 If AutoSu...

Page 80: ... the CLI You must leave the power supplies turned on at the end of this procedure to provide power to the healthy node Steps 1 Check the MetroCluster status to determine whether the impaired node has automatically switched over to the healthy node metrocluster show 2 Depending on whether an automatic switchover has occurred proceed according to the following table If the impaired node Then Has aut...

Page 81: ...eck the state of the aggregates by using the storage aggregate show command controller_A_1 storage aggregate show Aggregate Size Available Used State Vols Nodes RAID Status aggr_b2 227 1GB 227 1GB 0 online 0 mcc1 a2 raid_dp mirrored normal 6 Heal the root aggregates by using the metrocluster heal phase root aggregates command mcc1A metrocluster heal phase root aggregates Job 137 Job succeeded Heal...

Page 82: ...tration or the written steps to remove the controller module from the chassis Removing the controller module 1 If you are not already grounded properly ground yourself 2 Release the power cable retainers and then unplug the cables from the power supplies 3 Loosen the hook and loop strap binding the cables to the cable management device and then unplug the system cables and SFPs if needed from the ...

Page 83: ...chassis 7 Place the controller module on a stable flat surface Step 3 Replace the NVDIMM battery To replace the NVDIMM battery you must remove the failed battery from the controller module and install the replacement battery into the controller module See the FRU map inside the controller module to locate the NVDIMM battery The NVDIMM LED blinks while destaging contents when you halt the system Af...

Page 84: ...Align the battery module with the opening for the battery and then gently push the battery into slot until it locks into place 7 Plug the battery plug back into the controller module and then close the air duct Step 4 Install the controller module After you have replaced the component in the controller module you must re install the controller module into the chassis and then boot it to Maintenanc...

Page 85: ... not already done so reinstall the cable management device e Interrupt the normal boot process and boot to LOADER by pressing Ctrl C If your system stops at the boot menu select the option to boot to LOADER f At the LOADER prompt enter bye to reinitialize the PCIe cards and other components g Interrupt the boot process and boot to the LOADER prompt by pressing Ctrl C If your system stops at the bo...

Page 86: ... modify node local auto giveback true Step 7 Switch back aggregates in a two node MetroCluster configuration After you have completed the FRU replacement in a two node MetroCluster configuration you can perform the MetroCluster switchback operation This returns the configuration to its normal operating state with the sync source storage virtual machines SVMs on the formerly impaired site now activ...

Page 87: ...o finish you can check on the status of in progress baselines by using the metrocluster config replication resync status show command 6 Reestablish any SnapMirror or SnapVault configurations Step 8 Return the failed part to NetApp After you replace the part you can return the failed part to NetApp as described in the RMA instructions shipped with the kit Contact technical support at NetApp Support...

Page 88: ...l message MAINT number_of_hours_downh The following AutoSupport message suppresses automatic case creation for two hours cluster1 system node autosupport invoke node type all message MAINT 2h 2 Disable automatic giveback from the console of the healthy node storage failover modify node local auto giveback false 3 Take the impaired node to the LOADER prompt If the impaired node is displaying Then T...

Page 89: ...e console of the healthy node storage failover modify node local auto giveback false 3 Take the impaired node to the LOADER prompt If the impaired node is displaying Then The LOADER prompt Go to the next step Waiting for giveback Press Ctrl C and then respond y when prompted System prompt or password prompt enter system password Take over or halt the impaired node For an HA pair take over the impa...

Page 90: ...in If you are unable to resolve the issue contact technical support 3 Resynchronize the data aggregates by running the metrocluster heal phase aggregates command from the surviving cluster controller_A_1 metrocluster heal phase aggregates Job 130 Job succeeded Heal Aggregates is successful If the healing is vetoed you have the option of reissuing the metrocluster heal command with the override vet...

Page 91: ...ameter the system overrides any soft vetoes that prevent the healing operation 7 Verify that the heal operation is complete by using the metrocluster operation show command on the destination cluster mcc1A metrocluster operation show Operation heal root aggregates State successful Start Time 7 29 2016 20 54 41 End Time 7 29 2016 20 54 42 Errors 8 On the impaired controller module disconnect the po...

Page 92: ...oth latches downward at the same time The controller module moves slightly out of the chassis 6 Slide the controller module out of the chassis Make sure that you support the bottom of the controller module as you slide it out of the chassis 7 Place the controller module on a stable flat surface Step 3 Replace the NVDIMM To replace the NVDIMM you must locate it in the controller module using the FR...

Page 93: ...mation illustration or the written steps to replace the NVDIMM The animation shows empty slots for sockets without DIMMs These empty sockets are populated with blanks Replacing the NVDIMM 1 Open the air duct and then locate the NVDIMM in slot 11 on your controller module The NVDIMM looks significantly different than system DIMMs 91 ...

Page 94: ...NVDIMM squarely into the slot The NVDIMM fits tightly in the slot but should go in easily If not realign the NVDIMM with the slot and reinsert it Visually inspect the NVDIMM to verify that it is evenly aligned and fully inserted into the slot 6 Push carefully but firmly on the top edge of the NVDIMM until the ejector tabs snap into place over the notches at the ends of the NVDIMM 7 Close the air d...

Page 95: ...s by rotating the locking latches upward tilting them so that they clear the locking pins gently push the controller all the way in and then lower the locking latches into the locked position The controller module begins to boot as soon as it is fully seated in the chassis Be prepared to interrupt the boot process d If you have not already done so reinstall the cable management device e Interrupt ...

Page 96: ...ule and then reenable automatic giveback 1 Recable the system as needed If you removed the media converters QSFPs or SFPs remember to reinstall them if you are using fiber optic cables 2 Return the node to normal operation by giving back its storage storage failover giveback ofnode impaired_node_name 3 If automatic giveback was disabled reenable it storage failover modify node local auto giveback ...

Page 97: ...surviving cluster 5 Verify that the switchback operation has completed metrocluster show The switchback operation is still running when a cluster is in the waiting for switchback state cluster_B metrocluster show Cluster Configuration State Mode Local cluster_B configured switchover Remote cluster_A configured waiting for switchback The switchback operation is complete when the clusters are in the...

Page 98: ...rage system hardware configuration Option 1 Most configurations To shut down the impaired node you must determine the status of the node and if necessary take over the node so that the healthy node continues to serve data from the impaired node storage About this task If you have a cluster with more than two nodes it must be in quorum If the cluster is not in quorum or a healthy node shows false f...

Page 99: ... a healthy node shows false for eligibility and health you must correct the issue before shutting down the impaired node see the Administration overview with the CLI If you have a MetroCluster configuration you must have confirmed that the MetroCluster Configuration State is configured and that the nodes are in an enabled and normal state metrocluster node show Steps 1 If AutoSupport is enabled su...

Page 100: ... the CLI You must leave the power supplies turned on at the end of this procedure to provide power to the healthy node Steps 1 Check the MetroCluster status to determine whether the impaired node has automatically switched over to the healthy node metrocluster show 2 Depending on whether an automatic switchover has occurred proceed according to the following table If the impaired node Then Has aut...

Page 101: ...eck the state of the aggregates by using the storage aggregate show command controller_A_1 storage aggregate show Aggregate Size Available Used State Vols Nodes RAID Status aggr_b2 227 1GB 227 1GB 0 online 0 mcc1 a2 raid_dp mirrored normal 6 Heal the root aggregates by using the metrocluster heal phase root aggregates command mcc1A metrocluster heal phase root aggregates Job 137 Job succeeded Heal...

Page 102: ...tration or the written steps to remove the controller module from the chassis Removing the controller module 1 If you are not already grounded properly ground yourself 2 Release the power cable retainers and then unplug the cables from the power supplies 3 Loosen the hook and loop strap binding the cables to the cable management device and then unplug the system cables and SFPs if needed from the ...

Page 103: ...ard you must locate the failed PCIe card remove the riser that contains the card from the controller module replace the card and then reinstall the PCIe riser in the controller module You can use the following animation illustration or the written steps to replace a PCIe card Replacing a PCIe card 1 Remove the riser containing the card to be replaced a Open the air duct by pressing the locking tab...

Page 104: ...n seating it in the socket The PCIe card must be fully and evenly seated in the slot If you are installing a card in the bottom slot and cannot see the card socket well remove the top card so that you can see the card socket install the card and then reinstall the card you removed from the top slot 4 Reinstall the riser a Align the riser with the pins to the side of the riser socket lower the rise...

Page 105: ...er module d Lift the riser up and then set it aside on a stable flat surface 2 Replace the mezzanine card a Remove any QSFP or SFP modules from the card b Loosen the thumbscrews on the mezzanine card and gently lift the card directly out of the socket and set it aside c Align the replacement mezzanine card over the socket and the guide pins and gently push the card into the socket d Tighten the th...

Page 106: ...e opening in the chassis and then gently push the controller module halfway into the system Do not completely insert the controller module in the chassis until instructed to do so 3 Recable the system as needed If you removed the media converters QSFPs or SFPs remember to reinstall them if you are using fiber optic cables 4 Complete the installation of the controller module a Plug the power cord i...

Page 107: ...ed reenable it storage failover modify node local auto giveback true Step 6 Restore the controller module to operation To restore the controller you must recable the system give back the controller module and then reenable automatic giveback 1 Recable the system as needed If you removed the media converters QSFPs or SFPs remember to reinstall them if you are using fiber optic cables 2 Return the n...

Page 108: ...surviving cluster 5 Verify that the switchback operation has completed metrocluster show The switchback operation is still running when a cluster is in the waiting for switchback state cluster_B metrocluster show Cluster Configuration State Mode Local cluster_B configured switchover Remote cluster_A configured waiting for switchback The switchback operation is complete when the clusters are in the...

Page 109: ...e unplugging the power cable removing the old PSU and installing the replacement PSU and then reconnecting the replacement PSU to the power source The power supplies are redundant and hot swappable This procedure is written for replacing one power supply at a time It is a best practice to replace the power supply within two minutes of removing it from the chassis The system continues to function b...

Page 110: ...ower supply cabling a Reconnect the power cable to the power supply and the power source b Secure the power cable to the power supply using the power cable retainer Once power is restored to the power supply the status LED should be green 8 After you replace the part you can return the failed part to NetApp as described in the RMA instructions shipped with the kit Contact technical support at NetA...

Page 111: ... to the next step Waiting for giveback Press Ctrl C and then respond y when prompted System prompt or password prompt enter system password Take over or halt the impaired node For an HA pair take over the impaired node from the healthy node storage failover takeover ofnode impaired_node_name When the impaired node shows Waiting for giveback press Ctrl C and then respond y Option 2 Controller is in...

Page 112: ... failover takeover ofnode impaired_node_name When the impaired node shows Waiting for giveback press Ctrl C and then respond y Option 3 Controller is in a two node MetroCluster To shut down the impaired node you must determine the status of the node and if necessary switch over the node so that the healthy node continues to serve data from the impaired node storage About this task If you are using...

Page 113: ... is successful If the healing is vetoed you have the option of reissuing the metrocluster heal command with the override vetoes parameter If you use this optional parameter the system overrides any soft vetoes that prevent the healing operation 4 Verify that the operation has been completed by using the metrocluster operation show command controller_A_1 metrocluster operation show Operation heal a...

Page 114: ... using the metrocluster operation show command on the destination cluster mcc1A metrocluster operation show Operation heal root aggregates State successful Start Time 7 29 2016 20 54 41 End Time 7 29 2016 20 54 42 Errors 8 On the impaired controller module disconnect the power supplies Step 2 Remove the controller module To access components inside the controller module you must remove the control...

Page 115: ...roller module and set it aside 5 Press down on both of the locking latches and then rotate both latches downward at the same time The controller module moves slightly out of the chassis 6 Slide the controller module out of the chassis Make sure that you support the bottom of the controller module as you slide it out of the chassis 7 Place the controller module on a stable flat surface Step 3 Repla...

Page 116: ...tery away from the holder rotate it away from the holder and then lift it out of the holder Note the polarity of the battery as you remove it from the holder The battery is marked with a plus sign and must be positioned in the holder correctly A plus sign near the holder tells you how the battery should be positioned c Remove the replacement battery from the antistatic shipping bag d Note the pola...

Page 117: ...ning in the chassis and then gently push the controller module halfway into the system Do not completely insert the controller module in the chassis until instructed to do so 3 Recable the system as needed If you removed the media converters QSFPs or SFPs remember to reinstall them if you are using fiber optic cables 4 If the power supplies were unplugged plug them back in and reinstall the power ...

Page 118: ...mm dd yyyy command d If necessary set the time in GMT using the set time hh mm ss command e Confirm the date and time on the target node 7 At the LOADER prompt enter bye to reinitialize the PCIe cards and other components and let the node reboot 8 Return the node to normal operation by giving back its storage storage failover giveback ofnode impaired_node_name 9 If automatic giveback was disabled ...

Page 119: ...surviving cluster 5 Verify that the switchback operation has completed metrocluster show The switchback operation is still running when a cluster is in the waiting for switchback state cluster_B metrocluster show Cluster Configuration State Mode Local cluster_B configured switchover Remote cluster_A configured waiting for switchback The switchback operation is complete when the clusters are in the...

Page 120: ... failed part to NetApp as described in the RMA instructions shipped with the kit Contact technical support at NetApp Support 888 463 8277 North America 00 800 44 638277 Europe or 800 800 80 800 Asia Pacific if you need the RMA number or additional help with the replacement procedure 118 ...

Page 121: ...Y WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE NetApp reserves the right to change any products described herein at any time and without notice NetApp assumes no responsibility or liability arising from the use of products described herein except as expressly agree...