IBM Elastic Storage System 5147-102, Service Manual

Page 1: ...IBM Elastic Storage System 5147 102 Storage Enclosure 6 1 5 Service Guide IBM SC31 5732 01...

Page 2: ...roduct number 5765 DME IBM Spectrum Scale Data Access Edition for IBM ESS product number 5765 DAE IBM welcomes your comments see the topic How to submit your comments on page x When you send informati...

Page 3: ...four 5147 102 storage enclosures to an existing ESS 3500 performance configuration 13 Adding up to four 5147 102 storage enclosures to an existing ESS 3500 capacity configuration 19 Adding up to four...

Page 4: ...iv...

Page 5: ...e cover guide pin and flange 8 11 Installation rules to upgrade half populated to fully populated configuration 8 12 Slide cover guide pin and flange 9 13 Thumbscrews at the front edge of the chassis...

Page 6: ...vi...

Page 7: ...ed drives to capacity fully populated configuration 20 5 Model upgrade MES capacity half full populated drives to capacity fully populated configuration 24 6 MES upgrade from hybrid x to hybrid y 27 7...

Page 8: ...viii...

Page 9: ...Installing IBM Spectrum Scale and CES protocols with the installation toolkit see Installing IBM Spectrum Scale on Linux nodes with the installation toolkit Detailed information about the IBM Spectru...

Page 10: ...s to the key on your terminal or workstation that is labeled with the word Enter In command examples a backslash indicates that the command or coding example continues on the next line For example mkc...

Page 11: ...eps to prepare for a service action to release the CMA to access the faulty PSU 4 Unplug the power cord from the faulty PSU jack 5 Remove the faulty PSU 6 Replace the PSU FRU 7 Connect the power cord...

Page 12: ...in working with devices that contain live circuits should operate the system Take precautions when performing all other types of maintenance Use the following replacement parts to service a faulty po...

Page 13: ...e power cord from the faulty PSU jack Warning While some factory approved maintenance on FRUs e g hot swapping a PSU fan I O module regulator or drive does not require the user to disconnect power to...

Page 14: ...ease tab engages and the PSU is fastened securely Note The proper orientation allows the PSU to come into place once it is fully inserted into the enclosure If the PSU slides out of the enclosure easi...

Page 15: ...grep power_supply_config_mismatch Node name ess3500rw6a hs test net Timestamp Event Name Severity Details 2023 01 31 12 15 52 661278 MST power_supply_config_mismatch ERROR Enclosure has an inconsisten...

Page 16: ...e upgraded a Perform the following checks in the essutils menu and gather ESS 5147 102 serial number in the SSR Checks for ESS 3500 nodes section i Quick storage configuration check ii Check enclosure...

Page 17: ...ng the S N for the enclosure that only has 52 HDDs with the previous output 4 Extend the enclosure to serviceable position a Loosen the two left and two right thumbscrews that secure the enclosure to...

Page 18: ...ft the top cover up and away from the enclosure 6 Make sure it has a half populated configuration Important If it is NOT a half populated configuration STOP replace the top lid and return the ESS 5147...

Page 19: ...d right enclosure cover thumbscrews located at the front edge of the chassis cover The thumbscrews are identified in figure Figure 13 Thumbscrews at the front edge of the chassis cover 10 Returning th...

Page 20: ...inside enclosure INFO Enclosure 5147 102 found 1 INFO Disk ST10000NM0578 E found 102 OK Total disk found 102 expected 102 INFO Total ssd found 0 expected 0 INFO End of storage quick configuration chec...

Page 21: ...s3500rw6b_hs root ess3500rw6a b Verify the block size of the existing filesystem mmlsfs device B Example root ess3500rw6a mmlsfs all B File system attributes for dev fs3500hyper flag value description...

Page 22: ...GiB 22 GiB vs_fs3500hyper_1A 1513 MiB vs_fs3500hyper_1B 1513 MiB vs_fs3500hyper_2A 3091 MiB vs_fs3500hyper_2B 3091 MiB vs_fs3500hyper_ess3500rw6a_hs_ess3500rw6b_hs 8620 MiB e Create the new vdisk sets...

Page 23: ...adding up to four IBM Elastic Storage System 5147 102 Storage Enclosures to an existing ESS 3500 performance configuration using single and dual SAS adapter pairs Adding up to two 5147 102 storage enc...

Page 24: ...02 HDD H2 h f 12 NVM e 2 X 102 HDD H3 h f 12 NVMe 3 X 102 HDD H4 h f 12 NVM e 4 X 102 HDD H1 f h 24 NVMe 1 X 52 HDD H1 f f 24 NVM e 1 X 102 HDD H2 f f 24 NVMe 2 X 102 HDD H3 f f 24 NVMe 3 X 102 HDD H4...

Page 25: ...up ess3500_ess3500rw6a_hs_ess3500rw6b_hs mmvdisk Analyzing disk topology for node ess3500rw6b hs test net mmvdisk Analyzing disk topology for node ess3500rw6a hs test net mmvdisk Validating existing p...

Page 26: ...rage enclosures using dual SAS adapter pairs The hybrid configuration consists of one IBM Elastic Storage System 5141 FN2 Server Enclosure and one or more IBM Elastic Storage System 5147 102 Storage E...

Page 27: ...f 12 NVM e 1 X 102 HDD H2 h f 12 NVM e 2 X 102 HDD H3 h f 12 NVMe 3 X 102 HDD H4 h f 12 NVM e 4 X 102 HDD H1 f h 24 NVMe 1 X 52 HDD H1 f f 24 NVM e 1 X 102 HDD H2 f f 24 NVMe 2 X 102 HDD H3 f f 24 NV...

Page 28: ...w6a_hs_ess3500rw6b_hs mmvdisk Analyzing disk topology for node ess3500rw6b hs test net mmvdisk Analyzing disk topology for node ess3500rw6a hs test net mmvdisk Validating existing pdisk locations for...

Page 29: ...his MES procedure involves adding up to four IBM Elastic Storage System 5147 102 Storage Enclosures to an existing capacity configuration using single and dual SAS adapter pairs Adding up to two 5147...

Page 30: ...ty fully populated drives Table 4 Model upgrade MES capacity half full populated drives to capacity fully populated configuration Target Source C1 f 4 NVMe 1 X 102 HDD C2 f 4 NVMe 2 X 102 HDD C3 f 4 N...

Page 31: ...ore adding additional enclosures Refer Upgrading an IBM Elastic Storage System 5147 102 Storage Enclosure from a half populated to a fully populated configuration on page 6 b Use at least one SAS adap...

Page 32: ...sk set count size raw size created file system and attributes vs_fs3500hyper_2A 1 209 TiB 263 TiB no DA2 8 2p 4 MiB dataOnly dataX declustered capacity all vdisk sets defined recovery group array type...

Page 33: ...sk fs add fs filesystem vs vdisk set 1 vdisk set 2 g Recommendation Restripe the filesystem as needed mmrestripefs fsname b mmrestripefs fs3500hyper b Adding up to four 5147 102 storage enclosures usi...

Page 34: ...ty fully populated drives Table 5 Model upgrade MES capacity half full populated drives to capacity fully populated configuration Target Source C1 f 4 NVMe 1 X 102 HDD C2 f 4 NVMe 2 X 102 HDD C3 f 4 N...

Page 35: ...ulated configuration upgrade to a fully populated configuration before adding additional enclosures Refer Upgrading an IBM Elastic Storage System 5147 102 Storage Enclosure from a half populated to a...

Page 36: ...mmvdisk vs create vs all f Add the vdisk sets to the filesystem mmvdisk fs add fs fsname vs vsname1a vsname1b mmvdisk fs add fs filesystem vs vdisk set 1 vdisk set 2 g Recommendation Restripe the file...

Page 37: ...t Source P f 24 NVMe H1 h h 12 NVM e 1 X 52 HDD H1 h f 12 NVM e 1 X 102 HDD H2 h f 12 NVM e 2 X 102 HDD H3 h f 12 NVMe 3 X 102 HDD H4 h f 12 NVM e 4 X 102 HDD H1 f h 24 NVMe 1 X 52 HDD H1 f f 24 NVM e...

Page 38: ...Populate the storage enclosure s with HDDs making sure to follow the drive population rules a One to four fully populated 5147 102storage enclosures If the first enclosure is half populated configura...

Page 39: ...for dev fs3500hyper flag value description B 4194304 Block size root ess3500rw6a c Verify all DAs have appropriate number of drives Note You need to have either 52 or 102 drives if you are adding only...

Page 40: ...iB vs_fs3500hyper_2B 3091 MiB vs_fs3500hyper_ess3500rw6a_hs_ess3500rw6b_hs 8620 MiB e Create the new vdisk sets mmvdisk vs create vs all Example root ess3500rw6a mmvdisk vs create vs all mmvdisk All v...

Page 41: ...root ess3500rw6a g Recommendation Restripe the filesystem as needed mmrestripefs fsname b mmrestripefs fs3500hyper b Adding up to four 5147 102 storage enclosures using dual SAS adapter pairs The hyb...

Page 42: ...ll additional pair of SAS adapter if required Refer ESS 3500 adapter concurrent MES upgrade to follow the instructions 2 If you are adding a new SAS adapter pair re cabling is required Refer Re cablin...

Page 43: ...tion B 4194304 Block size root ess3500rw6a c Verify all DAs have appropriate number of drives Note You need to have either 52 or 102 drives if you are adding only one enclosure When installing two enc...

Page 44: ...iguration that includes four 3 8TB NVMe drives in its 5141 FN2 server enclosure as the system s initial configuration for journal information When the MES is successfully completed the ESS 3500 soluti...

Page 45: ...ses an issue with the customer configuration during MES upgrade 4 Identify the 5141 FN2 server enclosure to be upgraded 5 Remove the drive fillers from the slots 3 12 and 15 24 6 Verify that the 20 ne...

Page 46: ...00rw3a hs Rack Bodhi Rack 1520 U25 26 Enclosure 5141 FN1 78E400K Drive 10 drive a8241014065c 78E400K SN5ASN5A SN5ASN5A ess3200rw3a hs Rack Bodhi Rack 1520 U25 26 Enclosure 5141 FN1 78E400K Drive 11 dr...

Page 47: ...241014065c 78E400K SN5ASN5A SN5ASN5A ess3200rw3a hs Rack Bodhi Rack 1520 U25 26 Enclosure 5141 FN1 78E400K Drive 15 drive a8241014065c 78E400K SN5ASN5A SN5ASN5A ess3200rw3a hs Rack Bodhi Rack 1520 U25...

Page 48: ...SN5A dev nvme19n1 S54KNE0NA00118 3 84TB NVMe Tier 1 Flash 1 3 84 TB 3 84 TB 4 KiB 8 B SN5ASN5A dev nvme1n1 202228CD166E Micron_7300_MTFDHBA960TDF 1 960 20 GB 960 20 GB 4 KiB 0 B 95420100 dev nvme20n1...

Page 49: ...20210816 mmvdisk Restarting GPFS daemon on node ess3200rw3a hs gpfs ess mmvdisk Restarting GPFS daemon on node ess3200rw3b hs gpfs ess Important This command automatically stops and restarts GPFS on e...

Page 50: ...n mmvdisk vdiskset define vdisk set vs_fs3200_2 recovery group ess3200_78E400K code 8 2p block size 4m set size 40 date mmvdisk Vdisk set vs_fs3200_2 has been defined mmvdisk Recovery group ess3200_78...

Page 51: ...is no irrespective of the TRIM settings of the existing vdisk sets Issue the mmvdisk filesystem add command with the trim auto no option to select TRIM settings when you are adding a vdisk set to a fi...

Page 52: ...SAS adapter a Record the SAS adapter and the ports to which the SAS cables are currently connected This information is needed and will be used in later step to restore the connection b Refer ESS 3500...

Page 53: ...d with half population configurations FRU part number list The FRU part numbers are listed in the table FRU Part Number Table 11 FRU Part Numbers Description Part Number Enclosure Rail Kit 03JN421 Cab...

Page 54: ...44 IBM Elastic Storage System 5147 10 Storage Enclosure Service Guide...

Page 55: ...nly used by screen readers Keys that are discernible by touch but do not activate just by touching them Industry standard devices for ports and connectors The attachment of alternative input and outpu...

Page 56: ...46 IBM Elastic Storage System 5147 10 Storage Enclosure Service Guide...

Page 57: ...MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND EITHER EXPRESS OR IMPLIED INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF NON INFRINGEMENT MERCHANTABILITY OR...

Page 58: ...in any form without payment to IBM for the purposes of developing using marketing or distributing application programs conforming to the application programming interface for the operating platform f...

Page 59: ...s of these publications or reproduce distribute or display these publications or any portion thereof outside your enterprise without the express consent of IBM Rights Except as expressly granted in th...

Page 60: ...50 IBM Elastic Storage System 5147 10 Storage Enclosure Service Guide...

Page 61: ...e central processor complex CPC central processor complex CPC A physical collection of hardware that consists of channels timers main storage and one or more central processors cluster A loosely coupl...

Page 62: ...er that attaches to the storage enclosure drives In the case of multiple drawers in a storage enclosure the ESM attaches to drawer control modules ESM See environmental service module ESM F failback C...

Page 63: ...qualified domain name FQDN The complete domain name for a specific computer or host on the Internet The FQDN consists of two parts the hostname and the domain name G GPFS cluster A cluster of nodes de...

Page 64: ...that contains programs for such tasks as input output management and control of hardware and the scheduling of user tasks L LACP See Link Aggregation Control Protocol LACP Link Aggregation Control Pr...

Page 65: ...An individual operating system image within a cluster Depending on the way in which the computer system is partitioned it can contain one or more nodes In a Power Systems environment synonymous with l...

Page 66: ...involving either one s operating system This permits high throughput low latency networking which is especially useful in massively parallel computer clusters RGD See recovery group data RGD remote k...

Page 67: ...TCP See Transmission Control Protocol TCP Transmission Control Protocol TCP A core protocol of the Internet Protocol Suite that provides reliable ordered and error checked delivery of a stream of oct...

Page 68: ...58 IBM Elastic Storage System 5147 10 Storage Enclosure Service Guide...

Page 69: ...ts x D documentation on web ix I information overview ix L license inquiries 47 N notices 47 O overview of information ix P patent information 47 preface ix R resources on web ix S submitting x T trad...

Page 70: ...60 IBM Elastic Storage System 5147 10 Storage Enclosure Service Guide...

Page 71: ......

Page 72: ...IBM Product Number 5765 DME 5765 DAE SC31 5732 01...