Cray CX1000, Site Preparation Manual

Page 1: ...Site Preparation Guide ...

Page 2: ...trademarks mentioned in this manual All brand names and software and hardware product names are subject to trademark and or patent protection Quoting of brand and product names is for information purposes only and does not represent trademark misuse The information in this document is subject to change without notice Cray will not be liable for errors contained herein or for incidental or conseque...

Page 3: ...viii viii viii viii viii viii viii viii ix ix ix x x x xi xi xi Preface Intended Readers Highlighting xii xii xii Chapter 1 Fitting out the Premises 1 1 1 2 1 3 1 3 1 4 1 4 1 5 1 5 1 5 1 6 1 7 1 7 1 7 1 8 1 8 1 8 1 9 1 9 1 1 1 2 1 3 1 4 1 5 1 5 1 General Recommendations Delivery Conditions Carry In Route Data Center Layout Planning Installation Cabinet Cut outs 1 5 1 1 1 5 1 2 1 5 1 3 1 5 2 1 5 2 ...

Page 4: ...m Specifications Air Renewal Air Distribution Air Circulation Air Pollutants Checks and Alarms Chapter 3 Power Supply 3 1 3 2 3 3 3 3 3 3 3 4 3 6 3 6 3 7 3 8 3 9 3 9 3 9 3 1 3 2 3 2 1 3 2 2 3 3 3 4 3 4 1 3 4 2 3 5 3 6 3 6 1 3 6 2 Power Supply Cabling Requirements Power Cables Network Cables Power Supply Cable Sockets Grounding Circuit Electric Protection Wire Characteristics Grounding the Raised F...

Page 5: ...Appendix A Notices A 1 A 2 A 3 A 5 A 6 A 8 A 1 A 2 A 3 A 4 A 5 DANGER Notices CAUTION Notices WARNING Notices IMPORTANT Notices NOTES Notices Index x 1 Contents v ...

Page 6: ...inet service clearance single and dual Example of water cooled service clearance single and dual Raised floor cross section Raised floor structure Example of poor air distribution Example of obstacles that may obstruct air circulation Plug and socket US plug characteristics PDU UPS Power socket Raised floor equipotentiality method Grounding circuit diagram for a TN S neutral point connection Water...

Page 7: ...List of Tables Table 1 Table 2 Chilled water characteristics Water treatment recommendations 4 2 4 3 Contents vii ...

Page 8: ...unity EC Council Directives This product is in conformity with the protection requirements of the following EC Council Directives Electromagnetic Compatibility 2004 108 EC Low Voltage 2006 95 EC EC Conformity 93 68 EEC Telecommunications Terminal Equipment 1999 5 EC Neither the provider nor the manufacturer can accept responsibility for any failure to satisfy the protection requirements resulting ...

Page 9: ...must be used in order to meet FCC emission limits Neither the provider nor the manufacturer are responsible for any radio or television interference caused by using other than recommended cables and connectors or by unauthorized changes or modifications to this equipment Unauthorized changes or modifications could void the user s authority to operate the equipment Pursuant to Part 15 21 of the FCC...

Page 10: ...e observed when connecting or disconnecting devices to the system DANGER The Customer is responsible for ensuring that the AC electricity supply is compliant with national and local recommendations regulations standards and codes of practice An incorrectly wired and grounded electrical outlet may place hazardous voltage on metal parts of the system or the devices that attach to the system and resu...

Page 11: ...e hazardous The optical drive contains internally a Class 3B gallium arsenide laser that is nominally 30 milliwatts at 830 nanometers The design incorporates a combination of enclosures electronics and redundant interlocks such that there is no exposure to laser radiation above a Class 1 level during normal operation user maintenance or servicing conditions Data Integrity and Verification WARNING ...

Page 12: ...hting The following highlighting conventions are used in this guide Bold Identifies the following Interface objects such as menu names labels buttons and icons File directory and path names Keywords to which particular attention must be paid Italics Identifies references such as manuals or URLs monospace Identifies portions of program codes command lines or messages displayed in command windows Id...

Page 13: ...n page 1 3 Carry In Route on page 1 3 Data Center Layout on page 1 4 Planning Installation on page 1 4 Electrical Electromagnetic Interference on page 1 8 Fire Protection on page 1 8 Water Damage on page 1 10 Computer Room Floor on page 1 10 Computer Room Ceiling on page 1 12 Computer Room Environment on page 1 12 Media Storage Room on page 1 13 Printing Room on page 1 13 Chapter 1 Fitting out the...

Page 14: ...RNING The installation of an on line Uninterruptible Power Supply UPS is strongly recommended to ensure continuity of service and to protect the equipment in the event of a mains failure Note In the case of a water cooled equipement installation the customer is responsible for supplying the complete water installation from the water supply station through to the water pipe outlets at the base of t...

Page 15: ... only 1 3 Carry In Route The route from the unloading bay to the Computer Room must be checked to ensure that the equipment can be carried into the premises easily and that floors can support temporary overloads mportant Access doors must be at least 2 10 meters high to allow rack mounted equipment to be transported vertically CAUTION The equipment must be transported as indicated on packing label...

Page 16: ...hers must be provided around equipment and cabinets Note It is recommended to make room for the removal of packing items in the Computer Room and ancillary premises 1 5 Planning Installation Installation must be planned so that any changes and or extensions may be implemented with minimum service disruption It is recommended to plan installation of the following elements cabinets and equipment cab...

Page 17: ... Cut outs The standard cut out for cables is shown in the following figure 15 cm 6 in 35 cm 14 in Figure 1 Standard 1100 installation cable cut out 1 5 1 2 Standard 1200 Cabinet Cut outs The standard cut out for cables is shown in the following figure Figure 2 Standard 1200 installation cable cut out Chapter 1 Fitting out the Premises 1 5 Cable cut out ...

Page 18: ...water cooled cabinets cut outs must be made in the raised floor slabs to allow correct water circulation as shown in the following figure mportant The water cooled cabinet must be aligned with slab sides Figure 3 Water cooled 1200 cabinet cable and water supply cut outs 1 6 Site Preparation Guide ...

Page 19: ...o plan ahead and to provide space for extenstions 1 5 2 1 Standard Cabinet Service Clearance Single and Dual 150 cm 100 cm Figure 4 Example of standard cabinet service clearance single and dual 1 5 2 2 Water Cooled Cabinet Service Clearance Single and Dual 59 in cm Rear 59 in Figure 5 Example of water cooled service clearance single and dual Chapter 1 Fitting out the Premises 1 7 150 cm Front in 3...

Page 20: ...rotect equipment in the event of a mains failure The UPS may be installed in the Computer Room or in an ancillary room 1 7 Fire Protection For effective fire protection the fire detection and extinction system must be specially designed for the Data Processing Center which must be located in a fire area surrounded by fire protection walls partitions and doors 1 7 1 Fire Detection The Customer must...

Page 21: ... contained fires mportant If the equipment is installed in a room equipped with water sprinklers or sprays water pipes must only be loaded subsequent to a primary detection 1 7 3 Extinguishing Agent In compliance with APSAD R13 commonly used extinguishing agents are inert gases such as ARGONITE Argon based and INERGEN water in the form of sprinklers or sprays pending certification CAUTION It is st...

Page 22: ...pped with a drain 1 9 2 Raised Floor CAUTION A raised floor compliant with the Standards NF P67 103 1 and 103 2 in force is mandatory in particular for the installation of a Cooling Unit CAUTION If the air conditioning unit s is are placed on brackets in the room it is mandatory to electrically isolate these brackets from the raised floor slabs crossbeams and actuators A raised floor allows even d...

Page 23: ...g the system slabs galvanized steel or aluminium trays inside which high density particle boards are bonded and covered with plastic laminate or vinyl flooring volume resistivity 104 1010 ohms plenum space 30 cm 60 cm in the case of a cooling unit installation to allow the correct circulation of conditioned air slab surface flatness must be checked by laser Figures 6 and 7 show recommended raised ...

Page 24: ... the room improve the appearance of the room allow light fittings to be flush mounted conceal pipes ducts or channels glass or rock wool contained in a dust proof bag or cushion suspended ceiling height 2 50 m plenum space 20 cm 1 11 Computer Room Environment The Computer Room must also meet the environmental conditions set out below 1 11 1 Sound Proofing The sound level in the Computer Room may b...

Page 25: ...licable The Media Storage Room floor must be designed to support loads of up to 10 kN m2 Media is sensitive to high power electromagnetic fields Media must not be stored in the vicinity of electric ducts lightning rod chutes transformers etc 1 13 Printing Room It is strongly recommended to install all printers including non impact printers laser magnetographic printers etc in a dedicated room with...

Page 26: ...ollowing topics Air Conditioning System Features and Functions on page 2 2 Air Conditioning System Specifications on page 2 3 Checks and Alarms on page 2 5 CAUTION An air conditioning system compliant with the requirements of NF X44 101 Class 4 000 000 and with ISO 7730 is recommended Chapter 2 Air Conditioning 2 1 ...

Page 27: ...ssipated by these configurations ranges from 5 kW to several hundred kW mportant To ensure equipement reliability the climatic requirements for the computing environment are temperature 22 3 C gradient 3 C hour hygrometry 50 5 gradient 5 hour Optimum operation of water cooled equipment is reached when ambiant air temperature is set to 20 C 2 1 1 Temperature Control Sudden changes in temperature gr...

Page 28: ...h air supply is recommended for the comfort and health of personnel The fresh air supply must be directly input at air conditioning unit level and must be filtered in compliance with local health authority regulations D D A S S Regulations in France The Data Processing Center is over pressurized via the fresh air supply which must be strictly limited to 1 1 5 times room air per hour Over pressuriz...

Page 29: ...e raised floor plenum must not be obstructed by oversize pipes or cableways raised floor slabs must be correctly fitted WARNING Do not place perforated damper slabs in the immediate vicinity of one or more air conditioning units as shown in the following figure Air conditioning unit m Deflector if 350 mm Tubes to draught Figure 8 Example of poor air distribution Air Leaks Cables Piping H Figure 9 ...

Page 30: ...l However if there are chemical or industrial processing factories in the vicinity if chemical odors are present or if any other symptom or inherent problem is detected the site must be analyzed immediately 2 3 Checks and Alarms The air conditioning system is controlled automatically Variations outside the specified temperature and hygrometry ranges must be reported to support personnel Detectors ...

Page 31: ... on page 3 2 Cabling Requirements on page 3 3 Power Supply Cable Sockets on page 3 4 Grounding Circuit on page 3 6 Neutral Power System on page 3 8 Non Computing Power Supply on page 3 9 DANGER An AC power supply compliant with the Lightning Protection Standards NF C15 100 and IEC 62305 in force is mandatory Chapter 3 Power Supply 3 1 ...

Page 32: ...e corresponding electrical installation in compliance with specifications Provision and installation of the electric power supply panel and required protections Provision of power supply cables from the electricity distribution panel up to the hardware connection point Provision of grounded plug sockets for the connection of the system to the power supply Provision and installation of an emergency...

Page 33: ...in force In the case of a non raised floor equipped computer room this standard contains recommendations about the cabling way and protection WARNING Cables must be correctly labeled at both ends with the following information the equipment to which each cable end is being connected the physical location of the other end of the cable ID building floor column the length of the cable run 3 2 1 Power...

Page 34: ...and installing appropriate sockets for connection to the site power supply Note Power Cables are hard wired directly to the UPS without a plug except UPS 7 5 10 KVA models Figure 10 Plug and socket United States of America Equipment power cables may be equipped with ready mounted NEMA L6 30P plugs The Customer is responsible for supplying and installing appropriate sockets for connection to the si...

Page 35: ...s long enough Unscrew the base of the socket insert the cable through the socket base and wire to the socket head Figure 12 PDU UPS Power socket 5 6 Screw the socket base back to the socket head Check for 0 1 ohm resistance between the grounding pin on the power cable plug and the metal frame with the multimeter Connect the power supply cable socket to the power cable plug Use velcro fasteners to ...

Page 36: ...ndards in force A single equipotential grounding circuit dedicated to data processing systems is required If the building has several separate grounding shafts it is mandatory for them to be interconnected to eliminate the occurrence different transient potentials within the same unit 3 4 1 Electric Protection Wire Characteristics CAUTION Electric protection wire characteristics must be compliant ...

Page 37: ...aid under the raised floor structure Raised floor actuators must be grounded with 10 mm x 2 mm flat tin plated braiding Figure 13 Raised floor equipotentiality method Raised floor slab 60 cm x 60 cm Meshing braid 10 mm2 cross section tin plated copper braid At each strut the meshing braid is attached to the actuator via a metal collar Chapter 3 Power Supply 3 7 ...

Page 38: ...he power supply interface the Neutral point is directly connected to the grounding shaft measurement strip or to the nearest outlet This connection is called the Neutral reference wire DANGER When defining a neutral point connection the output Neutral point of a power source or power supply interface must always be directly connected referenced to the nearest grounding shaft strip and never to a g...

Page 39: ...ted from the data processing power supply by a specific distribution unit The position of lights must ensure homogeneous lighting It is preferable to flush mount lights in the suspended ceiling Average lighting measured at 0 85 m from the ground must be 500 Lux ISO 8995 future EN 12464 1 Lighting options are Instant lighting tubes equipped with interference suppressors to limit electromagnetic wav...

Page 40: ...ter Requirements on page 4 2 Water Cooled Cabinet Chilled Water Supply Requirements on page 4 3 Preparing the Cooling Unit for the Connection of Water Cooled Cabinets on page 4 4 Cooling Unit Fixtures for Water Cooled Cabinets on page 4 4 mportant The water supply installation should be performed by an air conditioning specialist plumber Chapter 4 Water Supply 4 1 ...

Page 41: ...lowing table lists recommended chilled water characteristics Table 1 Chilled water characteristics 4 2 Site Preparation Guide Chilled Water Characteristics Levels Unit of Measurement pH values 7 Carbonate hardness 3 8 dH Free carbon dioxide 8 15 mg dm3 Bound carbon dioxide 8 15 mg dm3 Agressive carbon dioxide 0 mg dm3 Sulphides 10 mg dm3 Oxygen 50 mg dm3 Chloride ions 250 mg dm3 Sulphate ions 10 m...

Page 42: ...nput water temperature 7 C Room temperature 25 C input water temperature 12 C Waterflow 2 liters per second Pressure drop 140 kPa typical at 2l second mportant For other site options please contact your Customer Service Representative Chapter 4 Water Supply 4 3 Impurity Treatment Mechanical pre treatment dp 1 mm Filter the water Excessive hardness Soften the water using ion exchange Moderate conte...

Page 43: ...e for connection and disconnection The following figure shows an example of plumbing installation for a water cooled cabinet Figure 15 Water cooled cabinet connection example mportant You are advised to clearly label the inlet and outlet water hoses 4 4 Cooling Unit Fixtures for Water Cooled Cabinets You are responsible for providing all required water supply circuit fixtures such as Armored hoses...

Page 44: ...ards that have the potential of causing death or serious personal injury CAUTION Notices call attention to the presence of hazards that have the potential of causing moderate or minor personal injury WARNING Notices call attention to actions that have the potential of causing damage to a program device system or data IMPORTANT Notices call attention to recommended practices NOTES Notices notices g...

Page 45: ...y to separate the power supply for the equipment from the non computing power supply DANGER For the protection of personnel and hardware it is mandatory for the grounding network to be compliant with Data Processing Center requirements and the standards in force A single equipotential grounding circuit dedicated to data processing systems is required If the building has several separate grounding ...

Page 46: ... to use CO2 gas hazard to personnel and to equipment The use of HALON 1301 gas is prohibited as from January 1 2003 CAUTION A raised floor compliant with the Standards NF P67 103 1 and 103 2 in force is mandatory in particular for the installation of a Cooling Unit CAUTION If the air conditioning unit s is are placed on brackets in the room it is mandatory to electrically isolate these brackets fr...

Page 47: ...ocated within the immediate vicinity of the system and to be easily accessible Each power line must be connected to a separate independent electrical panel and bipolar circuit breaker France Power sockets and plugs must be compliant with the Decree 88 1056 Article 20 IV in force CAUTION To be performed by a certified electrician only CAUTION Electric protection wire characteristics must be complia...

Page 48: ...oor prior to installing the system WARNING To avoid incorrect handling the equipment must be removed from its packing by authorized Customer Service Engineers only WARNING If the air conditioning system is shut down operational conditions must be re established in compliance with temperature 3 per hour and hygrometry 5 per hour gradients AT LEAST ONE HOUR before powering up equipment WARNING Do no...

Page 49: ...applied and complied with in other countries mportant Site layout must provide sufficient clearance around equipment for easy access addition of extensions and protection from direct sunlight Wall and floor carpets must not to be used in the Data Processing Center mportant Full access to equipment is mandatory for maintenance Customers are advised to plan ahead and to provide space for extensions ...

Page 50: ...s connected to the structures are hazardous to people to the structures themselves their contents and installations as well as to services The application of lightning protection measures in compliance with the Lightning Protection Standards NF C15 100 and IEC 62305 in force is essential mportant The raceways must comply with the European requirements of the Standard EN 50 174 1 2 in force In the ...

Page 51: ...lass 4 000 000 in force Notes The air conditioning system dries conditioned air It is therefore mandatory for the air conditioning system to include a steam generation humidifier Radiators fitted with thermostatic valves are acceptable in certain ancillary premises only Note The Computer Room and ancillary premises must be cleaned thoroughly at frequent intervals Note Modem cable connectors differ...

Page 52: ...er installation from the water supply station through to the water pipe outlets at the base of tha cabinets The water installation must comply with the standards in force Note For guidance an installation using100 mm section pipes allows chilled water to be supplied to four water cooled cabinets Appendix A Notices A 9 ...

Page 53: ... 3 1 3 2 ambient lighting 3 9 cable sockets 3 4 emergency lighting 3 9 grounding circuit 3 6 3 7 neutral power system 3 8 non computing 3 9 printing computer room 1 13 service clearance 1 7 standard cabinet 1 7 water cooled cabinet 1 7 sound proofing computer room 1 12 sunlight protection computer room 1 12 water damage 1 10 water supply 4 1 chilled water requirements 4 2 4 3 cooling unit fixtures...

Page 54: ...ectric protection wire 3 6 raised floor 3 7 grounding wire 3 6 neutral power system data center 3 8 non computing data center 3 9 PDU connecting 3 5 Preparing cooling unit water supply water cooled cabinet 4 4 Printing computer room data center 1 13 Protection fire 1 8 extinguishing agent 1 9 extinguishing procedure 1 9 G Grounding raised floor power supply 3 7 Grounding circuit power supply 3 6 e...

Page 55: ...ta center 4 1 chilled water requirements 4 2 4 3 cooling unit fixtures 4 4 preparing cooling unit 4 4 preparing cooling unit water cooled cabinet 4 4 Water cooled 1200 cabinet cut outs 1 6 Water cooled cabinet chilled water requirements water supply 4 3 cooling unit fixtures water supply 4 4 preparing cooling unit water supply 4 4 service clearance data center 1 7 Index x 3 ...

Page 56: ...Cray ...