Oracle Exalytics In-Memory Machine, Owner'S Manual

Page 1: ...Oracle Exalytics In Memory Machine Owner s Guide Release 2 0 0 2 0 E93626 01 September 2018 ...

Page 2: ... license restrictions applicable to the programs No other rights are granted to the U S Government This software or hardware is developed for general use in a variety of information management applications It is not developed or intended for use in any inherently dangerous applications including applications that may create a risk of personal injury If you use this software or hardware in dangerou...

Page 3: ...n 1 3 1 5 1 Front Panel Features 1 3 1 5 2 Back Panel Features 1 4 1 6 Connecting the Cables 1 7 1 7 Connecting the Power Cords 1 7 2 Introduction 2 1 About Oracle Exalytics In Memory Machine 2 1 2 2 Hardware Configuration 2 1 2 3 Assigning PCIe Slots 2 4 2 4 Key Features 2 5 2 5 Environmental Specifications 2 5 2 6 Safety and Compliance Information 2 6 3 Installing Oracle Exalytics In Memory Mach...

Page 4: ...0 IP Address 4 5 4 6 Connecting Exalytics In Memory Machine to Your Network 4 6 4 7 Configuring the Network for Oracle Exalytics In Memory Machine 4 7 4 7 1 Overview 4 7 4 7 1 1 InfiniBand 4 8 4 7 1 2 10 GbE Interfaces 4 8 4 7 1 3 1 GbE Interfaces 4 9 4 7 1 4 Host Name 4 9 4 7 1 5 Time 4 10 4 7 1 6 Domain Name Server DNS 4 10 4 7 2 State of the Oracle Exalytics In Memory Machine Before Running the...

Page 5: ...n SP Network Information Is Lost 6 5 6 5 4 Configuring ILOM IP Addresses Manually 6 6 7 Connecting Oracle Exalytics In Memory Machine to Oracle Exadata Database Machine 7 1 Understanding InfiniBand Card Cable and Switch Firmware Requirements 7 1 7 1 1 InfiniBand Card 7 1 7 1 2 InfiniBand Cables 7 1 7 1 3 InfiniBand Switches in the Oracle Exadata Database Machine 7 2 7 2 Physically Connecting the N...

Page 6: ...st ASR Software 9 1 About Oracle Auto Service Request ASR 9 1 9 2 Recommended Configuration 9 2 9 3 Before You Begin 9 2 9 4 Installing ASR Manager on a Standalone System 9 5 9 5 Activating Exalytics Assets Manually 9 5 9 6 Viewing and Verifying ASR Assets ASR Manager 9 5 9 7 Configuring Telemetry Sources 9 6 9 8 Enabling Exalytics Machine Assets in My Oracle Support 9 6 9 9 Sending Test Alerts 9 ...

Page 7: ... the Back of the Right Slide Rail 3 9 3 10 Inserting CMA Slide Rail Connector Into the Back of the Right Slide Rail 3 9 3 11 Inserting CMA Slide Rail Connector Into the Back of the Left Slide Rail 3 10 3 12 Installing CMA Cable Straps 3 11 4 1 Network Ports on the Back of the Oracle Exalytics In Memory Machine X6 4 X5 4 and X4 4 4 2 4 2 Network Ports on the Back of the Oracle Exalytics In Memory M...

Page 8: ...tal Specifications for Oracle Exalytics In Memory Machine 2 6 2 7 Product Safety Specifications Data Sheet Content 2 7 2 8 Electromagnetic Compatibility EMC Standards Regulations 2 7 2 9 Certification Markings on product 2 7 3 1 Rack Compatibility for Oracle Exalytics In Memory Machine 3 2 3 2 Figure Legend 3 3 3 3 Figure Legend 3 9 3 4 Figure Legend 3 10 3 5 Figure Legend 3 10 3 6 Figure Legend 3...

Page 9: ...paring your site for Oracle Exalytics In Memory Machine deployment consult your hardware specification This guide applies to Oracle Exalytics X2 4 X3 4 X4 4 X5 4 and X6 4 Release 2 2 2 This preface contains the following sections Audience Documentation Accessibility Related Documents Conventions Audience This guide is intended for Oracle Exalytics In Memory Machine customers and those responsible ...

Page 10: ...iness Intelligence Enterprise Edition Documentation Oracle Exalytics In Memory Machine Installation and Administration Guide for Linux Conventions The following text conventions are used in this document Convention Meaning boldface Boldface type indicates graphical user interface elements associated with an action or terms defined in text or the glossary italic Italic type indicates book titles em...

Page 11: ...r other pointed device for pushing front panel buttons You also need a system console device such as one of the following A laptop or Sun workstation ASCII terminal Terminal server 1 2 Installation Tasks This section summarizes a list of tasks that you must perform to install the Oracle Exalytics In Memory Machine Table 1 1 Server Installation Tasks Step Description For More Information 1 Unpack t...

Page 12: ...ctions Optional Oracle Server Documentation and Media Kit including the following License and safety documentation Tools and Drivers DVD includes drivers and additional software Oracle Hardware Installation Assistant CD and SunVTS CD 1 4 ESD Precautions Electronic equipment is susceptible to damage by static electricity Use a grounded antistatic wrist strap foot strap or equivalent safety equipmen...

Page 13: ...tics In Memory Machine Front Panel Table 1 2 describes the front panel buttons and LED indicators Table 1 2 Figure Legend Number Description 1 Locator LED Locator button white 2 Service Action Required LED amber 3 Main Power OK LED green 4 Power button 5 SP OK Fault LED green amber 6 Service Action Required LEDs 3 for Fan Module FAN Processor CPU and Memory amber 7 Power Supply PS Fault Service Ac...

Page 14: ...ard disk drive 4 16 Hard disk drive 5 1 5 2 Back Panel Features Figure 1 2 and Figure 1 3 shows the back panel of the Oracle Exalytics In Memory Machine X2 4 and Oracle Exalytics In Memory Machine X3 4 respectively Figure 1 2 Oracle Exalytics In Memory Machine X2 4 Back Panel Chapter 1 Oracle Exalytics In Memory Machine Description 1 4 ...

Page 15: ... supply unit 1 status indicator LEDs Service Action Required amber DC OK green AC OK green or amber 4 Power supply unit 1 AC inlet 5 System status LEDs Power OK green Attention amber Locate white 6 PCIe card slots 0 4 7 Cluster card slot 8 Network NET 10 100 1000 ports NET0 NET3 9 USB 2 0 connectors 2 10 PCIe card slots 5 9 11 Service processor SP network management NET MGT port 12 Serial manageme...

Page 16: ... Required amber DC OK green AC OK green or amber 2 Power supply unit 0 AC inlet 3 Power supply unit 1 status indicator LEDs Service Action Required amber DC OK green AC OK green or amber 4 Power supply unit 1 AC inlet 5 System status LEDs Power OK green Attention amber Locate white 6 PCIe card slots 1 6 7 Network NET 10 100 1000 ports NET0 NET3 8 USB 2 0 connectors 2 9 PCIe card slots 7 11 10 Serv...

Page 17: ...ine interface CLI using the serial management port connect a serial null modem cable to the RJ 45 serial port labeled SER MGT See the part 12 in Figure 1 2 1 7 Connecting the Power Cords Connect the power cords to the Oracle Exalytics In Memory Machine as follows 1 Connect two grounded server power cords to grounded electrical outlets 2 Connect the two server power cords to the AC connectors on th...

Page 18: ...lementation It eliminates the risk associated with procuring the system infrastructure from multiple vendors Oracle s technology is designed to scale seamlessly from small departmental installations to large scale enterprise Business Intelligence deployments Sun Oracle Database Machine combines the reliability of quality servers with the capabilities of Exadata Storage Servers Sun Oracle Database ...

Page 19: ... memory modules 32 GB DDR3 LRDIMM 1600 total system memory 2 TB per server upgradable to 3 TB For X4 4 Sixty four dual in line memory modules 32 GB DDR3 LRDIMM 1600 total system memory 2 TB per server upgradable to 3 TB For X3 4 Sixty four dual in line memory modules 32 GB DDR3 RDIMM 1066 total system memory 2 TB per server For X2 4 Sixty four dual in line memory modules LV DIMM 16 GB DDR3 1066 1 ...

Page 20: ...e 6 4TB V NAND Flash F640 NVMe AIC 19 2 TB total Aura 6 For X5 4 Three 1 6 TB NVMe PCIe Flash Cards 4 8 TB total For X4 4 Three 800 GB Sun FlashAccelerator F80 PCIe cards For X3 4 Six 400 GB Sun FlashAccelerator F40 PCIe cards Accessories Rack Mount Slide Rail Kit One tool less rack mount slide rail kit Cable Management Arm CMA One cable management arm Ports and I O Slots on the Server USB 2 0 Por...

Page 21: ...e PCIe Card 10 7 6 Aspen INT Sun Storage 12 Gb SAS PCIe HBA Internal 2 Sun Dual Port QDR Infiniband Host Channel Adapter for PCI Express Gen 3 Infiniband CX3 4 Sun Storage 16 GB Fiber Channel PCIe Universal HBA QLogic 11 8 Dual Port 10 Gigabit Ethernet PCIe 2 0 Copper Fiber SFP 3 5 Table 2 3 describes the assignment of PCIe cards in the slots on the Oracle Exalytics In Memory Machine X5 4 Table 2 ...

Page 22: ... 4 PCIe Card Slot Number for X2 4 Slot Number for X3 4 Sun FlashAccelerator F40 PCIe Card 400GB eMLC NA 1 2 3 5 7 8 8Gb Dual Port Fiber Channel PCI Express Card 0 0 Dual Port 10 Gigabit Ethernet PCIe 2 0 Copper Fiber SFP 2 9 SAS 2 8 port internal 2x4 6Gb RAID PCIe Gen 2 HBA 4 4 2 port QDR InfiniBand HCA 6 6 2 4 Key Features Business Intelligence Machine includes the following features Complete pre...

Page 23: ... 4 723W For X2 4 663W Regulations Safety IEC 60950 UL CSA 60950 CB Scheme with all country differences RFI EMI FCC CFR 47 Part 15 Class A EN 55022 Class A EN 61000 3 2 EN 61000 3 3 EN 300 386 Immunity EN 55024 EN 300 386 Certifications Safety UL cUL UL DEMKO CE BSMI CCC GOST R S Mark EMC CE FCC VCCI ICES C Tick MIC CCC GOST R BSMI Class A Other Complies with WEEE Directive 2002 96 EC and RoHS Dire...

Page 24: ...United States ICES 003 A NMB 3 A Class A Canada ETSI EN300386 V1 6 1 VCCI 2008 Class A Japan KN 22 RRL Public Notice 2009 9 Dec 21 2009 EN55024 Immunity EN61000 3 2 EN61000 3 3 Restriction of Hazardous Substances RoHS Directive 2011 65 EU Waste Electrical and Electronic Equipment WEEE Directive 2002 96 EC EMC Directive 2004 108 EC Table 2 9 Certification Markings on product Safety Marks UL Regulat...

Page 25: ...ng Brackets Onto Oracle Exalytics In Memory Machine Attaching the Slide Rail Assemblies to the Rack Installing Oracle Exalytics In Memory Machine into the Slide Rail Assemblies Installing the Cable Management Arm Verifying the Operation of the Slide Rails and CMA 3 1 Task Roadmap Installing your Oracle Exalytics In Memory Machine into a four post rack using the slide rail and cable management arm ...

Page 26: ...s 436 5 mm 17 19 inches Height 129 85 mm 5 11 inches 3 3 Before You Begin Before installing Oracle Exalytics In Memory Machine into a rack read the following cautions and important notes Always load equipment into a rack from the bottom up so that the rack will not become top heavy and tip over Deploy your rack s anti tip bar to prevent the rack from tipping during equipment installation If the Or...

Page 27: ...assembly until it reaches the stop as shown in Figure 3 1 4 Push the mounting bracket release button toward the front of the mounting bracket Figure 3 1 and simultaneously withdraw the mounting bracket from the slide rail assembly Figure 3 1 Disassembling the Slide Rails Before Installation Table 3 2 Figure Legend Number Description 1 Slide rail lock 2 Mounting bracket release button 3 5 Installin...

Page 28: ...e Exalytics In Memory Machine 3 6 Attaching the Slide Rail Assemblies to the Rack The slide rail assemblies support only racks with 9 5 mm square holes and M6 round holes All other racks including those racks with 7 2 mm M5 or 10 32 mounting holes are not supported See your rack documentation for information about the size of its rail holes To attach the slide rail assemblies to the rack complete ...

Page 29: ...he rack as shown in Figure 3 4 You will hear an audible click when the mounting pins engage the rack Note The slide assembly mounting pins accommodate either 9 5 mm square mounting holes or M6 round mounting holes No other mounting hole sizes are supported Figure 3 3 Installing Slide Rail Assemblies Onto the Rack Figure 3 4 Slide Rail Mounting Pins 4 Optional If you choose to attach the slide rail...

Page 30: ...ly 6 If available extend the anti tip bar at the bottom of the rack as shown in Figure 3 6 Refer to your rack documentation for instructions Figure 3 6 Extending the Anti tip Bar Caution If your rack does not have an anti tip bar the rack could tip over Chapter 3 Attaching the Slide Rail Assemblies to the Rack 3 6 ...

Page 31: ...slide rail assemblies in the rack as far as possible 2 Raise the Oracle Exalytics In Memory Machine so that the rear ends of the mounting brackets are aligned with the slide rail assemblies that are mounted in the rack as shown in Figure 3 7 3 Insert the mounting brackets into the slide rails then push the server into the rack until the mounting brackets encounter the slide rail stops approximatel...

Page 32: ...racle Exalytics In Memory Machine Into the Rack Caution Verify that the Oracle Exalytics In Memory Machine is securely mounted in the rack and that the slide rail locks are engaged with the mounting brackets before continuing 3 8 Installing the Cable Management Arm The cable management arm CMA is an optional assembly that you can use to route the server cables in the rack To install the optional C...

Page 33: ...as shown in Figure 3 9 Figure 3 9 Inserting the CMA Mounting Bracket Into the Back of the Right Slide Rail Table 3 3 Figure Legend Number Description 1 CMA mounting bracket 2 Right slide rail 5 Insert the right CMA slide rail connector into the right slide rail assembly until the connector locks into place with an audible click as shown in Figure 3 10 Figure 3 10 Inserting CMA Slide Rail Connector...

Page 34: ...t Slide Rail Table 3 5 Figure Legend Number Description 1 CMA slide rail connector 2 Left slide rail 7 Install and route cables to your Oracle Exalytics In Memory Machine as required 8 If required attach the cable hook and loop straps to the CMA and press them into place to secure the cables as shown in Figure 3 12 Note Cable hooks and loop straps are preinstalled on the CMA Perform the procedure ...

Page 35: ...he attached cables for any binding or kinks 3 Verify that the CMA extends fully from the slide rails 4 Push the Oracle Exalytics In Memory Machine back into the rack as described in the following sub steps When the Oracle Exalytics In Memory Machine is fully extended you must release two sets of slide rail stops to return the Oracle Exalytics In Memory Machine to the rack The first set of stops ar...

Page 36: ...l release buttons located near the front of each mounting bracket Figure 3 1 Simultaneously push both of the green slide rail release buttons and push the Oracle Exalytics In Memory Machine completely into the rack until both slide rail locks engage 5 Adjust the cable straps and CMA as required Chapter 3 Verifying the Operation of the Slide Rails and CMA 3 12 ...

Page 37: ...efore connecting the Oracle Exalytics In Memory Machine to your data center s existing network ensure that the following prerequisites are satisfied Switch Requirements Cable Requirements 4 1 1 Switch Requirements Before connecting the Oracle Exalytics In Memory Machine to your data center s network complete the following preparatory tasks Identify the network switch or connector that you will use...

Page 38: ... Exalytics In Memory Machine X6 4 X5 4 and X4 4 Call out Description 1 Power supply unit PSU 0 indicator panel 2 PSU 0 AC inlet 3 PSU 1 indicator panel 4 PSU 1 AC inlet 5 System status indicator panel 6 PCIe card slots 1 6 7 Network NET 10 GbE ports NET0 NET3 8 USB 2 0 ports 2 9 PCIe card slots 7 11 10 Service processor SP network management NET MGT port 11 Serial management SER MGT RJ 45 serial p...

Page 39: ... 10 100 1000 Gb Ethernet ports located on the rear of the Oracle Exalytics In Memory Machine are shown in a red circle of dashed line They are named NET 0 NET 1 NET 2 and NET 3 Note Although you can use any of these Gb Ethernet ports Oracle recommends that you use NET 0 The ILOM 10 100 Ethernet port is shown in a blue circle of solid line This port is named NET MGT Chapter 4 Overview of Network Po...

Page 40: ...acle Exalytics In Memory Machine Table 4 2 Cables Name Length Oracle Part Number TwinAx 1m 1m X2130 1M TwinAx 3m 3m X2130 3M TwinAx 5m 5m X2130 5M TwinAx 10m 10m X2130 10M For optical cables you must purchase SFP transceivers either Short Range SR or Long Range LR for each of the network ports and then plug in the appropriate optical cable Currently Oracle sells both the SR and LR SFP transceivers...

Page 41: ...les in the SFP connectors of the universal HBA enabling 16 Gb FC HBA functionality Table 4 4 Transceivers Name Part Number 2 Sun Storage 10 Gb FCoE short reach optics Qlogic for factory installation 7101677 2 Sun Storage 10 Gb FCoE short reach optics Qlogic 7101678 2 Sun Storage 16 Gb FC short wave optics Qlogic for factory installation 7101675 2 Sun Storage 16 Gb FC short wave optics Qlogic 71016...

Page 42: ...ther end of the cable to a network connector or switch if the network is already set up in your building or to a hub for a client server network 3 Ensure that you have a laptop with functional USB and network ports 4 Ensure that you have a Category 5E patch cable of maximum length 25 feet and a serial cable of maximum length 15 feet 5 Connect the network port of your laptop into an unused input po...

Page 43: ... netmask 255 255 255 0 up In this example the ifconfig command assigns the IPv4 address 192 0 2 10 with a network mask of 255 255 255 0 to the eth0 interface 4 7 Configuring the Network for Oracle Exalytics In Memory Machine This section describes how to use the exalytics_configure_network sh script to configure the network for Oracle Exalytics In Memory Machine It contains the following topics Ov...

Page 44: ...y Machine s connectivity with Exalogic machine or Oracle Exadata Database Machine You must specify an IP address and netmask for the script A gateway address for this InfiniBand interface is not necessary When you configure InfiniBand using the exalytics_configure_network sh script the following files are updated etc sysconfig network scripts ifcfg ib etc sysconfig network scripts ifcfg bond0 etc ...

Page 45: ...onded interface the script verifies whether you want to configure each interface separately You are prompted to enter an IP address netmask and gateway address for each individual interface configured When you configure 1 GbE interfaces using the exalytics_configure_network sh script the following files are updated etc sysconfig network scripts ifcfg eth etc sysconfig network scripts ifcfg bond2 e...

Page 46: ...nfiguration the script prompts you to set time and date For network time server configuration the script modifies the etc ntp conf file and it starts the ntpd daemon and adds it to start at boot For time zone selection the script creates a new soft link to etc localtime Note The script keeps a copy of the interface files in the tmp exalytics save directory and creates a flat backup for the files t...

Page 47: ...68 21 12 Use xx xxx xx xx for bond0 ip address y n y Verify the bond0 IP address you specified and enter y Enter bond0 netmask address Enter a bond0 netmask address For example 255 255 255 0 Use xx xxx xxx xx for bond0 netmask y n y Verify the bond0 netmask address you specified and enter y Do you want to configure 10G network y n n Depending on your configuration option enter y or n Configure bon...

Page 48: ...nter eth2 gateway address Enter an eth2 gateway address For example 10 1 1 1 Use xx x x x for eth2 gateway y n y y Verify the eth2 gateway address you specified and enter y Configure eth3 Interface y n n n Depending on your configuration option enter y or n Disable eth3 Depending on your configuration option enter y or n Configure eth4 Interface y n n n Depending on your configuration option enter...

Page 49: ...the above information OK If the information is correct enter 1 Here is that TZ value again this time on standard output so that you can use the usr bin tzselect command in shell scripts do you want to configure time server y n y y If you want to configure a time server enter y Enter network time server 10 162 39 1 Use 10 162 39 1 for timeserver y n y y Do you want to configure DNS s y n y y Enter ...

Page 50: ...e interfaces Verify the log file var log init exalytics init exalytics log Run the ifconfig a command to verify the network configuration 4 8 What Next After configuring network for the Oracle Exalytics In Memory Machine you can proceed to install the Oracle Business Intelligence software on the machine For more information see the Oracle Exalytics In Memory Machine Installation and Administration...

Page 51: ...figured on the Oracle Exalytics In Memory Machine Table 5 1 Disk Partitions Partition Capacity Hardware RAID Mount Point Description boot 99 MB RAID1 boot Boot partition base operating system logs and temp space X6 4 1 2 TB X5 4 1 2 TB X4 4 1 2 TB X3 4 900 GB X2 4 600 GB RAID1 Partition for the base operating system logs and temp space Software data files and so on X6 4 3 6 TB X5 4 3 6 TB X4 4 3 6...

Page 52: ...he Storage Area Network SAN feature is supported This section describes the procedure for connecting Exalytics to a Storage Area Network Connecting Exalytics to a Storage Area Network involves the following steps Before You Begin Preparing for Storage Area Network SAN Configuration Connecting the Optical Cable Installing Diagnostic Support for the Red Hat SUSE Linux OS Optional Verifying the Conne...

Page 53: ...ng an operating system boot by running the following commands as the root user chkconfig multipathd on chkconfig level 345 multipathd on service multipathd start 5 2 3 Connecting the Optical Cable To connect the optical cable complete the following steps Note The HBA does not allow normal data transmission on an optical link unless it is connected to another similar or compatible Fibre Channel pro...

Page 54: ... following steps 1 Check the status of the multipath device as follows root node1 service multipathd status multipathd pid xxxx is running 2 Run multipath ll command to view the detailed information of the multipath devices as shown in the following example multipath ll mpath2 3600144f0d153d65600004f90462c0003 dm 3 SUN Sun Storage 7410 size 100G features 0 hwhandler 0 rw _ round robin 0 prio 1 act...

Page 55: ...nt Note The multipath LUNs are available as block devices at dev mapper mpathXX 2 Add the mount point to etc fstab to auto mount during an operating system boot 5 3 For Additional Help For additional help or information regarding the storage configuration contact the Oracle support personnel Chapter 5 For Additional Help 5 5 ...

Page 56: ...oactively Learn about hardware errors and faults as they occur Remotely control the power state of your compute node View the graphical and non graphical consoles for the host View the current status of sensors and indicators on the system Determine the hardware configuration of your system Receive generated alerts about system events in advance via IPMI PETs SNMP Traps or E mail Alerts The ILOM s...

Page 57: ... static IP address for Service Processor ILOM access Note the following As a general convention root is used as the default user name The default value of the Service Processor SP ILOM password is welcome1 6 3 Connecting to ILOM via the Network Under normal circumstances you can access ILOM via the network You can use the Ethernet connection method to connect to the ILOM You must know the ILOM IP ...

Page 58: ... name is root and the default password is welcome1 3 Click Log In The web GUI is displayed 6 3 3 Launching a Remote KVM Session To access the ILOM console for your Oracle Business Intelligence Machine that is connected to ILOM do the following 1 Ensure that pop up blockers are disabled in your browser before you launch the remote console 2 In the address bar of a browser type the ILOM IP address o...

Page 59: ...sable hardware control 4 Verify that power is supplied to either PSU If there is power applied to either PSU then ILOM will be functional regardless of the power state of compute nodes 5 Press Enter on the terminal device A connection between the terminal device and the ILOM is established The ILOM login prompt is displayed 6 Log in to the CLI using the default user name and the password root and ...

Page 60: ...a the Network 2 Run the following commands at the ILOM command prompt cd SP network set pendingipdiscovery static set pendingipaddress ip_address set pendingipnetmask ip_netmask set pendingipgateway ip_gateway set commitpending true 6 5 3 Using ipmitool Commands When SP Network Information Is Lost If the service processor SP network information gets lost you can use an SSH shell to connect to the ...

Page 61: ...igure IP addresses manually complete the following steps 1 Obtain a free static IP address from the network to which the Exalytics machine is connected 2 Configure the IP address by using a serial port connection as follows a Establish a serial connection to the SP of a compute node by inserting a serial cable into the SER MGT port on the rear of the compute node b In the terminal window configure...

Page 62: ...ion e Run the following commands at the command prompt cd SP network set pendingipdiscovery static set pendingipaddress ip_address set pendingipnetmask ip_netmask set pendingipgateway ip_gateway set commitpending true Chapter 6 Configuring Network Access 6 7 ...

Page 63: ...irements This section contains the following topics InfiniBand Card InfiniBand Cables InfiniBand Switches in the Oracle Exadata Database Machine 7 1 1 InfiniBand Card Each Oracle Business Intelligence Machine includes an InfiniBand card such as the 2 port QDR InfiniBand Host Channel Adapter HCA For the Exalytics X2 4 and X3 4 models the InfiniBand card is installed in PCIe Slot 6 For the Exalytics...

Page 64: ...r s Guide 7 2 Physically Connecting the Network This section describes the following topics Physical Layout Considerations Scenario 1 Single Exalytics In Memory Machine with Oracle Exadata Database Machine Scenario 2 Clustered Exalytics In Memory Machine with Oracle Exadata Database Machine 7 2 1 Physical Layout Considerations It is assumed that the rack containing the Oracle Business Intelligence...

Page 65: ...xadata Database Machine In addition you should connect port 2 of the Oracle Business Intelligence Machine s InfiniBand card to the leaf 2 switch Sun Datacenter InfiniBand Switch 36 installed in the upper unit of the Oracle Exadata Database Machine Port 1 can function as the active port and port 2 can function as the passive port Note On the leaft switches of the Oracle Exadata Database Machine the...

Page 66: ...1 can function as the active port and port 2 can function as the passive port Note On the leaft switches of the Oracle Exadata Database Machine the following free ports can be used Ports 5B 6A 6B 7A 7B and 12A 7 3 Choosing a Subnet Manager Each InfiniBand network requires a Subnet Manager SM You can use the SM running on one of the Oracle Exadata Database Machine s leaf switches in your fabric For...

Page 67: ...basic status as in the following example ibstat CA mlx4_0 CA type MT26428 Number of ports 2 Firmware version 2 7 8130 Hardware version b0 Node GUID 0x0021280001cf577e System image GUID 0x0021280001cf5781 Port 1 State Active Physical state LinkUp Rate 40 Base lid 106 LMC 0 SM lid 54 Capability mask 0x02510868 Port GUID 0x0021280001cf577f Link layer IB Port 2 State Active Physical state LinkUp Rate ...

Page 68: ...splays the InfiniBand switch nodes in the fabric and the ibnetdiscover command discovers the InfiniBand topology For more information such monitoring and diagnostic commands see the Sun Datacenter InfiniBand Switch 36 Command Reference Chapter 7 Testing the Extended Network Connectivity 7 6 ...

Page 69: ...and HCA card but it does not include any InfiniBand switch or any instance of InfiniBand Subnet Manager To connect Oracle Exalytics to Oracle Big Data Appliance you need a pair of InfiniBand cables depending on the length requirements at the datacenter If the total distance between Exalytics and Oracle Big Data Appliance leaf switch ports is less than 10 meters you can use copper cables If the len...

Page 70: ... ib_addr ib_ipoib bonding root exalytics root lsmod grep rds rds_tcp 9460 0 rds_rdma 72457 1 rds 91561 2 rds_tcp rds_rdma rdma_cm 29058 3 rdma_ucm rds_rdma ib_sdp ib_core 61642 14 rdma_ucm rds_rdma ib_sdp rdma_cm iw_cm ib_uverbs ib_umad mlx4_vnic mlx4_ib ib_mthca i b_ipoib ib_cm ib_sa ib_mad root exalytics root 8 2 2 BDA Version Details To view software support of the Oracle Big Data Appliance run...

Page 71: ...5B 6A 6B Oracle recommends that you use ports 5A on both leaf switches If these ports are not available use any other available port on the leaf switches Note Never connect a host HCA system to the spine switch inside a rack If the Oracle Big Data Appliance rack is connected to other Oracle Engineered Systems port 5 6 A B may not be available In such a case it is recommended to use any other avail...

Page 72: ...racle Big Data Appliance machine is visible 3 On the Exalytics machine run the ibstat command to see the local ports status If an active Subnet Manager is present then the ports should be Active root exalytics ibstat CA mlx4_0 CA type MT26428 Number of ports 2 Firmware version 2 7 8130 Hardware version b0 Node GUID 0x0021280001efe984 System image GUID 0x0021280001efe987 Port 1 State Active Physica...

Page 73: ...bda10 egrep IPADDR NETMASK NETWORK BROADCAST etc sysconfig network scripts ifcfg bondib0 IPADDR 192 168 41 10 NETMASK 255 255 248 0 NETWORK 192 168 40 0 BROADCAST 192 168 47 255 Using the information obtained you can request for a free and unique IP address belonging to Oracle Big Data Appliance s IPoIB subnet To find out the CIDR prefix run the ipcalc command as follows root bda10 ipcalc p 192 16...

Page 74: ...le Big Data Appliance node s IPoIB address from Exalytics If the ping is successful you can use the connection for applications You can test the application usage between the Exalytics and BDA It may be necessary to install appropriate plugins and or clients for Hadoop on Exalytics Machine For further information refer to product owner s guide Chapter 8 IPoIB Validation 8 6 ...

Page 75: ...ating Exalytics Assets Manually Viewing and Verifying ASR Assets ASR Manager Configuring Telemetry Sources Enabling Exalytics Machine Assets in My Oracle Support Sending Test Alerts Fault Coverage 9 1 About Oracle Auto Service Request ASR Note ASR is available only for the Exalytics bare metal machines Auto Service Request ASR is a secure scalable customer installable software feature of warranty ...

Page 76: ...ly network away from the Data network The supported operating systems are Oracle Linux 5 3 and above and Solaris 10 Update 5 and above Note ASR Manager should not to be on the same subnet as the Bonded Data network interfaces ETH0 must be on a separate subnet that does not overlap the Bonded Data network 9 3 Before You Begin Before installing ASR ensure that the following conditions are met Ensure...

Page 77: ... file from the Oracle Software Delivery Cloud Web site at the following link http edelivery oracle com 2 Unzip the file into a temporary directory The zip file contains the files that are listed in Table 9 1 Table 9 1 Files in the Oracle Exalytics Configuration Utility Zip File File Name Notes exalytics_configure_network sh Oracle Exalytics network configuration script SUN HW TRAP MIB mib ASR Auto...

Page 78: ...server c At the command line execute the init q command to initialize and start the bda_mon server Note You can also download the patch p21565893_20000_Linux x86 64 zip patch 21565893 unzip the patch file unzip p21565893_20000_Linux x86 64 zip d tmp and follow the instructions in the tmp readme txt file For an OL6 based Exalytics Image the steps are a Go to the opt exalytics etc init d directory a...

Page 79: ... ASR Manager 1 Login as the root user in your ASR Manager Start the ASR menu by running the following command cd opt SUNWswasr bin asr 2 Activate ASR on Exalytics host machine by running the following command asr activate_asset i IP_address_of_exalytics_host_machine or asr activate_asset h host_name_exalytics_machine 3 Activate ILOM for Exalytics individually by running the following command asr a...

Page 80: ...My Oracle Support Dashboard click the More tab Then select Settings from the menu 3 In the Settings pane on the left of the window select Assets located under the Administrative submenu For your Support Identifier select ASR Status from the sort filter then select Pending from the All Statuses drop down menu 4 You can approve a Pending asset in two ways Activate via Asset Toolbar Click the asset s...

Page 81: ...ending Test Alerts To send test alerts complete the following 1 Login to the Exalytics Host as the root user 2 Send Test Alerts for ILOM telemetry by running the following command ipmitool sunoem cli set SP alertmgmt rules 1 testrule true 3 Send Test Alerts for Exalytics host machine by running the following command opt exalytics asr bda_mon_hw_asr pl validate_snmp_subscriber type asr You will rec...

Page 82: ...lowing situations in case you need to call service Is the Power OK LED flashing or constantly illuminated on the front of the system Ensure that the power cord is connected to the system and to a grounded power receptacle Does the wall outlet have power Test by plugging in another device Does the monitor sync within five minutes after power on The green LED on the monitor stops flashing and remain...

Page 83: ...ooting information see the Sun Fire X4470 M2 Server Service Manual 821 0703 and the Oracle x86 Servers Diagnostics Guide 820 6750 A 3 Locating the Machine Serial Number You might need to have the serial number of your Oracle Business Intelligence Machine when you ask for service on your system Record this number for future use Use one of the following methods to locate your machine s serial number...

Page 84: ...m Email address and phone number for you and a secondary contact Street address where the system is located Administrator password Summary of the problem and the work being done when the problem occurred Other useful information IP address Oracle Exalytics In Memory Machine name system host name Network or internet domain name Proxy server configuration Appendix A Locating the Machine Serial Numbe...

Page 85: ...Index H HP Oracle Database Machine 2 1 Index 1 ...