4.
Verify the rack master serial number is set correctly using the following command:
# dcli -l root -g group_file "ipmitool sunoem cli 'show \
/SP system_identifier'" > /tmp/show-rack-csn.out
# more /tmp/show-rack-csn.out
If none of the serial numbers are set, then contact Oracle Support Services. If one
serial number is not set correctly, then run the following command to correct it:
ipmitool sunoem cli 'set /SP system_identifier= \
"Exadata Database Machine rack_type xxxxAKyyyy"'
In the preceding command, rack_type is
X2-2
or
X2-8
, depending on the rack.
5.
Ensure disk cache policy is set to
Disabled
using the following command as the
root
user on the first database server in the cluster:
dcli -g /opt/oracle.SupportTools/onecommand/all_group -l root \
/opt/MegaRAID/MegaCli/MegaCli64 -LdPdInfo -aALL | grep -i 'Disk Cache Policy'
The following is an example of the output from the command:
dm01db01: Disk Cache Policy : Disabled
dm01db02: Disk Cache Policy : Disabled
dm01db03: Disk Cache Policy : Disabled
...
dm01cel01: Disk Cache Policy : Disabled
dm01cel02: Disk Cache Policy : Disabled
...
If any server shows different output, then identify the LUN and use the following
command to reset the policy to
Disabled
:
MegaCli64 -LDSetProp -DisDskCache -LUNn -a0
In the preceding command, LUNn is the LUN number, such as L2.
5.7 Verifying the InfiniBand Network
The following procedure describes how to verify the InfiniBand network:
1.
Visually check all the InfiniBand cable connections within the rack. The port lights
should be on, and the LEDs should be on. Do not press each connector to verify
connectivity.
2.
Log in as the
root
user on any component in the rack.
3.
Verify the InfiniBand topology using the following commands:
# cd /opt/oracle.SupportTools/ibdiagtools
# ./verify-topology [-t rack_size]
The following example shows the output when the network components are
correct.
[DB Machine Infiniband Cabling Topology Verification Tool ]
Is every external switch connected to every internal switch......[SUCCESS ]
Are any external switches connected to each other................[SUCCESS ]
Are any hosts connected to spine switch..........................[SUCCESS ]
Check if all hosts have 2 CAs to different switches..............[SUCCESS ]
Leaf switch check:cardinality and even distribution..............[SUCCESS ]
Chapter 5
Verifying the InfiniBand Network
5-60
