InfiniBand tests for end to end:
•
On listener host, run
ib_read_bw
.
•
On sender, run
ib_read_bw <ip address>
. You might need to specify a port number
with
-i
.
•
Run
ibnetdiscover
. This is a Voltaire tool that does a LID crawl.
Troubleshoot switches:
•
Run the Voltaire switch
port_verify
utility. This utility reports status for the ports and indicates
any problems. Also use
port_verify -v
.
•
Verify that Link Aggregation is set up to optimize switch redundancy and network load
optimization:
lag-mode show
To set up Link Aggregation, use this command:
lag-mode set full
NOTE:
If LAG is full on the InfiniBand switch, ensure that the connecting Ethernet switch
ports are also set up for aggregation.
Enabling client access
To enable client access to file systems, complete these steps:
•
Export the file systems to NFS.
•
On the client, mount the exported NFS shares over the InfiniBand network.
Setting up Voltaire InfiniBand
NOTE:
The installation procedure of the third-party InfiniBand drivers could be different than
what is shown here. Refer to the third party documentation for the most accurate information or
contact your vendor's support.
1.
Install the Voltaire OFED drivers on each couplet and client:
[root@ib VoltaireOFED-1.4.2_2-k2.6.18-128.el5-x86_64]# ./install.sh
installation will replace your iscsi-initiator-utils
RPM Uninstalling the previous version of OFED. This may take few moments.
Preparing to install
Verifying installation
Installing 64 bit RPMS
Preparing... ########################################### [100%]
1:libibverbs ########################################### [ 2%]
2:librdmacm ########################################### [ 5%]
3:libibcommon ########################################### [ 7%]
4:libibumad ########################################### [ 9%]
5:libibmad ########################################### [ 12%]
6:opensm-libs ########################################### [ 14%]
7:dapl ########################################### [ 16%]
8:ibvexdmtools ########################################### [ 19%]
9:compat-dapl ########################################### [ 21%]
10:libibcm ########################################### [ 23%]
11:libsdp ########################################### [ 26%]
...
IMPORTANT:
The installation overwrites pre-existing
/lib/modules
vital to X9000 software
and NFS protocol. You will need to rename the modules, run
depmod -a
, and reboot.
Complete steps 2–4 in the procedure
“Installing the driver” (page 144)
.
2.
Install the UFM Client software on the couplets and clients:
146
Setting up InfiniBand couplets
