RoCE – Configuration and Use Case Examples
NetXtreme-E User’s Manual
February 26, 2018 • NetXtreme-E-UG100
Page 66
RoCE – Configuration and Use Case Examples
This section provides configuration and use case examples for RoCE.
To enable RoCE for PFs or VFs, the user must enable the RDMA selection in the HII menu in the BIOS before
the RDMA option takes effect in the host or guest OS.
To enable RDMA in single function mode (if
Virtualization Mode
is
None
or
SR-IOV
):
1.
During the system boot, access the
System Setup -> Device Settings -> NetXtreme-E NIC -> Main
Configuration Page
and set
NIC+ RMDA Mode
to
Enabled
.
To enable RDMA if Virtualization Mode is NPAR or NPAR+SR-IOV:
1.
During the system boot, access the
System Setup -> Device Settings -> NetXtreme-E NIC ->NIC
Partitioning Configuration-> Partition 1 (or 2) Configuration
and set
NIC+ RMDA Mode
to
Enabled
.
Linux Configuration
Requirements
To configure RoCE in Linux, the following items are required:
•
bnxt_en-roce (RoCE supported bnxt_en driver which is part of released gzip compressed tar archive)
•
bnxt_re (RoCE driver)
•
libbnxtre (User mode RoCE library module)
BNXT_RE Driver Dependencies
The Bnxt_re driver requires a special RoCE enabled version of bnxt_en which is included in the netxtreme-
bnxt_en-1.7.9.tar.gz (or newer) package. The bnxt_re driver compilation depends on whether IB stack is
available along with the OS distribution or an external OFED is required.
•
Distros that have an IB Stack available along with OS distribution:
RH7.1/7.2/7.3/6.7/6.8, SLES12SP2 and Ubuntu 16.04
If not already installed, the IB stack and useful utils can be installed in Redhat with the following commands
prior to compiling bnxt_re:
yum -y install libibverbs* inifiniband-diag perftest qperf librdmacm utils
Note:
If using NPAR+SRIOV mode, only two VFs from each parent physical port can enable RDMA
support, or a total of four VFs + RDMA per physical device.
Note:
It is necessary to load the correct bnxt_en version that is included in the same netxtreme-
bnxt_en-1.7.x.tar.gz package. Bnxt_re and Bnxt_en function as a pair to enable RoCE traffic. Using
mismatching versions of these two drivers produces unreliable or unpredictable results.
