NetXtreme-UG100
45
NetXtreme-C/NetXtreme-E
User Guide
14.1.4 Limitations
In dual-port NICs, if both ports are on same subnet, RDMA perftest commands may fail. The possible cause is due to an arp
flux issue in the Linux OS. To workaround this limitation, use multiple subnets for testing or bring the second port/interface
down.
14.1.5 Known Issues
Bnxt_en
and
Bnxt_re
are designed to function in pair. Older
Bnxt_en
drivers prior to version 1.7.x do not support RDMA
and cannot be loaded at the same time as the
Bnxt_re
(RDMA) driver. The user may experience a system crash and reboot
if
Bnxt_re
is loaded with older
Bnxt_en
drivers. It is recommend that the user load the
Bnxt_en
and
Bnxt_re
module from
the same
netxtreme-bnxt_en-<1.7.x>.tar.gz
bundle.
To prevent mismatching a combination of
bnxt_en
and
bnxt_re
from being loaded, the following is required:
If RedHat/CentOS 7.2 OS was installed to the target system using PXEboot with
bnxt_en
DUD or a kernel module
RPM, delete the file
bnxt_en.ko
found in
/lib/modules/$(uname -r)/extra/bnxt_en/bnxt_en.ko
or edit
/etc/
depmod.d
/.
bnxt_en.conf to override to use updated version. Users can also erase the current BCM5741X Linux kernel driver using
the rpm -e kmod-bnxt_en command. RHEL 7.3/SLES 12 Sp2 has bnxt_en inbox driver (older than v1.7.x). This driver
must be removed and the latest bnxt_en be added before applying the bnxt_re (RoCE drivers).
14.2 Windows and Use Case Examples
14.2.1 Kernel Mode
Windows Server 2012 and beyond invokes the RDMA capability in the NIC for SMB file traffic if both ends are enabled for
RDMA. Broadcom NDIS miniport bnxtnd.sys v20.6.2 and beyond support RoCEv1 and RoCEv2 via the NDKPI interface.
The default setting is RoCEv1.
To enable RDMA:
1. Upgrade the NIC NVRAM using the appropriate board packages. In CCM or in UEFI HII, enable support for RDMA.
2. Go to the adapter
Advanced Properties
page and set
NetworkDirect Functionality
to
Enabled
for each BCM5741X
miniport, or using PowerShell window, run the following command:
Set-NetAdapterAdvancedProperty -RegistryKeyword *NetworkDirect -RegistryValue 1
3. The following Powershell commands returns true if
NetworkDriect
is enabled.
a. Get-NetOffLoadGlobalSetting
b. Get-NetAdapterRDMA
14.2.2 Verifying RDMA
To verify RDMA:
1. Create a file share on the remote system and open that share using Windows Explorer. To avoid hard disk read/write
speed bottleneck, a RAM disk is recommended as the network share under test.
2. From PowerShell, run the following commands:
Get-SmbMultichannelConnection | fl *RDMA*
ClientRdmaCapable : True
ServerRdmaCapable : True
