NPAR – Configuration and Use Case Example
NetXtreme-E User’s Manual
February 26, 2018 • NetXtreme-E-UG100
Page 65
Notes on Reducing NIC Memory Consumption
Because of the faster link speeds supported in this NIC, the default number of receive buffers is larger. More
packets can arrive within a given time interval when the link speed is higher, and if the host system is delayed
in processing the receive interrupts, the NIC must drop packets if all available receive buffers are in use.
The default value of receive buffers was selected to work well for typical configurations. But if you have many
NIC’s in a system, have enabled NPAR on multiple NIC’s, or if you have only a small amount of RAM, you may
see a Code 12 “yellow bang” in the Device Manager for some of the NIC’s. Code 12 means that the driver failed
to load because there were not enough resources. In this case, the resource is a specific type of kernel memory
called Non-Paged Pool (NPP) memory.
If you are getting a Code 12, or for other reasons wish to reduce the amount of NPP memory consumed by the
NIC:
•
Reduce the number of RSS queues from the default of 8 to 4 or 2. Each RSS queue has its own set of
receive buffers allocated, so reducing the number of RSS queues reduces the allocated NPP memory.
There can be performance implications from reducing the number of RSS queues, as fewer cores
participate in processing receive packets from that NIC. Per processor CPU utilization should be monitored
to ensure that there are no “hot” processors after this change.
•
Reduce memory allocation by reducing the number of receive buffers allocated. The default value of 0
means the driver should automatically determine the number of receive buffers. For typical configurations,
a setting of 0 (=auto) will map to XXXX receive buffers per queue. You can choose a smaller value such as
1500, 1000, or 500. (The value needs to be in multiples of 500 between the range of 500 and 15000.) As
mentioned above, a smaller number of receive buffers increases the risk of packet drop and a
corresponding impact to packet retransmissions and decreased throughput.
The parameters “Maximum Number of RSS Queues” and “Receive Buffers (0=Auto)” can be modified using the
Advanced
properties tab for each NIC in the
Device Manager
. If you want to modify multiple NIC’s at the same
time, it is faster to use the
Set-NetAdapterAdvancedProperty PowerShell cmdlet
. For example, to assign two
RSS queues for all NIC’s in a system whose NIC name starts with “Sl”, run the following command:
Set-NetAdapterAdvancedProperty Sl* -RegistryKeyword *NumRSSQueues -RegistryValue 2
Similarly, to set the number of Receive buffers to 1500, run the following command:
Set-NetAdapterAdvancedProperty Sl* -RegistryKeyword *ReceiveBuffers -RegistryValue 1500
See
https://blogs.technet.microsoft.com/wincat/2012/08/27/using-powershell-for-nic-configuration-tasks/
for an
overview of using PowerShell to modify NIC properties.
