Mellanox Technologies Innova IPsec, User Manual

Page 1: ...Mellanox Technologies www mellanox com Mellanox Innova IPsec Ethernet Adapter Card User Manual Rev 1 8...

Page 2: ...updated list of Mellanox trademarks visit http www mellanox com page trademarks NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHN...

Page 3: ...7 3 1 System Requirements 17 3 1 1 Hardware 17 3 1 2 Operating Systems Distributions 17 3 2 Safety Precautions 17 3 3 Pre installation Checklist 17 3 4 Bracket Installation Instructions 17 3 4 1 Remov...

Page 4: ...ems 32 5 1 5 2 Removing Signature from Kernel Modules 33 5 2 Installation of Kernel Module with IPsec Offload 34 5 2 1 Obtaining the Kernel Modules 34 5 2 2 Installing the Kernel and Driver 34 5 2 3 I...

Page 5: ...54 Appendix A Fast Installation and Update 56 A 1 Hardware Installation 56 A 2 Content of Mellanox Innova IPsec Bundle 56 A 3 Software Firmware and Tools Installation 56 A 4 Software Firmware and Too...

Page 6: ...nox Innova IPsec Active Cooling Adapter Card 11 Table 4 Features 12 Table 5 Documents List 15 Table 6 mlnxofedinstall Return Codes 31 Table 7 ethtool IPsec Offload Counters 39 Table 8 MNV101512A BCIT...

Page 7: ...s and Components 23 Figure 3 MNV101511A BCIT MNV101512A BCIT LEDs Placement Example 50 Figure 4 Mechanical Drawing of MNV101511A BCIT 52 Figure 5 Mechanical Drawing of MNV101512A BCIT 53 Figure 6 Sing...

Page 8: ...ation via MLNX_OFED on page 25 Updated Section 5 1 Installation via MLNX_OFED on page 25 Added Table 9 MNV101511A BCIT Specifications Table on page 49 Added Figure 5 Mechanical Drawing of MNV101512A B...

Page 9: ...page 52 Added Chapter 5 IPsec Offload Software Installation and Operation on page 25 Updated Section 5 2 2 Installing the Kernel and Driver on page 34 Updated Section 5 3 1 Loading Unloading the Modul...

Page 10: ...n session However the high computing power required by the IPsec algorithms consumes expensive CPU cycles and limits network connection performance The Mellanox Innova IPsec EN adapter offloads the pr...

Page 11: ...h Xilinx Kintex UltraScale XCKU060 Data Transmission Rate Ethernet 10 40Gb s Network Connector Types Single port QSFP PCI Express PCIe SerDes Speed PCIe 3 0 x8 8GT s RoHS R6 Adapter IC Part Number MT2...

Page 12: ...c applications with no required changes to the user s software IPsec offloading is handled by the combination of the ConnectX 4 Lx network controller and an on board FPGA providing high performance an...

Page 13: ...ad allowing more available CPU for computation tasks Quality of Service QoS Support for port based Quality of Service enabling various application requirements for latency and SLA Storage Acceleration...

Page 14: ...N Adapter Card Block Diagram 1 4 Operating Systems Distributions1 RHEL CentOS 1 Please refer to the driver release notes for feature availability Co n n e ctX D RA M x8 P C Ie G en3 FP G A C o n fig F...

Page 15: ...for Linux MLNX_OFED Performance Tuning Guidelines for Mellanox Network Adapters Document no 3368 User Manual describes important tuning parameters and settings that can improve performance for Mellan...

Page 16: ...an use a Mellanox QSA QSFP to SFP adapter module 2 2 PCI Express Interface The Mellanox Innova IPsec adapter card supports PCI Express 3 0 1 1 and 2 0 compatible through an x8 edge connector The devic...

Page 17: ...ystem if active 3 After shutting down the system turn off power and unplug the cord 4 Remove the card from its package Please note that the card must be placed on an antistatic surface 5 Check the car...

Page 18: ...ake sure that the LEDs are aligned onto the bracket holes 4 Use a torque driver to apply up to 2 9 lbs in torque on the screws 3 5 Card Installation Instructions 1 Open the system case 2 Place the ada...

Page 19: ...tor straight into the cage Do not apply any torque up or down to the connector cage in the adapter card d Make sure that the connector locks in place 3 After inserting a cable into a port the Amber LE...

Page 20: ...upward or downward in the rack 6 To remove a cable disengage the locks and slowly pull the connector away from the port receptacle LED indicator will turn off when the cable is unseated 3 7 Identify t...

Page 21: ...network stacks process more than once With these benefits IPsec offload allows the adapter to reach full wire speed with IPsec secured traffic on the wire while reducing CPU utilization IPsec offload...

Page 22: ...n the user can choose whether to enable the Mellanox Innova IPsec offload on the specific IPsec security association SA that is created once the connection is generated See Section 5 3 2 Setting up an...

Page 23: ...ova IPsec adapter currently supports offloading of the encryption decryption and authentication of IPsec traffic The key generation and exchange protocol whether done manually or through IKE protocol...

Page 24: ...oll Mode Driver PMD which makes use of this interface PMD provides a new API for DPDK applications to open close offloaded security associations control path while transmitting receiving traffic throu...

Page 25: ...nload the ISO image to your host The image s name has the format MLNX_OFED_LINUX ver OS label CPU arch iso An ISO image for the Mellanox Innova Flex adapter can be obtained through Mellanox support St...

Page 26: ...t be updated if you run the install script with the without fw update option mnt mlnxofedinstall OPTIONS Pre existing configuration files will be saved with the extension conf rpmsave On Redhat distri...

Page 27: ...ving OFED RPMs Created tmp MLNX_OFED_LINUX x x x rhel7 1 x86_64 ext tgz c config packages config_file Example of the configuration file can be found under docs n net network config_file Example of the...

Page 28: ...h uEFI and or tool will override this flag add kernel support Add kernel support Run mlnx_add_kernel_support sh skip distro check Do not check MLNX_OFED vs Distro matching hugepages overcommit Setting...

Page 29: ...lanox OFED components can be configured or reconfigured after the installation by modifying the relevant configuration files See the relevant chapters in this manual for details The list of the module...

Page 30: ...e kernel modules are installed under lib modules uname r extra mlnx ofa_kernel on RHEL and other RedHat like Distributions lib modules uname r updates dkms on Ubuntu Firmware The firmware of existing...

Page 31: ...URL to the software package tarball Example 2 With t flag to provide the path to the downloaded tarball Example 3 With p flag to provide the path to the downloaded and extracted tarball Example Table...

Page 32: ...ent request Step 3 Reboot the system The pending MOK key enrollment request will be noticed by shim efi and it will launch MokManager efi to allow you to complete the enrollment from the UEFI console...

Page 33: ...ing However please note that a similar message as the following will still be presented This message is presented once only for each boot for the first module that either has no signature or whose key...

Page 34: ...disk image has been created a Run ls boot and look for the relevant initramfs and vmlinuz files that match the kernel version you just installed names should match the RPM name 3 Please verify that th...

Page 35: ...e2 is a user space utilities package that controls TCP IP networking configuration in the kernel It includes commands such as ip for management of network tables and network interfaces It is also used...

Page 36: ...ec offload flags installed in your system Note There are several additional user space applications that provide an interface to configure IPsec policies and SAs Strongswan which has IPsec offload sup...

Page 37: ...anox see Section 5 2 3 Installing the Customized iproute2 Utility on page 35 In order to configure an IPsec secured connection between hosts it is necessary to 1 Configure the security association SA...

Page 38: ...the flag dir in dir out depending on the traffic direction of the state under the crypto offload parameters section will indicate that this state is offloaded by an offload device If these flags are n...

Page 39: ...the offload operation These counters are a part of the network interface counters and can be viewed using the ethtool S interface_name command Note The mlx5_core module must be loaded for the counter...

Page 40: ...added by FPGA ipsec_add_sa_fail Total amount of failed SA add commands by FPGA This can be a result of adding an already valid SA ipsec_del_sa_success Total amount of SAs successfully removed by FPGA...

Page 41: ...otes Extract the TGZ and run install sh Load mlx5_fpga_tools module See Section 4 2 2 mlx5_fpga_tools Module on page 23 Start mst service with the fpga lookup flag mst start with_fpga 6 2 mlx_fpga Syn...

Page 42: ...mst status MST modules MST PCI module is not loaded MST PCI configuration module is not loaded MST devices No MST devices were found nor MST modules were loaded You may need to run mst start to load...

Page 43: ...Range Default RW Description image_version 0x900000 31 00 00 0x0 RO Version of the image image_date 0x900004 31 00 00 0x0 RO Image date of creation The hex number is actually the decimal value i e 0x...

Page 44: ...the command mst status The mst device name will be of the form dev mst mt4117_pciconf0 d Get the PSID firmware identification and programmed firmware version using the command flint d mst device q wh...

Page 45: ...nox Innova IPsec Adapter Card Firmware Rev 1 8 45 Mellanox Technologies b To burn the firmware run c To load the firmware run mlxburn d dev mst mt4117_pciconf0 i fw bin mlxfwreset d dev mst mt4117_pci...

Page 46: ...ters stopped working after installing another adapter Try removing and re installing all adapters Check that cables are connected properly Make sure your motherboard has the latest BIOS Link indicator...

Page 47: ...grep i Mellanox Mellanox Firmware Tool MFT Download and install MFT http www mellanox com content pages php pg management_tools menu_section 34 Refer to the User Manual for installation instructions...

Page 48: ...wer Passive Cables 31 5W 1 5W Active Cables 33W Max power available through QSFP port 1 5W Temperature Operational 0 C to 55 Ca Non operational 40 C to 70 C a Ambient temperature may vary Please conta...

Page 49: ...bles 31W Max power available through QSFP port 1 5W Temperature Operational 0 C to 55 Ca Non operational 40 C to 70 C a Ambient temperature may vary Please contact Mellanox technical support if furthe...

Page 50: ...ls Group B LEDs Debug LEDs indicate memory calibration done memory BIST done ConnectX 4 Lx link up is with traffic Heartbeat and power good See Section 9 3 2 FPGA Debug LEDs on page 51 for details Gro...

Page 51: ...reen LED is lit and the Amber LED is off then the logical link has not been established Table 11 FPGA Debug LEDs LED Symbols LED Function D2 Power Good Or on all POWER GOOD inputs Expected LED ON D3 C...

Page 52: ...BCIT Table 12 FPGA Load Flow Debug LEDs LED LED Symbol and Function Green power good Off power issue D10 Power Good Red during configuration Green when complete D11 Configuration Done Indication Red f...

Page 53: ...Specifications Rev 1 8 53 Mellanox Technologies Figure 5 Mechanical Drawing of MNV101512A BCIT 167 65 68 90...

Page 54: ...Rev 1 8 54 Mellanox Technologies 9 5 Bracket Mechanical Drawing Figure 6 Single Port Tall Bracket 21 6 120 02...

Page 55: ...Specifications Rev 1 8 55 Mellanox Technologies Figure 7 Single Port Short Bracket 80 3 22 83...

Page 56: ...tion only If the bundle is already installed please refer to Appendix A 4 Software Firmware and Tools Update on page 58 Please make sure to install in the following order Step 1 Download the bundle fr...

Page 57: ...will install the FPGA image the FW and will also ask if to install the MFT and do a reset at the end modprobe mlx5_fpga_tools mst start with_fpga mst status MST modules MST PCI module is not loaded MS...

Page 58: ...p a modprobe mlx5_fpga_tools Step b mst start with_fpga Step c mst status To update the FPGA image Step 4 In the bundle folder directory look for the installation script mlnx_fpga_updater sh Step a Th...

Page 59: ...ollowing update script using one of the modes below 1 With u flag to provide URL to the software package tarball Example 2 With t flag to provide the path to the downloaded tarball Example 3 With p fl...

Page 60: ...dapter card has a different identifier printed on the label serial number and the card MAC for the Ethernet protocol Figure 8 MNV101511A BCIT Board Label Figure 9 MNV101512A BCIT Board Label The revis...

Page 61: ...1 F To guarantee proper air flow allow at least 8cm 3 inches of clearance around the ven tilation openings During periods of lightning activity do not work on the equipment or connect or dis connect c...

Page 62: ...se of controls or adjustment or performance of procedures other than those specified herein may result in hazardous radiation exposure CLASS 1 LASER PRODUCT and reference to the most recent laser stan...

Page 63: ...maximale est n cessaire En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouver tures de ventilation Pendant un orage il ne faut pas utiliser l...

Page 64: ...e en garde l utilisation de commandes ou de r glages ou l ex cution de proc dures autres que ce qui est sp cifi dans les pr sentes peut engendrer une exposition au rayonnement grave PRODUIT LASER DE C...

Page 65: ...gstemperatur erforderlich Au erdem sollten mindestens 8 cm 3 in Freiraum um die Bel ftungs ffnungen sein um einen einwandfreien Luftstrom zu gew hrleisten Arbeiten Sie w hrend eines Gewitters und Blit...

Page 66: ...ak Achtung Nutzung von Steuerungen oder Einstellungen oder Ausf hrung von Prozeduren die hier nicht spezifiziert sind kann zu gef hrlichem Strahlenkon takt f hren Klasse 1 Laserprodukt und Referenzen...

Page 67: ...ar una circulaci n de aire adecuada se debe dejar como m nimo un espacio de 8 cm 3 pulgadas alrededor de las aberturas de ventilaci n No utilizar el equipo ni conectar o desconectar cables durante per...

Page 68: ...ligrosos Precauci n el uso de controles o ajustes o la realizaci n de procedimientos distintos de los que aqu se especifican podr an causar exposici n a niveles de radiaci n peligrosos PRODUCTO L SER...