Dell Force 10, Manual

Page 1: ...Product Test Group Major Contributor Tony Molina Product Test Group Implementing FCoE in the Linux Operating System This white paper explains using Broadcom based network adapters and Dell Force10 and Cisco Nexus switches ...

Page 2: ...raphy or photography Dell the Dell logo and PowerEdge are trademarks of Dell Inc Intel and Xeon are registered trademarks of Intel Corporation in the U S and other countries Microsoft Windows and Windows Server are either trademarks or registered trademarks of Microsoft Corporation in the United States and or other countries Other trademarks and trade names may be used in this document to refer to...

Page 3: ...iagram 7 Figure 2 Results of show fip snooping enode command 10 Figure 3 Results of show fip snooping fcf command 10 Figure 4 Results of FCoE and LLDP query commands 11 Figure 5 FCoE support not installed 12 Figure 6 Entering Software Management menu 13 Figure 7 Selecting FCoE packages for installation 14 Figure 8 Network Services menu 15 Figure 9 FCoE client configuration screen with interfaces s...

Page 4: ...nstalled 25 Figure 25 Mounting the installation media 25 Figure 26 Installing FCoE support 25 Figure 27 Unmounting the installation media 26 Figure 28 Creating the FCoE Adapter Configuration file 26 Figure 29 Setting the DCB_REQUIRED variable to no 26 Figure 30 Checking the SUPPORTED_DRIVERS variable 27 Figure 31 Verifying proper configuration of FCoE VLAN 27 Figure 32 Verifying LUNs availability ...

Page 5: ... the various devices on the network such as the SAN switches network interface adapters and operating systems on the servers This Dell How To document is intended to assist you in setting up FCoE in your environment Due to the numerous variances in possible SAN switch network interface adapter and server choices it is not possible to write exact instructions for every conceivable supported configu...

Page 6: ...are version 7 2 14 was used 4 Install the appropriate Linux Operating System and make sure it is functioning on the Dell PowerEdge server In addition the Ethernet interface over which you want FCoE traffic to flow must be configured and online Verify that you can ping another device on the network using this Ethernet interface before attempting to setup FCoE 5 The driver for the Broadcom network a...

Page 7: ...cture for FCoE While performing these steps refer to the network diagram in Figure 1 Network infrastructure diagram Figure 1 1 Set up in NPIV mode and enable the FCoE feature on the Cisco Nexus 5020 distribution switch From the Global Configuration Mode of the distribution switch run the following commands feature fcoe feature lcap feature lldp feature npiv ...

Page 8: ... no ip address mtu 12000 portmode hybrid switchport spanning tree pvst edge port protocol lldp dcbx port role auto downstream no shutdown 4 Program the switch facing ports physically accessible ports on the back of the M1000e chassis on the edge switch by running the following commands from the switch command line interface TenGigabitEthernet 0 49 description LAG port to Cisco 5020 no ip address m...

Page 9: ... GUI Be aware of what port your server is on and what side of the fabric your server is on for example Fabric A side 2 You bind this address on the Cisco 5020 distribution switch that is running NPIV mode by running the following commands from the switch command line interface vfc XX this creates a Virtual Fibre Channel FVC interface XX is any number id that is available bind mac address a4 ba db ...

Page 10: ... see Figure 3 for results show fip snooping fcf Results of show fip snooping fcf command Figure 3 13 To verify on the Cisco Nexus 5020 distribution switch that your server is successfully connected using FCoE you can run the following commands in EXEC mode see Figure 4 for results show fcoe database this lists the vfcs and their respective active MAC address show lldp neighbors this shows all exte...

Page 11: ...Implementing FCoE in the Linux Operating System 11 Results of FCoE and LLDP query commands Figure 4 ...

Page 12: ... 1 Log in as root then the start YaST2 Control Center by running the yast command From YaST2 select Network Services in the left side menu then use the Tab key switch to the right side menu Use the arrow keys to scroll up and down the list of Network Services and look for FCoE Client Configuration if FCoE Client Configuration is not listed as in Figure 5 then FCoE support is not installed in SLES ...

Page 13: ...anagement in the right side menu and press Enter as in Figure 6 Entering Software Management menu Figure 6 3 Type fcoe into the Search Phrase box and then press Enter You should be presented with open fcoe and yast2 fcoe client as installable packages Use the Tab key to highlight each of these items and mark them for installation by using the Space Bar to put a in front of each of the packages nam...

Page 14: ... select OK to allow the installation to proceed After the packages install you are returned to the YaST2 Control Center When finished quit the YaST2 Control Center Configure the FCoE client VLAN Next configure the FCoE client VLAN by performing the following steps 1 Start the YaST2 Control Center using the yast command then select Network Services and then FCoE Client Configuration as in Figure 8 ...

Page 15: ...ces sub menu as in Figure 9 FCoE client configuration screen with interfaces sub menu Figure 9 Note that the supported Broadcom network adapters have yes listed in the FCoE column while unsupported Broadcom network adapters will not have any listing in the FCoE column 3 Highlight the adapter for which you would like to configure FCoE and then select Create FCoE VLAN Interface as in Figure 10 ...

Page 16: ...rface Figure 10 4 You are prompted to confirm whether or not you really want to create a new FCoE VLAN interface as in Figure 11 select Yes Confirmation of new FCoE VLAN interface Figure 11 You then see your FCoE VLAN appear in the FCoE VLAN Interface column as in Figure 12 ...

Page 17: ... System 17 New FCoE VLAN created Figure 12 5 Select OK and then Quit to return to the command prompt Run the command fcoeadm i to verify that the FCoE VLAN has been successfully brought online as in Figure 13 FCoE VLAN is now online Figure 13 ...

Page 18: ...displayed Figure 14 Partition and mount FCoE LUNs as any other hard disk device Configure the FCoE client to start at boot If you want the FCoE client to start and the FCoE LUNs to automatically be available after every reboot configure the appropriate services using YaST2 by performing the following steps 1 Start the YaST2 Control Center using the yast command select Network Service and then FCoE...

Page 19: ...erating System 19 Select the FCoE Client Configuration menu Figure 15 2 Configure FCoE to start when the system boots Select the Services menu Then set both FCoE Services Start and Lldpad Services Start to When Booting as in Figure 16 ...

Page 20: ...stem boots Troubleshooting Various conditions may exist on the network that interferes with FIPS snooping particularly on congested networks If the FCoE client is unable to see the FCoE VLAN after booting restart the lldpad and fcoe daemons as a troubleshooting step 1 To restart the lldpad and fcoe daemons run the following commands rcfcoe stop rclldpad stop rclldpad start rcfcoe start 2 Then chec...

Page 21: ...o detect installation media Figure 18 The failure to detect installation media is most commonly due to the SLES 11 SP2s configuration files pointing to the incorrect location of the installation media software repository To correct this issue perform the following steps 1 Start YaST2 using the yast command and then select Software in the left side menu and then select Software Repositories on the ...

Page 22: ...ux Operating System 22 Entering Software Repositories Menu Figure 19 2 Delete the existing installation media software repository and then add the SLES 11 SP2 installation media as a new software repository as in Figures 20 and 21 ...

Page 23: ...tory Figure 20 Adding new SLES Install Media Software Repository Figure 21 3 Select the appropriate installation media and then select Next as in Figure 22 4 Accept the License Agreement select Next and then select OK and you will be returned to the YaST2 Control Center as in Figure 23 ...

Page 24: ...Implementing FCoE in the Linux Operating System 24 Selecting the installation media Figure 22 Accepting the License Agreement Figure 23 ...

Page 25: ...install the appropriate FCoE support package for example install the fcoe utils package insert Disc 1 of the Red Hat Enterprise Linux 6 2 installation media into your CD DVD drive and then mount the disc to a directory of your choice in this example use media as in Figure 25 Mounting the installation media Figure 25 3 Next change directory to media Packages and install the appropriate packages for...

Page 26: ...Change directory to etc fcoe and copy the etc fcoe cfg ethx file to e4tc fcoe interface where interface is the name of the specific network interface over which FCoE traffic flows In this example we will use the interface name p3p1 See Figure 28 for details Creating the FCoE Adapter Configuration file Figure 28 2 Then edit the etc fcoe cfg p3p1 file and set DCB_REQUIRED variable to no as in Figure...

Page 27: ...running the service lldpad start and service fcoe start commands Then running the fcoeadm i command to verify that the FCoE VLAN is properly configured as in Figure 31 Verifying proper configuration of FCoE VLAN Figure 31 5 Finally verify LUNs device name by listing all available partitions using the cat proc partitions command as in Figure 32 ...

Page 28: ... Figure 33 2 Verify that the services are set to start at boot by running chkconfig list grep lldpad and chkconfig list grep fcoe as in Figure 34 Verifying that FCoE services are configured to start at boot time Figure 34 Troubleshooting It is possible for other various conditions to exist on the network that can interfere with FIPS snooping particularly on congested networks If the FCoE client is...

Page 29: ...mentations are likely be fairly similar on the various available FCoE compliant devices Use this document as a general guide to aid you in configuring FCoE in most situations If you do use this guide with hardware other than that listed in the Described Configuration section above verify that it is FCoE compliant and that all appropriate firmware and drivers are updated per vendor specifications p...