Brocade Communications Systems CNA 1010, Troubleshooting Manual

Page 1: ...53 1001582 01 22 June 2010 Brocade Adapters Troubleshooting Guide Supporting CNA models 1010 and 1020 Supporting HBA models 415 425 804 815 and 825 ...

Page 2: ...rams that accompany it The product described by this document may contain open source software covered by the GNU General Public License or other open source license agreements To find out which open source software is included in Brocade products view the licensing terms applicable to the open source software and obtain a copy of the programming source code please visit http www brocade com suppo...

Page 3: ...ystem support xii What s new in this document xiii Document conventions xiii Text formatting xiii Command syntax conventions xiv Command examples xiv Notes cautions and warnings xiv Key terms xv Notice to the reader xv Additional information xv Brocade resources xvi Other industry resources xvi Providing details for support xvii Document feedback xx Chapter 1 Introduction to Troubleshooting In thi...

Page 4: ...r scripts14 Files needed for bfad sys message appears 15 Cannot roll back driver on all adapter instances using Device Manager 15 Errors when installing brocade_driver_linux_ versions tar gz package 16 Operating system crashes after invalid HCM operand 16 UEFI boot problems 16 BIOS boot problems 19 HBA problems only 22 Quality of Service performance issues 22 Quality of Service not functioning 23 ...

Page 5: ...l HCM 38 Time on HCM screens does not match system time 39 Verifying Fibre Channel and CEE links stand up adapters 39 Adapter driver installation verification 40 Confirming driver package installation with HCM 41 Confirming driver package installation in Windows systems 41 Confirming driver package installation in Linux systems 42 Confirming driver package installation in Solaris systems 43 Confir...

Page 6: ...opback test 78 Memory test 79 Pinging Fibre Channel end points 80 Adapter temperature 81 Queue test 81 Trace route 82 Echo test 82 SCSI test 83 Test Logs 83 Collecting BIOS data 84 Displaying BIOS data through BCU 84 Display BIOS data through HCM 84 Collecting LLDP data CNAs only 84 Collecting SFP data stand up adapters 85 SFP properties 85 Predictive optical monitoring POM 85 Collecting port data...

Page 7: ...2 Determining target rate limiting settings through BCU 92 Determining settings through HCM 93 Persistent binding 93 Displaying Persistent Binding settings through BCU 93 Displaying Persistent Binding settings through HCM 94 Chapter 4 Performance Optimization In this chapter 95 Tuning storage drivers 95 Linux tuning 95 Solaris tuning 96 Windows tuning 96 VMware tuning 97 Tuning network drivers CNA...

Page 8: ...viii Brocade Adapters Troubleshooting Guide 53 1001582 01 ...

Page 9: ...tion to Troubleshooting provides an introduction and approach to troubleshooting adapter problems as well as tips for gathering problem information A checklist is also provided to verify that required procedures have been followed during installation Chapter 2 Isolating Problems provides information on common adapter problems and procedures to diagnose and recover from these problems Chapter 3 Too...

Page 10: ... Fabric OS and switch support Brocade CNAs must connect to Fibre Channel SANs and Ethernet data networks through a compatible FCoE switch For a current list of compatible switches refer to the latest compatibility matrices on the Brocade website at www brocade com Select Products Solutions and then select the Compatibility link Host operating system support This section describes operating systems...

Page 11: ...e detailed in the current HBA and CNA release notes for your adapter software version FCoE support The following lists operating systems that support FCoE for the CNAs Windows 2003 x86 x64 IA64 Windows Server 2008 x86 x64 IA64 Microsoft Hyper V for Windows 2008 x86 x64 IA64 Windows Server Core 2008 x86 x64 IA64 Microsoft WinPE 2 x for Windows 2008 x86 x64 IA64 Linux RHEL4 RHEL5 SLES10 and SLES11 x...

Page 12: ...is HBA installs in supported server blades that install in supported blade system enclosures Brocade 815 Single port stand up HBA with a per port maximum of 8 Gbps using an 8 Gbps SFP Brocade 825 Dual port stand up HBA with a per port maximum of 8 Gbps using an 8 Gbps SFP Brocade 415 Single port stand up HBA with a per port maximum of 4 Gbps using a 4 Gbps SFP Brocade 425 Dual port stand up HBA wi...

Page 13: ...ventions and important notice formats used in this document Text formatting The narrative text formatting conventions that are used are as follows bold text Identifies command names Identifies the names of user manipulated GUI elements Identifies keywords and operands Identifies text to enter at the GUI or CLI italic text Provides emphasis Identifies variables Identifies paths and Internet address...

Page 14: ...atement indicates potential damage to hardware or data CAUTION A Caution statement alerts you to situations that can be potentially hazardous to you or cause damage to hardware firmware software or data DANGER A Danger statement indicates conditions or situations that can be potentially lethal or extremely hazardous to you Safety labels are also attached directly to products to warn of these condi...

Page 15: ...esources for Brocade products is available Adapters For adapter resources such as product information software firmware and documentation visit the following websites HBA website at www brocade com hba CNA website at www brocade com cna For additional information on Brocade adapters refer to the following publications Brocade FCoE CNA Quick Installation Guide provided with your CNA model Brocade F...

Page 16: ...Hardware Compatibility section in Chapter 1 of the Brocade Adapters Installation and Reference Manual SAN information White papers online demonstrations and data sheets are available through the Brocade website at http www brocade com products solutions products index page For additional Brocade documentation visit the Brocade website http www brocade com Other industry resources For additional re...

Page 17: ...de are provided on the serial number label illustrated below This label is affixed to the adapter card You can also display the serial number through the following HCM dialog boxes and BCU commands Adapter Properties tab in HCM Select an adapter in the device tree then click the Properties tab in the right pane BCU adapter list command This command lists all adapters in the system and information ...

Page 18: ...c through a local CNA port is assigned a MAC address during FCoE Initialization Protocol FIP operations This MAC is assigned for the current FCoE communication only To find this MAC address perform one of the following tasks Select an FCoE port in the HCM device tree then click the Properties tab in the right pane to display the port Properties panel Look for the FCoE MAC field Enter the port quer...

Page 19: ...ade every effort to ensure the accuracy and completeness of this document However if you find an error or an omission or you think that a topic needs further development we want to hear from you Forward your feedback to documentation brocade com Provide the title and version number of the document and as much detail as possible about your comment including the topic heading and page number and you...

Page 20: ...xx Brocade Adapters Troubleshooting Guide 53 1001582 01 ...

Page 21: ...ormation in Chapter 2 Isolating Problems Adapter problems are organized under the following categories General HBA and CNA problems HBA problems only Ethernet network interface problems CNAs only FCoE and Fibre Channel problems CEE network problems CNAs only Search through the list of problems in Table 2 on page 5 Problems are organized in the table by problem title category and whether the proble...

Page 22: ...the problem documented in release notes for the switch and target storage system Is unexpected behavior intermittent or always present If the problem is in a Fibre Channel or FCoE switch storage device or in connectivity between these components refer to the documentation help systems or service providers of that equipment 5 If you cannot resolve the problem gather and provide problem information ...

Page 23: ... in Table 1 l The bfa_supportsave and FOS supportsave commands can provide current information for the topology map Also consider using the Brocade SAN Health products to provide information on your SAN environment including an inventory of devices switches firmware versions and SAN fabrics historical performance data zoning and switch configurations and other data Click the Services Support tab o...

Page 24: ...es that support adapter operation are installed in the appropriate enclosure bays Also check whether target storage devices are connected to the appropriate switch interconnect module or I O module in the blade system enclosure and are turned on Record the time and frequency of symptoms and the period of time symptoms have been observed Determine if unexpected behavior is intermittent or always pr...

Page 25: ...roblems HBA problems only Ethernet network interface problems CNAs only FCoE and Fibre Channel problems CEE network problems CNAs only Use Table 2 to quickly navigate to sections in this chapter that cover specific adapter problems Each problem section in this chapter contains a description of the problem possible causes and actions for resolution Click on the problem to link to the appropriate pr...

Page 26: ... problems UEFI boot problems All Driver and operating system installation failure on boot LUN General HBA and CNA problems UEFI boot problems All Driver and operating system installation failure on boot LUN General HBA and CNA problems BIOS boot problems All Target not visible from host General HBA and CNA problems BIOS boot problems Windows Ctlr B option does not display when booting host General...

Page 27: ...r VLAN operations through HCM fail Ethernet network interface problems CNAs only Windows 2003 Ping to remote host is failing Ethernet network interface problems CNAs only All Teaming errors Ethernet network interface problems CNAs only Windows Loss of sync and loss of signal errors in port statistics FCoE and Fibre Channel problems All Fabric authentication failures FCoE and Fibre Channel problems...

Page 28: ...em 1 Possible Cause Card not seated in host system connector properly Action Reseat the adapter 2 Possible Cause Host system connector issues Action Install an adapter of known working condition to determine whether there is a connector malfunction Action Try installing the adapter into a different slot if available 3 Possible Cause Adapter not compatible with host operating system or connected st...

Page 29: ...splaying the port list on page 86 for more information Refer to the following descriptions of possible causes and recommended actions or fixes for the problems 1 Possible Cause SFP or cable problems stand up adapters Action Ensure that the SFPs and cables are connected properly on both adapter and switch sides Check for any cable damage 2 Possible Cause Switch port is disabled or switch is disable...

Page 30: ...kage from the Brocade adapter website www brocade com hba or www brocade com cna Remove and reinstall the driver package using instructions in the Software Installation chapter of the Brocade Adapters Installation and Reference Manual Installer program does not autorun Windows only If the installer program does not automatically run from the CD that you create with the ISO file containing all supp...

Page 31: ...strator s Guide for procedures to configure adapter ports NOTE Observe the LEDs on adapter ports stand up adapters only Illuminated LEDs indicate connection link activity and connection speed negotiated with the attached device For the meaning of LED operation refer to the Brocade Adapters Installation and Reference Manual If the system freezes perform the following tasks a Verify whether the host...

Page 32: ...ARNING BCU and Driver versions don t match Possible Cause Installation may be incomplete Either the BCU or one or more driver instances were not upgraded to the latest version Action Remove the driver package then reinstall Refer to the Software Installation chapter in the Brocade Adapters Installation and Reference Manual NOTE Make sure that the server does not reboot before reinstalling the driv...

Page 33: ...nsceiver Faulty SFP or loopback connector 2 Possible Cause Fibre Channel or FCoE I O problems Action Run the loopback serdes test on the suspected Fibre Channel port HBAs only or FCoE port CNAs only using the BCU diag loopback port_id t serdes command If the test passes suspect the following external problems Faulty fiber Faulty software Target problem Action Run the BCU external loopback test sta...

Page 34: ...ation 4 Uninstall the driver using the driver installer program brocade_installer bat program Action To avoid the message when removing a driver do not uninstall the driver using the Device Manager if you have used Brocade installer driver installer programs to install driver instances Always use the Brocade installer programs If only one driver is present in the system then the Brocade programs a...

Page 35: ...and The Linux RHEL 4 update 7 system kernel 2 6 9 78 crashes after an invalid operand from the HCM agent process Possible Cause There may issues with the kernel regarding the system time Action Upgrade the kernel to the latest patch 2 6 9 78 0 5 UEFI boot problems This section describes problems that may occur when using the Brocade adapter and unified extensible firmware interface UEFI for bootin...

Page 36: ...m a specific LUN The host s operating system adapter driver and other necessary files are installed on the boot LUN Boot devices not available in host s Boot Manager menu Fibre Channel attached boot devices do not appear in the EFI Boot Manager menu or a boot device appears but it is not functioning Following are descriptions of possible causes and recommended actions or fixes for the problem 1 Po...

Page 37: ...any attached Fibre Channel disk devices For example you would enter the following for the HBA with device handle 29 Shell dh d 29 The following displays 29 PciIo ScsiPassThruExt BusSpecificDriverOverride DevPath P0A08 300 Pci 0 0 Pci 0 0 Managed by Drv 25 Brocade Fibre Channel Adapter Bus Driver Drv 26 SCSI Bus Driver Child Controllers Child 70 SCSI Disk Device Child 71 SCSI Disk Device Child 72 S...

Page 38: ...ate driver update disk DUD may not have been used for the host operating system and platform For example if the brocade_adapter_rhel53_x86_64 DUD is used during RHEL 5 3 x86 installation installation will fail Action Verify the DUD for your system using information in Chapter 1 of the Brocade Adapters Installation and Reference Manual Using the correct DUD reinstall appropriate drivers and the ope...

Page 39: ...n Brocade BIOS Configuration menu No target devices found or link down message displays on Brocade BIOS configuration menu during boot device discovery 1 Possible Cause There is no fabric connectivity between the Brocade adapter and target or the target is not online Action Execute the Fabric OS nsAllShow command on the attached switch to verify that the target and the host are online in the fabri...

Page 40: ... applications 3 Possible Cause Missing or improper storage array LUN mask setting Action Check LUN mapping and masking using storage array configuration tools 4 Possible Cause Adapter driver not loaded Action The adapter driver may not be loaded Refer to Adapter driver installation verification on page 39 for methods to verify driver installation Cannot boot over SAN from installed CNAs You cannot...

Page 41: ... some reason 1 Boot image is not present in the adapters 2 Adapter might have failed to initialize 3 Link did not come up Action Bring up the host system either using the Brocade live CD or boot from local disk Action Download the latest boot image zip file brocade_adapter_boot_fw_ version zip from the Brocade adapter website at www brocade com hba or www brocade com cna Extract the boot image fil...

Page 42: ...d low priority targets Action Verify that QoS is configured on the switch using instructions in the Fabric OS Administrator s Guide Quality of Service not functioning Quality of Service QoS is enabled on an adapter port connected to a supported edge switch but it does not appear to be functioning Possible Cause The following licenses must be installed Adaptive Networking AN license Server Applicat...

Page 43: ...nning loopback test 3 Symptom Loopback test returns a port not disabled error The error displays even after disabling the port Possible Cause Network load balancing service is enabled This will cause the adapter to disable and enable and the previously configured state port disable is lost Action Disable network load balancing and retest 4 Symptom The loopback test returns Device busy Retry operat...

Page 44: ... systems enter the following commands chkconfig NetworkManager off chkconfig NetworkManagerDispatcher off To disable NetworkManager for SLES systems perform the following steps a Open YaST b Select the Network Devices Network Card c On the first screen set the Network Setup Method option to Traditional Method with ifup Loss of adapter hardware address in Linux The ifconfig command displays HW Addr...

Page 45: ...pected on VMware systems After installing CNA drivers on VMware systems NIC numbering is not what is normally expected For example instead of vmnic32 or vmnic33 number is vmnic2 and vmnic3 Possible Cause CNA hardware was installed before drivers Action When installing a CNAs on a VMware systems it is advisable to install the driver before the CNA cards so that the NICs will be properly enumerated ...

Page 46: ... size if necessary Note that MTU size set on adapter must not be more than MTU size set on attached FCoE switch To set MTU size on the adapter refer to the Adapter Configuration Chapter in the Brocade Adapters Installation and Reference Manual Action Verify and set IP address and network mask if necessary Linux Run the ifconfig command to determine if port has proper IP address and network mask an...

Page 47: ...o the Adapter Configuration appendix in the Brocade Adapters Installation and Reference Manual VLAN creation and operation problems VLAN creation fails with BCU command or HCM or pass through VLAN stops working after creating with Device Manager These problems result when VLANs are created using HCM or BCU commands and also using Device Manager Follow these guidelines to avoid problems If you need...

Page 48: ...eating a team or VLAN through HCM on Windows 2003 systems the operation fails and an error may result Symptoms There are several symptoms of this problem VLAN or teaming operations fail through HCM or event 7030 occurs with the following text The Brocade HCM is marked as an interactive service However the system is configured not to allow interactive services This service may not function properly...

Page 49: ...ymptom Dynamic interrupt moderation is disabled Action For Windows verify if interrupt moderation is enabled using the Advanced tab on the Network Adapters Properties dialog box in Device Manager Action For Linux run the ethtool c interface ID command If interrupt moderation is enabled information similar to the following displays in the output Coalesce parameters for eth2 Adaptive RX on TX off Ac...

Page 50: ...ric authentication failures If failures in the authentication process between the adapter in host system and the connected switch occur refer to the following descriptions of possible causes and recommended actions to help resolve the problem Possible Cause Authenticating configuration is incorrect Action Check authentication settings on the switch and adapter For the switch execute the authutil s...

Page 51: ...ects and unknown name server port login response NS login unknown rsp These errors mean that the adapter most likely cannot log in to the name server 2 Possible Cause Adapter has a problem registering with the name server Action Display logical port statistics refer to Logical port statistics on page 68 for details on displaying these statistics Check for increasing errors of the following types T...

Page 52: ... the switch is set for a VLAN with FCF capability using appropriate Fabric OS command on the attached switch Refer to the Fabric OS Command Reference Manual for more information Action Set the FC MAP for a VLAN with FCF capability using the appropriate Fabric OS command on the attached switch Refer to the Fabric OS Command Reference Manual for more information 6 Possible Cause The FCoE Login group...

Page 53: ...e is enabled which implies zero Path TOV values or that fcpim MPIO mode is disabled with the expected Path TOV settings default is 30 seconds Disk I O requests cause low throughput and high latency on Linux If a high number of I O requests is causing low throughput and high latency on Linux systems refer to the following descriptions of possible causes and recommended actions to help resolve the p...

Page 54: ... fix the CEE configuration at the switch appropriately based on the displayed error reason code Action Check FCoE switch configuration using the appropriate Fabric OS command on the attached switch Refer to the Fabric OS Command Reference Manual for information Change configuration as necessary using appropriate Fabric OS command on the attached switch Refer to the Fabric OS Command Reference Manu...

Page 55: ...he hcmagent is stopped restarting the agent should resolve the problem To restart use the appropriate start command for your operating system which is also described in the Software Installation chapter in the Brocade Adapters Installation and Reference Manual under HCM Agent Operations c Note that one command described in the manual restarts the agent but the agent will not restart if the system ...

Page 56: ...ecuting the appropriate status command for your operating system described in the Brocade Adapters Installation and Reference Manual Refer to the section on modifying HCM Agent operation b If you receive a message that the hcmagent is stopped restarting the agent should resolve the problem To restart use the appropriate start command for your operating system which is also described in the Brocade...

Page 57: ...n using the Support Save feature refer to Support Save on page 48 Support data from the HCM application SupportSave feature Build information for the HCM application Select Help About in HCM to display the version build identification and build date To use the HCM SupportSave feature select Tools Support Save in HCM to generate a supportsave file If HCM cannot connect to the agent a message displa...

Page 58: ...ist of time zones that have changed together with their DST start dates and end dates Use the information in the following article located at http support microsoft com kb 981128 A hotfix is available to update the Daylight Saving Time for the UTC 04 00 Asuncion UTC 12 00 Fiji and UTC 04 00 Santiago time zone for Windows Operating Systems Verifying Fibre Channel and CEE links stand up adapters Che...

Page 59: ...ation verification Problems with adapter operation may be due to improper hardware or software installation incompatibility between the adapter and your host system unsupported SFPs installed on the adapter improper cable connected to the fabric or the adapter not operating within specifications Determine if problems may exist because of these factors by reviewing your installation with informatio...

Page 60: ... the following steps to display the adapter PWWN driver name and version firmware name and version and the BIOS version currently in operation 1 Launch HCM 2 Select the adapter in the device tree 3 Select the Properties tab in the right pane to display the Properties dialog box The dialog box displays adapter properties Confirming driver package installation in Windows systems Use the Device Manag...

Page 61: ...nstalled rpm qa grep i bna This command prints the names of the Brocade adapter network driver package bfa if installed lspci This utility displays information about all PCI buses in the system and all devices connected to them Fibre Channel Brocade Communications Systems Inc displays for an HBA Fibre Channel Brocade Communications Systems Inc and Ethernet Controller display for a CNA if driver pa...

Page 62: ...system ARCH sparc i386 VERSION alpha_bld31_20080502_1205 BASEDIR VENDOR Brocade DESC 32 bit 64 bit Device driver for Brocade Fibre Channel adapters PSTAMP 20080115150824 INSTDATE May 02 2008 18 22 HOTLINE Please contact your local service provider STATUS completely installed Confirming driver package installation in VMware systems Verify if the adapter driver installed successfully using the follo...

Page 63: ...peration Refer to your blade system enclosure and documentation for blade system enclosure components for help The blade server on which the adapter is installed is correctly configured for adapter operation and is correctly installed in the blade system enclosure Refer to your blade server and blade system enclosure documentation for help All modules or blades in that support adapter operation ar...

Page 64: ... for isolating problems 2 Fabric OS Troubleshooting and Diagnostic Guide Provides help with isolating problems in other Brocade SAN components Your host s operating system documentation and help system Provides details on commands for gathering information and isolating problems ...

Page 65: ... data CNAs only 82 Collecting SFP data 82 Collecting port data 83 Collecting teaming information CNAs only 87 Authentication settings 88 QoS settings HBAs only 89 Target rate limiting settings 90 Persistent binding 91 For detailed information This chapter provides basic instruction on tools useful for gathering information to isolate adapter problems For more detailed information on using these to...

Page 66: ...ics Port logs Port statistics and properties Adapter properties Host operating system error logs Data collection using host system commands Table 5 describes commands common to each supported operating system that you can use to gather information for troubleshooting problems For details on these commands refer to your system s online help and documentation NOTE Output from all of these commands i...

Page 67: ...twork adapter instances select Properties then select Driver tab For general driver information use vmkload_mod For network driver information use esxcfg nics Use modinfo options for bna or bfa driver Locating system log messages NOTE For more information refer to Host system logs on page 56 dmesg var log message System Category in Windows Event Viewer eventvwr exe var log vmkernel var log vmkwarn...

Page 68: ...BCU command prompt using the BCU shell mode For details on using BCU commands refer to the Brocade Adapters Administrator s Guide Through your internet browser Internet Explorer 6 or later or Firefox 2 0 or later you can collect bfa_supportsave output if you do not have root access do not have access to file transfer methods such as FTP and SCP or do not have access to the Host Connectivity Manage...

Page 69: ...net interface including IP address and mask CNAs only Status and states of all adapter ports including the Ethernet FCoE and CEE ports CNAs only CEE status and statistics CNAs only Network driver information Ethernet statistics offload parameters and flow control coalesce parameters CNAs only Ethernet offload flow control and coalescing parameters CNAs only NOTE Before collecting data through the ...

Page 70: ...ters complete the following steps 1 Select Monitor Technical Support SupportSave The Technical SupportSave dialog box displays 2 Click the Generate Now tab 3 Click the Hosts tab 4 Right click in the Available Hosts table and select Expand All 5 Select the host where adapters are installed and click the right arrow to move them to the Selected Switches and Hosts table 5 Click OK on the Technical Su...

Page 71: ...browser collects information for adapters installed on a selected host system Use the following steps to launch this feature 1 Open an Internet browser and type the following URL https localhost 34568 JSONRPCServiceApp SupportSaveController do where localhost is the IP address of the host system from which you want to collect the bfa_supportsave information 2 Log in using the factory default user ...

Page 72: ...te code MDC engineering date code EDC power rating PW disabled or enabled status model number temperature maximum speed supported installed driver name driver version firmware version hardware path and BIOS version This panel also displays PCI information for the adapter such as vendor ID device ID current number of lanes and PCI generation The Converged Network Adapter CNA Properties panel displa...

Page 73: ...n PWWN NWWN beacon status adapter query ad_id where ad_id ID of the adapter for which you want to query Data collection using Fabric OS commands Brocade switches only Use the following Fabric OS commands on attached Brocade switches to gather information and help isolate connectivity and other problems between the adapter switch and storage ports For details on using these commands refer to the Fa...

Page 74: ...his command displays sets and removes shared secret key information from the databases sfpShow Use this command to display detailed information about specific SFPs installed in a switch show vlan brief Displays information about a VLAN interface on the switch show cee maps Displays information about the configured CEE maps in the switch switchShow Use this command to display switch and port inform...

Page 75: ...lists the message file names and content for the message files TABLE 6 Message catalog location Operating System Catalog Location Linux opt bfa VMware opt bfa Solaris opt bfa Windows aen zip loaded to your driver installation directory Unzip this file to obtain all message catalog files TABLE 7 Event message files Event Catalog File Content bfa_aen_adapter html Adapter events such as adapter added...

Page 76: ...NA Brocade network adapter and BFAL Brocade fabric adapter library Table 8 describes the logs for each supported operating system where adapter event messages display and how to view these logs bfa_aen_rport html Remote port R_Port events hba_error_codes doc List of error codes and meanings for the following events Adapter events relating to the adapter Physical port L_Port logical port R_Port rem...

Page 77: ...n HCM discovery or HCM application issues Master Log The Master Log displays event information in seven fields Sr No Sequence number assigned to the event when it occurred in ascending order Severity Event severity level informational minor major or critical Critical level messages indicate that the software has detected serious problems that will eventually cause a partial or complete failure of ...

Page 78: ...aster Log by severity category and WWN of adapter using the Master Log Filter dialog box To display this dialog box click the Filter button in the Master Log section of the main HCM screen Select areas that you want to filter and click OK Application Log The Application Log displays all application related informational and error messages as well as the following attributes Date and time the messa...

Page 79: ...d port name This could also be the adapter index port index For example to specify adapter 1 port 1 you would use 1 1 as the port identification level Critical Error Warning Info Specifies the severity level Error is the default setting If no level is specified the current log level displays Adjusting the Ethernet logging level through HCM CNAs only This is available for CNAs only Adjust the loggi...

Page 80: ...d the HCM Agent HCM Debug Log where messages are logged locally To adjust the logging level use the following steps 1 Select Configure HCM Logging Levels 2 Select a level on the Agent Communication Log and HCM Debug Log lists Values are Trace Debug Info Warning Error and Fatal Statistics You can access a variety of statistics using BCU commands and HCM Use these statistics to monitor adapter perfo...

Page 81: ...mber of times a transmitted Fibre Channel authentication attempt was successful Tx DHCHAP Successes The number of rejected received Fibre Channel authentication attempts Rx Auth Rjts The number of received Fibre Channel authentication negotiation attempts Rx Auth Negs The number of completed received Fibre Channel authentication attempts Rx Auth Dones The number of received DH CHAP challenge attem...

Page 82: ...ort name This could also be the adapter index port index For example to specify adapter 1 port 1 you would use 1 1 as the port identification Displaying statistics through HCM Display the CEE Statistics dialog box using the following steps 1 From the device tree select a physical port of a CNA 2 Select Monitor Statistics CEE Statistics CEE query CNAs only Query the CEE information on a selected po...

Page 83: ...lticast octets Received multicast frames Received multicast VLANs Received broadcast octets Received broadcast frames Received broadcast VLANs Received frame drops Received packets Transmitted packets Received bytes Transmitted bytes Linux NetIf queue stops Linux NetIf queue wakups Linux TSO IPv4 packets Linux TSO IPv6 packets Linux errors TCP checksum offloads UDP checksum offloads Checksum help ...

Page 84: ...t the host level displays Ethernet IOC statistics CNAs only Use HCM options and BCU to display statistics relevant to the Ethernet IO controller IOC A variety of statistics display such as the following Mailbox interrupts Enable and disable events Heartbeat failures Firmware boots Ethernet ICO statistic timeouts Checksum help errors VLAN transmit and receive transactions You can also select option...

Page 85: ...only Use BCU and HCM to display statistical information related to a selected Fibre Channel over Ethernet FCoE port Statistics include the number of transmitted and received packets and transmitted and received bytes You can also select options to continue running data configure polling frequency and start polling Displaying FCoE statistics through HCM To display FCoE statistics through HCM use th...

Page 86: ...tatistics fabric stats port_id where port_id ID of the adapter port for which you want to display statistics This could be the PWWN port hardware path or user specified port name This could also be the adapter index port index For example to specify adapter 1 port 1 you would use 1 1 as the port identification Displaying fabric statistics through HCM Use the Fabric Statistics dialog box to monitor...

Page 87: ...ch as I Os waiting for circular queue space number of I O context requests I O abort requests number of task management I O context requests I O completions with OK status number of successful firmware I O underrun operations number of successful firmware I O overrun operations aborted I O requests I O timeouts I O selection timeouts I O protocol errors and host I O abort requests Displaying FCP i...

Page 88: ...stics for the following Name server NS port logins plogin activity Register symbolic port name RSPN_ID identifier activity Register FC4 type identifier RFT_ID activity Register FC4 type identifier RFT_ID activity Get all port ID requests for a given FC4 type NS_GID_FT activity Retries Timeouts Use these statistics to help determine if the adapter is not registering with the name server or cannot a...

Page 89: ...nd CNA statistics For HBAs statistics include transmitted and received frames and words received loop initialization primitive LIP event counts error frames received loss of synchronization link failure and invalid CRS counts end of frame EOF errors and encoding non frame errors Use these statistics to isolate link and frame errors For example loss of synch and loss of signal errors indicate a phy...

Page 90: ...he rport offline and rport online statistics The rport online counter should be one greater than the rport offline counter If not clear the counters and retry connecting to the remote port Verify the rport online and rport offline statistics again Displaying target statistics through HCM Launch the Target Statistics dialog box using the following steps to display target statistics 1 Launch the HCM...

Page 91: ...ld also be the adapter index port index For example to specify adapter 1 port 1 you would use 1 1 as the port identification Virtual port statistics Use HCM and BCU to display logical port statistics for fabric discovery FDISC activity logouts LOGO activity NPIV support number of fabrics online and offline and fabric cleanups Use these statistics to isolate NPIV login problems Following are exampl...

Page 92: ...s length of time between byte transmission and reception and correction status You can also use options to set the polling frequency start polling and reset statistics Displaying VLAN statistics through HCM To display the VLAN Statistics dialog box use the following steps 1 Select an Ethernet port from the device tree 2 Select Monitor Statistics VLAN Statistics from the main menu The VLAN Statisti...

Page 93: ...mands are sent to the other side of the link When the remote port receives these commands that port s LEDs flash The remote port sends a command back to the originating port When that port receives this command the port s LEDs flash NOTE To initiate link beaconing this feature must be available on the connected switch Toggle beaconing on and off and set beaconing duration using the BCU or HCM Enab...

Page 94: ... then returned without transmitting through the port The returned data is validated to determine port operation Errors may indicate a failed port External For this test a loopback connector is required for the port Random data patterns are sent to the adapter port The data transmits from the port and then returns The returned data is validated to determine port operation Errors may indicate a fail...

Page 95: ...A5A5A5A5 5 Click Start Ethernet port loopback test CNAs only Use BCU commands and HCM to test the Ethernet data path from the host to serdes or external loopback based on your selection You must disable the port before testing and use a loopback connector for the test NOTE For 64 bit platforms only you cannot perform Ethernet loopback tests on the port unless a VLAN is first created for the port P...

Page 96: ...M to perform a PCI loopback test for a specific port In this test a data pattern is sent from the host to adapter firmware through the PCI bus The returned data is validated to determine PCI operation NOTE You must disable the port before you run loopback tests Performing PCI loopback tests through BCU Use the diag pciloopback BCU command to perform a PCI loopback test diag pciloopback port_id p p...

Page 97: ...t disables the adapter Performing a memory test through BCU Use the diag memtest command to test the adapter s memory blocks diag memtest ad_id where ad_id ID of the adapter Performing a memory test through HCM Use the Hardware Tests tab on the Diagnostics dialog box as follows to perform a memory test 1 Launch the HCM 2 Select Configure Diagnostics 3 Click the Hardware Tests tab 4 Select Memory T...

Page 98: ...WN 0 indicates the base port Issuing a ping command to end points through HCM Use the FC Protocol Tests tab on the Diagnostics dialog box to test the connection to Fibre Channel end points Use the following steps to ping end points 1 Launch the HCM 2 Select Configure Diagnostics to display the Diagnostics dialog box 3 Click the FC Protocol Tests tab 4 Select FC Ping Test 5 Select the adapter port ...

Page 99: ... 3 Click the Hardware Tests tab 4 Select Queue Test 5 Click Start Trace route Use the BCU and HCM to trace the SAN path between the adapter and remote end point Tracing the route through BCU Use the fcdiag traceroute BCU command to trace the route between end points fcdiag fctraceroute port_id rpwwn l lpwwn where port_id ID of the port from which you want to trace the route This could be the PWWN ...

Page 100: ...rt fcdiag fcecho port_id rpwwn l lpwwn where port_id ID of the port for which you want to perform the test This could be the PWWN port hardware path or user specified port name This could also be the adapter index port index For example to specify adapter 1 port 1 you would use 1 1 as the port identification rpwwn Remote port WWN to which the echo command is being sent lpwwn Logical port WWN 0 ind...

Page 101: ...isplaying BIOS data through BCU Use the bios query BCU command to display such information as the boot over SAN enabled or disabled state port speed boot LUNs obtained from flash and boot LUN 0 identification and target WWN Preboot means that parameters were configured using a blade system management application Preboot configuration data can include port speed BIOS boot over SAN state and configu...

Page 102: ...m the following steps 1 Select a CNA in the device tree 2 Click the LLDP tab in the right pane Collecting SFP data This section provides an overview of BCU commands and HCM features that provide information on small form factor pluggable SFP transceivers SFP properties BCU and HCM provide detailed information on the SFP transceiver for a selected port such as its health status port speed connector...

Page 103: ... Collecting port data This section provides an overview of BCU commands and HCM features that provide information on adapter ports such as PWWN node WWN port type configured speed operating speed configured topology operating topology link and port beaconing state and other information Displaying port properties Use the port Properties tab to display information about a selected base adapter port ...

Page 104: ...ollowing steps 1 Select a CEE port in the device tree 2 Click the Properties tab in the right pane Displaying Ethernet port properties CNAs only The HCM Ethernet Port Properties panel enables you to display the properties associated with a selected Ethernet port Information such as the following displays Name of Ethernet device Port type Current FCoE and factory MAC address IOC identification WWN ...

Page 105: ...uthentication status algorithm group and error status Displaying FCoE port properties To display the HCM FCoE Port Properties panel use these steps 1 Select an FCoE port in the device tree 2 Click the Properties tab in the right pane Displaying remote port properties Use the HCM Remote Port Properties panel to display properties that are associated with the remote port such as WWN node WWN port na...

Page 106: ...rameters properties panel displays Displaying the port log Use the debug portlog BCU command to display a log of Fibre Channel frames and other main control messages that were sent out and received on a specific port You can use this information to isolate adapter and Fibre Channel protocol problems debug portlog port_id where port_id The ID of the port for which you want to display the port log T...

Page 107: ...g port speed Use port speed BCU command to display the current port speed setting port speed port_id where port_id ID of the port for which you want to display port speed This could be the PWWN port hardware path or user specified port name This could also be the adapter index port index For example to specify adapter 1 port 1 you would use 1 1 as the port identification Collecting teaming informa...

Page 108: ...ed teams by their configured IDs the team mode failback failover or 902 3ad and the team s MAC address bcu team list The bcu team query command displays the teaming configuration for a specific team including the team mode MAC address and number of ports Also included are statistics such as transmit and receive bytes connection status and connection duration bcu team query team name where team nam...

Page 109: ...wing commands to deter ming QoS settings Use the following BCU command to display QoS enabled status port query port id where port_id ID of the port for which you want to display port information Use the following command to display QoS status and other information for a port qos query port_id where port_id ID of the port for which you want to display target rate limiting settings Use the followin...

Page 110: ...ands to determine current status and configuration for target rate limiting settings Use the following BCU command to determine target rate limiting speed and enabled status ratelim query port id where port_id ID of the port for which you want to display target rate limiting settings This could be the PWWN port hardware path or user specified port name This could also be the adapter index port ind...

Page 111: ...binding is available in the Windows and VMware environments only Use HCM features or BCU commands to display target ID mapping for an adapter port Displaying Persistent Binding settings through BCU Use the pbind list BCU command to query the list of mappings for persistent binding on a specific port pbind list port_id pwwn where port_id ID of the port for which you want to query mappings pwwn Port...

Page 112: ...92 Brocade Adapters Troubleshooting Guide 53 1001582 01 Persistent binding 3 ...

Page 113: ...p deadline anticipatory and completely fair queuing Enabling the noop scheduler avoids any delays in queuing of I O commands This helps in achieving higher I O rates by queuing multiple outstanding I O requests to each disk To enable the noop scheduler run the following commands on your system for i in sys block sd b z queue scheduler do echo noop i done NOTE You must disable the default scheduler...

Page 114: ... LUN queue depth value of 32 This value is chosen to reflect the average operating I O load in most scenarios Storport manages the I O throttling at the per LUN level to guarantee the configured queue depth During performance testing with specific high end array LUNs it may be necessary to increase this I O queue depth to a much larger value Microsoft recommends a value of 96 during high performan...

Page 115: ...stem Performance Analysis for Windows Server 2003 optimizations located on the following website http download microsoft com Please see the sections Performance Tuning for Storage Subsystem and I O Priorities in Performance Tuning Guidelines for Windows Server 2008 located on the following website http www microsoft com VMware tuning For performance tuning on VMware refer to the following publicat...

Page 116: ...parameters TABLE 9 Default values for Windows tunable parameters Parameter Default Log Level 3 Interrupt Moderation Set for receive interrupts On Jumbo Packet Size 1500 TCP UDP Checksum Offload Enabled Network Address N A Receive Buffers 32 Transmit Buffers 16 Priority and VLAN Disabled Receive Side Scaling RSS Enabled Header Data Split HDS Enabled Large Segmentation Offload V1 IPv4 LSOv1 Enabled ...

Page 117: ...e using default values For details on configuring the parameters in the following table refer to the Adapter Configuration chapter in the Brocade Adapters Installation and Reference Manual Table 11 provides default values for VMware tunable parameters TABLE 10 Default values for Linux tunable parameters Parameter Default Interrupt moderation Enabled Log Level 3 Jumbo packet size 1 500 bytes TCP UD...

Page 118: ...y and improved host CPU utilization MSI X is enabled by default in VMware ESX Server 3 5 and must remain enabled for NetQueue to function Please make sure that bnad_msix 0 is not listed in VMware module parameters because that would disable NetQueue For the Brocade driver you cannot directly configure the number of NetQueue and filters per NetQueue By default these values are based on the number o...

Page 119: ...ll the adapters in the system using instructions in the Boot Code chapter of the Brocade Adapters Installation and Reference Manual Adapter no Version mismatch Card Ver XXX Disabling BIOS Current executing option ROM version is different from the one on the adapter Download the latest boot code file for installed Brocade adapters from your Brocade adapter s website The boot code file is designated...

Page 120: ...covery failed Disabling BIOS The boot option is set to Auto Discovery or First Discovered LUN through the Brocade BIOS Configuration Utility and the adapter is unable to discover any boot device If booting through this adapter is not desired then ignore this message If booting through this adapter is desired 1 Launch the Brocade BIOS Configuration Utility by pressing Ctrl B or Alt B 2 For the desi...

Page 121: ...out of fabric No action required Logical port lost fabric connectivity WWN logical port WWN Role initiator target IPFC mode etc Error Storage driver 3 LPORT Offline Logical port base or logical lost fabric connectivity Check switch and adapter configuration New logical port created WWN logical port WWN Role initiator target IPFC mode etc Information Storage driver 3 LPORT Created New logical port ...

Page 122: ...le initiator target IPFC mode etc Information Storage driver 3 LPORT Deleted Virtual port deleted No action required Virtual port deleted using standard interface WWN logical port WWN Role initiator target IPFC mode etc Information Storage driver 3 LPORT Deleted Virtual port deleted No action required Virtual port login failed Duplicate WWN logical port WWN reported by fabric Warning Storage drive...

Page 123: ...ection stand up adapters Heart Beat of IOC IOC instance number is good Information Network and storage driver 9 TOC Restart Successful restart of firmware after a failure No action required Heart Beat of IOC IOC instance number has failed Critical Network and storage driver 9 TOC Crash Firmware not responding Collect all error information and restart the firmware Invoke bfa_supportsave IOC IOC ins...

Page 124: ...POM level to aggregated SFP temperature voltage rx and tx power level port base port number Hardware Address base port WWN or MAC Warning Network and storage driver stand up adapters only 2 Port Unhealthy Change of current value against threshold of temperature voltage and rx tx power of SFP Pluggable optical module POM plugs into the SFP If POM level is not normal check the SFP TABLE 13 Driver ev...

Page 125: ...ion policy as the switch Please type the same CHAP Secret Occurs when a different CHAP secret is entered in the Retype Secret field Enter the same CHAP secret in both the CHAP Secret and Retype Secret fields Password cannot be blank Occurs when the Password field is left blank Enter a valid Password in the password field Please type a different password Occurs when user types in same value for old...

Page 126: ...Boot image update is in progress Do not close the dialog Occurs when user clicks the X button in the Update Boot Image dialog box while the boot image update is in progress Information message The adapter will be disabled for executing the Memory test This will also disable all the ports and their devices on the adapter Are you sure you want to continue Occurs when the user selects the Memory test...

Page 127: ...st name or IP address is given in Host Name field in the Setup for Discovery dialog box Enter a valid host name or IP address in the Host Name field Failed to connect to the Agent Occurs when the agent connection is lost Start the HCM agent service and then proceed Unable to find the host Please provide another host Occurs When scoped link local ipv6 address such as fe80 216 cbff fea9 c3b5 en0 or ...

Page 128: ...r Try again later Occurs when user enables or disables the adapter in the following scenario 1 HCM discovers the adapter and places it into the product tree 2 Another user removes the adapter before the next discovery 3 User now tries to enable disable the adapter Refresh the product tree and check if the adapter is still available The WWN or MAC address already exists in the All WWNs MACs table E...

Page 129: ...paration for adding a name but the name was not added Do you want to add that name before you exit Occurs when user configures names in the following scenario 1 Modifies name of any adapter or port 2 Enters WWN and name values in the Detached WWN MAC and Name fields 3 Clicks OK Verify if the detached WWN name is to be added to HCM alias store If yes click Yes to add or No Are you sure you wish to ...

Page 130: ...essfully updated the parameters for all ports Occurs when the configurations done in the Basic Port Configuration dialog box are successfully updated using the Apply button The configuration will be reflected only after a successful process QoS and Target rate limiting cannot be enabled at the same time Occurs when QOS and target rate limiting are enabled at the same time in the Basic Port Configu...

Page 131: ... you want to proceed Occurs when the user closes the any Statistics dialog box Information message Syslog configuration failed Occurs when the user tries to configure a syslog server using the Syslog Server Configuration dialog box Retry the operation A maximum of three IP addresses are allowed Occurs when more than three IP addresses are entered in the Syslog Server Configuration dialog box Confi...

Page 132: ...r selects Delete when there is no team configured Information message The team name is duplicated Please enter a qualified Team Name Occurs when the user tries to add a team with existing team name Provide different name for the new team You did not select any virtual port Select one or more virtual ports Occurs when the user selects Apply on the Virtual Port Deletion dialog box without selecting ...

Page 133: ...n Runtime Reason An error occurs while creating or deleting a VLAN Determine the problem with creating or deleting a VLAN and then proceed There are VLANs associated with the port you have selected to be added as a member of the team If you add the port all associated VLANS will be removed Are you sure you want to add this port to the team and remove all associated VLANs Occurs when the user creat...

Page 134: ...ready installed version Components list displays with tabular format HCM configuration preference option 1 Install with existing configuration option 2 Install with default configuration Two buttons displays like Continue Exit Occurs when you try to overwrite currently installed software with a newer version Select whether you want to install software using the default configuration or the configu...

Page 135: ...en you install drivers on a Linux system that does not contain the GCC and kernel rpm and when the variable allow_unsupported_mod ules in etc modprobe d unsupp orted modules file is set to 0 Change the 0 in variable allow_unsupported_mod ules present in etc modprobe d unsupp orted modules file to 1 Load the GCC and Kernel rpms on the system and install the drivers System dependencies not available...

Page 136: ...dules file is set to 0 please change it to 1 and retry the installation Occurs when the variable allow_unsupported_mod ules in etc modprobe d unsupp orted modules file is set to 0 Change the 0 to 1 in variable allow_unsupported_mod ules located in etc modprobe d unsupp orted modules file then proceed with installation Disk space calculation Available space is Available space Please free up at leas...

Page 137: ...n the driver is successfully removed Restart the system to complete removal ReStart Message For Linux SUCCESSFUL uninstallation of driver may require System Occurs on Linux systems when the driver is successfully removed Restart the system to complete removal Warning message If booting over Brocade HBA you must have a version of driver installed before you restart the system Occurs on Linux system...

Page 138: ...118 Brocade Adapters Troubleshooting Guide 53 1001582 01 HCM and Installer Message Reference B ...

Page 139: ...BCU version mismatch 12 beaconing 75 enabling through BCU 76 enabling through HCM 76 BIOS boot problems adapter BIOS not installed displays 22 boot from SAN stops on HP hosts 21 cannot boot from CNA 21 CTL B option does not display when booting 20 driver and operating system install failure 19 LUN not visible from host 21 no target devices found or link down 20 target not visible from host 19 unab...

Page 140: ...e parameters for Windows 97 E echo test 82 enable through HCM 83 enabling through BCU 82 enable and disable VLANs in Device Manager 29 Etherent loopback test problems 24 Ethernet 57 Ethernet IOC statistics 66 Ethernet link ports or LOM not coming up 25 Ethernet loopback tests enabling through BCU 78 enabling through HCM 78 Ethernet network interface problems 24 Ethernet port logging levels 61 Ethe...

Page 141: ...ws 41 confirming driver package with HCM 41 driver errors 16 verifying 40 IOC statistics 69 displaying through BCU 69 displaying through HCM 69 L Linux network driver tuning 99 Linux problem errors when installing driver package 16 loss of adapter hardware address 25 loss of adapter IP address 25 low throughput and high latency 34 poor network performance 30 Linux storage driver tuning 95 LLDP dat...

Page 142: ... boot from SAN stops on HP hosts 21 cannot from CNA 21 CEE network 35 CEE not enabled 35 CTL B option does not display when booting host 20 driver event messages in host logs 12 errors when installing Linux driver 16 Ethernet link ports or LOM not coming up 25 Ethernet loopback test problems 24 fabric authentication failures 31 failed to connect to agent on host error 35 FCoE and Fibre Channel 31 ...

Page 143: ... Ethernet 65 Ethernet IOC 66 fabric 68 displaying through BCU 68 displaying through HCM 69 FCIP IM module displaying through BCU 70 displaying through HCM 70 FCIP initiator mode displaying through BCU 70 FCoE 67 FCP IM module 70 IOC 69 displaying through BCU 69 displaying through HCM 69 logical port 71 displaying through BCU 71 displaying through HCM 71 port 72 display through BCU 72 display throu...

Page 144: ...e to boot from device 20 unable to create NPIV ports 23 unable to install HCM 38 V verifying Fibre Channel and CEE links 39 verifying installation 40 virtual devices not listed in name server 32 virtual port properties 88 virtual port statistics 74 displaying through BCU 74 displaying through HCM 74 VLAN creation and operation problems 28 VLAN statistics 74 VLANs enable and disable in Device Manag...