Page 3 - Contents
iii Contents Regulatory Compliance Statements xi Declaration of Conformity xv Preface xvii How This Book Is Organized xvii Using UNIX Commands xviii Shell Prompts xviii Typographic Conventions xix Related Documentation xix Documentation, Support, and Training xx Third-Party Web Sites xx Contacting S...
Page 4 - Adapter Installation; ibhcatest
iv Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 LED Displays 3 Hardware, Software, and Cable Requirements 5 2. Adapter Installation 7 IB-HCA Bus and Slot Selection 7 ▼ Installing the Adapter 7 ▼ Verifying the Installation 9 3. Configuring IP Over Infi...
Page 5 - Other Useful Utilities; Index
Contents v Other Useful Utilities 26 cfgadm 26 snoop 27 netstat 27 kstat 27 Index 29
Page 7 - Figures
vii Figures FIGURE 1-1 Sun Dual Port 4x IB Host Channel Adapter PCI-X Card 2 FIGURE 1-2 LED Layout 4 FIGURE 2-1 Typical Adapter-to-Switch Configuration 9 FIGURE 4-1 InfiniBand Software Stack 16 FIGURE A-1 Sun 4x-to-12x InfiniBand Cable Pin Assignments 21
Page 9 - Tables
ix Tables TABLE 1-1 IB-HCA LEDs 3 TABLE 1-2 Hardware, Software, and Cable Requirements 5 TABLE B-1 SunVTS Documentation 24
Page 11 - Regulatory Compliance Statements; FCC Class A Notice
xi Regulatory Compliance Statements Your Sun product is marked to indicate its compliance class: • Federal Communications Commission (FCC) — USA • Industry Canada Equipment Standard for Digital Equipment (ICES-003) — Canada • Voluntary Control Council for Interference (VCCI) — Japan • Bureau of Stan...
Page 12 - BSMI Class A Notice
xii Sun Dual Por t 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 ICES-003 Class B Notice - Avis NMB-003, Classe B This Class B digital apparatus complies with Canadian ICES-003. Cet appareil numérique de la classe B est conforme à la norme NMB-003 du Canada. BSMI Cl...
Page 13 - CCC Class A Notice
Regulatory Compliance Statements xiii CCC Class A Notice The following statement is applicable to products shipped to China and marked with “Class A” on the product’s compliancelabel. GOST-R Certification Mark
Page 15 - Declaration of Conformity; EMC
xv Declaration of Conformity EMC USA—FCC Class A This equipment complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions:1. This equipment may not cause harmful interference.2. This equipment must accept any interference that may cause undesired operation. Europea...
Page 17 - Preface; Note –; How This Book Is Organized
xvii Preface The Sun™ Dual Port 4x IB Host Channel Adapter PCI-X Installation and User ’sGuide provides installation instructions for the Sun Dual Port 4x IB Host ChannelAdapter PCI-X card. This manual includes an overview of the adapter, supportedplatforms, cable information, and related documentat...
Page 18 - Using UNIX Commands; Shell Prompts
xviii Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 Using UNIX Commands This document might not contain information about basic UNIX® commands andprocedures such as shutting down the system, booting the system, and configuringdevices. Refer to the foll...
Page 19 - Typographic Conventions
Preface xix Typographic Conventions Related Documentation The following documents provide additional information regarding the IB-HCA andrelated products. Typeface 1 1 The settings on your browser might differ from these settings. Meaning Examples AaBbCc123 The names of commands, files,and directori...
Page 20 - Third-Party Web Sites
xx Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 Documentation, Support, and Training Third-Party Web Sites Sun is not responsible for the availability of third-party web sites mentioned in thisdocument. Sun does not endorse and is not responsible or l...
Page 21 - Sun Welcomes Your Comments
Preface xxi Sun Welcomes Your Comments Sun is interested in improving its documentation and welcomes your comments andsuggestions. You can submit your comments by going to: http://www.sun.com/hwdocs/feedback Please include the title and part number of your document with your feedback: Sun Dual Port ...
Page 23 - Product Overview; Product Features; Hardware Description
1 C H A P T E R 1 Product Overview This chapter provides an overview of the Sun Dual Port 4x IB Host Channel AdapterPCI-X card and includes the following sections: ■ “Product Features” on page 1 ■ “Hardware Description” on page 1 ■ “I/O Interfaces” on page 3 ■ “LED Displays” on page 3 ■ “Hardware, S...
Page 24 - Low profile faceplate
2 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 The board characteristics are: ■ Power: approx. 12 watts total ■ Low profile PCI form-factor card (approx. 2.5 in. by approx. 6.7 in.) ■ Requires 4x InfiniBand cables ■ 3.3V PCI-X or PCI slot (Note that t...
Page 25 - InfiniBand Interface; LED Displays
Chapter 1 Product Overview 3 I/O Interfaces The IB-HCA includes the following interfaces: ■ Two 4x InfiniBand copper connectors ■ PCI-X 64-bit 133 MHz edge connector InfiniBand Interface The IB-HCA provides two 4x InfiniBand v1.1 connector (or ports) for externalcopper cables and is IBTA specificati...
Page 29 - IB-HCA Bus and Slot Selection; Caution –; or your system service manual.
7 C H A P T E R 2 Adapter Installation This chapter describes how to install the adapter in your system and verify that ithas been installed correctly. IB-HCA Bus and Slot Selection For optimal performance, the IB-HCA card should be installed in the fastestavailable bus, preferably a 133 MHz PCI-X b...
Page 30 - connector on the chassis.; appropriate ports on the InfiniBand switch.; Turn power back on to the system and allow the server to reboot.
8 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 2. Remove the cover from the unit to access the card slots and connectors. 3. Select an available 3.3 Volt PCI-X or PCI slot and remove the blank front panel. If you are replacing an existing card, remove...
Page 31 - Verifying the Installation
Chapter 2 Adapter Installation 9 This completes the hardware installation. Proceed to the verification instructions in “Verifying the Installation” on page 9 . FIGURE 2-1 Typical Adapter-to-Switch Configuration ▼ Verifying the Installation Before you can verify the installation, you must install the...
Page 33 - Configuring IP Over InfiniBand; IPoIB Configuration; Configuring IPoIB
11 C H A P T E R 3 Configuring IP Over InfiniBand This chapter contains the following sections: ■ “IPoIB Configuration” on page 11 ■ “Managing IB Devices for IPoIB” on page 13 IPoIB Configuration IP Over InfiniBand (IPoIB) is automatically configured when the system is booted.The following procedure...
Page 34 - Create the InfiniBand devices using the following command:
12 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 For example: Refer to dmesg (1M) for more information. 2. Create the InfiniBand devices using the following command: A /dev/ibd n device is created for each IB-HCA port in the system where n is the port ...
Page 35 - Check that data can be send over the InfiniBand link.; Managing IB Devices for IPoIB
Chapter 3 Configuring IP Over InfiniBand 13 5. Verify that the devices are plumbed. To do this, type the command: The output includes information on the IB devices. 6. Check that data can be send over the InfiniBand link. Use the ping command to send ICMP (ICMP6) ECHO_REQUEST packets and receive ICM...
Page 37 - InfiniBand Software Overview; InfiniBand Software
15 C H A P T E R 4 InfiniBand Software Overview This chapter includes the following topics: ■ “InfiniBand Software” on page 15 ■ “Related Software Documentation” on page 18 InfiniBand Software The InfiniBand software is included in the Solaris 10 Operating System. FIGURE 4-1 shows a block diagram of...
Page 38 - User Direct Access Programming Library; IPoIB
16 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 FIGURE 4-1 InfiniBand Software Stack User Direct Access Programming Library This library supports service providers written according to the uDAPL 1.2specification. Sun provides default service provider ...
Page 39 - InfiniBand Transport Framework; InfiniBand Management Framework
Chapter 4 InfiniBand Software Overview 17 InfiniBand Transport Framework IBTF is the Solaris implementation of the InfiniBand Transport Layer (IBTL). IBTFconsists of InfiniBand Device Manager (IBDM), InfiniBand CommunicationsManager (IBCM) and an InfiniBand Nexus driver. The Nexus driver along withI...
Page 40 - Related Software Documentation; Manuals; Man Pages
18 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 Related Software Documentation Manuals ■ System Administration Guide: Devices and Files System (817-5093) For information on InfiniBand devices, see the System Administration Guide: Devices and Files Sys...
Page 41 - InfiniBand Cable Pin Assignments
19 A P P E N D I X A InfiniBand Cable Pin Assignments FIGURE A-1 shows the InfiniBand cable connectors and pin assignments.
Page 43 - SunVTS Diagnostic Testing
23 A P P E N D I X B Diagnostic Software andTroubleshooting This appendix provides an overview of the SunVTS™ diagnostic application andtroubleshooting tips for checking the adapter. There is also a section outlining somecommon troubleshooting issues. This appendix contains the following sections: ■...
Page 45 - Using the; Unplumb the interface from the system, using the; for instructions on; Troubleshooting Tasks
Appendix B Diagnostic Software and Troubleshooting 25 ▼ Using the ibhcatest 1. Ensure that the SunVTS software and the Tavor driver are installed on your system. To do this, type the following command: If a SunVTS software package is not installed, refer to the SunVTS User’s Guide for installation i...
Page 46 - Check the system log or console for Tavor driver error messages.; cfgadm
26 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 If an InfiniBand software package is not installed, check your Solaris OperatingSystem documentation for software package information. ■ Check the system log or console for Tavor driver error messages. S...
Page 47 - snoop; netstat; kstat
Appendix B Diagnostic Software and Troubleshooting 27 snoop The snoop program captures and inspects network packets. See the snoop (1M) man page for details. For example: netstat netstat shows network status. See the netstat (1M) man page for details. For example: kstat kstat displays kernel statist...
