Page 3 - About This Document; Table of Contents
Table of Contents About This Document ....................................................................................................... 11 Intended Audience................................................................................................................................11New and ...
Page 4 - Preparing Individual Nodes
3.3.3 Switch Port Connections.........................................................................................................47 3.3.3.1 Switch Connections and HP Workstations.....................................................................49 3.3.4 Super Root Switch.........................
Page 6 - List of Figures
List of Figures 1-1 HP BladeSystem c7000 enclosure (Front and Rear Views)...........................................................22 1-2 HP BladeSystem c7000 Enclosure Bay Locations (Front View)....................................................22 1-3 HP BladeSystem c7000 Enclosure Bay Numbering ...
Page 8 - List of Tables
List of Tables 1-1 Supported Processor Architectures and Hardware Models.........................................................20 1-2 Supported HP ProLiant Server Blade Models...............................................................................21 1-3 Supported Console Management Devices....
Page 11 - “Supplementary Software Products” (page 14); Command; Intended Audience
About This Document This document describes how to prepare the nodes in your HP cluster platform before installingHP XC System Software. An HP XC system is integrated with several open source software components. Some open sourcesoftware components are being used for underlying technology, and their...
Page 12 - Ctrl; ENVIRONMENT VARIABLE; Key; Variable; HP XC and Related HP Products Information; HP XC System Software Release Notes
Ctrl+x A key sequence. A sequence such as Ctrl+x indicates that you must hold down the key labeled Ctrl while you press another key or mouse button. ENVIRONMENT VARIABLE The name of an environment variable, for example, PATH . [ERROR NAME] The name of an error, usually returned in the errno variable...
Page 13 - and other
Provides an overview of managing the HP XC user environmentwith modules, managing jobs with LSF, and describes how tobuild, run, debug, and troubleshoot serial and parallelapplications on an HP XC system. HP XC System Software User's Guide Provides a product overview, hardware requirements, software...
Page 14 - Related Information; Supplementary Software Products
• HP Server Blade c7000 Enclosure • HP BladeSystem c3000 Enclosure Related Information This section provides useful links to third-party, open source, and other related software products. Supplementary Software Products This section provides links to third-party and open source software products tha...
Page 15 - Linux Web Addresses; bash
• http://supermon.sourceforge.net/ Home page for Supermon, a high-speed cluster monitoring system that emphasizes lowperturbation, high sampling rates, and an extensible data protocol and programminginterface. Supermon works in conjunction with Nagios to provide HP XC system monitoring. • http://www...
Page 16 - MySQL, by Paul Debois
• http://www.linuxheadquarters.com Web address providing documents and tutorials for the Linux user. Documents containinstructions for installing and using applications for Linux, configuring hardware, and avariety of other topics. • http://www.gnu.org Home page for the GNU Project. This site provid...
Page 17 - MySQL Cookbook, by Paul Debois; If you are not sure about a command you need to use, enter the; HP Encourages Your Comments; Manpages
• MySQL Cookbook, by Paul Debois • High Performance MySQL, by Jeremy Zawodny and Derek J. Balling (O'Reilly) • Perl Cookbook, Second Edition, by Tom Christiansen and Nathan Torkington • Perl in A Nutshell: A Desktop Quick Reference , by Ellen Siever, et al. Manpages Manpages provide online reference...
Page 19 - Hardware and Network Overview; This chapter addresses the following topics:; An HP XC system is made up of interconnected servers.; Supported Processor Architectures and Hardware Models; Supported Cluster Platforms
1 Hardware and Network Overview This chapter addresses the following topics: • “Supported Cluster Platforms” (page 19) • “Server Blade Enclosure Components” (page 22) • “Server Blade Mezzanine Cards” (page 28) • “Server Blade Interconnect Modules” (page 28) • “Supported Console Management Devices” (...
Page 20 - Hardware Model
IMPORTANT: A hardware configuration can contain a mixture of Opteron and Xeon nodes, but not Itanium nodes. Table 1-1 Supported Processor Architectures and Hardware Models Hardware Model Processor Architecture Cluster Platform Server Type • HP ProLiant BL2x220c G5 • HP ProLiant BL260c G5 • HP ProLia...
Page 21 - Supported Server Blade Combinations
HP server blades offer an entirely modular computing system with separate computing andphysical I/O modules that are connected and shared through a common chassis, called anenclosure; for more information on enclosures, see “Server Blade Enclosure Components” (page 22) . Full-height Opteron server b...
Page 22 - Server Blade Enclosure Components; This section discusses the following topics:; Hardware and Network Overview
1.2 Server Blade Enclosure Components HP server blades are contained in an enclosure, which is a chassis that houses and connects bladehardware components. An enclosure is managed by an Onboard Administrator. The HPBladeSystem c7000 and c3000 enclosures are supported under HP XC. This section discus...
Page 24 - Figure 1-4 HP BladeSystem c7000 Enclosure Bay Locations (Rear View); General Configuration Guidelines
Figure 1-4 HP BladeSystem c7000 Enclosure Bay Locations (Rear View) 8 Interconnect Bay #7 1 Fan Bays 2 Interconnect Bay #1 9 Interconnect Bay #8 10 Onboard Administrator Bay 1 3 Interconnect Bay #2 4 Interconnect Bay #3 11 Onboard Administrator Bay 2 12 Power Supply Exhaust Vent 5 Interconnect Bay #...
Page 25 - Specific HP XC Setup Guidelines; The following enclosure setup guidelines are specific to HP XC:
Specific HP XC Setup Guidelines The following enclosure setup guidelines are specific to HP XC: • On every HP BladeSystem c7000 enclosure, an Ethernet interconnect module (either a switchor pass-through module) is installed in interconnect bay #1 (see callout 2 in Figure 1-4 ) for the administration...
Page 26 - Number of Fans
Figure 1-7 HP BladeSystem c3000 Enclosure Bay Locations (Front View) 1 Device bays 2 DVD drive (optional) 3 Insight Display. For more information, see “Insight Display” (page 28) . 4 Onboard Administrator (OA) The HP BladeSystem c3000 enclosure can house a maximum of 8 half-height or 4 full-heightse...
Page 27 - Figure 1-9 HP BladeSystem c3000 Enclosure Bay Locations (Rear View); “Cabling for the External Network” (page 41)
Figure 1-9 HP BladeSystem c3000 Enclosure Bay Locations (Rear View) 1 Interconnect bay #1 2 Fans 3 Interconnect bay #2 4 Enclosure/Onboard Administrator Link Module 5 Power Supplies 6 Interconnect bay #3 7 Interconnect bay #4 1 Enclosure Downlink port 2 Enclosure Uplink port 3 Onboard Administrator/...
Page 28 - Server Blade Mezzanine Cards; Interconnect Bay Port Mapping
You can access the Onboard Administrator through a graphical Web-based user interface, acommand-line interface, or the simple object access protocol (SOAP) to configure and monitorthe enclosure. You can add a second Onboard Administrator to provide redundancy. The Onboard Administrator requires a pa...
Page 29 - Supported Console Management Devices; HP workstation models do not have console ports.; Firmware Dependency
See the HP BladeSystem Onboard Administrator User Guide for illustrations of interconnect bayport mapping connections on half- and full-height server blades. 1.5 Supported Console Management Devices Table 1-3 lists the supported console management device for each hardware model within each cluster p...
Page 30 - Table 1-3 Supported Console Management Devices; Administration Network Overview
Table 1-3 Supported Console Management Devices (continued) Firmware Dependency Hardware Component iLO2, system BIOS, Onboard Administrator (OA) HP ProLiant BL460c iLO2, system BIOS, OA HP ProLiant BL480c iLO2, system BIOS, Onboard Administrator (OA) HP ProLiant BL680c G5 CP4000 LO-100i HP ProLiant D...
Page 31 - shows a graphical representation of the console branch.; Interconnect Network; QsNet; constitutes another interconnect family, and so on. For more; Administration Network: Console Branch
HP XC system, the administrative tools probe and discover the topology of the administrationnetwork. The administration network requires and uses Gigabit Ethernet. The administration network has at least one Root Administration Switch and can have multipleBranch Administration Switches. These switch...
Page 32 - Table 1-4 Supported Interconnects; Mixing Adapters; The QsNet; interconnect that is supported.
Table 1-4 Supported Interconnects INTERCONNECT FAMILIES QsNet II Myrinet® (Rev. D, E, and F) InfiniBand ConnectX Double Data Rate (DDR) 1 InfiniBand PCI Express SingleData Rate and Double Data Rate (DDR) InfiniBand® PCI-X Gigabit Ethernet ClusterPlatform andHardwareModel X X X X X CP3000 X X X CP300...
Page 33 - Cabling Server Blades; The following topics are addressed in this chapter:; to 16 blade servers; Blade Enclosure Overview
2 Cabling Server Blades The following topics are addressed in this chapter: • “Blade Enclosure Overview” (page 33) • “Network Overview ” (page 33) • “Cabling for the Administration Network” (page 37) • “Cabling for the Console Network” (page 38) • “Cabling for the Interconnect Network” (page 39) • “...
Page 34 - Small HP XC Cluster of Server Blades; a maximum of 64 nodes.; Cabling Server Blades
Chapter 3 (page 45) describes specific node and switch connections for non-blade hardware configurations. A hardware configuration with server blades does not have these specific cabling requirements;specific switch port assignments are not required. However, HP recommends a logical orderingof the c...
Page 35 - Medium Sized HP XC Cluster of Server Blades; and a maximum of 512 nodes.
Figure 2-1 Interconnection Diagram for a Small HP XC Cluster of Server Blades Enclosure 4 May be any managed GigE switch supported by CP ProCurve managed GigE Switch ProCurve managed GigE Switch Enclosure 1 Switch 1 Enclosure 1 Server Blades NIC1 Enclosure 2 Server Blades NIC1 Enclosure 3 Server Bla...
Page 36 - Large HP XC Cluster of Server Blades
Figure 2-2 Interconnection Diagram for a Medium Sized HP XC Cluster of Server Blades Enclosure 4 Smaller systems may use ProCurve 2626 ProCurve 2650 ProCurve 2848 Enclosure 1 Switch 1 Enclosure 1 Server Blades NIC1 Enclosure 2 Server Blades NIC1 Enclosure 3 Server Blades NIC1 Primary OA Ext Link Sec...
Page 37 - Chapter 3; Cabling for the Administration Network
Figure 2-3 Interconnection Diagram for a Large HP XC Cluster of Server Blades Enclosure 1 GigE Switch GigE Switch Region 1 Region 8 /4 /4 ProCurve 2650 Primary OA Ext Link Secondary OA Ext Link GigE Switch Enclosure 2 Primary OA Ext Link Secondary OA Ext Link GigE Switch Enclosure 3 Primary OA Ext L...
Page 38 - Figure 2-4 Administration Network Connections; Cabling for the Console Network; illustrates the connections that form the console network.
Figure 2-4 Administration Network Connections NIC 1NIC 2NIC 3NIC 4 MEZZ 1MEZZ 2MEZZ 3 iLO2 NIC 1NIC 2 MEZZ 1MEZZ 2 iLO2 Interconnect bay 1 Interconnect bay 2 Interconnect bay 3 Interconnect bay 4 Interconnect bay 5 Interconnect bay 6 Interconnect bay 7 Interconnect bay 8 ONBOARD ADMINISTRATOR MGT NI...
Page 39 - Figure 2-5 Console Network Connections; “Configuring a Gigabit Ethernet Interconnect” (page 39); Configuring a Gigabit Ethernet Interconnect; Cabling for the Interconnect Network
Figure 2-5 Console Network Connections NIC 1NIC 2NIC 3NIC 4 MEZZ 1MEZZ 2MEZZ 3 iLO2 NIC 1NIC 2 MEZZ 1MEZZ 2 iLO2 Interconnect bay 1 Interconnect bay 2 Interconnect bay 3 Interconnect bay 4 Interconnect bay 5 Interconnect bay 6 Interconnect bay 7 Interconnect bay 8 ONBOARD ADMINISTRATOR MGT NIC PCI S...
Page 40 - Figure 2-6 Gigabit Ethernet Interconnect Connections; Configuring an InfiniBand Interconnect; illustrates the connections for an InfiniBand interconnect.
Figure 2-6 Gigabit Ethernet Interconnect Connections NIC 1NIC 2NIC 3NIC 4 MEZZ 1MEZZ 2MEZZ 3 iLO2 NIC 1NIC 2 MEZZ 1MEZZ 2 iLO2 Interconnect bay 1 Interconnect bay 2 Interconnect bay 3 Interconnect bay 4 Interconnect bay 5 Interconnect bay 6 Interconnect bay 7 Interconnect bay 8 ONBOARD ADMINISTRATOR...
Page 41 - For information about configuring VLANs, see; Cabling for the External Network
2.5.3 Configuring the Interconnect Network Over the Administration Network In cases where an additional Gigabit Ethernet port or switch may not be available, the HP XCSystem Software enables you to configure the interconnect on the administration network. Whenthe interconnect is configured on the ad...
Page 42 - for information on which
server blades do not have three NICs, and therefore, half-height server blades are not includedin this example Because NIC1 and NIC3 on a full-height server blade are connected to interconnect bay 1, youmust use VLANs on the switch in that bay to separate the external network from theadministration ...
Page 44 - Enter
Figure 2-10 External Network Connections: Half and Full-Height Server Blades and NIC1 in Use NIC 1NIC 2NIC 3NIC 4 MEZZ 1MEZZ 2MEZZ 3 iLO2 NIC 1NIC 2 MEZZ 1MEZZ 2 iLO2 Interconnect bay 1 Interconnect bay 2 Interconnect bay 3 Interconnect bay 4 Interconnect bay 5 Interconnect bay 6 Interconnect bay 7 ...
Page 45 - Making Node and Switch Connections; The following topics are addressed:; Utility Cabinet; Trunking and Switch Choices
3 Making Node and Switch Connections This chapter provides information about the connections between nodes and switches that arerequired for an HP XC system. The following topics are addressed: • “Cabinets” (page 45) • “Trunking and Switch Choices” (page 45) • “Switches” (page 46) • “Interconnect Co...
Page 46 - Making Node and Switch Connections
to create a higher bandwidth connection between the Root Administration Switches and theBranch Administration Switches. For physically small hardware models (such as a 1U HP ProLiant DL145 server), a large numberof servers (more than 30) can be placed in a single cabinet, and are all attached to a s...
Page 47 - Table 3-1 Supported Switch Models; ProCurve Switch Model; Administrator Passwords on ProCurve Switches
Root Administration Switch This switch connects directly to Gigabit Ethernet portsof the head node, the Root Console Switch, BranchAdministration Switches, and other nodes in the utilitycabinet. Root Console Switch This switch connects to the Root Administration Switch,Branch Console Switches, and c...
Page 48 - Region 1
Figure 3-2 Node and Switch Connections on a Typical System Compute Nodes Br anch Switches Administration Root Console Root HeadNode Specialized Role Nodes ConsoleSwitches Administration Switches Br anch Switches Figure 3-3 shows a graphical representation of the logical layout of the switches and no...
Page 49 - Ports Used on Super Root Switch
3.3.3.1 Switch Connections and HP Workstations HP model xw workstations do not have console ports. Only the Root Administration Switchsupports mixing nodes without console management ports with nodes that have consolemanagement ports (that is, all other supported server models). HP workstations conn...
Page 50 - Figure 3-5 ProCurve 2848 Root Administration Switch; The callouts in the figure enumerate the following:
3.3.5 Root Administration Switch The Root Administration Switch for the administration network of an HP XC system can beeither a ProCurve 2848 switch or a ProCurve 2824 switch for small configurations. If you are using a ProCurve 2848 switch as the switch at the center of the administration network,...
Page 51 - Figure 3-6 ProCurve 2824 Root Administration Switch
Figure 3-6 ProCurve 2824 Root Administration Switch Po w er Fa u lt h p procurves witch 2824J 4903 A Ln kA c t FD xSp d St atu s R P S Fa n Te s t LE D Mode C ons o le R eset C lear 1 2 3 4 5 6 7 8 17 18 19 20 9 10 11 12 13 14 15 16 21 22 23 24 T M T M T M T M 1 3 4 6 2 5 The callouts in the figure ...
Page 55 - Figure 3-11 ProCurve 2848 Branch Administration Switch; Figure 3-12 ProCurve 2824 Branch Administration Switch
Figure 3-11 ProCurve 2848 Branch Administration Switch LED Mode Cl e a r Re s e t 4 5 43 44 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 Spd m ode : of f = 1 0 Mbps fl a s h = 1 0 0 Mbps o n = 1 0 0 0 Mbps 1 15 17 16 18 31 32 33 34 Po w e...
Page 56 - Figure 3-13 ProCurve 2650 Branch Console Switch; The callout in these figures enumerates the following:; Interconnect Connections; ) affects the naming of the nodes in the system. The documentation
Figure 3-13 ProCurve 2650 Branch Console Switch Po r t LED Vi e w Sel f Te s t Cl e a r Re s e t Fa n St atu s 4 5 48 47 46 45 43 44 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 Spd m ode : of f = 1 0 Mbp s , fl a s h = 1 0 0 Mbps , o n =...
Page 57 - For the QsNet; is the maximum number of nodes possible; Gigabit Ethernet Interconnect Connections; option on the
The method for wiring the administration network and interconnect networks allows expansionof the system within the system's initial interconnect fabric without recabling of any existingnodes. If additional switch chassis or ports are added to the system as part of the expansion,some recabling may b...
Page 58 - InfiniBand Interconnect Connections; or
If you do not specify the --ic=AdminNet option, the discover command attempts to locate the highest speed interconnect on the system with the default being a Gigabit Ethernet networkthat is separate from the administration network. 3.4.4 Myrinet Interconnect Connections The supported Myrinet interco...
Page 59 - Firmware Requirements and Dependencies; Table 4-1 Firmware Dependencies
4 Preparing Individual Nodes This chapter describes how to prepare individual nodes in the HP XC hardware configuration.The following topics are addressed: • “Firmware Requirements and Dependencies” (page 59) • “Ethernet Port Connections on the Head Node” (page 61) • “General Hardware Preparations f...
Page 60 - Switches
Table 4-1 Firmware Dependencies (continued) Firmware Dependency Hardware Component iLO2, system BIOS, Onboard Administrator (OA) HP ProLiant BL680c G5 CP4000 LO-100i, system BIOS HP ProLiant DL145 LO-100i, system BIOS HP ProLiant DL145 G2 LO-100i, system BIOS HP ProLiant DL145 G3 LO-100i, system BIO...
Page 61 - Ethernet Port Connections on the Head Node; Table 4-2 Ethernet Ports on the Head Node; All Other Interconnect Types; General Hardware Preparations for All Cluster Platforms; “Making Node and Switch
Table 4-1 Firmware Dependencies (continued) Firmware Dependency Hardware Component Firmware version Myrinet Interface card version Myrinet interface card Firmware version QsNet II Firmware version InfiniBand 4.2 Ethernet Port Connections on the Head Node Table 4-2 lists the Ethernet port connections...
Page 62 - Setting the Onboard Administrator Password; Administrator; Preparing Individual Nodes
that it is accessible to Nagios during system operation. For more information on Nagios, seethe HP XC System Software Administration Guide. 6. Review the documentation that came with the hardware and have it available, if needed. 7. If your hardware configuration contains server blades and enclosure...
Page 63 - Setup Procedure
4.5 Preparing the Hardware for CP3000 (Intel Xeon with EM64T) Systems Follow the procedures in this section to prepare each node before installing and configuring theHP XC System Software. Proceed to the following sections, depending on the hardware model: • “Preparing HP ProLiant DL140 G2 and G3 No...
Page 64 - Main; Table 4-3 BIOS Settings for HP ProLiant DL140 G2 Nodes; Set to This Value
2. Turn on power to the node. Watch the screen carefully during the power-on, self-test, andpress the F10 key when prompted to access the BIOS Setup Utility. The Lights-Out 100i (LO-100i) console management device is configured through the BIOS Setup Utility. The BIOS Setup Utility displays the foll...
Page 66 - Table 4-4 BIOS Settings for HP ProLiant DL140 G3 Nodes; Power; From the; Use the following menus to disable this option:; Parallel ATA; discover
Table 4-4 BIOS Settings for HP ProLiant DL140 G3 Nodes (continued) Set to This Value Option Name Submenu Name Menu Name Set the following boot order on all nodesexcept the head node: 1. CD-ROM 2. Removable Devices 3. Embedded NIC1 4. Hard Drive 5. Embedded NIC2 Enabled Embedded NIC1PXE Disabled Embe...
Page 67 - Record this information for future reference.; The BIOS settings for HP ProLiant DL160 G5 nodes are listed in
The callouts in the figure enumerate the following: 1. This port is used for the connection to the Console Switch. On the back of the node, this portis marked with LO100i . 2. This port is used for the connection to the Administration Switch (branch or root). On theback of the node, this port is mar...
Page 68 - Table 4-5 BIOS Settings for HP ProLiant DL160 G5 Nodes; Embedded NIC1; telnet; File
Table 4-5 BIOS Settings for HP ProLiant DL160 G5 Nodes (continued) Set to This Value Option Name Submenu Name Menu Name If this node is the head node, set thisvalue to: Embedded NIC1 Otherwise, set this value to: Hard Drive 2nd Boot Device Enabled Embedded NIC1PXE Disabled Embedded NIC2PXE 4. From t...
Page 69 - Table 4-6 iLO Settings for HP ProLiant DL360 G4 Nodes; On; Table 4-7 BIOS Settings for HP ProLiant DL360 G4 Nodes; Hard Disk; Esc
5. Watch the screen carefully, and press the F9 key when prompted to access the ROM-Based Setup Utility (RBSU). Table 4-6 iLO Settings for HP ProLiant DL360 G4 Nodes Set to This Value Option Name Submenu Name Menu Name Create a common iLO user name and password forevery node in the hardware configur...
Page 70 - Configuring Smart Arrays
Configuring Smart Arrays On the HP ProLiant DL360 G4 with smart array cards, you must add the disks to the smart arraybefore attempting to image the node. To do so, watch the screen carefully during the power-on self-test phase of the node, and pressthe F8 key when prompted to configure the disks in...
Page 71 - Table 4-8 iLO Settings for HP ProLiant DL360 G5 Nodes; The user; Make the following settings from the; lists the BIOS settings for HP; Disabled; Press the; key to exit the RBSU. Press the
Table 4-8 iLO Settings for HP ProLiant DL360 G5 Nodes Set to This Value Option Name Submenu Name Menu Name Create a common iLO user name and password forevery node in the hardware configuration. Thepassword must have a minimum of 8 characters bydefault, but this value is configurable. The user Admin...
Page 73 - Figure 4-6 HP ProLiant DL380 G5 Server Rear View; Table 4-10 iLO Settings for HP ProLiant DL380 G4 and G5 Nodes; key to save the
Figure 4-6 HP ProLiant DL380 G5 Server Rear View 1 3 4 5 3 1 2 2 The callouts in the figure enumerate the following: 1. This port is used for the connection to the external network. 2. This port is used for the connection to the Administration Switch (branch or root). 3. The iLO Ethernet port is use...
Page 74 - Table 4-11 BIOS Settings for HP ProLiant DL380 G4 Nodes; Table 4-12 BIOS Settings for HP ProLiant DL380 G5 Nodes
1. Make the following settings from the Main menu. The BIOS settings differ depending upon the hardware model generation: • BIOS settings for HP ProLiant DL380 G4 nodes are listed in Table 4-11 . • BIOS settings for HP ProLiant DL380 G5 nodes are listed in Table 4-12 . Table 4-11 BIOS Settings for H...
Page 75 - Disable
Table 4-12 BIOS Settings for HP ProLiant DL380 G5 Nodes (continued) Set to This Value Option Name Menu Name Disable Processor Hyper_threading Advanced Options COM1; IRQ4; IO: 3F8h - 3FFh BIOS Serial Console Port BIOS Serial Console &EMS 115200 BIOS Serial Console Baud Rate Disabled EMS Console C...
Page 76 - Make the iLO settings listed in
Figure 4-7 HP ProLiant DL580 G4 Server Rear View 1 2 3 The callouts in the figure enumerate the following: 1. NIC1 is used for the connection to the Administration Switch (branch or root). 2. NIC2 is used for the connection to the external network. 3. The iLO Ethernet port is used for the connection...
Page 77 - Table 4-13 iLO Settings for HP ProLiant DL580 G4 Nodes; Select; Watch the screen carefully, and press the
Table 4-13 iLO Settings for HP ProLiant DL580 G4 Nodes Set to This Value Option Name SubmenuName Menu Name Name Create a common iLO user name andpassword for every node in the hardwareconfiguration. The password must have aminimum of 8 characters by default, butthis value is configurable. The user A...
Page 78 - Table 4-14 BIOS Settings for HP ProLiant DL580 G4 Nodes
1. Make the following settings from the Main menu. The BIOS settings for HP ProLiant DL580 G4 nodes are listed in Table 4-14 . Table 4-14 BIOS Settings for HP ProLiant DL580 G4 Nodes Set to This Value Option Name Menu Name Set the following boot order on all nodesexcept the head node; CD-ROM does no...
Page 80 - Table 4-16 BIOS Settings for HP ProLiant DL580 G5 Nodes
5. Watch the screen carefully, and press the F9 key when prompted to access the ROM-Based Setup Utility (RBSU). Perform the following procedure from the RBSU for each HP ProLiant DL580 G5 node in the HPXC system: 1. Make the following settings from the Main menu. The BIOS settings for HP ProLiant DL...
Page 81 - Setup Utility; Network Controller
Figure 4-9 HP xw8200 and xw8400 Workstation Rear View 1 The callout in the figure enumerates the following: 1. This port is used for the connection to the administration network. Setup Procedure Use the Setup Utility to configure the appropriate settings for an HP XC system. Perform the following pr...
Page 82 - Table 4-18 BIOS Settings for xw8400 Workstations; SATA Emulation; to exit the Setup Utility.
Table 4-18 BIOS Settings for xw8400 Workstations Set to This Value Option Name Submenu Name Menu Name Separate IDE Controller After you make this setting, make sure the Primary SATA Controller and Secondary SATA Controller settings are set to Enabled . SATA Emulation Storage Options Storage Set the ...
Page 84 - Preparing the Hardware for CP3000BL Systems; “Setting the Onboard; Table 4-20 Boot Order for HP ProLiant Server Blades; All Other Nodes
4.6 Preparing the Hardware for CP3000BL Systems Perform the following tasks on each server blade in the hardware configuration after the headnode is installed and the switches are discovered: • Set the boot order • Create an iLO2 user name and password • Set the power regulator • Configure smart arr...
Page 85 - OA; Internet Explorer
f. The Onboard Administrator automatically creates user accounts for itself (prefixed withthe letters OA ) to provide single sign-on capabilities. Do not remove these accounts. 5. Enable telnet access: a. In the left frame, click Access . b. Click the control to enable Telnet Access . c. Click the A...
Page 86 - for each server blade in each enclosure.
10. Close the iLO2 utility Web page.11. Repeat this procedure from every active Onboard Administrator and make the same settings for each server blade in each enclosure. After preparing all the nodes in all the enclosures, return to the HP XC System Software InstallationGuide to discover all the nod...
Page 87 - Figure 4-11 HP ProLiant DL145 Server Rear View
4.7 Preparing the Hardware for CP4000 (AMD Opteron) Systems Follow the procedures in this section to prepare each node before installing and configuring theHP XC System Software. See the following sections depending on the hardware model: • “Preparing HP ProLiant DL145 Nodes” (page 87) • “Preparing ...
Page 88 - Table 4-21 BIOS Settings for HP ProLiant DL145 Nodes; Enabled; Boot Screen; admin
3. For each node, make the BIOS settings listed in Table 4-21 . Table 4-21 BIOS Settings for HP ProLiant DL145 Nodes Set to This Value Option Name Submenu Name Menu Name Enabled (for all nodes except the head node) Onboard NIC PXE OptionROM Boot SettingsConfiguration (for NIC1) Boot Disabled (for th...
Page 89 - dhcp
6. Ensure that all machines are requesting IP addresses through the Dynamic Host ControlProtocol (DHCP). Do the following to determine if DHCP is enabled:a. At the ProLiant> prompt, enter the following: ProLiant> net b. At the INET> prompt, enter the following: INET> state iface...ipsrc....
Page 90 - Figure 4-13 HP ProLiant DL145 G3 Server Rear View; Table 4-22 BIOS Settings for HP ProLiant DL145 G2 Nodes; Off
1. This port is used for the connection to the Administration Switch (branch or root). On therear of the node, this port is marked with the number 1 ( NIC1 ). 2. If a Gigabit Ethernet (GigE) interconnect is configured, this port is used for the interconnectconnection. Otherwise, it is used for an ex...
Page 91 - provides the BIOS settings for ProLiant DL145 G3 nodes
Table 4-22 BIOS Settings for HP ProLiant DL145 G2 Nodes (continued) Set to This Value Option Name Submenu Name Menu Name Enabled Disable Jitter bit Hammer Configuration Disabled page Directory Cache Enabled Device PCI Configuration/EthernetOn Board (for Ethernet 1 and 2) Enabled Option ROM Scan 40h ...
Page 92 - Table 4-23 BIOS Settings for HP ProLiant DL145 G3 Nodes
Table 4-23 BIOS Settings for HP ProLiant DL145 G3 Nodes Set to This Value Option Name Submenu Name Menu Name Off NumLock Boot Options Main BMC Serial Port Mode I/O Device Configuration Advanced Enabled Serial port A: 3F8 Base I/O address: IRQ 4 Interrupt: AUTO DRAM Bank Interleave Memory Controller ...
Page 94 - Table 4-24 BIOS Settings for HP ProLiant DL165 G5 Nodes; Exit; to exit the BIOS Setup Utility.
Table 4-24 BIOS Settings for HP ProLiant DL165 G5 Nodes Set to This Value Option Name Submenu Name Menu Name Disabled Bootup Num-Lock Boot Settings Configuration Main 3F8 Embedded Serial Port IRQ: I/O Device Configuration Advanced IRQ 4 Interrupt: S-ATA S-ATA Mode S-ATA Configuration Enabled INT13 s...
Page 95 - Table 4-25 iLO Settings for HP ProLiant DL365 Nodes; Add
Figure 4-15 HP ProLiant DL365 Server Rear View The callouts in the figure enumerate the following: 1. This port is the Ethernet connection to the Console Switch. On the back of the node, thisport is marked with the acronym iLO . 2. This port is the connection to the Administration Switch (branch or ...
Page 96 - Table 4-26 RBSU Settings for HP ProLiant DL365 Nodes; On the head node only, set this value to; key when prompted to configure the disks into the smart array.
1. Make the RBSU settings for the HP ProLiant DL365 nodes, as indicated in Table 4-26 Use the navigation aids shown at the bottom of the screen to move through the menus andmake selections. Table 4-26 RBSU Settings for HP ProLiant DL365 Nodes Set to This Value Option Name Menu Name On all nodes exce...
Page 98 - Table 4-27 iLO Settings for HP ProLiant DL365 G5 Nodes
Table 4-27 iLO Settings for HP ProLiant DL365 G5 Nodes Set to This Value Option Name Submenu Name Menu Name Create a common iLO user name and password for everynode in the hardware configuration. The password musthave a minimum of 8 characters by default, but this valueis configurable. The user Admi...
Page 99 - Table 4-28 RBSU Settings for HP ProLiant DL365 G5 Nodes
1. Make the RBSU settings for the HP ProLiant DL365 G5 nodes, as indicated in Table 4-28 Use the navigation aids shown at the bottom of the screen to move through the menus andmake selections. Table 4-28 RBSU Settings for HP ProLiant DL365 G5 Nodes Set to This Value Option Name Menu Name On all node...
Page 101 - provides the iLO settings for ProLiant DL385 nodes.; lists the iLO settings for ProLiant DL385 G2 nodes.
Figure 4-18 HP ProLiant DL385 G2 Server Rear View 1 3 4 5 3 1 2 2 The callouts in the figure enumerate the following: 1. If a Gigabit Ethernet (GigE) interconnect is configured, this port is used for the interconnectconnection. Otherwise, it is used for an external connection. On the back of the nod...
Page 102 - Table 4-30 iLO Settings for HP ProLiant DL385 G2 Nodes
Table 4-30 iLO Settings for HP ProLiant DL385 G2 Nodes Set to This Value Option Name Submenu Name Menu Name Create a common iLO user name and password for everynode in the hardware configuration. The password musthave a minimum of 8 characters by default, but this valueis configurable. The user Admi...
Page 103 - lists the RBSU settings for the HP ProLiant DL385 G2 nodes.
1. Make the RBSU settings accordingly. The settings differ depending on the hardware modelgeneration: • Table 4-31 provides the RBSU settings for the HP ProLiant DL385 nodes. • Table 4-32 provides the RBSU settings for the HP ProLiant DL385 G2 nodes. Use the navigation aids shown at the bottom of th...
Page 104 - Table 4-32 RBSU Settings for HP ProLiant DL385 G2 Nodes
Table 4-32 RBSU Settings for HP ProLiant DL385 G2 Nodes (continued) Set to This Value Option Name Menu Name COM2 Embedded Serial Port System Options COM1 Virtual Serial Port Set the following boot order on all nodes exceptthe head node; CD-ROM does not have to be firstin the list, but it must be lis...
Page 105 - Make the following iLO settings for each DL385 G5 node, as shown in
Figure 4-19 HP ProLiant DL385 G5 Server Rear View The callouts in the figure enumerate the following: 1. This port is the Ethernet connection to the Console Switch. On the back of the node, thisport is marked with the acronym iLO . 2. This port is the connection to the Administration Switch (branch ...
Page 106 - Figure 4-21 HP ProLiant DL585 G2 Server Rear View
4.7.8 Preparing HP ProLiant DL585 and DL585 G2 Nodes On HP ProLiant DL585 and DL585 G2 servers, use the following tools to configure the appropriatesettings for an HP XC system: • Integrated Lights Out (iLO) Setup Utility • ROM-Based Setup Utility (RBSU) HP ProLiant DL585 and DL585 G2 servers use th...
Page 107 - Table 4-34 iLO Settings for HP ProLiant DL585 Nodes; Table 4-35 iLO Settings for HP ProLiant DL585 G2 Nodes
1. NIC1 is the connection to the Administration Switch (branch or root). 2. If a Gigabit Ethernet (GigE) interconnect is configured, this port (labeled NIC2 ) is used for the interconnect connection. Otherwise, it is used for an external connection. 3. The iLO2 Ethernet port is the connection to the...
Page 109 - (all nodes except the head; lists the RBSU setting for the DL585 G2 nodes.
1. Make the RBSU settings accordingly for each node. The settings differ depending on thehardware model generation: • Table 4-36 provides the RBSU settings for the HP ProLiant DL585 nodes. • Table 4-37 provides the RBSU settings for the HP ProLiant DL585 G2 nodes. Table 4-36 RBSU Settings for HP Pro...
Page 110 - Table 4-37 RBSU Settings for HP ProLiant DL585 G2 Nodes; PCI Embedded HP NC373i
Table 4-37 RBSU Settings for HP ProLiant DL585 G2 Nodes (continued) Set to This Value Option Name Menu Name Set the following boot order on all nodes exceptthe head node; the CD-ROM must be listed beforethe hard drive: • IPL:1 CD-ROM • IPL:2 Floppy Drive (A:) • IPL:3 USB Drive Key (C:) • IPL:4 PCI E...
Page 111 - Figure 4-22 HP ProLiant DL585 G5 Server Rear View; Table 4-38 iLO Settings for HP ProLiant DL585 G5 Nodes
Figure 4-22 HP ProLiant DL585 G5 Server Rear View The callouts in the figure enumerate the following: 1. The iLO2 Ethernet port is the connection to the Console Switch. 2. NIC1 is the connection to the Administration Switch (branch or root). 3. If a Gigabit Ethernet (GigE) interconnect is configured...
Page 112 - Table 4-39 RBSU Settings for HP ProLiant DL585 G5 Nodes; Set the following boot order on the head node:
1. Make the RBSU settings for each node; Table 4-39 lists the RBSU setting for the DL585 G5 nodes. Table 4-39 RBSU Settings for HP ProLiant DL585 G5 Nodes Set to This Value Option Name Menu Name Disabled Embedded Serial Port System Options COM1 Virtual Serial Port On all nodes except the head node, ...
Page 114 - Table 4-40 iLO Settings for HP ProLiant DL785 G5 Nodes
Table 4-40 iLO Settings for HP ProLiant DL785 G5 Nodes Set to This Value Option Name Submenu Name Menu Name Create a common iLO user name and password for everynode in the hardware configuration. The password musthave a minimum of 8 characters by default, but this valueis configurable. The user Admi...
Page 115 - Table 4-41 RBSU Settings for HP ProLiant DL785 G5 Nodes
1. Make the RBSU settings for each node; Table 4-41 lists the RBSU setting for the DL785 G5 nodes. Table 4-41 RBSU Settings for HP ProLiant DL785 G5 Nodes Set to This Value Option Name Menu Name Disabled Embedded Serial Port System Options COM1 Virtual Serial Port On all nodes except the head node, ...
Page 117 - Table 4-42 Setup Utility Settings for xw9300 Workstations; Set the boot order on the head node as follows:
Setup Procedure Use the Setup Utility to configure the appropriate settings for an HP XC system. Perform the following procedure for each workstation in the hardware configuration. Changeonly the values described in this procedure; do not change any other factory-set values unlessyou are instructed ...
Page 118 - to install the HP XC System Software.
10. Follow the software installation instructions in the HP XC System Software Installation Guide to install the HP XC System Software. 118 Preparing Individual Nodes
Page 119 - Preparing the Hardware for CP4000BL Systems; Table 4-44 Boot Order for HP ProLiant Server Blades
4.8 Preparing the Hardware for CP4000BL Systems Perform the following tasks on each server blade in the hardware configuration after the headnode is installed and the switches are discovered: • Set the boot order • Create an iLO2 user name and password • Set the power regulator • Configure smart arr...
Page 121 - Set To This Value
9. Perform this step for HP ProLiant BL685c nodes; proceed to the next step for all otherhardware models. On an HP ProLiant BL685c node, watch the screen carefully during the power-on self-test,and press the F9 key to access the ROM-Based Setup Utility (RBSU) to enable HPET as shown Table 4-45 . Tab...
Page 122 - Preparing the Hardware for CP6000 (Intel Itanium) Systems; “Setting Static IP Addresses on Integrity Servers” (page 122); Set User Interface; Setting Static IP Addresses on Integrity Servers; LC; Gateway address is; 6xx ProCurve Port; assignments for an HP XC system.
4.9 Preparing the Hardware for CP6000 (Intel Itanium) Systems Follow the procedures in this section to prepare HP Integrity servers before installing andconfiguring the HP XC System Software. The following topics, including setup information forspecific hardware models, are discussed: • “Setting Sta...
Page 123 - Figure 4-26 HP Integrity rx1620 Server Rear View; LAN Gb A; Figure 4-27 HP Integrity rx2600 Server Rear View; LAN Gb; PC COM1
Figure 4-26 HP Integrity rx1620 Server Rear View PCI-X 133 PCI-X 133 SCSI LVD/SE LAN 10/100 LAN Gb A GSP RESETS SOFTHARD LAN Gb B CONSOLE / REMOTE / UPS SERIAL USB The callouts in the figure enumerate the following: 1. The port labeled LAN 10/100 is the MP connection to the ProCurve Console Sw...
Page 125 - Netboot; CM; Figure 4-28 HP Integrity rx2620 Server Rear View
13. Perform this step on all nodes except the head node. From the Boot Configuration menu, select the Edit OS Boot Order option. Do the following from the Edit OS Boot Order option: a. Use the navigation instructions shown on the screen to move the Netboot entry you just defined to the top of the bo...
Page 127 - Figure 4-29 HP Integrity rx2660 Server Rear View; MP LAN
13. Perform this step on all nodes except the head node. From the Boot Configuration menu, select the Edit OS Boot Order option. Do the following: a. Use the navigation instructions on the screen to move the Netboot entry you just defined to the top of the boot order. b. If prompted, press the Enter...
Page 129 - Figure 4-30 HP Integrity rx4640 Server Rear View
13. Perform this step on all nodes except the head node. From the Boot Configuration menu, select the Edit OS Boot Order option. Do the following: a. Use the navigation instructions on the screen to move the Netboot entry you just defined to the top of the boot order. b. If prompted, press the Enter...
Page 131 - General Hardware Preparation
e. Press the Enter key for no boot options. f. When prompted, save the entry to NVRAM. For more information about how to work with these menus, see the documentation thatcame with the HP Integrity server. 13. Perform this step on all nodes except the head node. From the Boot Configuration menu, sele...
Page 132 - Figure 4-31 HP Integrity rx8620 Core IO Board Connections
Figure 4-31 HP Integrity rx8620 Core IO Board Connections Connection to Administ ration Netw or k Connection to Console Net w or k (Management Processor) MP Se rial P or t SYS LAN MP LAN MP Reset Pin Preparing Individual Nodes Follow this procedure for each HP Integrity rx8620 node in the hardware c...
Page 133 - IPMI over LAN; CO
NOTE: Most of the MP commands of the HP Integrity rx8620 are similar to the HP Integrity rx2600 MP commands, but there are some differences. The two MPs for the HP Integrityrx8620 operate in a master/slave relationship. Only the master MP, which is on Core IOboard 0, is assigned an IP address. Core ...
Page 135 - reset
14. From the Boot Option Maintenance menu, add a boot option for the EFI Shell (if one does not exist). Follow the instructions in step 12a. 15. Exit the Boot Option Maintenance menu. 16. Choose the EFI Shell boot option and boot to the EFI shell. Enter the following EFI shell commands: EFI> acpi...
Page 136 - Preparing the Hardware for CP6000BL Systems; DVD boot; Procedure
4.10 Preparing the Hardware for CP6000BL Systems Use the management processor (MP) to perform the following tasks on each server blade in thehardware configuration after the head node is installed and the switches are discovered: • Clear all event logs • Enable IPMI over LAN • Create an MP login ID ...
Page 138 - RB; command to reset the BMC.; to exit the console mode and press the
16. Use the RB command to reset the BMC. 17. Press Ctrl - b to exit the console mode and press the x key to exit. After preparing all the nodes in all the enclosures, return to the HP XC System Software InstallationGuide to discover all the nodes and enclosures in the HP XC system. 138 Preparing Ind...
Page 139 - Troubleshooting; iLO2 Devices Can Become Unresponsive; reset map1
5 Troubleshooting This chapter describes known problems with respect to preparing hardware devices for use withthe HP XC System Software and their solutions. 5.1 iLO2 Devices 5.1.1 iLO2 Devices Can Become Unresponsive There is a known problem with the iLO2 console management devices that causes the ...
Page 141 - A Establishing a Connection Through a Serial Port; Enter a name for the connection, select an icon, and click; Select the; OK; Click; OK
A Establishing a Connection Through a Serial Port Follow this generic procedure to establish a connection to a server using a serial port connectionto a console port. If you need more information about how to establish these connections, seethe hardware documentation.1. Connect a null modem cable be...
Page 143 - B Server Blade Configuration Examples; In; KEY; B.2 InfiniBand Interconnect With Full-Height Server Blades; In the configuration shown in; B.1 Gigabit Ethernet Interconnect With Half-Height Server Blades
B Server Blade Configuration Examples This appendix contains illustrations and descriptions of fully cabled HP XC systems based oninterconnect type and server blade height.The connections are color-coded, so consider viewing the PDF file online or printing this appendixon a color printer to take adv...
Page 144 - B.3 InfiniBand Interconnect With Mixed Height Server Blades; The configuration shown in; Server Blade Configuration Examples
Figure B-2 InfiniBand Interconnect With Full-Height Server Blades NIC 1 NIC 2 NIC 3 NIC 4 MEZZ 1 MEZZ 2 MEZZ 3 iLO2 NIC 1 NIC 2 MEZZ 1 MEZZ 2 iLO2 Interconnect bay 1 Interconnect bay 2 Interconnect bay 3 Interconnect bay 4 Interconnect bays Interconnect bay 7 Interconnect bay 8 ONBOARD ADMINISTRATOR...
Page 147 - Glossary; compute
Glossary A administrationbranch The half (branch) of the administration network that contains all of the general-purposeadministration ports to the nodes of the HP XC system. administrationnetwork The private network within the HP XC system that is used for administrative operations. availability se...
Page 148 - A node that is connected to a network external to the HP XC system.
operating system and its loader. Together, these provide a standard environment for bootingan operating system and running preboot applications. enclosure The hardware and software infrastructure that houses HP BladeSystem servers. extensiblefirmwareinterface See EFI. external networknode A node tha...
Page 153 - Index; administration network
Index A administration network as interconnect, 41, 57console branch, 31defined, 30, 37 application cabinet, 45architecture (see processor architecture) B baseboard management controller (see BMC)BIOS settings HP ProLiant DL140 G2, 65HP ProLiant DL140 G3, 66HP ProLiant DL145, 88HP ProLiant DL145 G2,...
