IBM P5 520 - Manuals

© Copyright IBM Corp. 2006. All rights reserved. vii Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on th...

viii IBM System p5 520 and 520Q Technical Over view and Introduction Trademarks The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: Eserver ® Redbooks (logo) ™ pSeries®AIX 5L™AIX®Chipkill™DS4000™DS6000™DS8000™FICON® HA...

© Copyright IBM Corp. 2006. All rights reserved. ix Preface This IBM Redpaper is a comprehensive guide that covers the IBM® System p5™ 520 and 520Q UNIX® servers. It introduces major hardware offerings and discusses their prominent functions. Professionals who want to acquire a better understanding ...

x IBM System p5 520 and 520Q Technical Over view and Introduction expertise include Mainframe Channel Subsystem, FICON®, and pSeries RAS. He has written extensively on FICON. SahngShin Kim is a sales specialist of STG infra-solution sales team in Seoul, Korea. For three years, he was a sales special...

Preface xi 򐂰 Use the online Contact us review redbook form found at: ibm.com/redbooks 򐂰 Send your comments in an e-mail to: [email protected] 򐂰 Mail your comments to: IBM Corporation, International Technical Support OrganizationDept. HYTD Mail Station P0992455 South RoadPoughkeepsie, NY 12601-5400

xii IBM System p5 520 and 520Q Technical Over view and Introduction

© Copyright IBM Corp. 2006. All rights reserved. 1 Chapter 1. General description The IBM System p5 520 and IBM System p5 520Q rack-mount and deskside servers (9131-52A) give you new tools for managing on demand business, greater application flexibility, and innovative technology in 1-core, 2-core, ...

Chapter 1. General description 3 1.1 System specifications Table 1-1 lists the general system specifications of the p5-520 and p5-520Q systems. Table 1-1 IBM System p5 520 and IBM System p5 520Q specifications 1.2 Physical package This section discusses the major physical attributes of the p5-520 an...

Chapter 1. General description 5 1.2.2 Rack-mount model The IBM System p5 520 or IBM System p5 520Q can be configured as a 4U rack-mount model with the selected feature code. Table 1-3 lists the physical attributes and Figure 1-2 shows the system. Table 1-3 Physical attributes of the rack-mount mode...

6 IBM System p5 520 and 520Q Technical Over view and Introduction AIX 5L system tools for configuration management if the adapter is connected to a common maintenance console, such as the 7316-TF3 rack-mounted flat-panel display. 1.3 Minimum and optional features The systems are based on a flexible,...

Chapter 1. General description 7 Table 1-4 Processor feature codes 1.3.2 Memory features The minimum memory requirement for the p5-520 and p5-520Q servers is 1 GB, and the maximum capacity is 32 GB using 533 MHz DDR2 technology. The planar of each system has eight sockets for memory DIMMs. Table 1-5...

8 IBM System p5 520 and 520Q Technical Over view and Introduction Table 1-6 Hot-swappable disk drive options You can install any combination of the following DVD-ROM and DVD-RAM drives in the two slim-line bays: 򐂰 DVD-RAM drive, FC 1993 򐂰 DVD-ROM drive, FC 1994 A logical partition running a supporte...

Chapter 1. General description 9 four attached 7311 Model D20 drawers, the combined system supports up to 34 PCI-X adapters (in a maximum configuration, remote I/O expansion cards are required) and 56 hot-swappable SCSI disks, for a total internal capacity of 16.8 TB using 300 GB disks. PCI-X and PC...

10 IBM System p5 520 and 520Q Technical Over view and Introduction 1.3.6 Hardware Management Console models A p5-520 or p5-520Q can be either HMC-managed or non-HMC-managed. In HMC-managed mode, an HMC is required as a dedicated workstation that allows you to configure and manage partitions. The HMC...

Chapter 1. General description 11 1.4 Express Product Offerings The Express Product Offerings provide a convenient way to order any of several configurations that are designed to meet typical client requirements. Special reduced pricing is available when a system order satisfies specific configurati...

12 IBM System p5 520 and 520Q Technical Over view and Introduction Table 1-8 Express Product Offering standard set of feature codes A specific Express Product Offering ID or specific offering feature code is used to select the processor type and quantity, and the associated memory feature code and q...

14 IBM System p5 520 and 520Q Technical Over view and Introduction Optional Rear Door Heat eXchanger (FC 6858) Improved cooling from the heat exchanger enables the client to more densely populate individual racks freeing valuable floor space without the need to purchase additional air conditioning u...

Chapter 1. General description 15 򐂰 Width: 605 mm (23.8 in.) with side panels 򐂰 Depth: 1001 mm (39.4 in.) with front door 򐂰 Height: 1344 mm (49.0 in.) 򐂰 Weight: 100.2 kg (221.0 lb.) The S25 rack has a maximum load limit of 22.7 kg (50 lb.) per EIA unit for a maximum loaded rack weight of 667 kg (147...

16 IBM System p5 520 and 520Q Technical Over view and Introduction Table 1-11 Models supported in S11 and S25 racks 1.5.6 The ac power distribution unit and rack content For rack models T00 and T42, 12-outlet PDUs (FC 9188 and FC 7188) are available. For rack models S11 and S25, FC 7188 is available...

Chapter 1. General description 17 The S11 and S25 racks support as many PDUs as there is available rack space. For detailed power cord requirements and power cord feature codes, see IBM System p5, IBM Eserver p5 and i5, and OpenPower Edition Planning, SA38-0508. For an online copy, select Map of pSe...

18 IBM System p5 520 and 520Q Technical Over view and Introduction 1.5.7 Rack-mounting rules The primary rules that you should follow when you mount the server into a rack are: 򐂰 The p5-520 or p5-520Q is designed to be placed at any location in the rack. For rack stability, we advise that you start ...

Chapter 1. General description 19 reading and writing on Ultrium 2 tape cartridges, it is also read and write compatible with Ultrium 1 cartridges. 򐂰 SLR60 Tape Drive (QIC format) comes with 37.5 GB native data physical capacity per tape cartridge and a native physical data transfer rate of up to 4 ...

20 IBM System p5 520 and 520Q Technical Over view and Introduction 򐂰 Occupies only 1U (1.75 in) in a 19-inch standard rack The 1x8 Console Switch can be mounted in one of the following racks: 7014-T00, 7014-T42, 7014-S11, or 7014-S25. The 1x8 Console Switch supports GXT135P (FC 1980 and FC 2849) gra...

Chapter 1. General description 21 򐂰 The switch offers four ports for server connections. Each port in the switch can connect a maximum of 16 systems: – One KCO cable (FC 4268) or USB cable (FC 4269) is required for every four systems supported on the switch. – A maximum of 16 KCO cables or USB cable...

26 IBM System p5 520 and 520Q Technical Over view and Introduction 2.1 The POWER5+ processor The IBM POWER5+ processor capitalizes on all the enhancements brought by the POWER5 processor. For a detailed description of the POWER5 processor, refer to IBM System p5 520 Technical Overview and Introducti...

Chapter 2. Architecture and technical overview 27 򐂰 Double the SMP suppor t. Changes have been made in the fabric, L2 and L3 controller, memory controller, GX controller, and processor RAS to provide support for the QCM that allows the SMP system sizes to be double that which is available in POWER5 ...

30 IBM System p5 520 and 520Q Technical Over view and Introduction 2.2.4 Available processor speeds Table 2-1 lists the available processor capacities and speeds for the p5-520 and p5-520Q systems. Table 2-1 p5-520 and p5-520Q available processor capacities and speeds To determine the processor char...

Chapter 2. Architecture and technical overview 31 Figure 2-7 Memory placement for the p5-520 and p5-520Q servers 2.3.1 Memory placement rules Table 2-2 lists the memory features that are available at the time of writing for the p5-520 and p5-520Q servers. Table 2-2 Available memory features Memory c...

32 IBM System p5 520 and 520Q Technical Over view and Introduction To determine how much memory is installed in a system, use the following command: # lsattr -El sys0 | grep realmemrealmem 524288 Amount of usable physical memory in Kbytes False 2.3.2 OEM memory OEM memory is not supported or certifi...

Chapter 2. Architecture and technical overview 33 Table 2-3 Theoretical throughput rates 2.4 I/O buses This section provide additional information that is related to the internal RIO-2 buses and GX+ buses. The QCM or DCM provides a GX+ bus. In the past, the 6XX bus was the front end from the process...

34 IBM System p5 520 and 520Q Technical Over view and Introduction Figure 2-9 p5-520 or p5-520Q GX+ Bus connection overview According to the processor speed, the I/O subsystem is capable of supporting 5.6 GBps when using the 2.1 GHz processor, or capable of supporting 4.4 GBps when using a 1.65 GHz ...

36 IBM System p5 520 and 520Q Technical Over view and Introduction You also have the option to make the internal Ultra320 SCSI channel externally accessible on the rear side of the system by installing FC 4275. No additional SCSI adapter is required in this case. If FC 4275 is installed, a second 4-...

Chapter 2. Architecture and technical overview 37 The iSCSI protocol is implemented on top of the physical and data-link layers and presents to the operating system a standard SCSI Access Method command set. It suppor ts SCSI-3 commands and reliable delivery over IP networks. The iSCSI protocol runs...

38 IBM System p5 520 and 520Q Technical Over view and Introduction IBM iSCSI software Host Support Kit The iSCSI protocol can also be used over standard Gigabit Ethernet adapters. To utilize this approach, download the appropriate iSCSI Host Suppor t Kit for your operating system from the IBM NAS su...

Chapter 2. Architecture and technical overview 39 Table 2-7 Available Fibre Channel adapters 2.6.6 Graphic accelerators The p5-520 and p5-520Q suppor t up to four enhanced POWER GXT135P (FC 1980) 2D graphic accelerators. The POWER GXT135P is a low-priced 2D graphics accelerator for IBM System p5 ser...

40 IBM System p5 520 and 520Q Technical Over view and Introduction In many cases, the 5723 asynchronous adapter is configured to supply a backup HACMP heartbeat. In these cases, a serial cable (FC 3927 or FC 3928) must be also configured. Both of these serial cables and the 5723 adapter have 9-pin c...

Chapter 2. Architecture and technical overview 41 2.6.12 Ethernet ports The two built-in Ethernet ports provide 10/100/1000 Mbps connectivity over CAT-5 cable for up to 100 meters. Table 2-9 lists the attributes of the LEDs that are visible on the side of the jack. Table 2-9 Ethernet LED description...

42 IBM System p5 520 and 520Q Technical Over view and Introduction assigns the name ses0 to the first 4-pack, and ses1 to the second, if present). The two hot-swappable 4-pack disk drive backplanes can accommodate the devices listed in Table 2-11. Table 2-11 Available hot-swappable disk drives At th...

44 IBM System p5 520 and 520Q Technical Over view and Introduction Figure 2-11 7311 Model D20 internal SCSI cabling 2.8.2 7311 I/O drawer RIO-2 cabling As described in 2.8, “External I/O subsystem” on page 43, you can connect up to four I/O drawers in the same loop to the p5-520 or p5-520Q system. E...

Chapter 2. Architecture and technical overview 45 Figure 2-12 RIO-2 connections The RIO-2 cables used have different lengths to satisfy the different connection requirements: 򐂰 Remote I/O cable, 1.2 m (FC 3146) 򐂰 Remote I/O cable, 1.75 m (FC 3156) 򐂰 Remote I/O cable, 2.5 m (FC 3168) 򐂰 Remote I/O cab...

46 IBM System p5 520 and 520Q Technical Over view and Introduction Figure 2-13 SPCN cabling examples There are different SPCN cables to satisfy different length requirements: 򐂰 SPCN cable drawer to drawer, 2 m (FC 6001) 򐂰 SPCN cable drawer to drawer, 3 m (FC 6006) 򐂰 SPCN cable rack to rack, 6 m (FC ...

Chapter 2. Architecture and technical overview 47 36.4 GB, 73.4 GB or 146.8 GB 15K rpm drives. Each of the four six-packs disk drive enclosure can be attached independently to an Ultra320 SCSI or Ultra320 SCSI RAID adapter. For high available configurations, a dual bus repeater card (FC 5742) allows...

48 IBM System p5 520 and 520Q Technical Over view and Introduction 2.10 Logical partitioning Dynamic logical partitions (LPARs) and virtualization increase utilization of system resources and add a new level of configuration possibilities. This section provides details and configuration specificatio...

Chapter 2. Architecture and technical overview 49 2.11.1 POWER Hypervisor Combined with features designed into the POWER5 and POWER5+ processors, the POWER Hypervisor delivers functions that enable other system technologies, including Micro-Partitioning technology, virtualized processors, IEEE VLAN,...

50 IBM System p5 520 and 520Q Technical Over view and Introduction client adapter. Only the Virtual I/O Server partition can define virtual SCSI server adapters, other partitions are client partitions. The Virtual I/O Server is available with the optional Advanced POWER Virtualization feature (FC 79...

Chapter 2. Architecture and technical overview 51 2.12 Advanced POWER Virtualization feature The Advanced POWER Virtualization feature (FC 7940) is an optional, additional cost feature. This feature enables the implementation of more fine-grained vir tual partitions on IBM System p5 servers. The Adv...

52 IBM System p5 520 and 520Q Technical Over view and Introduction A partition can be defined with a processor capacity as small as 0.10 processing units. This represents one-tenth of a physical processor. Each physical processor can be shared by up to 10 shared processor partitions, and a partition...

Chapter 2. Architecture and technical overview 53 򐂰 Virtual processors do not introduce any additional abstraction level, they are really only a dispatch entity. When running on a physical processor, they run at the full speed of the physical processor. 򐂰 Each partition’s profile defines a CPU entit...

Chapter 2. Architecture and technical overview 55 Figure 2-15 Micro-Partitioning technology and VIOS Because the Virtual I/O Server is an operating system-based appliance server, redundancy for physical devices attached to the Virtual I/O Server can be provided by using capabilities such as Multipat...

56 IBM System p5 520 and 520Q Technical Over view and Introduction Shared Ethernet adapter A shared Ethernet adapter (SEA) is a Virtual I/O Server service that acts as a layer 2 network bridge between a physical Ethernet adapter or aggregation of physical adapters (EtherChannel) and one or more Virt...

Chapter 2. Architecture and technical overview 57 Virtual I/O Server version 1.3 Virtual I/O Server version 1.3 brings a host of new enhancements, including improved monitoring such as additional topas and viostat performance metrics and the bundling of the Performance ToolKit (PTX®) agent. Virtual ...

Chapter 2. Architecture and technical overview 59 򐂰 IVM cannot be used by HACMP software to activate Capacity on Demand (CoD) resources on machines that support CoD. IVM provides advanced vir tualization functionality without the need for an extra-cost workstation. For more information about IVM fun...

60 IBM System p5 520 and 520Q Technical Over view and Introduction Table 2-13 Operating system supported functions 2.13 Hardware Management Console The Hardware Management Console (HMC) is a dedicated workstation that provides a graphical user interface for configuring, operating, and performing bas...

Chapter 2. Architecture and technical overview 61 Figure 2-17 HMC to service processor and LPARs network connection The default mechanism for allocation of the IP addresses for the service processor HMC ports is dynamic. The HMC can be configured as a DHCP server, providing the IP address at the tim...

Chapter 2. Architecture and technical overview 63 unpartitioned system. You can create an entirely new system configuration, or you can create a system configuration based upon any of the following: 򐂰 Performance data from an existing system that the new system is to replace 򐂰 Performance estimates ...

64 IBM System p5 520 and 520Q Technical Over view and Introduction 2.14 Operating system support The p5-520 and p5-520Q are capable of running the AIX 5L and Linux operating systems. The AIX 5L operating system has been developed and enhanced specifically to exploit and to support the extensive RAS ...

Chapter 2. Architecture and technical overview 65 configuration changes are necessary to enable a system to use 64 KB pages; they are fully pageable, and the size of the pool of 64 KB page frames on a system is dynamic and fully managed by AIX 5L. The main benefit of a larger page size is improved p...

66 IBM System p5 520 and 520Q Technical Over view and Introduction Many of the features that are described in this document are operating system dependent and might not be available on Linux. For more information, see: http://www.ibm.com/systems/p/software/whitepapers/linux_overview.pdf Specially pr...

Chapter 2. Architecture and technical overview 67 2.15.1 Touch point colors Blue (IBM blue) or terra-cotta (orange) on a component indicates a touch point (for electronic parts) where you can grip the hardware to remove it from or install it into the system, open or close a latch, and so on. IBM def...

68 IBM System p5 520 and 520Q Technical Over view and Introduction Figure 2-20 Pull the server to the service position 3. Release the rack latches (C) on both the left and right sides, as shown in the Figure 2-20. 4. Review the following notes, and then slowly pull the system or expansion unit out f...

Chapter 2. Architecture and technical overview 69 2.15.5 Operator control panel The service processor provides an interface to the control panel that is used to display server status and diagnostic information. See Figure 2-21 for operator control panel physical details and buttons. Figure 2-21 Oper...

70 IBM System p5 520 and 520Q Technical Over view and Introduction Function 30 – CEC SP IP address and location Function 30 is one of the Extended control panel functions and is only available when Manual mode is selected. You can use this function to display the central electronic complex (CEC) Ser...

Chapter 2. Architecture and technical overview 71 The following examples are the output of the lsmcode command for AIX 5L and Linux, showing the firmware levels as they are displayed in the outputs: 򐂰 AIX 5L The current permanent system firmware image is SF220_005.The current temporary system firmwa...

72 IBM System p5 520 and 520Q Technical Over view and Introduction Receive server firmware fixes using an HMC If you use an HMC to manage your server and you need to configure several partitions on the server periodically, you need to download and install fixes for your server and power subsystem fi...

Chapter 2. Architecture and technical overview 73 2.15.7 Service processor The service processor is an embedded controller running the service processor internal operating system. The service processor operating system contains specific programs and device drivers for the service processor hardware....

74 IBM System p5 520 and 520Q Technical Over view and Introduction This interface is accessible using a Web browser on a client system that is connected directly to the service processor (in this case, you can use either a standard Ethernet cable or a crossed cable) or through an Ethernet network. U...

Chapter 2. Architecture and technical overview 75 3. Look for the power-on self-test (POST) indicators: memory, keyboard, network, SCSI, and speaker that appear across the bottom of the screen. Press the numeric 1 key after the word keyboard appears and before the word speaker appears. The SMS menus...

© Copyright IBM Corp. 2006. All rights reserved. 77 Chapter 3. RAS and manageability This chapter provides information about IBM System p5 design features that help lower the total cost of ownership (TCO). IBM reliability, availability, and service (RAS) technology allow you to improve your TCO arch...

78 IBM System p5 520 and 520Q Technical Over view and Introduction 3.1 Reliability, availability, and serviceability Excellent quality and reliability are inherent in all aspects of the IBM System p5 processor design and manufacturing. The fundamental objective of the design approach is to minimize ...

Chapter 3. RAS and manageability 79 Figure 3-1 Schematic of Fault Isolation Register implementation The FIRs are impor tant because they enable an error to be uniquely identified, thus enabling the appropriate action to be taken. Appropriate actions might include such things as a bus retry, ECC corr...

80 IBM System p5 520 and 520Q Technical Over view and Introduction The operating system cannot program or access the temperature threshold using the SP. EPOW events can, for example, trigger the following actions: 򐂰 Temperature monitoring, which increases the fan’s rotation speed when ambient temper...

Chapter 3. RAS and manageability 81 3.1.5 N+1 redundancy The use of redundant parts allows the p5-520 and p5-520Q to remain operational with full resources: 򐂰 Redundant spare memory bits in L1, L2, L3, and main memory 򐂰 Redundant fans 򐂰 Redundant power supplies (optional) 3.1.6 Fault masking If corr...

82 IBM System p5 520 and 520Q Technical Over view and Introduction If the output shows CPU Guard as disabled, enter the following command to enable it: chdev -l sys0 -a cpuguard='enable' Cache or cache-line deallocation is aimed at performing dynamic reconfiguration to bypass potentially failing com...

Chapter 3. RAS and manageability 83 internal LED diagnostics that identify the parts that require service. Attenuation of the error is provided through a series of light attention signals, starting on the exterior of the system (System Attention LED) which is located on the front of the system and e...

84 IBM System p5 520 and 520Q Technical Over view and Introduction manage the machine-wide settings using the ASMI and allows complete system and partition management from the HMC. Also, the surveillance function of the SP is monitoring the operating system to check that it is still running and has ...

Chapter 3. RAS and manageability 85 – Concurrent mode enables the normal system functions to continue while you are checking selected resources. Because the system is running in normal operation, some devices might require additional actions by the user or diagnostic application before testing can b...

86 IBM System p5 520 and 520Q Technical Over view and Introduction Service Agent sends outbound transmissions only and does not allow any inbound connection attempts. Only hardware machine configuration, machine status, or error information is transmitted. Service Agent does not access or transmit a...

Chapter 3. RAS and manageability 87 You can download the latest version of Service Agent at: ftp://ftp.software.ibm.com/aix/service_agent_code Service Focal Point Traditional service strategies become more complicated in a partitioned environment. Each logical par tition reports errors it detects, w...

88 IBM System p5 520 and 520Q Technical Over view and Introduction cannot be concurrently activated, are deferred. Figure 3-4 shows the system firmware file naming convention. Figure 3-4 System firmware file naming convention An installation is disruptive if: 򐂰 The release levels (XXX) of currently ...

© Copyright IBM Corp. 2006. All rights reserved. 91 Related publications The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics that are covered in this Redpaper. IBM Redbooks For information about ordering these publications, see “H...

92 IBM System p5 520 and 520Q Technical Over view and Introduction 򐂰 IBM Sserver Hardware Management Console for pSeries Installation and Operations Guide, SA38-0590, provides information to operators and system administrators about how to use a IBM Hardware Management Console for pSeries (HMC) to m...

Related publications 93 How to get IBM Redbooks You can search for, view, or download Redbooks, Redpapers, Hints and Tips, draft publications and Additional materials, as well as order hardcopy Redbooks or CD-ROMs, at this Web site: ibm.com/redbooks Help from IBM IBM Support and downloads ibm.com/su...

® INTERNATIONAL TECHNICALSUPPORTORGANIZATION BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on re...