Page 3 - International Technical Suppor t Organization; IBM; Sserver; p5 570 Technical Overview and
International Technical Suppor t Organization IBM Sserver p5 570 Technical Overview and Introduction July 2004
Page 5 - iii; Contents
© Copyright IBM Corp. 2004. All rights reserved. iii Contents Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiTrademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...
Page 6 - iv
iv p5-570 Technical Overview and Introduction 2.5.4 SCSI adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.6 Internal storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...
Page 9 - vii; Notices
© Copyright IBM Corp. 2004. All rights reserved. vii Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on th...
Page 10 - viii; Trademarks; The following terms are trademarks of other companies:
viii p5-570 Technical Over view and Introduction Trademarks The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: Advanced Micro-Partitioning™AIX®AIX 5L™Chipkill™Electronic Service Agent™Enterprise Storage Server® Eserve...
Page 11 - ix; Preface; The team that wrote this Redpaper
© Copyright IBM Corp. 2004. All rights reserved. ix Preface This document is a comprehensive guide covering the IBM® Sserver ™ p5 570 UNIX ® servers. It introduces major hardware offerings and discusses their prominent functions. Professionals wishing to acquire a better understanding of IBM Sserver...
Page 12 - The project that produced this document was managed by:; Become a published author; Use the online Contact us review redbook form found at:; Mail your comments to:
x p5-570 Technical Overview and Introduction The project that produced this document was managed by: Scott VetterIBM U.S. Thanks to the following people for their contributions to this project: Ron Arroyo, John Banchy, Barb Hewitt, Thoi Nguyen, Jan Palmer, Charlie Reeves, Craig Shempert, Scott Smyli...
Page 13 - General description; The IBM
© Copyright IBM Corp. 2004. All rights reserved. 1 Chapter 1. General description The IBM ^ p5 570 rack-mount server is designed for greater application flexibility, with innovative technology, to capitalize on the e-business revolution at the midrange level for server environments. Introduced with ...
Page 14 - slots with Enhanced Error Handling
2 p5-570 Technical Overview and Introduction combined system. Additional processor configurations are possible with the installation of Capacity on Demand (CoD) features. Main memory starting at 2 GB can be expanded to 128 GB in a single drawer, based on the available DIMMs, for higher performance a...
Page 15 - One p5-570 drawer is packaged in a 4U; Description
Chapter 1. General description 3 1.1 System specifications Table 1-1 lists the general system specifications of a single p5-570 drawer. Table 1-1 p5-570 specifications 1.2 Physical package One p5-570 drawer is packaged in a 4U 2 rack-mounted enclosure, and it is available only in the rack-mounted fo...
Page 16 - Minimum and optional features; Eight slim-line media bays for optional optical storage devices
4 p5-570 Technical Overview and Introduction Figure 1-1 Views of the p5-570 1.3 Minimum and optional features The p5-570 full configuration system is made of four p5-570 building blocks. It features: Up to eight processor books using the POWER5 chip, for a total of 16 processors From 2 GB to 512...
Page 17 - Two SPCN ports; CUoD; activation of prepaid processor, and; Processor card FC
Chapter 1. General description 5 Two SPCN ports In addition, the p5-570 building block features two internal Ultra320 SCSI controllers, redundant hot-swap power supply and redundant hot-swap cooling fans, and redundant processor power regulators (FC 7875). There is a CUoD card as part of the hardw...
Page 19 - Feature Code
Chapter 1. General description 7 In a full configuration, with four connected p5-570 building blocks, the combined system supports up to eight slim-line media device bays. To support two slim-line devices in each p5-570 building block, the optional media enclosure and backplane (FC 7869) is required...
Page 20 - The drawer has the following attributes:; 311 Model D10 I/O drawer physical package
8 p5-570 Technical Overview and Introduction The drawer has the following attributes: 4U rack-mount enclosure that can host one or two D10 drawers Six adapter slots – Five PCI-X slots: 3.3 V, keyed, 133 MHz blind-swap hot-plug – One PCI slot: 5 V, keyed, 33 MHz blind-swap hot-plug Default redu...
Page 21 - 311 Model D11 I/O drawer physical package
Chapter 1. General description 9 7311 Model D11 I/O drawer The 7311 Model D11 I/O drawer is very similar to the 7311 Model D10, except that it features six long PCI-X slots and uses an improved blind-swap cassette design. Only the improved blind-swap cassettes are supported (FC 7862, for full-sized ...
Page 22 - 311 Model D20 I/O drawer physical package; The I/O drawer has the following physical characteristics:; I/O drawers and usable PCI slots; Total number of PCI-X slots
10 p5-570 Technical Over view and Introduction 7311 Model D20 I/O drawer physical package The I/O drawer has the following physical characteristics: Width: 482 mm (19.0 in) Depth: 610 mm (24.0 in) Height: 178 mm (7.0 in) Weight: 45.9 kg (101 lb) Figure 1-3 on page 10 shows the different view...
Page 23 - Hardware Management Console models; The system must have at least two disk drives of at least 73.4 GB each.
Chapter 1. General description 11 1.3.6 Hardware Management Console models The Hardware Management Console (HMC) provides a set of functions that is necessary to manage the p5-570 system when LPAR, Capacity on Demand without reboot, inventory and microcode management, and remote power control functi...
Page 24 - Model type conversion
12 p5-570 Technical Over view and Introduction Table 1-8 Value Pak configuration 1.5 Model type conversion Customers who own an IBM pSeries 650 system may convert their system to an IBM Sserver p5 570 system. System hardware for the new model will consist of one or more drawers to replace the old sy...
Page 25 - Optional removable side panels.
Chapter 1. General description 13 1.6.1 IBM RS/6000 7014 Model T00 Enterprise Rack The 1.8-meter (71-inch) Model T00 is compatible with past and current p5, pSeries, and RS/6000 racks, and is designed for use in all situations that previously used the older rack models R00 and S00. The T00 rack has ...
Page 26 - AC power support only; AC Power Distribution Unit and rack content; For rack models T00 and T42 nine-outlet PDUs are available.
14 p5-570 Technical Over view and Introduction 1.6.2 IBM RS/6000 7014 Model T42 Enterprise Rack The 2.0-meter (79.3-in) Model T42 is the rack that will address the special requirements of customers who want a tall enclosure to house the maximum amount of equipment in the smallest possible floor spac...
Page 27 - Flat panel display options; Flat panel color monitor.
Chapter 1. General description 15 Any remaining space in the rack can be used to install other systems or peripherals, provided that the maximum permissible weight of the rack is not exceeded and the installation rules for these devices are followed. Before placing a p5-570 into the service posi...
Page 28 - Hardware Management Console 7310 Model CR2
16 p5-570 Technical Over view and Introduction Hardware Management Console 7310 Model CR2 The 7310 Model CR2 is a 1U, 19-inch rack-mountable drawer supported in the 7014 Model T00 and T42 racks. The 7310 Model CR2 provides one serial port, two integrated Ethernet ports, and two additional PCI slots....
Page 29 - Statement of direction; IBM plans to extend the capabilities of the IBM
Chapter 1. General description 17 1.7 Statement of direction IBM plans to extend the capabilities of the IBM Sserver p5 product line by introducing support for the i5/OS™ operating system. This support is planned for selected IBM Sserver p5 570 and future high-end IBM Sserver p5 models. i5/OS suppor...
Page 32 - Because of the higher transistor density of the POWER5 0.13-
20 p5-570 Technical Over view and Introduction 2.1 The POWER5 chip The POWER5 chip features single-threaded and multi-threaded execution, providing higher performance in the single-threaded mode than its POWER4 predecessor provides at equivalent frequencies. POWER5 maintains both binary and architec...
Page 33 - smtctl; Enhanced SMT features; Dynamic resource balancing; ST operation; In current CMOS
Chapter 2. Architecture and technical overview 21 2.1.1 Simultaneous multi-threading As a permanent requirement for performance improvements at the application level, simultaneous multi-threading (SMT) functionality is embedded in the POWER5 chip technology. Developers are familiar with process-leve...
Page 35 - IBM Systems and; and; IBM Microelectronics; that enables IBM; SMI II
Chapter 2. Architecture and technical overview 23 2.1.4 CMOS, copper, and SOI technology The POWER5 processor design is a result of a close collaboration between IBM Systems and Technology Group and IBM Microelectronics technologies that enables IBM Sserver p5 systems to give customers improved perf...
Page 36 - Processor drawer interconnect cables
24 p5-570 Technical Over view and Introduction Figure 2-5 Processor card with DDR1 memory socket layout view There are two system backplanes in the p5-570 system. A GX+ bus planar, which docks vertically into the system planar, is always present in the system. The processor cards dock directly into ...
Page 37 - X Where; pmcycles; command is par t of the bos.pmapi fileset. First check whether that
Chapter 2. Architecture and technical overview 25 Figure 2-6 Logical p5-570 building blocks connection 2.2.2 Processor clock rate The p5-570 system features base 2-way, 4-way, 8-way, 12-way, and 16-way configurations with the POWER5 processor running at 1.5 GHz, 1.65 GHz, and 1.9 GHz. The processor ...
Page 38 - Memory subsystem; Heat Sink
26 p5-570 Technical Over view and Introduction 2.3 Memory subsystem The p5-570 memory controller is internal to the POWER5 chip. It interfaces to either two (DDR1) or four (DDR2) SMI-II buffer chips and 8 pluggable DIMMs per processor card, as described in 2.2, “Processor cards” on page 23. The mini...
Page 41 - pSeries Information Center at:; Feature code
Chapter 2. Architecture and technical overview 29 The PCI-X slots in the p5-570 system support hot-plug and Extended Error Handling (EEH). In the unlikely event of a problem, EEH-enabled adapters respond to a special data packet that is generated from the affected PCI-X slot hardware by calling syst...
Page 42 - Internal storage
30 p5-570 Technical Over view and Introduction Table 2-2 Available SCSI adapters 2.6 Internal storage Two Ultra320 SCSI controllers under EADS-X chips that are integrated into the system planar are used to drive the internal disk drives. The six internal drives plug into the disk drive backplane, wh...
Page 43 - Hot-swap disks and Linux
Chapter 2. Architecture and technical overview 31 planning has been given to any operating-system-related disk layout, such as the AIX Logical Volume Manager, when using disk hot-swap capabilities. For more information, see Problem Solving and Troubleshooting in AIX 5L, SG24-5496. After the SCSI dis...
Page 47 - SPCN
Chapter 2. Architecture and technical overview 35 Each RIO-2 port can operate at 1 GHz in bidirectional mode and is capable of passing data in each direction on each cycle of the port. Therefore, the maximum data rate is 4 GBps per I/O drawer in double barrel mode. There is one default primary RIO-2...
Page 48 - IBM 2104 Expandable Storage Plus
36 p5-570 Technical Over view and Introduction the p5-570 system. If you have more than one I/O drawer, you continue the loop, connecting the next drawer (or drawers) with the same rule. Figure 2-14 SPCN cabling examples There are different SPCN cables to satisfy different length requirements: SPC...
Page 49 - For further information about SSA, visit the 7133 pages at:; IBM TotalStorage FAStT Storage servers; For further information about ESS refer to the following Web site:
Chapter 2. Architecture and technical overview 37 IBM 7133 Serial Disk Subsystem (SSA) The p5-570 system supports the Advanced Serial RAID adapter (FC 6230), to attach the IBM 7133 Serial Disk Subsystem Model D40, if migrated from a pSeries system. The IBM 7133 Serial Disk Subsystem Model D40 provid...
Page 50 - Dynamic logical partitioning; Advanced POWER Virtualization feature
38 p5-570 Technical Over view and Introduction 2.8 Dynamic logical partitioning The logical partition (LPAR) was introduced with the POWER4 processor product line and the AIX 5L Version 5.1 operating system. The technology offered the capability to divide a pSeries system into separate systems, wher...
Page 51 - Micro-Partitioning technology
Chapter 2. Architecture and technical overview 39 The Advanced POWER Virtualization feature includes: Firmware enablement for Micro-Partitions Installation image for the Virtual I/O server software that supports: – Ethernet adapter sharing – Virtual SCSI Server Partition Load Manager – Automat...
Page 52 - Shared Ethernet adapter; POWER Hypervisor; Storage Network; HMC; POWER5 Partitioning
40 p5-570 Technical Over view and Introduction Figure 2-15 Micro-Partitioning and LPAR Virtual I/O Server The Virtual I/O Server is a special-purpose partition that provides virtual I/O resources to client partitions. The Virtual I/O Server owns the real resources that are shared with the other clie...
Page 53 - Partition Load Manager; Memory-pages-steal high thresholds and low thresholds; Service processor
Chapter 2. Architecture and technical overview 41 Partition Load Manager The Partition Load Manager (PLM) is part of the Advanced POWER Virtualization feature. It provides automated processor and memory distribution between dynamic LPAR and Micro-Partitioning capable logical partition running AIX 5L...
Page 54 - The SP base unit offers the following connections:
42 p5-570 Technical Over view and Introduction Figure 2-16 Service processor logic diagram 2.10.1 Service processor - base The PPC405 core features a five-stage pipeline instruction processor and contains 32-bit general purpose registers. The flash ROM contains a compressed image of a software load....
Page 55 - IPL flow without an HMC attached to the system
Chapter 2. Architecture and technical overview 43 2.11 Boot process From the earlier RS/6000 systems, through the previous pSeries systems, the boot process passed through several enhancements. With the implementation of the POWER5 technology, the boot process is enhanced to accommodate the flexibil...
Page 56 - IPL flow with an HMC attached to the system
44 p5-570 Technical Over view and Introduction – Boot to open firmware prompt – Boot to System Management Service (SMS) to further select the boot devices or network boot options Boot to server firmware – Select the state for the server firmware: Standby or Running. – When the server is in the ser...
Page 57 - Managed systems
Chapter 2. Architecture and technical overview 45 – When the Partition Standby power-on is completed, the operator panel on the managed system displays LPAR... , indicating that the managed system is ready for you to use the HMC to partition its resources and, possibly, activate them. – When a parti...
Page 58 - Hardware requirements for partitioning; Processors; The default uncapped weight is 128.
46 p5-570 Technical Over view and Introduction – To enhance the flexibility to use the system within several different logical configurations, a System profile could be defined to collect more than one Partition profile to provide requested system behavior. 2.11.5 Hardware requirements for partition...
Page 59 - keyboard
Chapter 2. Architecture and technical overview 47 The p5-570 (or the logical partition) must be equipped with either a graphic adapter that is connected to a graphics display, keyboard, and mouse device, or an ASCII display terminal that is connected to one of the native serial ports or the attached...
Page 60 - Select Install or Boot a Device; External tape drives
48 p5-570 Technical Over view and Introduction partition displays the SMS menus, all of the necessary tests have been performed and the machine is scanning the bus for a boot source. Most system backplanes are designed such that the drive in the first slot spins up immediately after power-on, and ot...
Page 61 - available to the specific partition, using; mksysb; The; admin; password is usually used by the system administrator, and the; general ID password; Operating system requirements
Chapter 2. Architecture and technical overview 49 available to the specific partition, using mksysb , for example. SCSI disk, and Virtual SCSI disk The more common method of booting the system is to use a disk situated in one of the hot-swap bays in the front of the machine. However, any external SC...
Page 62 - suma; command, which helps the administrator; Linux support
50 p5-570 Technical Over view and Introduction IBM periodically releases maintenance packages for the AIX 5L operating system. These packages are available on CD-ROM (FC 0907) and can be downloaded from the Internet at: http://techsupport.services.ibm.com/server/fixes You can also get individual ope...
Page 64 - Capacity on Demand; -way 1.9 GHz POWER5 processor card with DDR1 memory slots; Processor Capacity Upgrade on Demand methods
52 p5-570 Technical Over view and Introduction 3.1 Capacity on Demand p5-570 systems can be shipped with non-activated resources (processors and memory), which may be added as they are needed. Processors and memory can be brought online to meet increasing workload demands without affecting system op...
Page 65 - Capacity Upgrade on Demand for memory; Permanent activation
Chapter 3. Capacity on Demand, RAS, and manageability 53 Capacity Upgrade on Demand In Capacity Upgrade on Demand (CUoD), processors are shipped to the customer as installed in the p5-570, and can be activated later in increments of one processor. Additional options deliver the possibility to tempor...
Page 66 - On/Off Capacity on Demand; On/Off memory enablement can be activated with FC 7954.; Capacity Upgrade on Demand for memory feature codes; The feature codes to activate the memory are:; How to report temporary activation resources; Using Electronic Service AgentTM
54 p5-570 Technical Over view and Introduction On/Off Capacity on Demand The On/Off memory enablement feature is ordered and the associated enablement code is entered into the system. On/off usage must be reported to IBM at least monthly. This information, which is used to compute your billing data,...
Page 67 - Activate Immediate
Chapter 3. Capacity on Demand, RAS, and manageability 55 Using e-mail 3.1.4 Trial Capacity on Demand Customers with CUoD featured systems must purchase the activation codes from IBM before the non-activated CUoD resources can be activated to meet the increased workload. However with the Trial Capa...
Page 68 - diagela; ) routines analyze the error log; Mutual surveillance; Disk
56 p5-570 Technical Over view and Introduction 3.2.2 First Failure Data Capture If a problem should occur, the ability to diagnose it correctly is a fundamental requirement upon which improved availability is based. The p5-570 incorporates advanced capability in start-up diagnostics and in run-time ...
Page 70 - Dynamic or persistent deallocation; CPU Guard
58 p5-570 Technical Over view and Introduction ( dynamic bit-steering ). Memory scrubbing is the process of reading the contents of the memory during idle time and checking and correcting any single-bit errors that have accumulated by passing the data through the ECC logic. This function is a hardwa...
Page 71 - Error indication and LED indicators
Chapter 3. Capacity on Demand, RAS, and manageability 59 If the output shows CPU Guard as disabled, enter the following command to enable it: chdev -l sys0 -a cpuguard='enable' Cache or cache-line deallocation is aimed at performing dynamic reconfiguration to bypass potentially failing components. T...
Page 72 - System Attention LED; Concurrent Maintenance; Disk drives; Advanced System Management Interface
60 p5-570 Technical Over view and Introduction starting on the exterior of the system (System Attention LED) and ending with an LED near the failing Field Replaceable Unit. For more information about replaceable units, including videos, see: http://publib16.boulder.ibm.com/pseries/en_US/infocenter/b...
Page 73 - Automatic problem analysis
Chapter 3. Capacity on Demand, RAS, and manageability 61 Figure 3-2 Advanced System Management main menu 3.3.2 Service Agent Service Agent is an application program that operates on a p5, pSeries, or IBM RS/6000 computer and monitors them for hardware errors. It reports detected errors, assuming tha...
Page 74 - Service focal point; You can download the latest version of Service Agent at:; Service Update Management Assistant; Benefits provided by SUMA include:; smitty suma; ) or a command line interface
62 p5-570 Technical Over view and Introduction checks the current customer service status from the IBM entitlement database; if this reveals that you are not under warranty or MA, then the service call is refused and posted back using e-mail. Service focal point Service Focal Point is used by servic...
Page 75 - trusted
Chapter 3. Capacity on Demand, RAS, and manageability 63 3.4 Cluster 1600 Today's IT infrastructure requires that systems meet increasing demands while offering the flexibility and manageability to rapidly develop and deploy new services. IBM clustering hardware and software provide the building blo...
Page 76 - CSM value points
64 p5-570 Technical Over view and Introduction Some customers expanded the use of their dedicated systems and consequently model more business processes. This often caused an increased number of dedicated systems that were used and a stronger demand on flexibility. In addition, the life cycle of the...
Page 77 - Related publications; IBM Redbooks; Other publications
© Copyright IBM Corp. 2004. All rights reserved. 65 Related publications The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this Redpaper. IBM Redbooks For information about ordering these publications, see “How to get...
Page 78 - Online resources
66 p5-570 Technical Over view and Introduction Online resources These Web sites and URLs are also relevant as further information sources: AIX 5L operating system maintenance packages downloads http://www.ibm.com/servers/eserver/support/pseries/aixfixes.html Autonomic computing on IBM Sserver pS...
Page 79 - SSA boot FAQ; How to get IBM Redbooks; IBM Support and downloads
Related publications 67 SSA boot FAQ http://www.storage.ibm.com/hardsoft/products/ssa/faq.html#microcode SUSE LINUX Enterprise Server 8 for pSeries information http://www.suse.de/us/business/products/server/sles/i_pseries.html The LVT is a PC based tool intended assist you in logical partition...
Page 82 - server; Back cover
® INTERNATIONAL TECHNICALSUPPORTORGANIZATION BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on re...
