Page 2 - Legal Notices
2 Legal Notices © Copyright 2006-2008 Hewlett-Packard Development Company, L.P. Publication Date: 2008 Valid license from HP required for possession, use, or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commerci...
Page 3 - Contents; Disaster Tolerance and Recovery in a Serviceguard Cluster
3 Contents 1. Disaster Tolerance and Recovery in a Serviceguard Cluster Evaluating the Need for Disaster Tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14What is a Disaster Tolerant Architecture? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Und...
Page 4 - Disaster Scenarios and Their Handling
Contents 4 Creating a Multiple Disk Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 To Create and Assemble an MD Device. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Creating Volume Groups and Configuring VG Exclusive Activa...
Page 7 - Printing History; NOTE
7 Printing History The printing date and part number indicate the current edition. The printing date changes when a new edition is printed. (Minor corrections and updates which are incorporated at reprint do not cause the date to change.) The part number changes when extensive technical changes are ...
Page 8 - HP Printing Division:
8 HP Printing Division: Business Critical Computing Business Unit Hewlett-Packard Co. 19111 Pruneridge Ave. Cupertino, CA 95014
Page 9 - Preface; Intended Audience; It is assumed that you are familiar with the following topics; Document Organization; The chapters of this guide include:
9 Preface This guide introduces the concept of Extended Distance Clusters (XDC). It describes how to configure and manage HP Serviceguard Extended Distance Clusters for Linux and the associated Software RAID functionality. In addition, this guide includes information on a variety of Hewlett-Packard ...
Page 11 - The following documents contain additional useful information:; Problem Reporting
11 Preface Related Publications The following documents contain additional useful information: • Clusters for High Availability: a Primer of HP Solutions, Second Edition. Hewlett-Packard Professional Books: Prentice Hall PTR, 2001 (ISBN 0-13-089355-2) • Designing Disaster Tolerant HA Clusters Using ...
Page 13 - Disaster Tolerance and Recovery in a Serviceguard Cluster; Chapter 1
Disaster Tolerance and Recovery in a Serviceguard Cluster Chapter 1 13 1 Disaster Tolerance and Recovery in a Serviceguard Cluster This chapter introduces a variety of Hewlett-Packard high availability cluster technologies that provide disaster tolerance for your mission-critical applications. It is...
Page 14 - Evaluating the Need for Disaster Tolerance
Disaster Tolerance and Recovery in a Serviceguard Cluster Evaluating the Need for Disaster Tolerance Chapter 1 14 Evaluating the Need for Disaster Tolerance Disaster tolerance is the ability to restore applications and data within a reasonable period of time after a disaster. Most people think of fi...
Page 16 - What is a Disaster Tolerant Architecture?; High Availability Architecture.; pkg A fails
Disaster Tolerance and Recovery in a Serviceguard Cluster What is a Disaster Tolerant Architecture? Chapter 1 16 What is a Disaster Tolerant Architecture? In a Serviceguard cluster configuration, high availability is achieved by using redundant hardware to eliminate single points of failure. This pr...
Page 18 - Understanding Types of Disaster Tolerant Clusters; Extended Distance Clusters
Disaster Tolerance and Recovery in a Serviceguard Cluster Understanding Types of Disaster Tolerant Clusters Chapter 1 18 Understanding Types of Disaster Tolerant Clusters To protect against multiple points of failure, cluster components must be geographically dispersed: nodes can be put in different...
Page 26 - — Maximum distance that guarantees a network latency of no more
Disaster Tolerance and Recovery in a Serviceguard Cluster Understanding Types of Disaster Tolerant Clusters Chapter 1 26 • Disk resynchronization is independent of CPU failure (that is, if the hosts at the primary site fail but the disk remains up, the disk knows it does not have to be resynchronize...
Page 27 - Continental Cluster
Disaster Tolerance and Recovery in a Serviceguard Cluster Understanding Types of Disaster Tolerant Clusters Chapter 1 27 "objective" can be set for the recovery point such that if data is updated for a period less than the objective, automated failover can occur and a package will start. If ...
Page 28 - Data Center A
Disaster Tolerance and Recovery in a Serviceguard Cluster Understanding Types of Disaster Tolerant Clusters Chapter 1 28 Figure 1-6 Continental Cluster Continentalclusters provides the flexibility to work with any data replication mechanism. It provides pre-integrated solutions that use HP StorageWo...
Page 29 - — Storage subsystems implemented by CLX are also pre-integrated
Disaster Tolerance and Recovery in a Serviceguard Cluster Understanding Types of Disaster Tolerant Clusters Chapter 1 29 Benefits of Continentalclusters • You can virtually build data centers anywhere and still have the data centers provide disaster tolerance for each other. Since Continentalcluster...
Page 30 - Continental Cluster With Cascading Failover
Disaster Tolerance and Recovery in a Serviceguard Cluster Understanding Types of Disaster Tolerant Clusters Chapter 1 30 replicate the data between two data centers. HP provides a supported integration toolkit for Oracle 8i Standby DB in the Enterprise Cluster Management Toolkit (ECMT). • RAC is sup...
Page 31 - Key Benefit
Disaster Tolerance and Recovery in a Serviceguard Cluster Understanding Types of Disaster Tolerant Clusters Chapter 1 31 Table 1-1 Comparison of Disaster Tolerant Cluster Solutions Attributes Extended Distance Cluster CLX Continentalclusters (HP-UX only) Key Benefit Excellent in “normal” operations,...
Page 34 - nodes for this release.
Disaster Tolerance and Recovery in a Serviceguard Cluster Understanding Types of Disaster Tolerant Clusters Chapter 1 34 Application Failover type Automatic (no manual intervention required). Automatic (no manual intervention required). Semi-automatic (user must “push the button” to initiate recover...
Page 35 - Dark Fiber
Disaster Tolerance and Recovery in a Serviceguard Cluster Understanding Types of Disaster Tolerant Clusters Chapter 1 35 Data Replication Link Dark Fiber Dark Fiber Continuous Access over IP Continuous Access over ATM WAN LAN Dark Fiber (pre-integrated solution) Continuous Access over IP (pre-integr...
Page 36 - CLX XP or CLX EVA
Disaster Tolerance and Recovery in a Serviceguard Cluster Understanding Types of Disaster Tolerant Clusters Chapter 1 36 DTS Software/Licenses Required SGLX + XDC SGLX + CLX XP or CLX EVA SG + Continentalclusters + (Metrocluster Continuous Access XP or Metrocluster Continuous Access EVA or Metroclus...
Page 37 - Disaster Tolerant Architecture Guidelines; Protecting nodes through geographic dispersion; Protecting Nodes through Geographic Dispersion
Disaster Tolerance and Recovery in a Serviceguard Cluster Disaster Tolerant Architecture Guidelines Chapter 1 37 Disaster Tolerant Architecture Guidelines Disaster tolerant architectures represent a shift away from the massive central data centers and towards more distributed data processing facilit...
Page 38 - Protecting Data through Replication; Off-line Data Replication
Disaster Tolerance and Recovery in a Serviceguard Cluster Disaster Tolerant Architecture Guidelines Chapter 1 38 Protecting Data through Replication The most significant losses during a disaster are the loss of access to data, and the loss of data itself. You protect against this loss through data r...
Page 40 - Replication consumes no additional CPU.; Disadvantages of physical replication in hardware are:
Disaster Tolerance and Recovery in a Serviceguard Cluster Disaster Tolerant Architecture Guidelines Chapter 1 40 Figure 1-7 Physical Data Replication MD Software RAID is an example of physical replication done in the software; a disk I/O is written to each array connected to the node, requiring the ...
Page 42 - Writes are synchronous unless the link or disk is down.; Disadvantages of physical replication in software are:; Additional hardware is required for the cluster.; Logical Data Replication
Disaster Tolerance and Recovery in a Serviceguard Cluster Disaster Tolerant Architecture Guidelines Chapter 1 42 • Because there are multiple read devices, that is, the node has access to both copies of data, there may be improvements in read performance. • Writes are synchronous unless the link or ...
Page 43 - node 1a; Network
Disaster Tolerance and Recovery in a Serviceguard Cluster Disaster Tolerant Architecture Guidelines Chapter 1 43 Figure 1-8 Logical Data Replication Advantages of using logical replication are: • The distance between nodes is limited only by the networking technology. • There is no additional hardwa...
Page 44 - For performance and data currency—physical data replication.; Using Alternative Power Sources
Disaster Tolerance and Recovery in a Serviceguard Cluster Disaster Tolerant Architecture Guidelines Chapter 1 44 • If the primary database fails and is corrupt, which results in the replica taking over, then the process for restoring the primary database so that it can be used as the replica is comp...
Page 45 - Creating Highly Available Networking
Disaster Tolerance and Recovery in a Serviceguard Cluster Disaster Tolerant Architecture Guidelines Chapter 1 45 Figure 1-9 Alternative Power Sources Housing remote nodes in another building often implies they are powered by a different circuit, so it is especially important to make sure all nodes a...
Page 46 - — T1 and T3: low end
Disaster Tolerance and Recovery in a Serviceguard Cluster Disaster Tolerant Architecture Guidelines Chapter 1 46 Disaster Tolerant Local Area Networking Ethernet networks can also be used to connect nodes in a disaster tolerant architecture within the following guidelines: • Each node is connected t...
Page 47 - Disaster Tolerant Cluster Limitations
Disaster Tolerance and Recovery in a Serviceguard Cluster Disaster Tolerant Architecture Guidelines Chapter 1 47 Disaster Tolerant Cluster Limitations Disaster tolerant clusters have limitations, some of which can be mitigated by good planning. Some examples of MPOF that may not be covered by disast...
Page 48 - Managing a Disaster Tolerant Environment; Implement automated or manual recovery?
Disaster Tolerance and Recovery in a Serviceguard Cluster Managing a Disaster Tolerant Environment Chapter 1 48 Managing a Disaster Tolerant Environment In addition to the changes in hardware and software to create a disaster tolerant architecture, there are also changes in the way you manage the en...
Page 49 - Who manages the nodes in the cluster and how are they trained?
Disaster Tolerance and Recovery in a Serviceguard Cluster Managing a Disaster Tolerant Environment Chapter 1 49 Even if recovery is automated, you may choose to, or need to recover from some types of disasters with manual recovery. A rolling disaster, which is a disaster that happens before the clus...
Page 50 - Additional Disaster Tolerant Solutions Information; On-line versions of HA documentation are available at
Disaster Tolerance and Recovery in a Serviceguard Cluster Additional Disaster Tolerant Solutions Information Chapter 1 50 Additional Disaster Tolerant Solutions Information On-line versions of HA documentation are available at http://docs.hp.com -> High Availability -> Serviceguard for Linux. ...
Page 51 - Chapter 2; “Types of Data Link for Storage and Networking” on page 52
Building an Extended Distance Cluster Using Serviceguard and Software RAID Chapter 2 51 2 Building an Extended Distance Cluster Using Serviceguard and Software RAID Simple Serviceguard clusters are usually configured in a single data center, often in a single room, to provide protection against fail...
Page 52 - Gigabit Ethernet Twisted Pair
Building an Extended Distance Cluster Using Serviceguard and Software RAID Types of Data Link for Storage and Networking Chapter 2 52 Types of Data Link for Storage and Networking Fibre Channel technology lets you increase the distance between the components in an Serviceguard cluster, thus making i...
Page 54 - Quorum Server running in a Serviceguard cluster
Building an Extended Distance Cluster Using Serviceguard and Software RAID Two Data Center and Quorum Service Location Architectures Chapter 2 54 • Fibre Channel Direct Fabric Attach (DFA) is recommended over Fibre Channel Arbitrated loop configurations, due to the superior performance of DFA, espec...
Page 56 - arbitration is not allowed in this configuration.
Building an Extended Distance Cluster Using Serviceguard and Software RAID Two Data Center and Quorum Service Location Architectures Chapter 2 56 There are no requirements for the distance between the Quorum Server Data center and the Primary Data Centers, however it is necessary to ensure that the ...
Page 57 - Rules for Separate Network and Data Links; No routing is allowed for the networks between the data centers.
Building an Extended Distance Cluster Using Serviceguard and Software RAID Rules for Separate Network and Data Links Chapter 2 57 Rules for Separate Network and Data Links • There must be less than 200 milliseconds of latency in the network between the data centers. • No routing is allowed for the n...
Page 61 - Configuring your Environment for Software RAID; Chapter 3; “Understanding Software RAID” on page 62
Configuring your Environment for Software RAID Chapter 3 61 3 Configuring your Environment for Software RAID The previous chapters discussed conceptual information on disaster tolerant architectures and procedural information on creating an extended distance cluster. This chapter discusses the proce...
Page 62 - Understanding Software RAID; Install the extended distance cluster software.
Configuring your Environment for Software RAID Understanding Software RAID Chapter 3 62 Understanding Software RAID Redundant Array of Independent Disks (RAID) is a mechanism that provides storage fault tolerance and, occasionally, better performance. Software RAID is designed on the concept of RAID...
Page 63 - Installing the Extended Distance Cluster Software; Supported Operating Systems; Red Hat 4 U3 or later; Prerequisites; HP Serviceguard for Linux A 11.16.07 or higher; Installing XDC
Configuring your Environment for Software RAID Installing the Extended Distance Cluster Software Chapter 3 63 Installing the Extended Distance Cluster Software This section discusses the supported operating systems, prerequisites and the procedures for installing the Extended Distance Cluster softwa...
Page 64 - Verifying the XDC Installation
Configuring your Environment for Software RAID Installing the Extended Distance Cluster Software Chapter 3 64 Complete the following procedure to install XDC: 1. Insert the product CD into the drive and mount the CD. 2. Open the command line interface. 3. If you are installing XDC on Red Hat 4, run ...
Page 65 - will be listed. The
Configuring your Environment for Software RAID Installing the Extended Distance Cluster Software Chapter 3 65 In the output, the product name, xdc -A.01.00-0 will be listed. The presence of this file verifies that the installation is successful.
Page 66 - Configuring the Environment; Configure multipath for storage
Configuring your Environment for Software RAID Configuring the Environment Chapter 3 66 Configuring the Environment After setting up the hardware as described in the Extended Distance Cluster Architecture section and installing the Extended Distance Cluster software, complete the following steps to ...
Page 69 - Configuring Multiple Paths to Storage; You need to register with Hewlett-Packard to access this site.
Configuring your Environment for Software RAID Configuring Multiple Paths to Storage Chapter 3 69 Configuring Multiple Paths to Storage HP requires that you configure multiple paths to the storage device using the QLogic HBA driver as it has inbuilt multipath capabilities. Use the install script wit...
Page 70 - The value to be set for; second
Configuring your Environment for Software RAID Configuring Multiple Paths to Storage Chapter 3 70 The QLogic cards are configured to hold up any disk access and essentially hang for a time period which is greater than the cluster reformation time when access to a disk is lost. This is achieved by al...
Page 71 - Using Persistent Device Names; may be renamed to
Configuring your Environment for Software RAID Using Persistent Device Names Chapter 3 71 Using Persistent Device Names When there is a disk related failure and subsequent reboot, there is a possibility that the devices are renamed. Linux names disks in the order they are found. The device that was ...
Page 72 - Creating a Multiple Disk Device; To Create and Assemble an MD Device
Configuring your Environment for Software RAID Creating a Multiple Disk Device Chapter 3 72 Creating a Multiple Disk Device As mentioned earlier, the first step for enabling Software RAID in your environment is to create the Multiple Disk (MD) device using two underlying component disks. This MD dev...
Page 75 - Found duplicate PV 9w3TIxKZ6lFRqWUmQm9tlV5nsdUkTi4i: using
Configuring your Environment for Software RAID Creating Volume Groups and Configuring VG Exclusive Activation on the MD Mirror Chapter 3 75 Found duplicate PV 9w3TIxKZ6lFRqWUmQm9tlV5nsdUkTi4i: using /dev/sde not /dev/sdf With this error, you cannot create a new volume group on /dev/md0 . As a result...
Page 77 - Creating and Editing the Package Control Scripts; To Create a Package Control Script; xdcmd
Configuring your Environment for Software RAID Configuring the Package Control Script and RAID Configuration File Chapter 3 77 # Specify the method of activation and deactivation for md. # Leave the default (RAIDSTART="raidstart", "RAIDSTOP="raidstop") if you want # md to be star...
Page 78 - Editing the raid.conf File
Configuring your Environment for Software RAID Configuring the Package Control Script and RAID Configuration File Chapter 3 78 To Edit the XDC_CONFIG FILE parameter In addition to modifying the DATA_REP variable, you must also set XDC_CONFIG_FILE to specify the raid.conf file for this package. This ...
Page 80 - has elapsed, a package in; values. The discussion below is based on the timeline; RPO Target Definitions
Configuring your Environment for Software RAID Configuring the Package Control Script and RAID Configuration File Chapter 3 80 For example, let us assume that the data storage links in Figure 1-4 fail before the heartbeat links fail. In this case, after the time specified by Link_Down_Timeout has el...
Page 81 - parameter is greater
Configuring your Environment for Software RAID Configuring the Package Control Script and RAID Configuration File Chapter 3 81 Now consider an XDC configuration such as that shown in Figure 1-3 (DWDM links between data centers). If DC1 fails such that links A and B both fail simultaneously, and DC1'...
Page 83 - file to all nodes in the cluster. All the
Configuring your Environment for Software RAID Configuring the Package Control Script and RAID Configuration File Chapter 3 83 • RAID_MONITOR_INTERVAL This parameter defines the time interval, in seconds, the raid monitor script waits between each check to verify accessibility of both component devi...
Page 85 - Disaster Scenarios and Their Handling; Chapter 4
Disaster Scenarios and Their Handling Chapter 4 85 4 Disaster Scenarios and Their Handling The previous chapters provided information on deploying Software RAID in your environment. In this chapter, you will find information on how Software RAID addresses various disaster scenarios. All the disaster...
Page 86 - Clusters” on page 18 of this document.; Disaster Scenario
Disaster Scenarios and Their Handling Chapter 4 86 The following table lists all the disaster scenarios that are handled by the Extended Distance Cluster software. All the scenarios assume that the setup is the same as the one described in “Extended Distance Clusters” on page 18 of this document. Ta...
Page 88 - Disaster Scenarios and Their Handling (Continued)
Disaster Scenarios and Their Handling Chapter 4 88 A package (P1) is running on a node (Node 1). The package uses a mirror (md0) that consists of two storage components - S1 (local to Node 1 - /dev/hpdev/mylink-sde ) and S2 (local to Node 2) Data center 1 that consists of Node 1 and P1 experiences a...
Page 97 - Appendix A; Managing an MD Device; Following are the topics discussed in this chapter:
Managing an MD Device Appendix A 97 A Managing an MD Device This chapter includes additional information on how to manage the MD device. For the latest information on how to manage and MD device, see The Software-RAID HOWTO manual available at: http://www.tldp.org/HOWTO/Software-RAID-HOWTO.html Foll...
Page 98 - Viewing the Status of the MD Device; file. Following is the output; A status report obtained using the
Managing an MD Device Viewing the Status of the MD Device Appendix A 98 Viewing the Status of the MD Device After creating an MD device, you can view its status. By doing so, you can remain informed of whether the device is clean, up and running, or if there are any errors. To view the status of the...
Page 99 - Stopping the MD Device; file. To stop the MD; Stopping the MD Device; To stop the MD device
Managing an MD Device Stopping the MD Device Appendix A 99 Stopping the MD Device After you create an MD device, it begins to run. You need to stop the device and add the configuration into the raid.conf file. To stop the MD device, run the following command: # mdadm -S <md_device name> When y...
Page 100 - Starting the MD Device; To start the MD device, run the following command:; Starting the MD Device; To start the MD device
Managing an MD Device Starting the MD Device Appendix A 100 Starting the MD Device After you create an MD device, you would need to stop and start the MD device to ensure that it is active. You would not need to start the MD device in any other scenario as this is handled by the XDC software. To sta...
Page 101 - Removing and Adding an MD Mirror Component Disk
Managing an MD Device Removing and Adding an MD Mirror Component Disk Appendix A 101 Removing and Adding an MD Mirror Component Disk There are certain failure scenarios, where you would need to manually remove the mirror component of an MD device and add it again later. For example, if links between...
Page 102 - To remove a failed MD component disk from; Adding a Mirror Component Device; Adding a new disk as an MD component to; Following is an example of the status message displayed in the
Managing an MD Device Removing and Adding an MD Mirror Component Disk Appendix A 102 Example A-3 Removing a failed MD component disk from /dev/md0 array To remove a failed MD component disk from /dev/md0 , run the following command: # mdadm –-remove /dev/md0 /dev/hpdev/sde Following is an example of...
Page 103 - Index
Index 103 Aasynchronous data replication , 39 Ccluster extended distance , 22 FibreChannel , 52 metropolitan , 23 wide area , 27 cluster maintenance , 49 configuring , 46 disaster tolerant Ethernet networks , 46 disaster tolerant WAN , 46 consistency of data , 38 continental cluster , 27 currency of...
