2.1. pulse

This is the controlling process that starts the other daemons as needed. It is started on the LVS routers by the /etc/rc.d/init.d/pulse script, normally at boot time . Through pulse, which implements a simple heartbeat, the inactive LVS router determines the health of the active router and whether to initiate failover.

2.2. lvs

The lvs daemon runs on the LVS routers. It reads the configuration file and calls ipvsadm to build and maintain the IPVS routing table.

2.3. nanny

The nanny monitoring daemon runs on the active LVS router. Through this daemon, the active router determines the health of each real server and gets its workload. A separate process runs for each real server used by each virtual server.

2.4. /etc/lvs.cf

This is the LVS cluster configuration file. Directly or indirectly, all daemons get their configuration information from this file.

2.5. piranha

The GUI tool for monitoring, configuring, and administering an LVS cluster. Normally this is the tool you will use to maintain /etc/lvs.cf, restart the running daemons, and monitor an LVS cluster.

2.6. ipvsadm

This tool updates the IPVS routing table in the kernel. The lvs daemon sets up and administers an LVS cluster by calling ipvsadm to add, change or delete entries in the IPVS routing table.

4.1. LVS Routers

A Linux server that will be the primary LVS router is required. This machine needs two network adapters, one connecting it to the public network and the other to a private network of real servers.

If you want failover capability, a second Linux server that will be the backup LVS router is required. This machine also needs two network adapters connecting it to the public network and to the private network of real servers. The adapter devices must match on the two LVS routers. Thus, if devices eth0 and eth1 on the primary LVS router connect it to public and private network, respectively, then these same devices on the backup LVS router must connect it to the public and private network. Note that the backup LVS router is a purely hot standby machine.

4.2. Real Servers

The private network to which the LVS routers are connected includes a variable number of Web/FTP server hosts. Jobs addressed to virtual servers are redirected to these real servers, which may be any kind of computing platform running any operating system or Web server.

During configuration, you assign a weight to each real server. This is an integer reflecting each server's processing capacity (based on memory, processor speed, number of processors, etc.) relative to that of the others. It is the ratio (2/1, 20/10, 200/100) that is significant. For example, a weight of 2000 assigned to a server indicates that it has twice the computing power of a server assigned a weight of 1000. The assigned weight, which may be dynamically adjusted based on load information, is used by two available job scheduling algorithms (described in Table 1). You should be prepared to assign accurate weights.

8.1. Editing the Configuration File

The /etc/lvs.cf file has three sections. The global section, described in Table 3, sets up the LVS routers and specifies networking and heartbeat parameters. There is one set of parameters for the cluster. The per-virtual-server section, described in Table 4, defines virtual server addresses, sets up the associations between virtual servers and real servers, and specifies job-scheduling parameters. There is a separate set of parameters for each defined virtual server. The per-real-server section, described in Table 5, defines the real servers routed to from each virtual server. There is one set of these parameters for each virtual server.

8.2. Using the Piranha Configuration Tool

To start piranha, become root and type piranha &. When you do, its main window opens. The window has four tabs:

Controls/Monitoring -- The initial tab. Used to start/stop/restart the cluster daemons and monitor the runtime status. See Section 8.2.1.

Global Settings -- Use to set the IP address of the primary LVS router and NAT router. See Section 8.2.2.

Redundancy -- Use to set the address of the backup LVS router and set the heartbeating parameters. See Section 8.2.3.

Virtual Servers -- Use to set service addresses and set up routing between service addresses and real Web/FTP server hosts. See Section 8.2.4.

Piranha windows have one or more of these buttons along the bottom:

OK -- Apply changes and exit from the window.

Apply -- Apply changes without exiting the window.

Close -- Exit from the window without applying changes.

9.1. Preliminary Setup

1. >From the webmaster, obtain a virtual server IP address. In our example this will be 1.2.3.1. Requests for service at the LVS cluster will be addressed to a fully-qualified domain name associated with this address.

2. Locate 5 servers and designate their roles: 1 primary LVS router, 1 backup LVS router, 3 real servers. The LVS routers must be Linux boxes running Red Hat 6.1 or later. The real servers may be any platform running any operating system and Web server.

   Steps 3-7 set up the LVS routers.

3. On each LVS router, install two ethernet adapter cards, eth0 and eth1. Create a public IP interface on eth0 and a private IP interface on eth1. The public interface device (eth0) is the heartbeat device. The virtual server address is aliased to this device.

   	Primary node	Backup node
   eth0	1.2.3.2	1.2.3.3
   eth1	192.168.1.1	192.168.1.2

4. Designate an IP address (192.168.1.254) for the router device (eth1) connecting the active LVS router to the private network. This floating IP address will be aliased to the router device as eth1:1, and will be the gateway to the private network and the default route used by each real server to communicate with the active router.

5. On each LVS router:

   1. Enable packet forwarding. To do this at system boot, make sure the file /etc/sysconf/network contains the line FORWARD_IPV4=yes. To enable packet forwarding without rebooting, as root issue this command:

      
          echo "1" > /proc/sys/net/ipv4/ip_forward

      
      
      

   2. Enable packet defragmenting. To do this at system boot, make sure the file /etc/sysconf/network contains the line DEFRAG_IPV4=yes. To enable packet defragmenting without rebooting, as root issue this command:

      
          echo "1" > /proc/sys/net/ipv4/ip_always_defrag

      
      
      

   3. Masquerade the private network. Issue this command and put it in /etc/rc.d/rc.local:

      
           ipchains -A forward -j MASQ -s 192.168.1.0/24 -d 0.0.0.0
      

      
      
      

6. Decide whether to use the rsh or ssh families for synchronizing LVS cluster files. Verify that your choice is installed such that the LVS routers can log in to one another as root without administrator intervention (see Table 2). In this example, we will choose rsh.

7. On each LVS router, verify that the LVS cluster software is installed (see Section 7.

   Steps 8-11 set up real servers.

8. On each real server, install an ethernet network card, eth0, create an IP address on the same private subnet as in Step 3, and assign a weight to each server indicating its processing capacity relative to that of the others. In this example, rs1 has twice the capacity (two processors) of rs2 and rs3.

   	rs1	rs2	rs3
   eth0	192.168.1.3	192.168.1.4	192.168.1.5
   weight	2000	1000	1000

9. On each real server, verify that the address named in Step 4 (192.168.1.254) is its default route for communicating with the active LVS router.

10. Decide which program (uptime, ruptime, rup) will be used by the active router to monitor the workload on the real servers. If you choose uptime, each LVS router must be able to connect with each real server without administrator intervention, using the tool family you selected in Step 6. See Table 2 for general enablement instructions. If the selected tool cannot be enabled (for example, one of the real servers is an NT box), the scheduling algorithms that use dynamic load information will still work but the user-assigned weights will be statically applied rather than dynamically adjusted based on load.

11. Verify that each real server runs an installed and configured httpd server. Note that the real servers must listen on the same port (80 in the example) as the corresponding virtual server.

12. Verify (for example, using telnet or ping) that each real server can reach hosts on the public LAN. If a real server on the private network cannot reach a host on your LAN, this probably indicates a communication failure between the server and the active router. See Steps 8 and 9.

13. Determine the runtime parameters. For some of these, you may need to experiment over time to obtain optimal values. In this example, we will use the values listed.

    Value	Parameter Description
    1050	Number of the heartbeat listening port on the primary and backup routers.
    2	Number of seconds between heartbeats.
    10	Number of seconds to wait for a non-responding router to respond before initiating failover.
    10	Number of seconds to wait for a non-responding real server to respond before removing it from the routing table.
    180	When a real server that has been removed from the routing table starts responding again, wait this number of seconds before re-adding the server to the routing table.
    wlc	Use the Weighted least-connections load-balancing algorithm (assign more jobs to servers that are least busy relative to their load-adjusted weight). See Table 1 for a description of the choices.
    http	Application. The alternative is ftp.
    80	Virtual server port number. The listening port selected for the virtual server is used on the real servers as well.

Now we are ready to implement the example. You can do this from a shell as explained in the next section. Or you can use the GUI configuration tool as explained in Section 9.3.

9.2. Implementing the Example from the Shell

1. With your favorite editor, open /etc/lvs.cf and set the values shown below. The number on the right is a link to the step in Section 9.1 that discusses this setting.

   # Global section
   primary = 1.2.3.2                    3
   backup =  1.2.3.3                    3
   keepalive = 2                        13
   deadtime = 10                        13
   heartbeat_port = 1050                13
   rsh_command = rsh                    6
   network = nat
   nat_router = 192.168.1.254 eth1:1    4
   # Per-virtual-server section
   virtual server vs1 {                  8      
    address = 1.2.3.1                   1
    active = 1
    load_monitor = ruptime              10
    timeout = 10                        13
    reentry = 180                       13
    port = 80                           13
    scheduler = wlc                     13
   # Per-real-server section
    server rs1 {                        8
     address = 192.168.1.3              8
     active = 1
     weight = 2000                      8
    }
    server rs2 {                        8
     address = 192.168.1.4              8
     active = 1
     weight = 1000                      8
    }
    server rs3 {                        8
     address = 192.168.1.5              8
     active = 1
     weight = 1000                      8
    }
   }
           

   
   
   

2. Copy the edited configuration file to the backup router.

3. On the primary router, start the pulse daemon with this command:

                 /etc/rc.d/init.d/pulse start                
             

4. Start pulse on the backup router.

9.3. Implementing the Example with Piranha

1. On the primary LVS router, as root, type piranha & to start the GUI configuration tool.

2. Click the Global Settings tab and enter the values shown:

   Field	Enter:	See Step:
   Primary LVS server IP	1.2.3.2	3
   NAT Router IP	192.168.1.254	4
   NAT Router Device	eth1:1	4
   Sync tool	rsh	6

3. Click the Redundancy tab and enter the values shown:

   Field	Enter:	See Step:
   Enable Redundant server	(select)
   Redundant LVS server IP	1.2.3.3	3
   Heartbeat interval	2	13
   Assume dead after	10	13
   Heartbeat port	1050	13

4. Click the Virtual Servers tab. On the tab, click Add, then Edit, and enter the values shown:

   Field	Enter:	See Step:
   Name	vs1
   Application	http	13
   Port	80	13
   Address	1.2.3.1	1
   Device	eth0:1	3
   Re-entry time	180	13
   Service timeout	10	13
   Load monitoring tool	ruptime	10
   Scheduling	weighted least-connections	13

5. Click the Real Servers tab. On the tab, click Add, then Edit, and enter the values shown:

   Field	Enter:	See Step:
   Name	rs1	8
   Address	192.168.1.3	8
   Weight	2000	8

6. Repeat Step 5 for rs2 (192.168.1.4) and rs3 (192.168.1.5). For them, enter a weight of 1000, which will indicate that rs1 has twice the computing capacity of rs2 and rs3.

7. Click Close to return to the Real Servers tab. There, select each real server and click Activate to activate it.

8. Click Close to return to the Virtual Servers tab. There, select the vs1 virtual server and click Activate to activate it.

9. Click the Controls/Monitoring tab and:

   1. Click the Start button (name changes to Stop).

   2. If you check "Add pulse daemon to this runlevel" the cluster will be started at boot time and, on startup of piranha, the button above will usually be labeled Stop.

10. Log in to the backup router and start pulse with this command:

    
                  /etc/rc.d/init.d/pulse start
              

    
    
    

If you enter the values exactly as shown above, piranha generates at the end of the configuration file the structures shown below for this virtual server and its real servers.

network = nat
nat_router = 192.168.1.254 eth1:1

virtual vs1 {
        address = 1.2.3.1 eth0:1
        active = 1
        scheduler = wlc
        load_monitor = ruptime
        timeout = 5

        server rs1 {
                address = 192.168.1.3
                active = 1
                weight = 2000
        }
        server rs2 {
                address = 192.168.1.4
                active = 1
                weight = 1000
        }
        server rs3 {
                address = 192.168.1.5
                active = 1
                weight = 1000
        }
}