How to setup Linux loadbalancer using: LVS + ldirectord + heartbeat 2 on CentOS

Steps to setup heartbeat-2.1.4 on centos 5.2.

In this document you will find procedure how to setup lb01 and lb02 as loadbalancer for 2 or more web servers. In this example I have used only 2 webservers i.e web01 and web02. In total there are 4 virtual servers. All four servers are virtual servers. I am using Xen as dom0. 

After installing base operating system (I have installed CentOS 5.2) we need to install following packages on both loadbalancer linux boxes. As heartbeat is not available with standard packages you will need to download it.

you can also download it from http://download.opensuse.org/repositories/home:/ikedaj:/ossvert-stable-hb214/RHEL_5/x86_64/

[root@lb01]#yum install xorg-x11-fonts-truetype bitmap-fonts xorg-x11-xauth libglade2 pygtk2-libglade
[root@lb01]#yum install xorg-x11-fonts-75dpi redhat-lsb perl-libwww-perl perl-Net-SSLeay ipvsadm net-snmp-libs
[root@lb01]#rpm -Uvh Perl-MailTools
[root@lb01]#rpm -Uvh heartbeat-stonith-2.1.4-2.1.x86_64.rpm heartbeat-pils-2.1.4-2.1.x86_64.rpm
[root@lb01]#rpm -Uvh heartbeat-2.1.4-2.1.x86_64.rpm heartbeat-ldirectord-2.1.4-2.1.x86_64.rpm
 

Host name and IP addresses for servers are as below.

xen01 – a citrix xen host – 10.10.10.99
lb01 – a virtual machine – 10.10.10.100
web01 – a virtual machine – 10.10.10.101
web02 – a virtual machine – 10.10.10.102
 
After installing required packages now its time to configure heartbeat. in this particular case we have only 1 loadbalancer. our ha.cf configuration file will be as below.
crm on
udpport 695
bcast eth0
node lb01 lb02

Also we will need to generate authkey to setup /etc/ha.d/authkeys file required by heartbeat. to do so use this shell hack.

#( echo -ne “auth 1\n1 md5 “; dd if=/dev/urandom bs=512 count=1| openssl md5) > /etc/ha.d/authkeys

your authkeys config file will look like.

auth 1
1 md5  50eba111c1d6b915cec28ff

Note : Also enable ip_forwarding on loadbalancer(s) and actual web servers to allow packet forwarding. Now lets configure ldirector. configuration file for ldirectord is /etc/ha.d/ldirectord.cf. For this particular scenario ldirectord.cf looks like.

checktimeout=10
checkinterval=20
autoreload=yes
logfile=”/var/log/ldirectord.log”
quiescent=yes
virtual=10.10.10.91:80
      real=10.10.10.101:80 gate
      real=10.10.10.102:80 gate
      service=http
      request=”/lb.html”
      receive=”still alive”
      scheduler=wlc
      protocol=tcp
      checktype=negotiate
      persistent = 10
      netmask = 255.255.255.255

 

Now start ldirectord daemon. /etc/init.d/ldirectord start . Correct if it displays any error.

Now configure heartbeat gui interface using hb_gui. login to loadbalancer using ssh with user hacluster.

ssh -X hacluster@lb01 #Note : -X is for X forwarding -bash-3.2$ hb_gui& 
#Note: This will popup another window with Title “Linux HA Management Client”.

 we need to configure few things using this gui interface. Click on Connection -> Login on top left corner & enter password for hacluster user to login.

Create resource group “load_balancer”

A resource group places constraints on the resources to make their management easier. It enforces that resources within the group run on the same node and have to start in a specific order; from the top to the bottom and stop in the reverse order.

 

  1.  Create a group named ” load_balancer “, leave “colocation” and “ordered” as “true”.
  2. Add the resource ” IPAddr2
  3. Add the value for the parameter “ip” 10.10.10.91. This is the virtual IP address 10.10.10.91 that clients will connect to.
  4. Add the parameter ” lvs_support ” with a value of ” true “
  5. Add an operation named ” monitor “, interval “20”, timeout “10” start delay “0” On fail “restart”.

 

 

Now add Resources.

  1. Add a native resource named ” ldirectord ” with the class ” ocf/heartbeat ” that belongs to the group “load_balancer“.
  2. Add the parameter ” configfile ” with the value of ” /etc/ha.d/ldirectord.cf
  3. Add an operation name ” monitor “, interval “20”, timeout “10” start delay “0” On fail “restart”.
  4. Start the resource group. Highlight the “load_balancer” group and click on the ” Play ” button on the top bar.
The resource group should come up with a green light. After you complete this, our loadbalancer is almost ready now.

Following steps are to setup webservers configuration that is configuration for real servers

Virtual interface :

Edit /etc/sysconfig/network/ifcfg-lo:1 and add: DEVICE=lo:1 IPADDR=10.10.10.91 NETMASK=255.255.255.255 ONBOOT=yes

On webservers, we need to create lb.html file with “still alive” in it under document root. In this case /var/www/html/lb.html file is set with “still alive”.

arptables configuration :

We need to install package called arptables_jf so that we can setup arptables for virtual IP (loadbalanced IP Address) on both webservers we need to install arptables_jf yum install arptables_jf after installing it setup arptable using following commands.

On web01 following rules are required.

arptables -A IN -i lo -d 10.10.10.91 -j DROP
arptables -A OUT -o eth0 -s 10.10.10.91 -j mangle –mangle-ip-s 10.10.10.101
arptables -A OUT -o any -s 10.10.10.91 -j DROP

On web02 following rules are required.

arptables -A IN -i lo -d 10.10.10.91 -j DROP
arptables -A OUT -o eth0 -s 10.10.10.91 -j mangle –mangle-ip-s 10.10.10.102
arptables -A OUT -o any -s 10.10.10.91 -j DROP

Also you can see more details by using crm_mon command on loadbalance shell prompt.

======== Last updated: Wed Oct 14 13:39:35 2009 Current DC: lb01 (777c9229-00af-00a4-9dc1-6a55f4f000ea) 1 Nodes configured. 1 Resources configured. ======== Node: ukibistagelb01 (777c9229-00af-00a4-9dc1-6a55f4f000ea): online Resource Group: load_balancer vip97 (ocf::heartbeat:IPaddr2): Started lb01 ldirector (ocf::heartbeat:ldirectord): Started lb01

To findout more about which webserver is serving how much traffic use “ipvsadm –l –stats” on loadbalancer command prompt.

[root@lb01 ha.d]# ipvsadm -l –stats IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes -> RemoteAddress:Port TCP 10.10.10.91:http 37 185 0 19683 0 -> 10.10.10.101:http 17 85 0 11963 0 -> 10.10.10.102:http 20 100 0 7720 0

Thanks useful links http://www.novell.com/communities/node/4846/load-balancing-howto-lvs-ldi…

Share this post

Post Comment