Lab 5 Explicit Proxy Performance, Load Balancing & Redundancy
Objectives
The purpose of this lab is to demonstrate both high availability and performance using virtual IPs coupled with DNS round robin as a load sharing technique including failover.
Throughout the lab all three WCGs will be configured in a managed cluster, correctly allocated virtual IPs and subsequently configured to be addressed via DNS round robin, allowing load balancing of the WCGs with active/active clustering. You will test
functionality and configuration at each stage of the process.
Lab Environment
Managed vs. Full Clustering
Management Clustering
Management-only mode enables you to administer all the nodes in a cluster at the same time. Nodes automatically share configuration information.
Full Clustering
Full-clustering mode uses one logical cache spread across the machines – all cluster members communicate to see if any of them have a copy of a requested page. Includes management-only mode.
A fully clustered Websense Content Gateway maps cached content to specific nodes in the cluster. When a node receives a request, it checks to see if the request is a hit somewhere in the cluster.
If the request is a hit on a different node, the node handling the request fetches the object from the hit node and serves it to the client. Websense Content Gateway uses a proprietary communication protocol to fetch an object from other cluster nodes.
If a node fails or is shut down and removed, Websense Content Gateway removes
references to the missing node on all nodes in the cluster. If virtual IP failover is enabled requests destined for the missing node are handled by another node.
Configuring a Managed Cluster
Configure WSG-1, WSG-2, WSG-3 as a Managed Cluster using eth0 as the clustering interface.
Do the following steps 2-9 on WSG-1, WSG-2 and WSG-3 1. At the command line, enter the following:
route add 224.0.1.37/32 dev eth0
2. You need the multicast group address and interface name to configure through Websense Content Manager.
7. Apply the settings and restart the gateway.
8. On each gateway, use the following command to verify the cluster communication is working correctly:
tcpdump -v -ni eth0 ip multicast
You should see the multicast communication displaying on each of the WSG servers, showing the source IP of the WCG interface and the destination multicast group address.
Note when implementing clustering with appliance based WCGs, you do not require a dedicated interface.
Active/Active & Active/Passive - HA, Failover, Load Balancing and Redundancy
High Availability
High availability refers to a system or component that is continuously operational for a desirable percentage of total time.
Failover
Failover involves the functions of a system being assumed by a secondary system when the primary becomes unavailable through either failure or scheduled down time.
Load balancing
Load balancing spreads work between two or more computers, network links or other resources in order to get optimal resource utilization, maximize throughput and minimize response time.
Redundancy
Redundancy may take two forms:
Spare components /systems may be inactive but available for use in case of the failure of the active system.
Multiple components/instances may be implemented with load balancing to provide increased throughput if all are available and redundancy if any of them fail.
Virtual IP
Virtual IP is used in clustering - virtual IP failover assures that if a node in the cluster fails, other nodes can assume the failed node's responsibilities, if virtual IP addressing is enabled, several virtual IP addresses could be assigned to this node. The virtual IP
should be tied to the proxy interface.
How it works
Virtual IP addresses are not bound to any one machine and can be moved about. It works via ARP; when a machine assumes a virtual IP address it tells the router/switch that its MAC address is the address for that virtual IP address. Incoming packets are sent to the VIP address, but all packets travel through real network interfaces. Browser clients are configured to connect to the virtual IP address of the proxy. Load balancing may then be provided by DNS Round Robin
It is not recommended to combine Virtual IP with hardware load balancing e.g. Layer 4 Switch
Using a proprietary management protocol, Websense Content Gateway nodes
communicate their status with their peers. If a node fails, its peers recognize the failure and negotiate which of the remaining nodes will mask the fault by taking over the failed node’s virtual interface.
Virtual IP Cluster recovery process
When a cluster node fails, the processes performed during cluster creation are re-run to redistribute Virtual IP addresses and URLs to be cached. The remaining nodes may or may not retain current Virtual IP addresses or URLs - the process is not a redistribution of the failed node’s associations, it is a completely new distribution
Choosing the number of VIPs dependent on number of nodes
When choosing the number of VIPs to allocate to a cluster, for load balancing purposes it’s important to recognize how the VIPs will be re-distributed after a node failure.
In the following two node cluster, two VIPs addresses provide a balanced configuration on failover.
For the following three node cluster, six VIPs provide a balanced configuration on failover.
If each node had only one VIP, upon re-distribution three VIPs would be distributed to the remaining two active nodes. One node would take 2 VIPs leaving the other with a single VIP – the result of which would be that load balancing would in fact be unbalanced. The calculation for determining the number of VIPs required to balance a cluster in the event of one node failing is N x (N-1) where N is the number of nodes.
Configuring HA for WCG Using Virtual IP Addresses
Recall you have 3 gateways in a management cluster, assuming the event of one cluster node failing you want to distribute the remaining IPs evenly. We will assign 6 VIPs across the 3 cluster nodes. This results in 2 VIPs per gateway assuming full availability balancing to 3 VIPs per node on the failure of 1 node.
On WCG-1
9. Navigate to Configure > My Proxy > Basic and Enable Virtual IPs 10. Restart the proxy
11. Navigate to Configure > Networking > Virtual IPs 12. Click on the Edit file button
13. Type the following Virtual IP details adding each one 172.29.1.150 eth0 Sub Interface 2
172.29.1.151 eth0 Sub Interface 3 172.29.1.152 eth0 Sub Interface 4 172.29.1.153 eth0 Sub Interface 5 172.29.1.154 eth0 Sub Interface 6 172.29.1.155 eth0 Sub Interface 7
14. Click the Apply button and then the Close button.
15. Verify on WCG-2 and WCG-3 the VIPs are shown under Configure > Networking > Virtual IPs
Note the sub interface number must be unique for the entire cluster:
-bash-3.1# ifconfig eth0...
eth0:2...
eth0:3...
On WCG-1, WCG-2 and WCG-3 Systems
17. Run ifconfig on WCG-1, WCG-2 and WCG-3
18. View the sub interfaces to determine which virtual IP(s) have been assigned to each proxy.
You can also get IP / Node mappings from Monitor > Networking > Virtual IP (this does not provide sub interface details)
19. Note the IP addresses mappings below.
WCG-1 WCG-2 WCG-3
Sub Interface VIP Sub Interface VIP
Testing VIP directly
20. Switch to the XP-1 VM
21. Set Internet Explorer to explicitly proxy to one of the VIPs assigned to WCG-3 22. Test Connectivity by using http://www.google.com
23. Test Filtering by using http://www.concordiahomes.ca
Shutting down WCG-3 to redistribute VIPs
24. On WCG-3, run ifconfig and verify sub interfaces are listed
25. Using /opt/WCG/WCGAdmin stop, stop the WCG services on WCG-3 hosting the explicit proxy VIP in the previous section
26. Await all services to stop
Verifying re-assignment of VIP to WCG-1 and WCG-2
Sub Interface VIP
29. You should still be set to proxy through the VIP previously allocated to WCG-3 30. From the WXP VM test Connectivity to http://www.google.com and filtering using
http://www.concordiahomes.ca
31. If not yet functioning, wait 30 seconds 32. Test connectivity and test filtering again.
33. Time permitting, start the WCG services again on WCG-3 and verify the time it takes to redistribute VIPs to the node, either via a WCG GUI or verifying with ifconfig on WCG-3.
Configuring round robin
Now you have successfully configured and tested the cluster with VIPs, you will now configure DNS round robin to load balance traffic across the VIPs.
Round robin is a local balancing mechanism used by DNS servers to share and distribute network resource loads. You can use it to rotate all resource record (RR) types contained in a query answer if multiple RRs are found.
By default, DNS uses round robin to rotate the order of resource records (RR) data returned in a query where multiple RRs of the same type exist for a queried DNS domain name. DNS resource records (RRs) describe the characteristics of a zone (or domain). Round robin provides a simple method for load balancing client use of Web servers and other frequently queried multi-homed computers.
Example: Round-robin rotation
A forward lookup-type query (for all A RRs that match a DNS domain name) is made for a multihomed computer (wcgproxyrr.wstrain.com) that has three IP addresses. Separate A RRs are used to map the host's name to each of these IP addresses in the zone. In the stored wstrain.com zone, the RRs appear in this fixed order:
wcgproxyrr IN A 10.0.0.1 wcgproxyrr IN A 10.0.0.2 wcgproxyrr IN A 10.0.0.3
The first DNS client that queries the server to resolve this host's name receives the list in default order. When a second client sends a subsequent query to resolve this name, the list is rotated as follows:
wcgproxyrr IN A 10.0.0.2 wcgproxyrr IN A 10.0.0.3 wcgproxyrr IN A 10.0.0.1
Resource Record A: An IPv4 Address record. An IPv4 address for a host
IN: Query class used for Internet
34. Start the Content Gateway services on WCG-3.
Verify Round Robin is enabled on the Domain Controller
35. On the domain controller VM, open DNS from Start > Administrative Tools > DNS 36. Right click on the domain controller name DC-WSG and click Properties
37. Choose the Advanced tab.
38. Ensure Enable Round Robin is checked
Defining A Hosts
39. In the navigation tree on the left, drill down into the domain, expand Forward Lookup Zones, and select wstrain.com
40. Right click in the right frame and choose New Host (A)
41. Add the six VIPs selected for the WCG A Host records as wcgproxyrr.wstrain.com, one for each of the VIPs assigned to the Gatwway, i.e. 172.29.1.150 – 172.29.1.155
Testing Proxy Addressing via hostname
42. Configure XP-1 VM to proxy to wcgproxyrr.wstrain.com
43. Verify you can still browse the internet using the new proxy settings.
44. To verifying VIP and DNS RR are both working together, from a command prompt on the DC run, and leave the command running
ping wcgproxyrr.wstrain.com –t
45. This should resolve to one of the IP from one of the A Host IPs entered in the Forward Lookup Zone in the DNS server.
46. Open a browser on the DC, configuring the proxy details to proxy through
wcgproxyrr.wstrain.com on the required ports for HTTP/HTTPS and verify you can browse the internet.
47. Establish the WCG that owns the VIP resolved from wcgproxyrr.wstrain.com on the DC, and on the WCG owning that VIP, use WCGAdmin to stop the WCG services
48. You should see connection drop, then eventually come back up.
49. Verify you can still browse from the DC, and use ifconfig to verify which Content Gateway now owns that IP
50. You have now verified that both DNS Round Robin and VIPs are working for the managed cluster.
