vRealize Log Insight: Internal vs External Load Balancer

Most of the times when talking about high availability or scaling a product like vRealize Log Insight is easily forgotten. This is a shame, especially since vRealize Log Insight is so easy to setup as a cluster with load balancing capabilities. You might wonder why it could be important to have Log Insight highly available, it’s only log information right?

Compliancy is becoming more and more a company standard. And part of the compliancy can be audit trails, which you store in Log Insight or any other syslog system. Not every application or service is able to resend messages to a syslog, so it is importent to have your Log Insight higly available.

Or what about troubleshooting? It can be really frustrating to miss a piece of information because something happend within the timeframe that your Log Insight servers where not available.

So I believe that having Log Insight high available is important, but how can we accomplish that?

Setup a cluster

Now before we dive into creating a load balanced vRealize Log Insight system let’s see how easy it is to setup a Log Insight cluster.

To start with you will need to deploy at least three Log Insight appliances. You might ask why not start with two? Well, Log Insight uses Cassandra and this database requires a minimum of three nodes for failover purposes. Changes made to node 1 will be written in it’s own database, then send the change to node 2. Node 2 will in turn send it to node 3.

Once you deployed three appliances you will be directed to the webinterface of the appliance, which will start with a first time setup.  The first choose you have to make in the setup is to “Join Existing Deployment” or “Start New deployment”. For your first node you will logically choose for a new deployment. Just follow to wizard, you can’t go wrong :)

For the 2nd and 3th node you choose to join a existing deployment. You are required to enter the FQDN of the Log Insight master, which is the first node that you configured.

Once succesfull go to the webinterface of the master node and under management go to “Cluster”. There you will see a request from the other node(s) to be added as a worker. Accepting this request will add the node to the cluster.

To enable the Internal Load Balancer (ILB) for a cluster you simply click on “New virtual address”. This will present a new window where you can enter the load balancer Virtual IP (VIP) and it’s FQDN. Click save and you will have a load balanced cluster, it’s that easy.

In the cluster overview you can see which node is the master and which node is holding the VIP for the ILB.

Internal vs External

So now back to the titel of the article Internal vs External. As the title suggests it is possible to use either a Internal Load Balancer (ILB) or External Load Balancer (ELB). But what are the pro’s or cons for either setup?

Internal Load Balancer

Pro’s for the ILB are:

• The load balancing software is already included, no need to purchase or download extra software
• As shown above the ILB is really easy to configure, specially compared to an ELB, which in turn can help lower operation costs
• Natively ILB provides layer 4 support and automatic message rebalancing across nodes (layer 7 functionality)
• Health check and automatic failover are included

Downside of using the ILB is that the nodes must be on the same layer 2 network segment. This is because the load balancer VIP is claimed by one node using ARP.

External Load Balancer

Pro’s for an ELB are:

• Compared to the ILB you can use multiple layer 2 network segments allowing for more options in placement of the nodes
• As an organization you might already have a load balancer and want to have this as your default within the infrastructure

As mentioned configuring an ELB is more complex than the ILB and altough the next bullets aren’t really a con, they are things to keep in mind configuring an ELB:

• If UDP is used by the application / service to send syslog then make sure your load balancer supports this. Not all load balancers do this, so you might need to switch to TCP if available for the application or service
• Each load balancer has it’s own choice of algorithms. To avoid issues caused by unbalanced nodes it is adviced to use the “Least Connections” algorithm.
• Make sure your load balancer can meet the throughput for your needs. Check both the technical limits as potential licensing limits.
• Try to avoid “long-lived” TCP sessions in the laod balancer configuration. This can cause issues when a node is down for a longer period.

Altough VMware doesn’t provide a recommendation on which ELB to use. This list provides some vendors that can provide the required load balancing services:

• VMware NSX edge
• F5
• Kemp LoadMaster
• Riverbed
• Barracuda

Conclusion

As with almost every design the right choice depends on the situation. The ILB is nice and simple, but maybe your organization needs an ELB to spread out the nodes more in the network. So in the end it’s up to the requirements that will determine which load balancer comes out a winner.