4.4 Ingestion Rates
When architecting a vRealize Log Insight solution to handle log file collection, the design must be able to handle the ingestion rate of all the configured source devices. vRealize Log Insight 3.0 can ingest more than twice the data per second than the vRealize Log Insight 2.5 release, with an ingestion rate of up to 15,000 events per second. With a maximum 12-node cluster, that comes to 2.7 TB of data per day.
Table 8. Ingestion Rates by vRealize Log Insight Cluster Deployment Size
Option | Log Ingest Rate | vCPUs | Memory | IOPS | Syslog Connections | Events per Sec |
Extra Small | 3 GB/day | 2 | 4 GB | 75 | 20 | 200 |
Small | 15 GB/day | 4 | 8 GB | 500 | 100 | 1000 |
Medium | 37.5 GB/day | 8 | 16 GB | 1000 | 250 | 2500 |
Large | 112.5 GB/day | 16 | 32 GB | 1500 | 750 | 7500 |
Note Adding additional vCPU resources to vRealize Log Insight virtual appliances will improve query and ingestion performance.
With vRealize Log Insight 3.0 supporting up to 12 nodes in a cluster, the architecture can scale out to likely meet the needs of the most demanding VMware Cloud Providers that deploy VMware infrastructure across numerous geographically dispersed data centers.
Scaling out and providing high availability are the primary reasons to take a clustered approach to deployment. For example, it might be more appropriate to design a solution that employs three or four small nodes as opposed to a single medium or large node. This method provides the same or higher ingestion rate but has the added benefit of supporting high availability.
Designers should architect a solution to meet the current ingestion rate needs of the infrastructure, while also allowing for growth in the provider’s requirements. In addition, always deploy the vRealize Log Insight virtual appliance with thick-provisioned, eager-zeroed disks. This allows for improved performance and operation of the virtual appliance.