vRealize Log Insight supports the use of NFS mounts for data archiving, which might be necessary to include in a solution design if the service provider’s design has requirements for long-term auditing, compliance, or data retention. The process of data archiving preserves old logs that would otherwise be removed from the vRealize Log Insight appliance due to storage limitations. Where the service consumer’s business requirements demand it, plan to include vRealize Log Insight data archiving in the design.

Under normal operating conditions, vRealize Log Insight never runs out of disk space, because it checks the current storage status every minute, and if there is less than three percent of disk space remaining, it retires old “data buckets”. If archiving is enabled, vRealize Log Insight archives the data buckets before retiring them.

As part of any vRealize Log Insight design proposal, include archiving calculations that outline the required NFS disk space needed each year to meet the provider’s regulatory, compliance or data retention requirement.

vRealize Log Insight itself does not manage the NFS mount used, because this is maintained through the storage management tools typically provided by the service provider’s storage vendor. To receive system notifications, you must configure vRealize Log Insight so it can send out an email alert when the NFS mount is getting low on available space. More importantly, if the NFS mount runs out of space or is not available for a period of time that is longer than the retention period of the vRealize Log Insight appliance, data ingestion will stop until the NFS mount has been restored and enough free space becomes available for archiving. Alternatively, you can disable archiving temporarily.

Log events that have been archived are no longer searchable. To search archived logs, you must first import the logs into a vRealize Log Insight instance. For more information on vRealize Log Insight data archiving, see the VMware vRealize Log Insight Administration Guide at http://pubs.vmware.com/log-insight-30/topic/com.vmware.ICbase/PDF/log-insight-30-administration-guide.pdf.

Another design factor to be considered is not to import archived data into the existing production vRealize Log Insight instance. This is due to the fact that data from the archive import will force the oldest ingested data to be deleted to make room, which might affect your inability to maintain the desired retention period for events. VMware recommends that you import archives into a dedicated vRealize Log Insight instance that is not ingesting message events and is dedicated to this function.

The following table lists sample log archival storage requirements for general guidance. Actual requirements will vary considerably from environment to environment depending on hardware, software, vSphere features used, and configuration settings.

 

After data archiving is configured, it is the operational team’s responsibility to verify that the archive destination is cleaned up and maintained, and does not run out of space. vRealize Log Insight does not have any mechanism to manage or monitor the NFS destination, and will continue to attempt to archive data, even after the destination becomes full. Typically, a simple cleanup script can be leveraged to manage this.