3.4.2 Syslog Aggregators and vRealize Log Insight Forwarding
The ability to forward or aggregate syslog data is an important design feature when the architecture requires syslog data to traverse WAN interconnects across sites. This capability also allows you to process or store syslog events, as well as forward events, to another upstream syslog server. Syslog aggregators are typically used in multi data center environments or where message traffic needs to be controlled, for reasons typically relating to security.
vRealize Log Insight 2.5 introduced the ability for a vRealize Log Insight appliance to be used for message forwarding. Any vRealize Log Insight 2.5 or later instance can be employed to forward event messages, whether standalone or clustered. However, even when vRealize Log Insight forwards an event, the vRealize Log Insight instance still ingests and retains those events locally and even archives event messages if it has been configured to do so. In addition, a vRealize Log Insight instance that has been configured to forward events can still be used to run queries for locally stored data. This can be a significant design factor for storage and security when architecting a large scale, multi data center infrastructure for a service provider. There is no way to configure vRealize Log Insight to only forward events and not store them locally.
In environments where this might be an issue, and the provider does not want to store potentially sensitive event message data in specific locations, the two most commonly third-party syslog aggregators are syslog-NG and rsyslog. These daemons allow syslog sources to send them log messages and the aggregator’s configuration determines how those messages are forwarded to the central vRealize Log Insight instance. Syslog-NG is available as either an open-source edition (OSE), or a premium edition (PE) at a cost per agent. The open-source edition is sufficient for most implementations. The following site provides a comparison of the two options:
http://www.balabit.com/network-security/syslog-ng/comparing/detailed.
Both vRealize Log Insight forwarding and syslog aggregators provide some flexibility and control of how events are forwarded, the format they are forwarded in, the protocol used, and where they are forwarded to. They can also provide design advantages with their ability to limit the configuration changes required when reconfiguring syslog targets. For example, you might need to change only the configuration of the syslog aggregators, as opposed to every source device in the infrastructure, which can be useful from a BCDR design and planning perspective. Syslog aggregators also provide granular control and can be configured to send some or all of the messages that they receive and the messages they generate internally. The following figure shows a typical use case for employing vRealize Log Insight to forward syslog message events in the data center.
Figure 4. vRealize Log Insight Event Forwarder
When implementing this type of forwarding solution, configure the syslog aggregators to maintain the original source IP or hostname of the device when forwarding the messages to the centralized syslog server. This is extremely important to provide successful auditing, querying, and analysis of logs.
Finally, syslog aggregators can also be used effectively in a scenario where the syslog target is located on a different network segment (VLAN) than the source host and a firewall sits between the two devices. This allows for significantly more granularity when it comes to configuring a firewall’s access control list (ACL). For example, a network security team is likely to be far more comfortable configuring a firewall ruleset that allows traffic from a single source IP address to pass through to a single destination IP address, than allowing a whole network segment to pass traffic over the security appliance on port 514.
The following figure shows a use case where third-party syslog aggregators are employed.
Figure 5. Third-Party Syslog Aggregator
There is one final design consideration for the use of forwarders and syslog aggregators. In a smaller environment, sending syslog traffic directly to a vRealize Log Insight cluster would typically be fine. However, in a multi-site, multi-zone complex service provider environment, VMware recommends that designs employ forwarders in front of the core vRealize Log Insight instances at whatever points are appropriate in the architecture. For instance, forwarders might be deployed in every data center, at a minimum, including the local data center facility where the core vRealize Log Insight instance is located. Also, you might provide an additional level of granularity, depending on the solution design and network architecture, by including additional forwarder instances per network segment, per tenant, or per business unit.
Although syslog forwarders and aggregators increase architecture complexity with their initial setup and configuration (and the potential involvement of more third-party systems), they are often an architectural necessity to meet a service provider’s complex mix of challenging requirements and environment circumstances such as network routing, firewalling, or providing a globally distributed architecture.
An additional complication to the solution design is that the VMware Cloud Provider might want such forwarders or aggregators to be highly available. In this case, vRealize Log Insight forwarders would have to be clustered for event message ingestion, providing a highly available configuration, requiring a minimum of three nodes per forwarder instance. When considering the virtual appliance sizing for dedicated forwarding instances of vRealize Log Insight, small deployments are typically sufficient to meet the needs of most environments, because the forwarders themselves are not going to be used as the primary mechanism to store ingested events or to perform operational queries of the environment.
