Open Log Stack

Log collection and translation at scale

Overview

With increasing automation across the entire spectrum of business operations, and with the continued deployment and expansion of IT infrastructure, the amount of logs that these systems generate has become humongous. It has become almost impossible to manually process these logs in the event of a failure to take corrective measures. Besides, with increasing scale, the log collection mechanism itself comes under stress, resulting in loss of important information that could have helped in better analysis.

Lumina Open Log Stack is an open source based application that facilitates log collection at scale, while aiding the in-memory translation from unstructured log to structured data. The structured data is then stored in analytics db like Elastic which aides further automated analysis and detection of anomalies in operations of systems. It renders itself for horizontal scaling, thereby bringing the ability to parse/analyze millions of logs per second in a clustered deployment

While the OLS app can be deployed for any domain, it finds its key applications in security domain, aiding in analysis of millions of firewall logs, thereby assisting in threat detection and enabling effective counter measures

Benefits

  • Enables IT departments to collect granular log messages, without worry about loss of logs in translation
  • In-memory translation at scale permits seamless conversion from unstructured to structured data, facilitating better offline analysis
  • Identification of patterns from large data sets facilitates automating corrective actions and closed-feedback loops

Features

  • Hardened, open source based log ingestion
  • Translators for security log analysis
  • Log push support for Elastic DB
  • Dashboards
  • Policy query language and feedback triggers (future)
  • Tuned to operate at scale

Related Solutions

Pin It on Pinterest