Skip to content

Architecture

To give comprehensive security coverage, a log ingestion platform must collect data flexibly and process it intelligently. SenseOn designed its architecture on these principles. This document first gives the flexible deployment models that bring data into the platform from all of your digital estate. It then explains the tiered log pipelines. The log pipelines make sure that the platform processes each log cost-effectively and in relation to its security value. The processing can be simple storage, immediate alerts or deep contextual enrichment.

Deployment Models

Modern IT environments are diverse, from cloud APIs to legacy on-premises systems. Thus SenseOn gives three flexible deployment models. Each model solves a different collection problem. You can use the models together in any combination to make an ingestion solution for your environment. Thus you can integrate any log source into the platform, wherever the source is and however it sends data.

  • Receive. The simplest deployment. You send logs over the Internet to a log sink that SenseOn manages.
  • Relay. You use a log collector in your estate. The collector can transform and filter logs.
  • Retrieve. SenseOn gets access to logs in another system. An example is an S3 bucket or a different object storage bucket.

Receive

Receive is the easiest deployment approach. You change your log destinations to a receiver that SenseOn manages. Thus you can see value immediately, and data flows within minutes of configuration.

Also, you do not have new infrastructure to configure or manage. SenseOn manages the full log ingestion pipeline, with the technologies and approaches that suit you. SenseOn can receive data over any standard log transport protocol or method. Examples are Syslog, OpenTelemetry, Filebeat, and cloud-native pipelines such as the AWS Kinesis Data Firehose.

Relay

SenseOn recommends relay mode for high-volume deployments. Examples are deployments that monitor containerised workloads. Other examples are deployments that must filter logs before transmission, to decrease transmission or storage costs.

In relay mode, you set up and manage one or more log forwarders in your environment. It is possible that security compliance makes this mode necessary, because your forwarders transform or filter the data inside your boundary.

Retrieve

SenseOn can pull in logs from any location. Usually, SenseOn uses this mode for API-only data sources that do not transport logs natively. Use Retrieve data sources only when there is no alternative. SenseOn professional services make it possible to ingest these logs, and they manage the necessary infrastructure. This is a premium service for unique needs.

Log Pipelines

SenseOn gives three log pipelines. Each pipeline gives a different level of processing and analysis. The correct level for a log depends on its importance and its intended use. With this tiered approach, high-value logs get the attention that they deserve. Less critical logs go to efficient storage for future reference.

💡 Note: If necessary, you can send logs to more than one pipeline at the same time. This gives flexible data management and better cost management. For example, you can keep logs in long-term archival storage for compliance. At the same time, the platform can analyse the same logs for threat detection for a shorter period.

Detection & Response

SenseOn actively analyses and correlates high-value logs across your environment. It uses them to detect threats and make intelligent alerts. Thus you can respond quickly to security events.

Observability

SenseOn keeps logs in warm, queryable storage for visualisation, search and investigation.

Compliance

Long-term immutable archival storage keeps logs in their original format. This storage is for audits, regulatory obligations and historical retrieval.