Overview

Background

SenseOn provides a highly curated and correlated data fabric using telemetry from its Universal Sensor, Integrations and network probes which provides high value immediate threat detection and response. This visibility can be extended further through the ingestion of logs which provide additional context and coverage of your digital estate.

Why?

Compliance and Auditing: You may need to comply with regulations, legal requirements, or internal policies which mandate the retention of specific logs. Having a separate archive of raw logs ensures you can meet these compliance requirements and provide auditors with the exact data they require.

Bespoke Application Coverage: Some applications, especially legacy or custom-built systems, may not have direct integrations with SenseOn. Collecting raw logs from these applications ensures that you still have visibility into their activity and can correlate events with other data sources.

Comprehensive Coverage: While SenseOn's Universal Sensor and integrations cover a vast range of data sources, there may be some systems which we can't directly integrate with. Collecting raw logs from these sources ensures that no part of your digital estate is a complete blind spot. For advanced users, all data within the data lakehouse that powers SenseOn's Intelligence Cloud is accessible using SQL, allowing for deep, customised queries on this comprehensive dataset. This capability enables users to perform intricate analyses and extract specific insights tailored to their unique requirements.

Choosing the right logs to ingest

We don't recommend that every possible log is ingested into the SenseOn platform. Focusing on the most valuable data sources first will ensure you get rapid visibility and actionable intelligence without being overwhelmed. This guide outlines the key log sources to prioritise, why they are important, and the common methods for collecting them.

1. What are the gaps in your current visibility?

Before you start ingesting logs, it's crucial to understand your current visibility gaps. This involves assessing your existing visibility which may be provided by the SenseOn Universal Sensor, existing integrations, and any other security tools you have in place. Identify which critical systems, applications, or network segments are not adequately covered. Consider the types of attacks you are likely to experience and which logs would provide the most relevant data to detect and respond to these threats.

2. Is there an opportunity to reduce costs by sending logs to SenseOn instead of other platforms?

With the logs and tooling you already have in place, consider whether there is an opportunity to consolidate your log management within SenseOn. This can include saving on infrastructure, licensing, and operational overhead by not having to maintain multiple logging solutions. This analysis should be financially and operationally driven, considering costs and benefits around your current products, people and processes and how this compares to SenseOn.

3. How will you get these logs to SenseOn?

SenseOn has multiple methods for ingesting logs, each with its own advantages and considerations. The most common methods are for you to host a log collector within your environment but for smaller deployments or specific use cases, you may also be able to use a cloud-hosted collector or direct API integration. More details of the methods are defined in the Architecture document.

4. What is the estimated volume of logs?

You need to understand the volume of logs you expect to ingest from each source and understand the business value required from each source. This is important for determining the appropriate sizing for your deployment and ensuring that your log ingestion process is efficient and cost effective. High log volumes can lead to increased costs. Estimate the daily log volume for each source you plan to ingest, SenseOn solution engineers can help you calculate this if required. This will help you determine the appropriate sizing for your deployment and ensure that your log ingestion process is efficient and cost-effective.

5. Prioritise what logs need to have what level of service.

As logs are ingested into the SenseOn platform, SenseOn has different paths for the logs depending on what processing is required of them. Each path has a different pricing associated with it allowing you to prioritise the most important logs to have the highest level of capability and manage costs effectively. For example logs required for audit purposes may be just sent to cold storage, but logs required for real-time detection should be sent to paths where analytics can run on the data. More details on the different paths and their capabilities can be found in the Architecture document.