Skip to content

Configuration Overview

There are three ways to send logs to SenseOn, depending on your environment and requirements. These correspond to the three deployment models: Receive, Relay, and Retrieve.

Requirements

SenseOn accepts any valid JSON sent via HTTPS POST to the /log_sink endpoint.

That's the only technical requirement. While metadata fields like timestamp, source, and severity are useful for enriching your data, they are not mandatory.

Minimum Valid Log

{"event": "connection"}

{
  "timestamp": "2025-10-14T15:30:45Z",
  "source": "nginx",
  "level": "info",
  "remote_addr": "203.0.113.42",
  "request": "GET /api/health HTTP/1.1",
  "status": "200"
}

See also: Configuration Examples for detailed setup guides for common log sources.

Receive Mode

Sending your logs to SenseOn is the easiest deployment approach and recommended for low volume deployments. In this deployment model SenseOn manages the log collector. You simply configure log destinations to point to the SenseOn managed collector. This means you can see immediate value, with data flowing within minutes of configuration.

For this configuration SenseOn will provide the URL of the collector endpoint: 

https://<code-name>-collector.snson.net/log_sink

This endpoint can receive logs from any source that supports HTTPS POST with JSON payloads, including:

  • Syslog (via rsyslog HTTP output)
  • Open Telemetry
  • Filebeat
  • FluentBit
  • Custom applications
  • Cloud-native pipelines (AWS Kinesis Firehose, GCP Pub/Sub)

Testing Your Connection

Once you have your collector endpoint, test the connection:

curl -X POST "https://<code-name>-collector.snson.net/log_sink" \
  -H "Content-Type: application/json" \
  --data '{"timestamp": "2025-10-14T15:30:45Z", "source": "test", "message": "Connection test"}'

If successful, you'll see a 200 or 201 response.

Next steps: See Configuration Examples for step-by-step setup guides for your specific log source.

Relay Mode

In the Relay deployment model, you deploy and manage your own log collectors within your environment. This is recommended for:

  • High-volume deployments
  • Containerised workloads (Kubernetes, Docker)
  • Environments requiring pre-filtering to reduce costs
  • Security compliance requirements for data transformation within your boundary

Example: Forwarding VM Logs to SenseOn using Fluent Bit

This guide shows how to forward logs from a VM to the SenseOn Log Collector using Fluent Bit.

The steps here are meant as a baseline setup example — actual configuration will vary depending on the log source (firewall, VM syslog, GCP, etc.).

1. Prerequisites

  • Fluent Bit installed on the system which will act as the log collector (latest version, not td-agent-bit)
  • Outbound HTTPS access to SenseOn-provided log collection endpoint: 
    https://<code-name>-collector.snson.net
  • Access to your desired system logs (e.g. /var/log/syslog)

2. Fluent Bit Configuration

Logs must be JSON formatted and forwarded over HTTPS in order to be successfully ingested. To achieve this using Fluent Bit, create or edit /etc/fluent-bit/fluent-bit.conf with:

[SERVICE]
    Parsers_File /etc/fluent-bit/parsers.conf

[INPUT]
    Name         tail
    Path         /var/log/syslog  # or path to logs
    Parser       syslog-rfc3164
    Read_from_Head  true

[FILTER]
    Name         modify
    Match        *
    Rename       message log
    Rename       Message log

[OUTPUT]
    Name         http
    Match        *
    Host         <code-name>-collector.snson.net  # collection endpoint provided by SenseOn
    Port         443
    URI          /log_sink  # or other URI provided by SenseOn
    Format       json_lines
    json_date_key   false
    tls          On
    Header       Content-Type application/json

Restart Fluent Bit:

sudo systemctl restart fluent-bit
sudo systemctl status fluent-bit

3. Testing the Connection

Option A — Check Fluent Bit logs 

journalctl -u fluent-bit -f

Look for successful flush messages. If you see repeated errors such as:

  • Domain name not found → confirm you're using snson.net (not senseon.net).

Option B — Send a test log with curl 

curl -v -X POST "https://<code-name>-collector.snson.net/log_sink" \
  -H "Content-Type: application/json" \
  --data '{"timestamp": "2025-10-14T15:30:45Z", "source": "test", "log": "hello test"}'

If accepted, you'll see a 200 or 201 response.

4. Common Pitfalls

  • DNS resolution issues: If <code-name>-collector.snson.net doesn't resolve, check with your network team or SenseOn support.
  • Content-Type errors: Double-check your Header setting in the configuration.
  • Disk space errors (No space left on device): Not related to forwarding — make sure /var/log has enough space, otherwise local logs may be dropped before forwarding.

5. Additional Configuration Examples

For more detailed examples including:

  • Nginx web server logs with JSON formatting
  • Syslog (rsyslog) with HTTP output module
  • Kubernetes container logs with DaemonSet deployment
  • Application integration (Python, Node.js code examples)

See the comprehensive Configuration Examples guide.

6. Additional Resources

7. Next Steps

Once logs are flowing from a VM, repeat the same pattern for other log sources (firewalls, GCP integrations, etc.).

Each will differ slightly in how logs are collected, but all forward them in the same way to the collector endpoint. That is, logs sent in JSON format over HTTPS.

Retrieve Mode

Some systems do not support log forwarding. In these cases, SenseOn can pull logs from a location where they are stored. This is a premium service and requires professional services engagement to set up and manage.

Common use cases:

  • API-only data sources with no native log forwarding
  • Legacy systems without modern log export capabilities
  • Object storage buckets (S3, GCS, Azure Blob)
  • Custom data sources requiring specialized connectors

Contact SenseOn professional services to discuss Retrieve mode implementation for your specific requirements.

Troubleshooting

Logs Not Being Received

  1. Verify endpoint URL - Check you're using the correct collector endpoint provided by SenseOn
  2. Test with curl - Use the curl command above to test connectivity
  3. Check JSON format - Ensure your logs are valid JSON: 
    echo '{"test": "log"}' | jq .
  4. Review forwarder logs - Check your log forwarder (FluentBit, rsyslog) for errors
  5. Network connectivity - Ensure outbound HTTPS (port 443) is allowed
  6. DNS resolution - Verify you can resolve the collector domain

High Resource Usage

  1. Reduce buffer sizes in your forwarder configuration
  2. Increase flush frequency to send logs more often
  3. Check for parsing errors that might cause log buildup
  4. Monitor disk I/O for source log files

Connection Timeouts

  1. Verify collector endpoint is correct
  2. Check firewall rules allow HTTPS (443) outbound
  3. Test DNS resolution: nslookup <code-name>-collector.snson.net
  4. Check proxy settings if using a corporate proxy

Additional Documentation