Skip to content

SenseOn Logship

SenseOn Logship is a lightweight, cross-platform log collector that tails files, Windows Event Log channels, and (optionally) network syslog on an endpoint, parses and filters the events, and forwards them over HTTPS to your SenseOn tenant for ingestion. It runs as a single static binary on Linux, macOS, and Windows with no runtime dependencies, and can be installed and managed alongside the Universal Sensor.

Use Logship when you need to bring telemetry into SenseOn from sources the Universal Sensor does not natively cover — for example application log files, custom syslog producers, or specific Windows Event Log channels.

🚧 Pre-release testing. Logship is currently in pre-release testing. The configuration schema and CLI surface may change in incompatible ways before general availability. Use it in production at your own risk and pin the installed version where stability matters. Contact support@senseon.io if you encounter problems.

💡 How is this different from log ingestion? The other articles in this section describe the platform side of log ingestion — how SenseOn receives, parses, and surfaces the events. Logship is the agent that runs on your hosts to deliver those events. The two are designed to work together but Logship is not the only way to feed events into SenseOn. See Log Ingestion Overview for the broader picture.

Supported Platforms

OS Architectures Package format
Debian / Ubuntu x86_64, arm64 .deb
RHEL / CentOS / Fedora / Rocky / AlmaLinux x86_64, arm64 .rpm
macOS (Catalina 10.15+) x86_64, Apple Silicon .pkg (launchd daemon)
Windows (10, 11, Server 2016+) x86_64, arm64 .msi (Windows service)

Prerequisites:

  • Outbound HTTPS to your tenant's log-sink URL — typically https://<your-code-name>-collector.snson.net/log_sink. Your SenseOn contact provides the exact URL.
  • Read access to the log files or Windows Event Log channels you want to collect. On Linux and macOS the daemon usually runs as root; on Windows it runs as LocalSystem by default.
  • No Go runtime or other dependencies — the binary is statically linked.

Installation

The install command and any tenant-specific values (sink URL, code name) are provided by your SenseOn implementation contact.

Debian / Ubuntu

sudo dpkg -i senseon-logship_<version>_amd64.deb

Installs:

  • Binaries: /usr/bin/senseon-logship, /usr/bin/senseon-logship-ctl
  • Config directory: /etc/senseon-logship/
  • Service unit: senseon-logship.service (systemd)
  • State directory: /var/lib/senseon-logship/

RHEL / CentOS / Fedora / Rocky / AlmaLinux

sudo rpm -i senseon-logship-<version>.x86_64.rpm

Installs to the same paths as the .deb package.

macOS

sudo installer -pkg senseon-logship_<version>_darwin_arm64.pkg -target /

Installs:

  • Binaries: /usr/local/bin/senseon-logship, /usr/local/bin/senseon-logship-ctl
  • Config directory: /etc/senseon-logship/
  • LaunchDaemon: /Library/LaunchDaemons/io.senseon.logship.plist
  • State directory: /Library/Application Support/senseon-logship/
  • Log file: /Library/Logs/senseon-logship.log

💡 macOS Gatekeeper: The PKG ships unsigned during pre-release testing. Install from the command line as shown above or push it via your MDM (Jamf Pro / Kandji / Intune for Mac). Double-clicking the PKG may trigger a Gatekeeper warning; right-click and choose Open to bypass it for now.

💡 Full Disk Access: For log paths outside the user-readable area (for example /var/log/), grant senseon-logship Full Disk Access in System Settings > Privacy & Security > Full Disk Access.

Windows

msiexec /i senseon-logship_<version>_windows_amd64.msi /qn

Installs:

  • Binaries: C:\Program Files\senseon-logship\senseon-logship.exe and senseon-logship-ctl.exe
  • Config directory: C:\ProgramData\senseon-logship\
  • Service: senseon-logship (registered but not started until configured)
  • Service log: C:\ProgramData\senseon-logship\senseon-logship.log

Configuration

Logship is configured via a single YAML file:

OS Path
Linux /etc/senseon-logship/senseon-logship.yaml
macOS /etc/senseon-logship/senseon-logship.yaml
Windows C:\ProgramData\senseon-logship\senseon-logship.yaml

Override the location with --config <path> if you need to.

Minimal configuration

The shortest valid config sends Logship-collected system and auth logs to your sink:

sink:
  url: https://example-collector.snson.net/log_sink

presets: [system, auth]

The presets field expands to OS-appropriate file paths so the same config works across Debian, RHEL, macOS, and Windows.

Full reference

sink:
  url: https://<your-code-name>-collector.snson.net/log_sink
  gzip: true                                  # default true; set false to send uncompressed
  auth:                                       # optional; only needed for proxy authentication
    type: bearer
    token: ${LOGSHIP_BEARER_TOKEN}
  tls:
    verify: true                              # never disable in production
    ca_bundle: /etc/ssl/private/corp-root.pem # optional, appended to system trust roots

presets: [system, auth]                       # optional bundle of OS-appropriate defaults

sources:
  - name: app_logs
    enabled: true
    paths:
      - /var/log/myapp/*.log
    format: json                              # raw | json | syslog | regex | wineventlog
    tags:
      app: myapp
      environment: production
    multiline:
      start_pattern: '^\d{4}-\d{2}-\d{2}'    # regex marking the start of each entry

  - name: windows_security
    channel: Security
    format: wineventlog
    query: '*[System[(Level=1 or Level=2)]]' # XPath filter (optional)

  - name: edge_appliances
    listen:
      udp: ':514'                             # accept syslog over UDP on port 514
      tcp: ':514'                             # and TCP
    format: syslog

filters:
  redact:
    - pattern: '(?i)password=\S+'
      replacement: 'password=***'
  include:
    - source: app_logs
      pattern: '^(ERROR|WARN)'                # keep only these levels for this source
  exclude:
    - source: app_logs
      pattern: '^DEBUG '                      # drop debug noise

buffer:
  dir: /var/lib/senseon-logship/spool          # default <state>/spool
  max_disk_mb: 256                             # cap on persisted backlog

limits:
  max_events_per_sec: 5000                     # per-source soft cap
  max_batch_bytes: 1048576                     # batches above this are split
  flush_interval_ms: 1000                      # how long a partial batch waits

Environment variables of the form ${VAR} are interpolated at load time inside sink.url, sink.auth.token, sink.tls.ca_bundle, buffer.dir, and source paths. Unset variables are an error.

Validating your configuration

Before applying changes, validate the file:

sudo senseon-logship-ctl validate-config /etc/senseon-logship/senseon-logship.yaml

Errors include the line number and a snippet. Exit code 0 means the configuration is valid.

Reloading without a restart

sudo senseon-logship-ctl reload

The daemon re-reads the configuration in place. File offsets, the on-disk spool, and the host ID are preserved.

Log Sources

Files

Each file source needs name, paths, and format:

- name: nginx_access
  paths:
    - /var/log/nginx/*.log
  format: regex
  pattern: '^(?P<remote>\S+) \S+ \S+ \[(?P<timestamp>[^\]]+)\] "(?P<request>[^"]+)" (?P<status>\d+)'
  time_layout: '02/Jan/2006:15:04:05 -0700'

Logship reads files using persistent byte offsets stored in the state directory, keyed by file identity (device + inode on Linux/macOS, volume serial + file index on Windows). This means renames and rotations are handled correctly.

  • Files present at startup are followed from their current end. Existing content is not re-ingested when Logship is first deployed.
  • Files appearing later (for example after a log rotation) are read from the beginning.
  • Truncation and rename-and-create rotation patterns are both supported.
  • UTF-16 BOMs and CRLF line endings are decoded transparently.
  • Glob patterns are expanded at load time and re-evaluated for new matches.

Supported formats:

Format Use for
raw Treat each line as the message verbatim
json Newline-delimited JSON objects; time/timestamp/@timestamp/ts, level/severity, and message/msg are lifted into the top-level event; the rest become fields
syslog RFC 3164 / RFC 5424 with optional <PRI> prefix
regex Supply pattern with named captures; timestamp, level, and message captures are lifted into the event
wineventlog Windows Event Log channels only (see below)

Windows Event Log

- name: windows_security
  channel: Security
  format: wineventlog
  query: '*[System[(Level=1 or Level=2)]]'
  • channel is the channel name as it appears in Event Viewer (Security, System, Application, or a custom channel).
  • query is an optional XPath filter. The example above keeps only Critical (1) and Error (2) events.
  • Delivery starts from the moment Logship subscribes. Events raised while the service was stopped are not replayed.
  • The Security channel requires LocalSystem or membership of Event Log Readers for the Logship service account.

Syslog listener (UDP and TCP)

- name: edge_appliances
  listen:
    udp: ':514'
    tcp: ':514'
  format: syslog
  • Listeners are off by default — Logship only opens ports when a source explicitly declares them.
  • UDP treats each datagram as one message. TCP is newline-framed (octet-counted RFC 6587 framing is not supported yet).
  • Binding ports below 1024 requires root on Linux/macOS or administrator on Windows.
  • Per-source rate limits apply.

Presets

Curated per-OS bundles save you from writing the same source list for every host:

presets: [system, auth]
Preset Linux macOS Windows
system /var/log/syslog, /var/log/messages unified log via files System channel
auth /var/log/auth.log, /var/log/secure /var/log/asl/*.asl Security channel

Paths that don't exist on a given host are watched for creation rather than treated as an error, so the same preset works across distributions.

Filtering and Redaction

Logship can drop or rewrite events on the host before they leave the machine.

filters:
  redact:
    - pattern: '(?i)password=\S+'
      replacement: 'password=***'
    - pattern: '\b\d{4}-\d{4}-\d{4}-\d{4}\b'
      replacement: '[REDACTED-CARD]'
  include:
    - source: app_logs
      pattern: '^(ERROR|WARN)'
  exclude:
    - source: app_logs
      pattern: '^DEBUG '
  • redact applies to the message and to all string fields, before serialisation.
  • include is an allow-list. When present for a source, only matching lines pass.
  • exclude is a deny-list. Exclude wins over include when both match.

Redaction is unconditional and runs first. There is no way to bypass it for individual sources today.

Service Management

Linux (systemd)

# Enable and start at boot
sudo systemctl enable --now senseon-logship

# Stop / restart
sudo systemctl stop senseon-logship
sudo systemctl restart senseon-logship

# Reload config without dropping events
sudo systemctl reload senseon-logship

# Tail the daemon's own logs
sudo journalctl -u senseon-logship -f

macOS (launchd)

# Load and start (at boot too)
sudo launchctl bootstrap system /Library/LaunchDaemons/io.senseon.logship.plist

# Stop
sudo launchctl bootout system /Library/LaunchDaemons/io.senseon.logship.plist

# Tail the daemon log
sudo tail -f /Library/Logs/senseon-logship.log

Windows

# Start / stop the service
sc.exe start senseon-logship
sc.exe stop senseon-logship

# Tail the service log
Get-Content C:\ProgramData\senseon-logship\senseon-logship.log -Wait -Tail 50

The MSI registers the service with automatic startup. The first manual start enables boot-time launches thereafter.

Operational Behaviour

Delivery semantics

Logship offers at-least-once delivery. Every event is written to the on-disk spool before it is dispatched; the spool cursor only advances after the sink acknowledges the batch (HTTP 200 or 201). Crashes, restarts, and sink outages cause spooling, not loss.

In rare cases (network blip plus an interrupted ACK), the same event may be delivered twice. Downstream processing on the platform side is responsible for any de-duplication.

Buffering

  • The on-disk spool lives at <state>/spool (/var/lib/senseon-logship/spool on Linux, /Library/Application Support/senseon-logship/spool on macOS, %ProgramData%\senseon-logship\spool on Windows).
  • The default cap is 256 MiB (buffer.max_disk_mb). Beyond it, the oldest events are evicted to make room and the eviction is counted in the operational metrics.
  • When the sink is unreachable, Logship continues collecting and buffering. When the sink recovers, the backlog drains automatically.

Resource use

Logship is designed to be light. Typical steady-state numbers on a Linux host with a handful of sources at moderate throughput:

  • CPU: Under 1% on a single core.
  • Memory: ~30–60 MB resident.
  • Network: Proportional to event volume. Gzip compression is enabled by default and typically gives 5–10× shrink for JSON payloads.

Heavier configurations (busy syslog listener, many sources, high rate limits) will use more. Use senseon-logship-ctl status to see per-source rates.

Where Logship writes its own diagnostics

Platform Path
Linux (systemd) journalctl -u senseon-logship
macOS /Library/Logs/senseon-logship.log
Windows C:\ProgramData\senseon-logship\senseon-logship.log

The format is structured JSON. By default the daemon is intentionally quiet — typically one line when an error starts and one line when it clears. To increase verbosity, run the daemon with --log-level debug (set the systemd / launchd / service argument to include this flag).

Troubleshooting

The fastest first step on any host is to run the built-in health check:

sudo senseon-logship-ctl doctor

It runs every diagnostic in one pass and prints concrete fix hints for anything that's not healthy.

If you need to drill in further, the commands below cover the four most common failure shapes.

Is the configuration valid?

sudo senseon-logship-ctl validate-config /etc/senseon-logship/senseon-logship.yaml

Validation errors include the file, line number, and a snippet of the offending region. Exit 0 means the config parses and references real files / channels.

Can the host reach the sink?

sudo senseon-logship-ctl test-sink --config /etc/senseon-logship/senseon-logship.yaml

Exit codes:

Code Meaning
0 OK
2 Config error
3 DNS resolution failed
4 TLS handshake failed (check tls.ca_bundle if you use a custom CA)
5 Got an HTTP error response from the sink
6 Network error before any HTTP response

Is the daemon healthy and ingesting events?

sudo senseon-logship-ctl status

Per-source output shows:

  • Active tailers and read offsets
  • Event counts and rates
  • Time of the last event seen
  • Five-minute error tallies

Common patterns:

Symptom Likely cause
ACTIVE 0 for a file source Paths don't exist or are unreadable
LAST EVENT never but the file has content Files present at startup are followed from the end — pre-existing content is not re-ingested
Sink shows FAILING, spooling Sink unreachable; events are spooling; will drain when connectivity returns
dropped counter rising Outage outlasted max_disk_mb. Raise the cap if you have disk
Parse errors counted but events arriving Source format does not match the actual content. Lines that fail their format fall back to raw

What's actually being sent?

sudo senseon-logship-ctl tail app_logs --lines 20 --show-wire

Replaces a guessing game with the exact JSON Logship is about to send — after parsing, filtering, and redaction.

Common situations

  • Syslog listener silent. Confirm the bind with ss -ulnp | grep <port> (Linux) or Get-NetUDPEndpoint (PowerShell). Open the host firewall. Test from another host with logger -n <ip> -P <port> -d "test".
  • Whole file re-sent after a restart. The state directory was wiped or moved. Offsets live in <state>/offsets.json. Restore it to keep offsets stable across upgrades.
  • Windows event channel shows ACTIVE 0. The service account does not have read access. Add it to the Event Log Readers group, or run as LocalSystem.
  • Rotation locked on Windows. Logship opens files with delete-sharing enabled. If a rotator still reports the file as locked, another process is holding a handle. Use Sysinternals handle.exe to identify it.
  • macOS permission denied on /var/log. Grant Full Disk Access to the binary at /usr/local/bin/senseon-logship or run the daemon as root via launchd (default).
  • Antivirus / EDR interference. Add the Logship install directory and senseon-logship.exe / senseon-logship-ctl.exe to your AV product's exclusions, particularly if you are running Microsoft Defender for Endpoint. See EPP Exclusions for the SenseOn-side equivalent.

If none of the above resolves the issue, run the daemon by hand with debug logging and a local diagnostics endpoint:

sudo senseon-logship \
  --config /etc/senseon-logship/senseon-logship.yaml \
  --log-level debug \
  --debug-addr 127.0.0.1:6060

This exposes /health and /debug/pprof/ on loopback and prints structured JSON logs to stderr. Capture that output and send it to support@senseon.io along with your config.

Updating

senseon-logship-ctl version

Logship does not auto-update during pre-release testing. Upgrade by installing a newer package over the existing one:

# Debian / Ubuntu
sudo dpkg -i senseon-logship_<new-version>_amd64.deb

# RHEL / CentOS / Fedora
sudo rpm -Uvh senseon-logship-<new-version>.x86_64.rpm

# macOS
sudo installer -pkg senseon-logship_<new-version>_darwin_arm64.pkg -target /

# Windows
msiexec /i senseon-logship_<new-version>_windows_amd64.msi /qn

The state directory (offsets, host ID, spool) is preserved across upgrades. The service picks up the new binary on its next restart, which the installer triggers automatically.