Skip to content

SenseOn Log Ingestion - FAQ and Quick Reference

Quick Reference

The Essentials

  • Format: JSON (any valid JSON object)
  • Endpoint: /log_sink on your provided collector URL
  • Required: Valid JSON only - metadata is optional

Quick Test

curl -X POST https://your-collector-endpoint.senseon.io/log_sink \
  -H "Content-Type: application/json" \
  -d '{"test": "connection"}'

Frequently Asked Questions

Q: What log format does SenseOn require?

A: Any valid JSON object sent via HTTPS POST. That's the only requirement. While fields like timestamp, source, and severity are useful, they're not mandatory.

Q: Do I need specific fields in my logs?

A: No. Any valid JSON is accepted. However, including timestamps, source identifiers, and severity levels helps with analysis and correlation.

Q: What's the minimum valid log?

A: The absolute minimum is any valid JSON: 

{"event": "test"}

Q: What's a recommended log format?

A: For best results, include key context: 

{
  "timestamp": "2025-10-14T15:30:45Z",
  "source": "nginx",
  "level": "info",
  "remote_addr": "203.0.113.42",
  "request": "GET /api/health HTTP/1.1",
  "status": "200"
}

Q: How do I get my collector endpoint URL?

A: SenseOn provides your unique collector endpoint during onboarding. Contact your SenseOn representative if you need this information.

Q: Can I send batched logs?

A: Yes, if your forwarder supports it. However, each log entry should still be a valid JSON object. Some forwarders send one JSON object per request, others can batch.

Q: Can I send logs from multiple sources to the same endpoint?

A: Yes. All your log sources can send to the same collector endpoint. Use a source or type field to distinguish between different log sources.

Q: What network access do I need?

A: Your log sources need: - Outbound HTTPS (port 443) to your collector endpoint - DNS resolution for the collector domain - Stable internet connectivity

Q: Is my data encrypted in transit?

A: Yes. All logs are transmitted over HTTPS with TLS encryption.

Q: Can I test before sending production logs?

A: Yes. Use curl to send test logs and verify they're received: 

curl -X POST https://your-collector-endpoint.senseon.io/log_sink \
  -H "Content-Type: application/json" \
  -d '{"test": "verification", "timestamp": "2025-10-14T15:30:45Z"}'

Troubleshooting

Q: My logs aren't appearing in SenseOn. What should I check?

A: 1. Verify your forwarder service is running 2. Test the endpoint with curl 3. Verify JSON format is valid 4. Check firewall/security group rules 5. Contact SenseOn support with example logs

Q: I'm getting connection timeout errors. What's wrong?

A: 1. Verify the collector endpoint URL is correct 2. Check HTTPS (443) outbound is allowed 3. Check DNS resolution works 4. Review proxy settings if applicable

Q: How do I validate my JSON is correct?

A: Use jq or online JSON validators: 

echo '{"test": "log"}' | jq .

Q: FluentBit returns HTTP 500 error: "Could not add time to JSON for FluentBit"

A: This error occurs when using FluentBit's json output format. The SenseOn collector adds timestamps server-side and cannot parse FluentBit's JSON array wrapper format.

Solution: Change your FluentBit OUTPUT configuration to use json_lines format:

[OUTPUT]
    Name            http
    Match           *
    Host            your-collector-endpoint.senseon.io
    Port            443
    URI             /log_sink
    Format          json_lines    # Must be json_lines, not json
    json_date_key   false         # Let collector add timestamps
    tls             On
    Header          Content-Type application/json

Why this works: - The json format sends logs as an array with FluentBit metadata wrapper - The json_lines format sends newline-delimited JSON (NDJSON) - Each log is sent as a separate plain JSON object - The collector can parse and add timestamps to each individual object

After changing the configuration, restart FluentBit and verify logs show HTTP 201 responses.

Q: Can I send logs from cloud platforms (AWS, GCP, Azure)?

A: Yes. Use cloud-native log forwarding: - AWS: CloudWatch Logs + Lambda or Kinesis Firehose - GCP: Cloud Logging + Pub/Sub + Cloud Run - Azure: Azure Monitor + Event Hub Or run FluentBit as a VM/container in your cloud environment.

Q: What information should I provide when requesting support?

A: Include:

  • Your organization name
  • Collector endpoint URL
  • Example log entries (sanitized)
  • Error messages or symptoms
  • Log source type (nginx, syslog, etc.)
  • When the issue started
  • Configuration files (if applicable)