Skip to content

Troubleshooting

Use this page to troubleshoot a problem yourself before you contact support@senseon.io.

There are two generations of the SenseOn agent. SenseOn supports both generations:

  • Version 7.0 and later has two components. The bootstrapper manages installation and updates. The sensor (senseon-seed) collects telemetry.
  • Version 6 and earlier is the legacy agent. It has the sensor only. These devices do not have a bootstrapper.

Most issues have one of two causes. A component does not run, or a connectivity problem prevents its connection to the SenseOn platform.

Do the steps that follow in sequence. Each step ends with a fix, or it tells you what to send to support. Step 2 tells you which generation a device has. It also sends you to the steps that apply to that generation.

💡 Want background on what the agent does? Refer to How the Universal Sensor Works.


Step 1: Is the system clock correct?

Make sure that your system clock is correct. If possible, use NTP. If the clock is ahead of or behind real time, installs can stop with errors such as token expired. The SenseOn platform can also reject TLS connections. Do this check first, because an incorrect clock can look like any of the problems in the later steps.

Run these commands in an elevated PowerShell terminal:

w32tm /query /status      # shows the time source and last successful sync
w32tm /resync             # force a sync with the configured time source
timedatectl                       # shows the clock, timezone, and whether NTP is synced
sudo timedatectl set-ntp true     # enable NTP sync if it is reported as off
sudo systemsetup -getusingnetworktime      # reports "Network Time: On" or "Off"
sudo systemsetup -setusingnetworktime on   # enable network time if it is off

For the token expired install error, refer to Install fails with "token expired".

Clock was wrong? Correct it. Then try again the operation that did not work.

If the install did not work, do the check in Step 2 before you run the install command again. The command in Settings > Universal Sensor installs version 7.0. Thus, on a legacy device, the command replaces the agent and does not repair it. To rebuild a legacy device, use Installing the legacy agent as an alternative.

If the agent is already installed and could not connect, do not use the installer first. Do the steps that follow, and make sure that the connection works in Step 7. Some of those steps end with a new installation, but only after they find a cause that makes it necessary.

Clock is correct? Continue to Step 2.


Step 2: Is the bootstrapper installed?

This step is also the test that shows which generation a device has. A device with a bootstrapper has version 7.0 or later.

A device with a sensor and no bootstrapper usually has the legacy agent, version 6 or earlier. But a version 7.0 uninstall that stopped before completion gives the same result. The sensor version shows the difference, and Step 4 reads the version.

Open Add/Remove Programs (Start → search "add or remove programs"). Look for SenseOn Bootstrapper.

systemctl status senseon-bootstrapper
sudo launchctl list io.senseon.bootstrapper

Installed? This device has version 7.0 or later. Continue to Step 3.

Not installed? Do not install over the device at this time. Go to Step 4: Is the sensor installed?. Do the check there first:

  • A sensor is present, on version 6 or earlier. This is a legacy device. SenseOn supports this state. It is not a fault. Step 3 applies to version 7.0 and later only. Skip Step 3 and continue at Step 5: Is the sensor running?. Steps 5 to 8 apply to the legacy agent without changes.
  • A sensor is present, on version 7.0 or later. This is not a legacy device. A version 7.0 sensor with no bootstrapper usually shows that an uninstall stopped before completion. Remove the sensor (refer to Uninstalling SenseOn). Then install again with the command from Settings > Universal Sensor.
  • No sensor either. Nothing is installed. Go to the installation guide. If the install script stopped with an error, refer to Running the Install Script in Verbose Mode for detailed output.

âš  Do not install over a legacy device that works, unless you want to move the device to version 7.0. The install replaces the agent of the device. The install command in Settings > Universal Sensor installs the version 7.0 bootstrapper. The bootstrapper then replaces the legacy sensor with a version 7.0 sensor. To rebuild a device on the legacy agent, refer to Installing the legacy agent.


Step 3: Is the bootstrapper running?

ℹ Note: Version 7.0 and later only. A legacy device has no bootstrapper service. Skip this step and go to Step 5.

Open Services (Start → search services.msc). Make sure that senseon-bootstrapper shows the status Running.

To start the service, right-click it and select Start.

systemctl status senseon-bootstrapper

To start it:

sudo systemctl start senseon-bootstrapper
sudo launchctl list io.senseon.bootstrapper

To start it:

sudo launchctl start io.senseon.bootstrapper

Not running? Start it. Wait a few minutes. Then continue to Step 4.

Running? Continue to Step 4.


Step 4: Is the sensor installed?

This check applies to both generations. On version 7.0 and later, the bootstrapper installs the sensor when the bootstrapper runs for the first time. Usually, the installation completes within approximately ten minutes. If the bootstrapper runs but the sensor is not installed, a connectivity issue is the probable cause. The issue prevented the connection from the bootstrapper to the SenseOn servers during setup. On a legacy device, a direct installation of the sensor package installs the sensor. Thus the sensor is there or it is not there.

Open Add/Remove Programs and look for SEE, published by SenseOn. As an alternative, make sure that C:\Program Files\senseon-see\ exists.

systemctl status senseon-seed
sudo launchctl list io.senseon.endpoint-sensor

You can read the sensor version on the device. This method works even if the device never registered. Use the method for your operating system:

Find SEE in Add/Remove Programs. The entry shows the version.

dpkg -l senseon-see    # Debian and Ubuntu
rpm -q senseon-see     # RHEL and CentOS
pkgutil --pkg-info io.senseon.endpoint-sensor

A registered device also shows its version in Assets > Devices.

Not installed, and the bootstrapper is? The most probable cause is a DNS or TLS connectivity issue. The issue prevented the connection from the bootstrapper to the SenseOn servers. Refer to Step 7: Check connectivity.

On Windows, if the installation fails with MSI Installer Error 0x643, refer to Installation fails with MSI error 1603 (0x643).

If connectivity looks correct and the sensor is still not installed, send the bootstrapper log to support@senseon.io:

  • Windows: %ProgramData%\senseon-bootstrapper\logs\bootstrapper.log
  • Linux / macOS: /var/senseon-bootstrapper/logs/bootstrapper.log

Not installed, and no bootstrapper either? Nothing is installed on this device. Go to the installation guide.

Installed? Continue to Step 5.


Step 5: Is the sensor running?

Open Services (Start → search services.msc). Make sure that senseon-seed shows the status Running.

To start the service, right-click it and select Start.

systemctl status senseon-seed

To start it:

sudo systemctl start senseon-seed
sudo launchctl list io.senseon.endpoint-sensor

To start it:

sudo launchctl start io.senseon.endpoint-sensor

Not running? Start it. Wait a few minutes. Look for the device in the platform. If the device does not appear, continue to Step 6.

Running but not appearing in the platform? After a new install, wait a few minutes. The device can take a short time to register. If the device still does not appear after five minutes, continue to Step 6.


Step 6: Check the configuration

The sensor must have a valid installer key to connect to the platform. Examine the contents of the flags file:

Open C:\Program Files\senseon-see\see.flags in Notepad. A correct file contains several lines that start with flags such as --tls_hostname and --api_key. If the file is empty or contains only {}, the installation of the sensor did not have an installer key.

see.flags file in Windows

sudo cat /etc/senseon-see/see.flags
sudo cat /var/senseon-see/see.flags

A file with a correct configuration looks like this:

--tls_hostname=ep.<omitted>.snson.net:443
--tls_secondary_hostname=
--api_key=<omitted>
--tls_server_certs=...
--tls_client_cert=...
--tls_client_key=...
--logger_plugin=tls
--verbose=false
--allow_unsafe=false

Empty or contains only {}? The installation of the sensor did not have an installer key. Thus the sensor has nothing to enrol with. The correct action is different for each generation.

On version 7.0 and later, uninstall SenseOn (refer to Uninstalling SenseOn). Make sure that you remove the bootstrapper and the sensor. Then run the install command from Settings > Universal Sensor again with SENSEON_INSTALLER_KEY set. On Windows, if you run the install command again over a bootstrapper that is already installed, the command does not register the agent again.

On a legacy device, it is usually not necessary to install the agent again. Each time the sensor starts, it looks for the install key file. If the sensor finds a key file, it consumes the file. Put a valid key file in the correct location. Then restart the sensor:

Write the key file to C:\Windows\Temp\senseon_install.txt. Then restart the senseon-seed service from Services.

Put the key file at /tmp/senseon_install.txt. Then run this command:

sudo systemctl restart senseon-seed

Put the key file at /tmp/senseon_install.txt. Then run this command:

sudo seectl stop && sudo seectl start

Install the agent again only if the device still does not register after that. Refer to Installing the legacy agent. The Linux purge procedure in Uninstalling a legacy device also deletes the sensor data.

Looks correct? Continue to Step 7.


Step 7: Check connectivity

This step applies to both generations.

On version 7.0 and later, a successful install shows that the bootstrapper connected to the SenseOn servers. The bootstrapper downloaded the sensor through that path. It does not show that the sensor can connect. When the sensor runs, it reads its proxy from proxy.json. The install writes that file only if you pass --proxy on Universal Sensor 7.0.50 or later. A device that installed through a system proxy, or on an earlier version, can install without errors and then not connect. Do a check of the DNS and TLS rules that follow. If the device uses a proxy, also do a check of Proxy Configuration.

On a legacy device, the install shows nothing about the network. Enrolment reads a key file on the device, and a connection is not necessary. A legacy device installed behind a proxy that is not configured, or behind a firewall that blocks, never connected before this step. Thus, also apply the rules that follow to a device that never worked. Do not apply them only to a device that worked and then stopped.

DNS check

Run the command that follows to make sure that basic DNS resolution works:

nslookup avmirror.snson.net
nslookup avmirror.snson.net
dscacheutil -q host -a name avmirror.snson.net

A correct result gives one or more IP addresses. If there is no response, DNS resolution does not work. Examine the DNS configuration of the host.

💡 Tenant-specific hostnames: The sensor connects to a subdomain that is unique to your tenant. You can see the subdomain in see.flags as --tls_hostname. If you must do a check of those specific hostnames and no longer have your installer key, contact support. We can give you the hostnames. Send your request to support@senseon.io.

On a device that already has version 7.0 or later, there is an alternative. You can run the install script again with your installer key in diagnostics mode. This mode automatically does a check of DNS resolution and TCP connectivity to your tenant. Do not do this on a legacy device. The install script installs version 7.0 and replaces the agent that you troubleshoot.

TLS interception

The sensor uses mutual TLS 1.2+ for all communication. If your network does TLS interception, add a bypass for *.snson.net.

Refer to Endpoint Requirements — TLS Interception for details.

Connectivity looks fine but still not working? Continue to Step 8.


Step 8: Generate diagnostic logs

If the earlier steps did not correct the issue, generate logs. Send the logs to support@senseon.io.

bootstrapper logs

ℹ Note: Version 7.0 and later only. A legacy device has no bootstrapper log. Go directly to the section 'Sensor debug logs'.

Look in the bootstrapper log first for installation and connectivity failures:

  • Windows: %ProgramData%\senseon-bootstrapper\logs\bootstrapper.log
  • Linux / macOS: /var/senseon-bootstrapper/logs/bootstrapper.log

Sensor debug logs

The sensor streams telemetry directly to SenseOn. By default, the sensor does not write logs locally. If the sensor has no network connectivity, you can edit see.flags to make the sensor write local logs:

  1. Stop the sensor service (Services → senseon-seed → Stop).
  2. Open C:\Program Files\senseon-see\see.flags in a text editor.
  3. Change --logger_plugin=tls to --logger_plugin=tls,filesystem.
  4. Add a new line: --logger_path=C:\Program Files\senseon-see\log\.
  5. Change --verbose=false to --verbose=true.
  6. Save the file.
  7. Start the service again.
  8. After the issue occurs again, send the files in C:\Program Files\senseon-see\log\ to support.

💡 After you troubleshoot: When the issue is corrected, revert the see.flags changes.

  1. Stop the sensor: sudo systemctl stop senseon-seed
  2. Edit /etc/senseon-see/see.flags.
  3. Change --logger_plugin=tls to --logger_plugin=tls,filesystem.
  4. Add a new line: --logger_path=/var/log/senseon-see/.
  5. Change --verbose=false to --verbose=true.
  6. Save the file.
  7. Start the sensor: sudo systemctl start senseon-seed.
  8. After the issue occurs again, send the files in /var/log/senseon-see/ to support.

💡 After you troubleshoot: When the issue is corrected, revert the see.flags changes.

  1. Stop the sensor: sudo launchctl stop io.senseon.endpoint-sensor
  2. Edit /var/senseon-see/see.flags.
  3. Change --logger_plugin=tls to --logger_plugin=tls,filesystem.
  4. Add a new line: --logger_path=/var/log/senseon-see/.
  5. Change --verbose=false to --verbose=true.
  6. Save the file.
  7. Start the sensor: sudo launchctl start io.senseon.endpoint-sensor.
  8. After the issue occurs again, send the files in /var/log/senseon-see/ to support.

💡 After you troubleshoot: When the issue is corrected, revert the see.flags changes.


Other issues

Install fails with "token expired"

ℹ Note: Version 7.0 and later only. The legacy agent enrols from an install key file, not from a token. Thus the legacy agent does not give this error.

If the install stops with an error like the one that follows, the host system clock is almost always not in sync. The cause is almost never a token that is really expired.

ERROR ... Error validating token: token expired
Error: Failed to install token. Please re-run the installer again with a valid INSTALL_TOKEN set

The installer key that you paste into the command does not expire. SenseOn uses the key to mint a short-lived install token. The token is valid for one hour from the moment of its generation on the appliance. The host validates that token locally against its own system clock. Thus, if the clock is incorrect, the token can look expired although it is only moments old.

Two clock conditions give the same token expired message:

  • Clock is in the future. In this condition, the host clock is more than an hour ahead of real time. For the host, the one-hour validity window of the token is already complete.
  • Clock is in the past. If the host clock is behind real time, the host sees the token generation time as in the future. The host rejects the token. The host also reports this as token expired, which can cause confusion.

How to fix it: make sure that the host clock is correct and synced with NTP.

On Linux and macOS, you can compare the host clock against a known good time source. Use these commands:

date +%s        # Unix epoch seconds, compare against a trusted source
date -u         # human-readable UTC, rules out timezone confusion

On Linux, use these commands to examine and correct NTP sync:

timedatectl                       # shows the clock, timezone, and whether NTP is synced
sudo timedatectl set-ntp true     # enable NTP sync if it is reported as off

On macOS, use these commands to examine and correct network time:

sudo systemsetup -getusingnetworktime      # reports "Network Time: On" or "Off"
sudo systemsetup -setusingnetworktime on   # enable network time if it is off

After you correct the clock, run the install command again.

💡 If the host cannot use NTP: the one-hour validity window gives you some tolerance for a clock that is set slightly ahead of real time. But a clock that is set in the past always gives token expired. If you must set the time manually, set it accurately. Do not set it behind real time deliberately.

âš  Before you try again on RHEL/RPM hosts, remove the bootstrapper package. A token install that did not work can leave the bootstrapper package registered. If you then run the installer again, it reports package senseon-bootstrapper ... is already installed and does nothing. Use these commands:

sudo rpm -e senseon-bootstrapper     # remove the partially installed bootstrapper
rpm -q senseon-bootstrapper          # confirm it reports "not installed"

Then, after you correct the clock, run the install command again.

Installation fails with MSI error 1603 (0x643)

On Windows, the bootstrapper can install correctly, and then the sensor installation fails with MSI Installer Error 0x643. This error is Windows Installer error 1603. One possible cause is a Device Installation Restrictions policy that blocks the SenseOn driver. This cause is frequent where an organisation uses Group Policy or Intune to block USB devices. Usually, the installation fails on standard user workstations, and is successful on servers and IT workstations.

To find and correct this cause, refer to Windows Installer Error 1603 (0x643) and Device Installation Restrictions.

TLS interception can also cause error 1603. Refer to Endpoint Requirements: TLS Interception.

High memory usage

If the agent uses abnormally high memory and a service restart does not correct this, a process memory dump can help. The support team can use the dump to find the root cause.

Refer to Capturing a Process Memory Dump.

EPP exclusion issues

If you see false positives or blocked processes related to the Endpoint Protection (EPP) feature, refer to EPP Exclusions.


Running the Install Script in Verbose Mode

If the install script stops with an error, run it again with verbose output enabled. Send the output to support@senseon.io, together with the log files that this section lists for your operating system.

ℹ Note: The install script installs version 7.0 and later. The legacy agent does not use the install script. If a legacy install does not work, contact support@senseon.io and send the package manager or MSI output.

ℹ For network, proxy or certificate failures, use diagnostics mode instead of plain verbose. Pass -Diagnostics (Windows) or --diagnostics (Linux and macOS). Diagnostics mode does everything that verbose mode does. It also prints the resolved proxy and environment. It runs extra DNS, TCP and TLS connectivity checks against your appliance, and it reports the certificate that the host sees. Diagnostics mode makes more outbound connections. Thus, use it to troubleshoot, and not for routine or unattended installs. For example:

  • Windows: $env:SENSEON_INSTALLER_KEY = "<your-installer-key>"; powershell -ExecutionPolicy Bypass -File install.ps1 -Diagnostics
  • Linux / macOS: sudo SENSEON_INSTALLER_KEY="<your-installer-key>" bash install.sh --diagnostics

VERBOSE also works as an environment variable on both platforms. DIAGNOSTICS=true is different. It is an environment variable alternative on Linux and macOS only. On Windows, only the -Diagnostics switch works.

  1. Download the install script:
curl -fsSL https://<your-tenant>.senseon.io/install.sh -o install.sh
  1. Run it with verbose output:
sudo SENSEON_INSTALLER_KEY="<your-installer-key>" VERBOSE=true bash install.sh

Send us the terminal output and the file /var/log/apt/term.log.

  1. Download the install script:
curl -fsSL https://<your-tenant>.senseon.io/install.sh -o install.sh
  1. Run it with verbose output:
sudo SENSEON_INSTALLER_KEY="<your-installer-key>" VERBOSE=true bash install.sh

Send us the terminal output.

  1. Download the install script:
curl -fsSL https://<your-tenant>.senseon.io/install.sh -o install.sh
  1. Run it with verbose output:
sudo SENSEON_INSTALLER_KEY="<your-installer-key>" VERBOSE=true bash install.sh

Send us the terminal output and the file /var/log/install.log.

💡 Verbose output not appearing? The current release has a known issue: verbose mode does not work on macOS. Before you run the downloaded script, apply this one-line patch to it:

sed -i '' 's|echo "INFO: \$\*"|echo "INFO: $*" >\&2|' install.sh

âš  Run these steps in an elevated PowerShell terminal. Open Start and search for PowerShell. Then right-click it and select Run as administrator.

  1. Download the install script:
iwr -useb https://<your-tenant>.senseon.io/install.ps1 -OutFile install.ps1
  1. Set your installer key and run the script with verbose output:
$env:SENSEON_INSTALLER_KEY = "<your-installer-key>"
$env:VERBOSE = "true"
powershell -ExecutionPolicy Bypass -File install.ps1

Send us the terminal output. If the error occurs at the install stage, also send the MSI log. The error message gives the path. Usually, the path is:

C:\Users\<username>\AppData\Local\Temp\senseon-install.log