OS Process

Configuration

Process Abnormal Termination (Technical Preview)

Note: This section describes a functionality that is currently in Technical Preview.

The Instana agent can automatically detect abnormal termination of processes (e.g., crashes) of relevant processes on the host, and their exit status code. To be able to detect process crashes you will need to enable a dedicated sensor, which is disabled by default durring the Technical Preview phase:

com.instana.plugin.ebpf:
  enabled: true

The abnormal process termination of a process will be displayed on the dashboards of that process’s dashboard, as well as in the Event view.

Abnormal process termination displayed on the process's dashboard.

Known limitations:

  • The detection of abnormal process termination is only supported on Linux. A 4.7 or later Linux kernel is required or, in the case of RHEL, a Linux kernel 3.10.0-957 or later.
  • The detection of abnormal process termination does not currently work in container-optimized images on Google Cloud.
  • The detection of abnormal process termination does not currently work on Ubuntu-based Cloud Foundry stemcells.
  • The detection of abnormal process termination does not catch the termination the processes with pid 1; on most Linux hosts, this is not an issue, because pid 1 is the init system and if that dies, all processes like the Instana agent or any application follows suite. However, pid 1 processes are sometimes used in Docker containers to run applications, depending on how the image is built. (In such Docker containers, pid 1 is not the actual pid of your application, it is just the pid the process has in the container’s cgroup, but the limitation still applies.)

Custom Processes

Instana will automatically monitor process metrics of higher level sensors like Java or MySQL by default. Should you want to monitor an OS process which is not covered by Instana automatically, you can configure it like this:

com.instana.plugin.process:
  processes:
  - 'sshd'
  - 'slapd'

Voluntary and non-voluntary context switches (only supported on Linux hosts)

You can manually enable monitoring of context switches by editing the host’s agent configuration file (/opt/instana/agent/etc/instana/configuration.yaml):

...
com.instana.plugin.process:
  ctx_switches_enabled: true

OS Process Environment Variables

Instana’s process sensor automatically captures all the environment variables of any monitored process. Because environments often contains sensitive or secret data, the process sensor will take any configured secrets into account when filtering.

More about configuring secrets can be found at Agent Configuration secrets

configuring secrets

Windows Services

Instana supports monitoring of Windows Services and its child processes. You can configure it like this:

...
com.instana.plugin.process:
  services:
    - 'WindowsService1'
    - 'WindowsService2'

Metrics collection

Configuration data

  • PID
  • Executable
  • Started At
  • User
  • Group
  • Max Open Files
  • Arguments

Performance metrics

CPU usage

CPU usage values as a percentage; user and system. The values are displayed on a graph over a selected time period.

Data point: Filesystem

Granularity: 3 seconds

Memory

Memory usage values as a byte; virtual, resident and share. The values are displayed on a graph over a selected time period.

Data point: Filesystem

Granularity: 3 seconds

Open Files

Open files values used as a total number and current as a percentage. The values are displayed on a graph over a selected time period.

Open files current vs max will be visible when they are available on the operating system.

Data point: Filesystem

Granularity: 3 seconds

Number of context switches

Number of times the process was context-switched; voluntary and nonvoluntary. The values are displayed on a graph over a selected time period.

Data point: Filesystem

Granularity: 3 seconds

Health Signatures

For each sensor, there is a curated knowledgebase of health signatures that are evaluated continuously against the incoming metrics and are used to raise issues or incidents depending on user impact.

Built-in events trigger issues or incidents based on failing health signatures on entities, and custom events trigger issues or incidents based on the thresholds of an individual metric of any given entity.

For information about the built-in event for the OS process sensor, see the Built-in events reference.