Data Collection

Concepts

Instana uses a single agent with multiple sensors. Currently, Instana supports over fifty sensors. These sensors are not extensions, they are completely updated, loaded, and unloaded by the agent without human intervention. There is even an optional command line interface which provides access to the agent state, individual sensors, and agent logs.

A sensor is designed to automatically discover and monitor a specific technology, and pass its data to the agent. The agent manages all communication to the Instana Service Quality Engine. After discovery, the sensor collects the details and metric data to provide an accurate representation of the component’s state. A specific sensor will gather specific types of data about their respective technologies, which will vary depending on the technology. Sensors collect the following:

  • Configuration – Catalogues current settings and states in order to keep track of any change;
  • Events – Initial Discovery and all state changes (online, offline);
  • Traces – Traces are captured based upon the programming language platform;
  • Metrics – Qualitative attributes of the technology that indicate performance.

In addition, discovery is recursive within a sensor. For example, the Java Machine sensor continues up the stack and discovers frameworks running on it (like Tomcat or Springboot), then assists the agent to load the appropriate additional sensors.

The intelligence combining this data into traces, discovering dependencies and services, and detecting changes and issues is done on the server. The agent is therefore lightweight, and can be injected into thousands of hosts.

Pipeline

The Instana backend utilizes streaming technology able to process millions of events per second streamed from the agents. Our streaming engine is effectively real-time, taking only 3 seconds to process the situation and display to the user.

Terminology

Agent

Instana’s Agent takes care of discovery, monitoring, and upgrading - all automatically. After installing the agent, it continuously discovers what runs on the machine, automatically using the appropriate sensors to monitor the components. In addition, it automatically maintains those sensors and its own internal components through self-updating capabilities.

The agent ships with a daily auto update that occurs around 4:30 am. However, users can configure this to update on specified days or times.

Sensor

The Instana Agent performs its monitoring function of the various technologies by deploying sensors into the system infrastructure. These sensors are custom built to best monitor their specified technology.

A list of sensors can be found on our agent sensors page.

Usage

To get started with data collection, head over to our agent setup docs, or learn more about our technology coverage in the agent sensors section.

Data Retention

Depending on the type of data, we apply varying data retentions. The following list describes for how long we are storing each data type.

  • Configuration & Graph Data: Kept for the lifetime of the contract
  • Event Data: Kept for the lifetime of the contract
  • Trace Data: Kept for 5 days
  • Metric Data: Retention is defined per granularity

    • 1 second granularity: Kept for 10 minutes
    • 5 second granularity: Kept for 24 hours
    • 1 minute granularity: Kept for 1 month
    • 5 minute granularity: Kept for 3 months
    • 1 hour granularity: Kept for 13 months