Collecting metrics data with Prometheus is becoming more popular. With Instana, it’s easy to capture Prometheus metrics and correlate them using our extensive knowledge graph. A typical example is custom business metrics.
The Instana Prometheus sensor doesn’t require a Prometheus server. The sensor captures metrics directly from the endpoints that are exposed by the monitored systems. For each Instana agent, specify which Prometheus endpoints you want to poll and which metrics should be collected from them using regular expressions.
For the Prometheus sensor to be able to collect data, metric sources must be defined in the agent configration file;
<agent_install_dir>/etc/instana/configuration.yaml. The sensor picks up any type of metric. With a regular expression, you can define which metrics should captured for a specific metric endpoint.
# Static Prometheus Configuration com.instana.plugin.prometheus: customMetricSources: - url: 'http://localhost:8080/metrics' # endpoint with exposed metrics metricNameIncludeRegex: '^sample_app_request' - url: 'http://localhost:9100/prometheus' # endpoint with exposed metrics metricNameIncludeRegex: '^sample_app_request'
Regex in the example above, configures a sensor to gather all metrics starting with
sample_app_request_duration. In the case that
metricNameIncludeRegex is not defined, the sensor collects all the exposed metrics until the defined limit.
Our Prometheus sensor detects if Kubernetes is being used and automatically gathers all the available pods on the node the sensor is running on. The URL field in the configuration is still required, but instead of specifying the whole URL, the path part is sufficient, e.g.
# Kubernetes Discovery Prometheus Configuration com.instana.plugin.prometheus: customMetricSources: - url: '/metrics' # url path part which expose metrics metricNameIncludeRegex: '^sample_app_request' - url: '/prometheus-metrics' # url path part which expose metrics metricNameIncludeRegex: '^sample_app_request'
Prometheus custom metrics
Prometheus metrics appear as “Prometheus Apps” associated with the host from where they are collected. You can query Prometheus custom metrics using the “Dynamic Focus”, “Event & Alerts”, and the “Grafana Plugin” with
- Counters (max. 700)
- Gauges (max. 700)
- Histograms (max. 700)
- Summaries (max. 700)