Build 124

3 May, 2017

Features

Instana had search capabilities for some time now. You could search through entities, events and traces. Unfortunately though, combining these results was a manual process. This left much to be desired. With this release, we are giving you a feature which we love and call dynamic search. Dynamic search enables you to search through all the Instana data contexts, i.e. entities, traces and events, and to search across these contexts. This enables you to express interesting queries such as:

search movie

  • Show entities with open issues.
  • Show traces in which entities are involved that have open issues.
  • Show entities that were involved in slow traces.
  • Much, much more…

query Or to even extend the previous query to cross entity / trace / event queries: Spring Boot apps with open issues involved in slow traces.

On top of this, we redesigned the search bar from the ground up. The search bar will now aide you in writing lucene queries. We wrote custom tooling so that syntax highlighting of lucene queries works. The search bar is also no longer a plain input, but under the hood a full blown editor. Thanks to this, many lucene query string pitfalls become easier to avoid, e.g. the free text query and vs. the boolean operator AND (size does matter after all).

query Syntax highlighting even helps understand uppercase AND / lowercase and differences in lucene query strings. Also note the new keyword hierarchy which, in combination with auto completion, makes it easier to discover keywords.

Next, the search bar now features an IDE grade proposal of search fields and field values. This makes it much easier to learn about Instana’s search capabilities for new users. For advanced users, writing queries is much faster. Also, you no longer need to remember all those keywords. Just start typing! Selection of proposal works again, just like your IDE. Use arrow up/down to navigate through proposal or enter to select a proposal.

query Show all traces which interacted with some kind of database. Note how the database value is proposed based on the current input.

The last major feature is the ability to search through span data. This often requested feature is the extension of the trace search feature. With the help of the new span… keywords and much more span categories under span.type, e.g. database, http and messaging, this makes it much easier to describe ad hoc queries to find slow calls. You even can search for content in spans using the span.content keyword (e.g. for all update statements taking longer than 200 ms).

query All traces with JDBC calls slower than 50ms.

Sortable Tables

A common request we received was that, while the tables within the dashboards provide very helpful information, users were lacking capabilities to make sense of this data. With this release we reworked all tables within Instana with the following major goals:

  1. Allow sorting of tables.
  2. Sort tables according to columns with live updating metrics.
  3. Make it easier to understand that table rows can be expanded by adding a + character before each expandable row.
  4. Fix issues with inverted row colors that made it hard to read spark charts.
  5. Render percentages as bar charts within cells to make percentages easier to compare visually.
  6. Support customization of table update and sorting frequency so that large dashboards remain responsive on old machines.
  7. Add paging for large tables to further performance boosts.

We are happy to announce that we all this made it into this release. On the following screenshots, you can see a few use cases for the new table. And in case you are working on old or modern hardware, don’t forget to check out the settings to tune the table refresh rate to your needs.

refresh rate JVM memory pools sorted by current pool size with expanded old gen row.

process list Which process is consuming most CPU resources? Host process top list sorted by CPU usage.

docker list Which Docker container is transmitting the largest amount of data? Docker container table sorted by mean number of bytes transmitted per second over a 10 minute window.

ui settings Setting to tune down the table refresh rate.

Comparison table for processes

top and htop and similar utilities are very helpful for day-to-day tools for administrators. With the process comparison table, you now get a top-like list that helps you understand resource utilization across many hosts or on a single hosts. And of course the search functionality can be used to restrict the number of matches as you see fit.

comparison table Process comparison table showing the new table pagination in the top-right corner.

Performance Improvements

We are continuously improving Instana’s performance. In this release however, we did focus on it a lot.

  • Performance optimizations of the agent daemon process
  • Performance optimizations of the trace instrumentations
  • Performance optimizations of the backend services
  • Performance optimizations of the 3D and 2D user interfaces

LDAP for Authorization for On-Premises Installations

Instead of Instana’s built-in authorization, On-Premises installations can use LDAP with this release.

Improved Incident Detection

With this release we introduce an improved incident detection.

First of all we improved the algorithms detecting incidents on the service KPIs (load, response time, error rate) to better cope with volatile metrics.

As another improvement we changed the logic when an incident is created. With this release incidents are triggered only if an edge service is affected. Those services are what customers or other unmonitored systems access and what your monitored application delivers. Take for example this situation:

incident

The Service “Shop App Shop” is the edge service (shown by the unknown service calling it or by an End User Application) that is calling the HTTP service “productsearch” which is then calling the Elasticsearch index “products”.

The incident is triggered the moment an issue is detected on the “Shop App Shop” service - resulting in automatically traversing the graph and all related events (issues, changes) that are related and thereby assisting root cause analysis.

A third improvement is that also critical infrastructure issues now trigger an incident. After a lot of feedback we start with the issues detecting and forecasting disk saturation and the split brain situation of Elasticsearch clusters because most likely result in data loss.

Easier agent installation

We have improved the agent download.

Currently you have three different possibilities to download and install the Instana agent. The manual download provides a full Instana agent specific for your operating system. The one-liner is an easier method to install the agent on Linux systems. You can configure the package that will be installed right on the new view.

Last but not least, we provide a docker based installation.

The needed information like base url and agent key is automatically pre filled for you.

agent download

Node.js health check support

The Instana Node.js sensor will now pick up custom health check and execute these health checks every ten seconds. Once these checks are failing for at least 30 seconds, an issue is raised to inform you about this problem.

Health checks are gathered from installed admin-plugin-healthcheckmodules.

health check Health checks listed in the Node.js dashboard.

health check Issue raised about failing health checks.

Newly supported Technologies

.NET Tracing

With the first release of .NET Tracing, we shed light on the internal workings and communications of your .NET based applications. Our aim is to deliver the same insights as we do for other stacks. This release already supports several technologies and will enable you to gather information about your applications. However, we are still working on instrumenting more libraries and frameworks to make this module valuable for everyone.

Supported trace technologies

Entries

  • Requests to IIS/IISExpress hosted applications (using the Instana.ManagedTracing.Modules library)
  • Requests to WCF-Service methods

Intermediates

  • Requests to ASP.NET WebForms pages
  • Requests to ASP.NET WebAPI controllers
  • Requests to ASP.NET MVC controllers

Exits

  • Queries against MSSQL-Server databases (using System.Data.SqlClient.SqlCommand)
  • Queries against PostgreSQL-Databases (using Devart.Data.PostgreSql.PgCommand)
  • Queries against Elasticsearch (NEST.ElasticClient)
  • HTTP-Calls (using System.Web.Http.HttpClient)
  • Requests going out to WCF-Services

We currently support the Classic/Full Framework on Windows only. The tracing for the CLR is disabled by default. You will find information on how to enable tracing for your applications in the docs.

Hadoop YARN

Monitored configuration

  • Cluster ID
  • Version
  • State
  • Started At

Monitored metrics, events, and overviews

  • Cluster Nodes
  • Cluster Apps
  • Cluster Containers
  • Cluster Memory
  • Cluster Virtual Cores
  • Nodes
  • Most recent Apps

OpenLDAP

Many more new metrics.

Improvements

  • Host sidebar shows uptime/start timestamp of server
  • Docker container table shows creation and start date and time
  • Support monitoring of the same Docker container ID on different hosts.
  • Notifications always show entity type and name.
  • Support for OpenStack Nova virtualization
  • Prevent misdetection of proxy responses as valid Google - - Compute or Amazon Ec2 metadata responses
  • Improved permission checking in mysql sensor
  • Improved distributed tracing in Mule ESB when vm transport is used.
  • Initial support for garden/runc containers running in Cloud Foundry
  • Improved discovery of Websphere Servers, support jmx requiring authentication.
  • Extended Memcached Sensor support to memcached 1.2
  • Go: Now reports using SDK span types; better display and mapping
  • Go: New Events API 🎆 report events directly from Go to your dashboard
  • PHP: Support for PHP compiled with ZTS
  • PHP: Batching of large amounts of SQL spans
  • PHP: Error reporting for PDO
  • Redis Slow log will now discard binary data (and replace it with )
  • EUM application error counts are now broken down into XMLHttpRequest (AJAX) errors and general uncaught JavaScript errors. The old error_rate metric was deprecated for EUM service due to its unpredictable behavior.
  • Support Amazon SES for on prem backends
  • Slack notifications of incidents and issues now contain the same detailed information as email notifications.
  • Unmonitored Hosts table now has a separate column for the reverse lookup
  • Trace Table marks traces that ended with an error
  • Go is now named Go instead of Go
  • Configurable Service Extraction for Message Brokers
  • ActiveMQ Dashboard is now sorted consistently
  • Support searching docker containers by their Id
  • UI login cookies live now 2 weeks instead of one day
  • Improved compatibility of the agent with Kubernetes CNI networking
  • Reverse resolved DNS names no longer have the trailing dot
  • Support CPU profiling in Node.js >=7.0.0.
  • Grouping of hosts in the infrastructure map according to their EC2 / GCE instance type. grouping

Fixed Issues

  • Agent could leave temp files behind when trying to monitor an incompatible JVM.
  • Agent could stop communicating with apps running in docker Slf4j loggers are no longer counted multiple times against error totals.
  • Fixed crash in Php 5.3 when Suhosin patch is used.
  • Go: Fixed ID generation
  • Processes that had a filtered command line argument could not be monitored if that command line argument was important for configuration or authentication.

Known Issues

  • Search for terms that include white spaces and wildcards is not supported yet (e.g. ‘entity.label:”Shop *Service”’ does not work). If you remove the whitespace the search works (e.g. ‘entity.label:”Shop*Service”’ results in a working query).