AWS Lambda Native Tracing

Instana offers native tracing for AWS Lambda functions. This feature is currently available for AWS Lambda functions based on the Node.js and Python runtimes; the X-Ray integration is deprecated and will not be made Generally Available.

Native tracing of AWS Lambda functions provides large advantages over X-Ray based tracing:

  • The execution Lambda functions now seamlessly correlates with tracing of other monitored components. Calls from Hosts processes to Serverless and back are now linked, correlated and analyzed as such with no additional steps required.
  • Instana can now provide rich performance analysis between specific versions of the same AWS Lambda function.
  • Reduces Cost: Completely eliminates the need of pulling costly X-Ray data from AWS.
  • Provides automatic and deep trace visibility into the activities of your Lambda functions. A big improvement over the limited and very manual X-Ray alternative.

Overview

As described in the following sections, the steps required to enable native tracing of your AWS Lambda functions are:

  1. Ensure you are monitoring your AWS region
  2. Add the Instana Lambda Layer to your AWS Lambda function
  3. Add a custom handler to your AWS Lambda function
  4. Add required environment variables

Also refer to the Node.js and Python specific pages for details about those runtimes.

Supported Versions

AWS Lambda Runtimes

Runtime Versions
Node.js 8.10 (see note), 10.x, 12.x
Python 2.7, 3.6, 3.7, 3.8

Node.js 8.1.0: Please use the approach outlined in the section Instana Lambda Layer & Manual Wrapping.

Instana On-Premises

Instana Versions
On-Premises v170 and newer

Instana Agent Prerequisites

  1. Ensure you have set up Instana agents that monitor the AWS regions in which you deploy Lambda functions.
  2. Activate the monitoring of individual Lambda versions in the AWS Agent as described in the following section.

Enabling Monitoring of Individual Lambda Versions in the Agent

See our general AWS docs and the AWS Agent Installation docs for details on how to monitor AWS services with the Instana agent.

By default, the Instana AWS Lambda sensor will try to monitor each version of a Lambda function individually. This is a requirement for linking Lambda calls correctly to Lambda infrastructure. However, this requires the permission lambda:ListVersionsByFunction so please make sure that the IAM role you use in the AWS agent has that permission.

Configuring an AWS Lambda For Native Tracing

Instana AutoTrace

The preferred way to enable Instana tracing for your AWS Lambda functions is to use the Instana Lambda layer and a custom function handler. This approach requires no modification of the Lambda function code and is done purely through configuration. It is also suited to be automated or included in a Lambda deployment pipeline.

The following instructions are valid for both Node.js and Python AWS Lambda functions.

1. Add the Instana Lambda layer to your function

In the configuration page for your Lambda function, click on the “Layers” box and then on “Add a layer”.

layer
In the popup that opens, select “Provide a layer version ARN” Copy and paste the ARN for your AWS region into the ARN text input field.

layer selection

The list of Instana Lambda Layer ARNS are available per runtime. Select the appropriate layer for your region and runtime:

2. Configure the Lambda Handler

If you are using the default Lambda handler, then simply replace that value with a custom value.

For Node.js, set the handler to instana-aws-lambda-auto-wrap.handler. The Instana Lambda layer will automatically trigger the default Node.js runtime handler: index.handler.

For Python, set the handler to instana.lambda_handler. The Instana Lambda layer will automatically trigger the default Python runtime handler: lambda_function.lambda_handler.

If you are using a custom handler value (not the default runtime handler), specify that value in an environment variable LAMBDA_HANDLER to notify the Instana Lambda Layer.

new handler

3. Configure Environment Variables

Add the environment variables INSTANA_ENDPOINT_URL and INSTANA_AGENT_KEY with the required values as described below.

env vars

The value for INSTANA_ENDPOINT_URL depends on where your Instana tenant unit is deployed:

Your Instana Unit’s Region INSTANA_ENDPOINT_URL
US https://serverless-red-saas.instana.io/
EU https://serverless-blue-saas.instana.io/

The agent key for INSTANA_AGENT_KEY is the same that you use in all installed Instana agents. If you do not know your key, please contact your Instana representative to help you out.

As mentioned in the previous section, if you are using a custom handler for your Lambda function, make sure to specify that custom handler value in LAMBDA_HANDLER.

4. Save the Lambda function definition

save

See Also

  • See also the Node.js and Python Lambda runtime pages for more options and details specific to those runtimes.

Other Tools

All of this can be done either via the AWS web console or any of the usual AWS management tools, like

Here is an example aws CLI command that might serve as a starting point if you want to automate the Instana integration of your AWS Lambdas:

# Do not copy and paste this verbatim!
# It will overwrite any previously defined collection of layers and
# environment variables.
aws --region $YOUR_REGION lambda update-function-configuration \
  --function-name $YOUR_LAMBDA_FUNCTION_NAME \
  --layers $INSTANA_LAYER_ARN \
  --handler instana-aws-lambda-auto-wrap.handler
  --environment ""Variables={LAMBDA_HANLDER=yourActual.handler,INSTANA_ENDPOINT_URL=...,INSTANA_AGENT_KEY=...}""

Other Configuration Options

Environment Variables

The following is a list of optional environment variables that are generally supported in native Lambda tracing:

Environment Variable Meaning
INSTANA_TIMEOUT Timeout for the HTTP requests reporting data to the Instana back end
INSTANA_EXTRA_HTTP_HEADERS Semicolon-separated list of HTTP headers to be captured
INSTANA_SERVICE_NAME Custom service name
INSTANA_LOG_LEVEL The log level for the Instana package, possible values are debug, info, warn, and error (default: info)
INSTANA_DEBUG Set this to any value to set the log level to debug.
INSTANA_DISABLE_CA_CHECK Set this to true to disable verifying the server certificate against the list of CAs baked into Lambda runtime when connecting to the Instana back end. Enabling this makes your lambda vulnerable to MITM attacks for this connection. This setting should never be used, unless you use Instana On-Premises and are unable to operate the Instana back end with a certificate with a known root CA. (Available since @instana/aws-lambda@1.93.0/layer version 25.)

Capturing HTTP Headers

To capture HTTP headers (for trigger types API gateway with Lambda proxy integration or application load balancer), you need to provide the environment variable INSTANA_EXTRA_HTTP_HEADERS with a semicolon-separated list of headers to capture.

Additional Information

HTTP Call Attributes, API Gateway & Lambda Proxy Integration

Instana offers detailed capturing of HTTP attributes for Lambda executions that are triggerd by a trigger of type “API Gateway” or “Application Load Balancer”. This includes extracting the URL, path templates, the status code, query parameters etc. The standard enpoint extraction uses this attributes, too.

However, for API Gateway calls, HTTP attributes can only be captured if the option “Use Lambda Proxy integration” is used when defining the API Gateway’s methods. After the creation of a an API Gateway’s methods, this can be checked by inspecting the “Integration Request” box on the API Gateway configuration page. If it says “Type: LAMBDA PROXY”, it uses the Lambda Proxy integration.

This constraint does not apply to “Application Load Balancer” triggers.

Captured Meta Data For Triggers

A Lambda invocation will be traced no matter how it is triggered. Native Lambda tracing will capture additional meta data for the following triggers:

Trigger Meta Data Supported in Lambda Runtimes
API Gateway1 HTTP method, URL, Path Template, Query Parameters, Headers2 Node.js, Python
Application Load Balancer HTTP method, URL, Query Parameters, Headers2 Node.js, Python
Cloudwatch Event Event Resources Node.js, Python
Cloudwatch Logs Log Groups, Log Stream, Log Events Node.js, Python
S3 S3 Event Name, Bucket Name, Object Key Node.js, Python
SQS SQS Queue ARN Node.js, Python

  1. Additional meta data will only be captured if the Lambda Proxy Integration option is used, see HTTP Call Attributes, API Gateway & Lambda Proxy Integration.

  2. Only headers configured via INSTANA_EXTRA_HTTP_HEADERS will be captured, see Capturing HTTP Headers.