Monitoring AWS Lambda Python Functions
The Python APM Agent can be used with AWS Lambda to monitor the execution of your AWS Lambda functions.
The Centralized Agent Configuration on the Elasticsearch APM currently does NOT support AWS Lambda.
You need an APM Server to send APM data to. Follow theAPM Quick start if you have not set one up yet. For the best-possible performance, we recommend setting up APM on Elastic Cloud in the same AWS region as your AWS Lambda functions.
Both theElastic APM AWS Lambda extension and the Python APM Agent are added to your Lambda function asAWS Lambda Layers. Therefore, you need to add the corresponding Layer ARNs (identifiers) to your Lambda function.
To add the layers to your Lambda function through the AWS Management Console:
Navigate to your function in the AWS Management Console
Scroll to the Layers section and click theAdd a layer button
Choose theSpecify an ARN radio button
Copy and paste the following ARNs of the Elastic APM AWS Lambda extension layer and the APM agent layer in theSpecify an ARN text input:
APM Extension layer:
arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-extension-{{apm-lambda-ext-v}}-{ARCHITECTURE}:1
- Replace
{AWS_REGION}
with the AWS region of your Lambda function and{ARCHITECTURE}
with its architecture.
- Replace
APM agent layer:
arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-python-{{apm-python-v}}:1
- Replace
{AWS_REGION}
with the AWS region of your Lambda function.
- Replace
Click theAdd button
To add the Layer ARNs of the Elastic APM AWS Lambda extension and the APM agent through the AWS command line interface execute the following command:
aws lambda update-function-configuration --function-name yourLambdaFunctionName \--layers arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-extension-{{apm-lambda-ext-v}}-{ARCHITECTURE}:1 \arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-python-{{apm-python-v}}:1
- Replace
{AWS_REGION}
with the AWS region of your Lambda function and{ARCHITECTURE}
with its architecture. - Replace
{AWS_REGION}
with the AWS region of your Lambda function.
In your SAMtemplate.yml
file add the Layer ARNs of the Elastic APM AWS Lambda extension and the APM agent as follows:
...Resources: yourLambdaFunction: Type: AWS::Serverless::Function Properties: ... Layers: - arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-extension-{{apm-lambda-ext-v}}-{ARCHITECTURE}:1 - arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-python-{{apm-python-v}}:1...
- Replace
{AWS_REGION}
with the AWS region of your Lambda function and{ARCHITECTURE}
with its architecture. - Replace
{AWS_REGION}
with the AWS region of your Lambda function.
In yourserverless.yml
file add the Layer ARNs of the Elastic APM AWS Lambda extension and the APM agent to your function as follows:
...functions: yourLambdaFunction: handler: ... layers: - arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-extension-{{apm-lambda-ext-v}}-{ARCHITECTURE}:1 - arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-python-{{apm-python-v}}:1...
- Replace
{AWS_REGION}
with the AWS region of your Lambda function and{ARCHITECTURE}
with its architecture. - Replace
{AWS_REGION}
with the AWS region of your Lambda function.
To add theElastic APM AWS Lambda extension and the APM agent to your function add the ARNs to thelayers
property in your Terraform file:
...resource "aws_lambda_function" "your_lambda_function" { ... layers = ["arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-extension-{{apm-lambda-ext-v}}-{ARCHITECTURE}:1", "arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-python-{{apm-python-v}}:1"]}...
- Replace
{AWS_REGION}
with the AWS region of your Lambda function and{ARCHITECTURE}
with its architecture.
To add the Elastic APM AWS Lambda extension and the APM agent to your container-based function extend the Dockerfile of your function image as follows:
FROM docker.elastic.co/observability/apm-lambda-extension-{IMAGE_ARCH}:latest AS lambda-extensionFROM docker.elastic.co/observability/apm-agent-python:latest AS python-agent# FROM ... <-- this is the base image of your Lambda functionCOPY --from=lambda-extension /opt/elastic-apm-extension /opt/extensions/elastic-apm-extensionCOPY --from=python-agent /opt/python/ /opt/python/# ...
- Replace
{IMAGE_ARCH}
with the architecture of the image.
The Elastic APM AWS Lambda extension and the APM Python agent are configured through environment variables on the AWS Lambda function.
For the minimal configuration, you will need theAPM Server URL to set the destination for APM data and anAPM Secret Token. If you prefer to use anAPM API key instead of the APM secret token, use theELASTIC_APM_API_KEY
environment variable instead ofELASTIC_APM_SECRET_TOKEN
in the following configuration.
For production environments, we recommendusing the AWS Secrets Manager to store your APM authentication key instead of providing the secret value as plaintext in the environment variables.
To configure APM through the AWS Management Console:
- Navigate to your function in the AWS Management Console
- Click on theConfiguration tab
- Click onEnvironment variables
- Add the following required variables:
AWS_LAMBDA_EXEC_WRAPPER = /opt/python/bin/elasticapm-lambdaELASTIC_APM_LAMBDA_APM_SERVER = <YOUR-APM-SERVER-URL>ELASTIC_APM_SECRET_TOKEN = <YOUR-APM-SECRET-TOKEN>ELASTIC_APM_SEND_STRATEGY = background
- Use this exact fixed value.
- This is your APM Server URL.
- This is your APM secret token.
- TheELASTIC_APM_SEND_STRATEGY defines when APM data is sent to your Elastic APM backend. To reduce the execution time of your lambda functions, we recommend to use the background strategy in production environments with steady load scenarios.
To configure APM through the AWS command line interface execute the following command:
aws lambda update-function-configuration --function-name yourLambdaFunctionName \ --environment "Variables={AWS_LAMBDA_EXEC_WRAPPER=/opt/python/bin/elasticapm-lambda,ELASTIC_APM_LAMBDA_APM_SERVER=<YOUR-APM-SERVER-URL>,ELASTIC_APM_SECRET_TOKEN=<YOUR-APM-SECRET-TOKEN>,ELASTIC_APM_SEND_STRATEGY=background}"
- TheELASTIC_APM_SEND_STRATEGY defines when APM data is sent to your Elastic APM backend. To reduce the execution time of your lambda functions, we recommend to use the background strategy in production environments with steady load scenarios.
In your SAMtemplate.yml
file configure the following environment variables:
...Resources: yourLambdaFunction: Type: AWS::Serverless::Function Properties: ... Environment: Variables: AWS_LAMBDA_EXEC_WRAPPER: /opt/python/bin/elasticapm-lambda ELASTIC_APM_LAMBDA_APM_SERVER: <YOUR-APM-SERVER-URL> ELASTIC_APM_SECRET_TOKEN: <YOUR-APM-SECRET-TOKEN> ELASTIC_APM_SEND_STRATEGY: background...
- TheELASTIC_APM_SEND_STRATEGY defines when APM data is sent to your Elastic APM backend. To reduce the execution time of your lambda functions, we recommend to use the background strategy in production environments with steady load scenarios.
In yourserverless.yml
file configure the following environment variables:
...functions: yourLambdaFunction: ... environment: AWS_LAMBDA_EXEC_WRAPPER: /opt/python/bin/elasticapm-lambda ELASTIC_APM_LAMBDA_APM_SERVER: <YOUR-APM-SERVER-URL> ELASTIC_APM_SECRET_TOKEN: <YOUR-APM-SECRET-TOKEN> ELASTIC_APM_SEND_STRATEGY: background...
- TheELASTIC_APM_SEND_STRATEGY defines when APM data is sent to your Elastic APM backend. To reduce the execution time of your lambda functions, we recommend to use the background strategy in production environments with steady load scenarios.
In your Terraform file configure the following environment variables:
...resource "aws_lambda_function" "your_lambda_function" { ... environment { variables = { AWS_LAMBDA_EXEC_WRAPPER = /opt/python/bin/elasticapm-lambda ELASTIC_APM_LAMBDA_APM_SERVER = "<YOUR-APM-SERVER-URL>" ELASTIC_APM_SECRET_TOKEN = "<YOUR-APM-SECRET-TOKEN>" ELASTIC_APM_SEND_STRATEGY = "background" } }}...
- TheELASTIC_APM_SEND_STRATEGY defines when APM data is sent to your Elastic APM backend. To reduce the execution time of your lambda functions, we recommend to use the background strategy in production environments with steady load scenarios.
Environment variables configured for an AWS Lambda function are passed to the container running the lambda function. You can use one of the other options (through AWS Web Console, AWS CLI, etc.) to configure the following environment variables:
AWS_LAMBDA_EXEC_WRAPPER = /opt/python/bin/elasticapm-lambdaELASTIC_APM_LAMBDA_APM_SERVER = <YOUR-APM-SERVER-URL>ELASTIC_APM_SECRET_TOKEN = <YOUR-APM-SECRET-TOKEN>ELASTIC_APM_SEND_STRATEGY = background
- Use this exact fixed value.
- This is your APM Server URL.
- This is your APM secret token.
- TheELASTIC_APM_SEND_STRATEGY defines when APM data is sent to your Elastic APM backend. To reduce the execution time of your lambda functions, we recommend to use the background strategy in production environments with steady load scenarios.
You can optionallyfine-tune the Python agent or theconfiguration of the Elastic APM AWS Lambda extension.
That’s it. After following the steps above, you’re ready to go! Your Lambda function invocations should be traced from now on. Spans will be captured forsupported technologies. You can also usecapture_span
to capture custom spans, and you can retrieve theClient
object for capturing exceptions/messages usingget_client
.