Instrument Azure App Service with serverless-init - Linux Containers
Overview
This instrumentation method uses serverless-init and provides the following additional monitoring capabilities for containerized Linux Azure App Service workloads:
- Fully distributed APM tracing using automatic instrumentation.
- Customized APM service and trace views showing relevant Azure App Service metrics and metadata.
- Support for manual APM instrumentation to customize spans.
Trace_ID injection into application logs.- Support for submitting custom metrics using DogStatsD.
Prerequisites
Make sure you have a Datadog API Key and are using a programming language supported by a Datadog tracing library.
Instrument your application
Dockerfile
Datadog publishes new releases of the serverless-init container image to Docker Hub, Google’s gcr.io, and AWS’s ECR:
| dockerhub.io | gcr.io | public.ecr.aws |
|---|
| datadog/serverless-init | gcr.io/datadoghq/serverless-init | public.ecr.aws/datadog/serverless-init |
Images are tagged based on semantic versioning, with each new version receiving three relevant tags:
1, 1-alpine: use these to track the latest minor releases, without breaking changes1.x.x, 1.x.x-alpine: use these to pin to a precise version of the librarylatest, latest-alpine: use these to follow the latest version release, which may include breaking changes
Add the following instructions and arguments to your Dockerfile.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
RUN npm install --prefix /dd_tracer/node dd-trace --save
ENV DD_SERVICE=datadog-demo-run-nodejs
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
CMD ["/nodejs/bin/node", "/path/to/your/app.js"]
Explanation
Copy the Datadog serverless-init into your Docker image.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
Copy the Datadog Node.JS tracer into your Docker image.
RUN npm install --prefix /dd_tracer/node dd-trace --save
If you install the Datadog tracer library directly in your application, as outlined in the manual tracer instrumentation instructions, omit this step.
(Optional) Add Datadog tags.
ENV DD_SERVICE=datadog-demo-run-nodejs
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
Change the entrypoint to wrap your application in the Datadog serverless-init process.
Note: If you already have an entrypoint defined inside your Dockerfile, see the alternative configuration.
ENTRYPOINT ["/app/datadog-init"]
Execute your binary application wrapped in the entrypoint. Adapt this line to your needs.
CMD ["/nodejs/bin/node", "/path/to/your/app.js"]
Alternative configuration
If you already have an entrypoint defined inside your Dockerfile, you can instead modify the CMD argument.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
RUN npm install --prefix /dd_tracer/node dd-trace --save
ENV DD_SERVICE=datadog-demo-run-nodejs
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
CMD ["/app/datadog-init", "/nodejs/bin/node", "/path/to/your/app.js"]
If you require your entrypoint to be instrumented as well, you can swap your entrypoint and CMD arguments instead. For more information, see How serverless-init works.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
RUN npm install --prefix /dd_tracer/node dd-trace --save
ENV DD_SERVICE=datadog-demo-run-nodejs
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
CMD ["/your_entrypoint.sh", "/nodejs/bin/node", "/path/to/your/app.js"]
As long as your command to run is passed as an argument to datadog-init, you will receive full instrumentation.
Add the following instructions and arguments to your Dockerfile.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
RUN pip install --target /dd_tracer/python/ ddtrace
ENV DD_SERVICE=datadog-demo-run-python
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
CMD ["/dd_tracer/python/bin/ddtrace-run", "python", "app.py"]
Explanation
Copy the Datadog serverless-init into your Docker image.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
Install the Datadog Python tracer.
RUN pip install --target /dd_tracer/python/ ddtrace
If you install the Datadog tracer library directly in your application, as outlined in the manual tracer instrumentation instructions, omit this step.
(Optional) Add Datadog tags.
ENV DD_SERVICE=datadog-demo-run-python
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
Change the entrypoint to wrap your application in the Datadog serverless-init process.
Note: If you already have an entrypoint defined inside your Dockerfile, see the alternative configuration.
ENTRYPOINT ["/app/datadog-init"]
Execute your binary application wrapped in the entrypoint, launched by the Datadog trace library. Adapt this line to your needs.
CMD ["/dd_tracer/python/bin/ddtrace-run", "python", "app.py"]
Alternative configuration
If you already have an entrypoint defined inside your Dockerfile, you can instead modify the CMD argument.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
RUN pip install --target /dd_tracer/python/ ddtrace
ENV DD_SERVICE=datadog-demo-run-python
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
CMD ["/app/datadog-init", "/dd_tracer/python/bin/ddtrace-run", "python", "app.py"]
If you require your entrypoint to be instrumented as well, you can swap your entrypoint and CMD arguments instead. For more information, see How serverless-init works.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
RUN pip install --target /dd_tracer/python/ ddtrace
ENV DD_SERVICE=datadog-demo-run-python
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
CMD ["your_entrypoint.sh", "/dd_tracer/python/bin/ddtrace-run", "python", "app.py"]
As long as your command to run is passed as an argument to datadog-init, you will receive full instrumentation.
Add the following instructions and arguments to your Dockerfile.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ADD 'https://dtdg.co/latest-java-tracer' /dd_tracer/java/dd-java-agent.jar
ENV DD_SERVICE=datadog-demo-run-java
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
CMD ["./mvnw", "spring-boot:run"]
Explanation
Copy the Datadog serverless-init into your Docker image.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
Add the Datadog Java tracer to your Docker image.
ADD 'https://dtdg.co/latest-java-tracer' /dd_tracer/java/dd-java-agent.jar
If you install the Datadog tracer library directly in your application, as outlined in the manual tracer instrumentation instructions, omit this step.
(Optional) Add Datadog tags.
ENV DD_SERVICE=datadog-demo-run-java
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
Change the entrypoint to wrap your application in the Datadog serverless-init process.
Note: If you already have an entrypoint defined inside your Dockerfile, see the alternative configuration.
ENTRYPOINT ["/app/datadog-init"]
Execute your binary application wrapped in the entrypoint. Adapt this line to your needs.
CMD ["./mvnw", "spring-boot:run"]
Alternative configuration
If you already have an entrypoint defined inside your Dockerfile, you can instead modify the CMD argument.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ADD 'https://dtdg.co/latest-java-tracer' /dd_tracer/java/dd-java-agent.jar
ENV DD_SERVICE=datadog-demo-run-java
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
CMD ["/app/datadog-init", "./mvnw", "spring-boot:run"]
If you require your entrypoint to be instrumented as well, you can swap your entrypoint and CMD arguments instead. For more information, see How serverless-init works.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ADD 'https://dtdg.co/latest-java-tracer' /dd_tracer/java/dd-java-agent.jar
ENV DD_SERVICE=datadog-demo-run-java
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
CMD ["your_entrypoint.sh", "./mvnw", "spring-boot:run"]
As long as your command to run is passed as an argument to datadog-init, you will receive full instrumentation.
Manually install the Go tracer before you deploy your application. Add the following instructions and arguments to your Dockerfile.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ENTRYPOINT ["/app/datadog-init"]
ENV DD_SERVICE=datadog-demo-run-go
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
CMD ["/path/to/your-go-binary"]
Explanation
Copy the Datadog serverless-init into your Docker image.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
Change the entrypoint to wrap your application in the Datadog serverless-init process.
Note: If you already have an entrypoint defined inside your Dockerfile, see the alternative configuration.
ENTRYPOINT ["/app/datadog-init"]
(Optional) Add Datadog tags.
ENV DD_SERVICE=datadog-demo-run-go
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
Execute your binary application wrapped in the entrypoint. Adapt this line to your needs.
CMD ["/path/to/your-go-binary"]
Alternative configuration
If you already have an entrypoint defined inside your Dockerfile, you can instead modify the CMD argument.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ENV DD_SERVICE=datadog-demo-run-go
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
CMD ["/app/datadog-init", "/path/to/your-go-binary"]
If you require your entrypoint to be instrumented as well, you can swap your entrypoint and CMD arguments instead. For more information, see How serverless-init works.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ENV DD_SERVICE=datadog-demo-run-go
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
CMD ["your_entrypoint.sh", "/path/to/your-go-binary"]
As long as your command to run is passed as an argument to datadog-init, you will receive full instrumentation.
Note: You can also use Orchestrion, a tool for automatically instrumenting Go code. Orchestrion is in private beta. For more information, open a GitHub issue in the Orchestrion repo, or contact Support.
Add the following instructions and arguments to your Dockerfile.
# For alpine or arm64 builds, refer to the explanation section
COPY --from=datadog/serverless-init:1 / /app/
RUN chmod +x /app/dotnet.sh && /app/dotnet.sh
ENV DD_SERVICE=datadog-demo-run-dotnet
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
CMD ["dotnet", "helloworld.dll"]
Explanation
Copy the Datadog serverless-init into your Docker image.
COPY --from=datadog/serverless-init:1 / /app/
Copy the Datadog .NET tracer into your Docker image.
For linux/amd64, include the following:
RUN chmod +x /app/dotnet.sh && /app/dotnet.sh
For other architecture types, configure your Dockerfile like so:
# For arm64 use datadog-dotnet-apm-2.57.0.arm64.tar.gz
# For alpine use datadog-dotnet-apm-2.57.0-musl.tar.gz
ARG TRACER_VERSION
ADD https://github.com/DataDog/dd-trace-dotnet/releases/download/v${TRACER_VERSION}/datadog-dotnet-apm-${TRACER_VERSION}.tar.gz /tmp/datadog-dotnet-apm.tar.gz
RUN mkdir -p /dd_tracer/dotnet/ && tar -xzvf /tmp/datadog-dotnet-apm.tar.gz -C /dd_tracer/dotnet/ && rm /tmp/datadog-dotnet-apm.tar.gz
If you install the Datadog tracer library directly in your application, as outlined in the manual tracer instrumentation instructions, omit this step.
(Optional) Add Datadog tags.
ENV DD_SERVICE=datadog-demo-run-dotnet
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
Change the entrypoint to wrap your application in the Datadog serverless-init process.
Note: If you already have an entrypoint defined inside your Dockerfile, see the alternative configuration.
ENTRYPOINT ["/app/datadog-init"]
Execute your binary application wrapped in the entrypoint. Adapt this line to your needs.
CMD ["dotnet", "helloworld.dll"]
Alternative configuration
If you already have an entrypoint defined inside your Dockerfile, you can instead modify the CMD argument.
# For alpine or arm64 builds, refer to tracer installation of the explanation section
COPY --from=datadog/serverless-init:1 / /app/
RUN chmod +x /app/dotnet.sh && /app/dotnet.sh
ENV DD_SERVICE=datadog-demo-run-dotnet
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
CMD ["/app/datadog-init", "dotnet", "helloworld.dll"]
If you require your entrypoint to be instrumented as well, you can swap your entrypoint and CMD arguments instead. For more information, see How serverless-init works.
# For alpine or arm64 builds, refer to tracer installation of the explanation section
COPY --from=datadog/serverless-init:1 / /app/
RUN chmod +x /app/dotnet.sh && /app/dotnet.sh
ENV DD_SERVICE=datadog-demo-run-dotnet
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
CMD ["your_entrypoint.sh", "dotnet", "helloworld.dll"]
As long as your command to run is passed as an argument to datadog-init, you will receive full instrumentation.
Manually install the Ruby tracer before you deploy your application. See the example application.
Add the following instructions and arguments to your Dockerfile.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ENV DD_SERVICE=datadog-demo-run-ruby
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENV DD_TRACE_PROPAGATION_STYLE=datadog
ENTRYPOINT ["/app/datadog-init"]
CMD ["rails", "server", "-b", "0.0.0.0"]
Explanation
Copy the Datadog serverless-init into your Docker image.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
(Optional) add Datadog tags
ENV DD_SERVICE=datadog-demo-run-ruby
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
This environment variable is needed for trace propagation to work properly in Cloud Run. Ensure that you set this variable for all Datadog-instrumented downstream services.
ENV DD_TRACE_PROPAGATION_STYLE=datadog
Change the entrypoint to wrap your application in the Datadog serverless-init process.
Note: If you already have an entrypoint defined inside your Dockerfile, see the alternative configuration.
ENTRYPOINT ["/app/datadog-init"]
Execute your binary application wrapped in the entrypoint. Adapt this line to your needs.
CMD ["rails", "server", "-b", "0.0.0.0"]
Alternative configuration
If you already have an entrypoint defined inside your Dockerfile, you can instead modify the CMD argument.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ENV DD_SERVICE=datadog-demo-run-ruby
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENV DD_TRACE_PROPAGATION_STYLE=datadog
CMD ["/app/datadog-init", "rails", "server", "-b", "0.0.0.0"]
If you require your entrypoint to be instrumented as well, you can swap your entrypoint and CMD arguments instead. For more information, see How serverless-init works.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ENV DD_SERVICE=datadog-demo-run-ruby
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENV DD_TRACE_PROPAGATION_STYLE=datadog
ENTRYPOINT ["/app/datadog-init"]
CMD ["your_entrypoint.sh", "rails", "server", "-b", "0.0.0.0"]
As long as your command to run is passed as an argument to datadog-init, you will receive full instrumentation.
Add the following instructions and arguments to your Dockerfile.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ADD https://github.com/DataDog/dd-trace-php/releases/latest/download/datadog-setup.php /datadog-setup.php
RUN php /datadog-setup.php --php-bin=all
ENV DD_SERVICE=datadog-demo-run-php
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
# use the following for an Apache and mod_php based image
RUN sed -i "s/Listen 80/Listen 8080/" /etc/apache2/ports.conf
EXPOSE 8080
CMD ["apache2-foreground"]
# use the following for an Nginx and php-fpm based image
RUN ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log
EXPOSE 8080
CMD php-fpm; nginx -g daemon off;
Note: The datadog-init entrypoint wraps your process and collects logs from it. To get logs working properly, ensure that your Apache, Nginx, or PHP processes are writing output to stdout.
Explanation
Copy the Datadog serverless-init into your Docker image.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
Copy and install the Datadog PHP tracer.
ADD https://github.com/DataDog/dd-trace-php/releases/latest/download/datadog-setup.php /datadog-setup.php
RUN php /datadog-setup.php --php-bin=all
If you install the Datadog tracer library directly in your application, as outlined in the manual tracer instrumentation instructions, omit this step.
(Optional) Add Datadog tags.
ENV DD_SERVICE=datadog-demo-run-php
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
Change the entrypoint to wrap your application in the Datadog serverless-init process.
Note: If you already have an entrypoint defined inside your Dockerfile, see the alternative configuration.
ENTRYPOINT ["/app/datadog-init"]
Execute your application.
Use the following for an apache and mod_php based image:
RUN sed -i "s/Listen 80/Listen 8080/" /etc/apache2/ports.conf
EXPOSE 8080
CMD ["apache2-foreground"]
Use the following for an nginx and php-fpm based image:
RUN ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log
EXPOSE 8080
CMD php-fpm; nginx -g daemon off;
Alternative configuration: CMD argument
If you already have an entrypoint defined inside your Dockerfile, and you are using an Apache and mod_php based image, you can instead modify the CMD argument.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ADD https://github.com/DataDog/dd-trace-php/releases/latest/download/datadog-setup.php /datadog-setup.php
RUN php /datadog-setup.php --php-bin=all
ENV DD_SERVICE=datadog-demo-run-php
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
RUN sed -i "s/Listen 80/Listen 8080/" /etc/apache2/ports.conf
EXPOSE 8080
CMD ["/app/datadog-init", "apache2-foreground"]
If you require your entrypoint to be instrumented as well, you can swap your entrypoint and CMD arguments instead. For more information, see How serverless-init works.
COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init
ADD https://github.com/DataDog/dd-trace-php/releases/latest/download/datadog-setup.php /datadog-setup.php
RUN php /datadog-setup.php --php-bin=all
ENV DD_SERVICE=datadog-demo-run-php
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
# use the following for an Apache and mod_php based image
RUN sed -i "s/Listen 80/Listen 8080/" /etc/apache2/ports.conf
EXPOSE 8080
CMD ["your_entrypoint.sh", "apache2-foreground"]
# use the following for an Nginx and php-fpm based image
RUN ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log
EXPOSE 8080
CMD your_entrypoint.sh php-fpm; your_entrypoint.sh nginx -g daemon off;
As long as your command to run is passed as an argument to datadog-init, you will receive full instrumentation.
Once the container is built and pushed to your registry, the last step is to set the required environment variables for the Datadog Agent:
DD_API_KEY: Datadog API key, used to send data to your Datadog account. It should be configured as an Azure Secret for privacy and safety.DD_SITE: Datadog endpoint and website. Select your site on the right side of this page. Your site is: DD_TRACE_ENABLED: Set to true to enable tracing.
For more environment variables and their function, see Additional Configurations.
3. Results
Once the deployment is completed, your metrics and traces are sent to Datadog. In Datadog, navigate to Infrastructure->Serverless to see your serverless metrics and traces.
Deployment
To update your Datadog instrumentation with zero downtime, use deployment slots. You can create a workflow that uses GitHub Action for Azure CLI.
See the sample GitHub workflow.
Additional configurations
Advanced Tracing: The Datadog Agent already provides some basic tracing for popular frameworks. Follow the advanced tracing guide for more information.
Logs: If you use the Azure integration, your logs are already being collected. Alternatively, you can set the DD_LOGS_ENABLED environment variable to true to capture application logs through the serverless instrumentation directly.
Custom Metrics: You can submit custom metrics using a DogStatsD client. For monitoring Cloud Run and other serverless applications, use distribution metrics. Distributions provide avg, sum, max, min, and count aggregations by default. On the Metric Summary page, you can enable percentile aggregations (p50, p75, p90, p95, p99) and also manage tags. To monitor a distribution for a gauge metric type, use avg for both the time and space aggregations. To monitor a distribution for a count metric type, use sum for both the time and space aggregations.
Trace Sampling: To manage the APM traced request sampling rate for serverless applications, set the DD_TRACE_SAMPLE_RATE environment variable on the function to a value between 0.000 (no tracing of Container App requests) and 1.000 (trace all Container App requests).
Metrics are calculated based on 100% of the application’s traffic, and remain accurate regardless of any sampling configuration.
Environment Variables
Troubleshooting
If you are not receiving traces or custom metric data as expected, enable App Service logs to receive debugging logs.
Share the content of the Log stream with Datadog Support.
Further reading
Additional helpful documentation, links, and articles:
