For details about open-source contributions to the PHP tracer, refer to the contributing guide.
Configure the Datadog Agent for APM
Install and configure the Datadog Agent to receive traces from your now instrumented application. By default the Datadog Agent is enabled in your datadog.yaml file under apm_config with enabled: true and listens for trace traffic at localhost:8126. For containerized environments, follow the links below to enable trace collection within the Datadog Agent.
See the specific setup instructions to ensure that the Agent is configured to receive traces in a containerized environment:
After having instrumented your application, the tracing client sends traces to Unix domain socket /var/run/datadog/apm.socket by default. If the socket does not exist, traces are sent to http://localhost:8126. If this is not the correct host and port change it by setting DD_TRACE_AGENT_URL, for example:
Similarly, where DD_TRACE_HEALTH_METRICS_ENABLED is set true, the trace client attempts to send stats to the /var/run/datadog/dsd.socket Unix domain socket. If the socket does not exist then stats are sent to http://localhost:8125.
If a different configuration is required, use the DD_DOGSTATSD_URL environment variable. Some examples:
This command installs the extension to all the PHP binaries found in the host or container. If --php-bin is omitted, the installer runs in interactive mode and asks the user to select the binaries for installation. The value of --php-bin can be a path to a specific binary in case dd-trace-php should be installed only to such binary.
Restart PHP (PHP-FPM or the Apache SAPI) and visit a tracing-enabled endpoint of your application. For traces, see the APM Service List.
When you do not specify --enable-appsec, the AppSec extension loads shortly at startup, and is not enabled by default. It immediately short-circuits, causing negligible performance overhead.
Note:
It may take a few minutes before traces appear in the UI. If traces still do not appear after a few minutes, create a phpinfo() page from the host machine and scroll down to the `ddtrace`. Failed diagnostic checks appear in this section to help identify any issues.
Apache ZTS:
If the PHP CLI binary is built as NTS (non thread-safe), while Apache uses a ZTS (Zend thread-safe) version of PHP, you need to manually change the extension load for the ZTS binary. Run /path/to/php-zts --ini to find where Datadog's .ini file is located, then add the -zts suffix from the file name. For example, from extension=ddtrace-20210902.so to extension=ddtrace-20210902-zts.so.
Automatic instrumentation
Tracing is automatically enabled by default. Once the extension is installed, ddtrace traces your application and sends traces to the Agent.
Datadog supports all web frameworks out of the box. Automatic instrumentation works by modifying PHP’s runtime to wrap certain functions and methods to trace them. The PHP tracer supports automatic instrumentation for several libraries.
Automatic instrumentation captures:
Method execution time
Relevant trace data, such as URL and status response codes for web requests or SQL queries for database access
Unhandled exceptions, including stacktraces if available
A total count of traces (for example, web requests) flowing through the system
Configuration
If needed, configure the tracing library to send application performance telemetry data as you require, including setting up Unified Service Tagging. Read Library Configuration for details.
Tracing short- and long-running CLI scripts
Additional steps are required for instrumenting CLI scripts. Read Trace PHP CLI Scripts for more information.
Once the installation is completed restart PHP (PHP-FPM or the Apache SAPI).
Note: If you are using second level caching in OPcache by setting the parameter opcache.file_cache, remove the cache folder.
Removing
To remove the PHP tracer:
For php-fpm, stop the php-fpm service, otherwise stop the Apache web server.
Unlink files 98-ddtrace.ini and 99-ddtrace-custom.ini from your php configuration folder.
For php-fpm, restart php-fpm service, otherwise restart the Apache web server.
Note: If you are using second level caching in OPcache by setting the parameter opcache.file_cache, remove the cache folder.
Troubleshooting an application crash
In the unusual event of an application crash caused by the PHP tracer, typically because of a segmentation fault, the best thing to do is obtain a core dump or a Valgrind trace and contact Datadog support.
Install debug symbols
For the core dumps to be readable, debug symbols for the PHP binaries have to be installed on the system that runs PHP.
To check if debug symbols are installed for PHP or PHP-FPM, use gdb.
Install gdb:
apt|yum install -y gdb
Run gdb with the binary of interest. For example for PHP-FPM:
gdb php-fpm
If the gdb output contains a line similar to the text below, then debug symbols are already installed.
...
Reading symbols from php-fpm...Reading symbols from /usr/lib/debug/path/to/some/file.debug...done.
...
If the gdb output contains a line similar to the text below, then debug symbols need to be installed:
...
Reading symbols from php-fpm...(no debugging symbols found)...done.
...
Centos
Install package yum-utils that provides the program debuginfo-install:
yum install -y yum-utils
Find the package name for your PHP binaries, it can vary depending on the PHP installation method:
yum list installed | grep php
Install debug symbols. For example for package php-fpm:
debuginfo-install -y php-fpm
Note: If the repository that provides the PHP binaries is not enabled by default, it can be enabled when running the debuginfo-install command. For example:
# ... leave here all the pre-existing packages
# add a `deb` deb http://deb.debian.org/debian-debug/ $RELEASE-debug main
# For example for buster
debhttp://deb.debian.org/debian-debug/buster-debugmain
Update apt:
apt update
Try canonical package names for debug symbols, first. For example, if the package name is php7.2-fpm try:
apt install -y php7.2-fpm-dbgsym
# if the above does not work
apt install -y php7.2-fpm-dbg
If debug symbols cannot be found, use the utility tool find-dbgsym-packages. Install the binary:
apt install -y debian-goodies
Attempt finding debug symbols from either the full path to the binary or the process id of a running process:
If PHP was installed from the ppa:ondrej/php, edit the apt source file /etc/apt/sources.list.d/ondrej-*.list by adding the main/debug component.
Before:
deb http://ppa.launchpad.net/ondrej/php/ubuntu <version> main
After:
deb http://ppa.launchpad.net/ondrej/php/ubuntu <version> main main/debug
Update and install the debug symbols. For example, for PHP-FPM 7.2:
apt update
apt install -y php7.2-fpm-dbgsym
PHP installed from a different package
Find the package name for your PHP binaries, it can vary depending on the PHP installation method:
apt list --installed | grep php
Note: In some cases php-fpm can be a metapackage that refers to the real package, for example php7.2-fpm in case of PHP-FPM 7.2. In this case the package name is the latter.
Try canonical package names for debug symbols, first. For example, if the package name is php7.2-fpm try:
apt install -y php7.2-fpm-dbgsym
# if the above does not work
apt install -y php7.2-fpm-dbg
If the -dbg and -dbgsym packages cannot be found, enable the ddebs repositories. Detailed information about how to install debug symbols from the ddebs can be found in the Ubuntu documentation.
For example, for Ubuntu 18.04+, enable the ddebs repo:
echo "deb http://ddebs.ubuntu.com $(lsb_release -cs) main restricted universe multiverse" | tee -a /etc/apt/sources.list.d/ddebs.list
echo "deb http://ddebs.ubuntu.com $(lsb_release -cs)-updates main restricted universe multiverse" | tee -a /etc/apt/sources.list.d/ddebs.list
Obtaining a core dump for PHP applications can be tricky, especially on PHP-FPM. Here are a few tips to help you obtain a core dump:
Determine whether the PHP-FPM generated a core dump by looking in the application error log:
Search for (SIGSEGV - core dumped) because a message like this means it has been dumped: WARNING: [pool www] child <pid> exited on signal 11 (SIGSEGV - core dumped) after <duration> seconds from start.
Search for (SIGSEGV) because a message like this indicates that the core was not dumped: WARNING: [pool www] child <pid> exited on signal 11 (SIGSEGV) after <duration> seconds from start.
Locate the core dump by running cat /proc/sys/kernel/core_pattern. The default value is typically core, meaning that a file named core will be generated in the web root folder.
If no core dump was generated, check the following configurations and change them as needed:
If /proc/sys/kernel/core_pattern contains a path including nested directories, ensure the full directory path exists.
If the user running the PHP-FPM pool workers is something other than root (a common user name is www-data) give that user write permissions to the core dumps directory.
Ensure that the value of /proc/sys/fs/suid_dumpable is not 0. Set it to 1 or 2 unless you run PHP-FPM workers pool as root. Check your options with your system administrator.
Ensure you have a suitable rlimit_core in the PHP-FPM pool configuration section. You can set it to unlimited: rlimit_core = unlimited.
Ensure you have a suitable ulimit set in your system. You can set it to unlimited: ulimit -c unlimited.
If your application runs in a Docker container, changes to /proc/sys/* have to be done to the host machine. Contact your system administrator to know the options available to you. If you are able to, try recreating the issue in your testing or staging environments.
Obtaining a Valgrind trace
To gain more details about the crash, run the application with Valgrind. Unlike core dumps, this approach always works in an unprivileged container.
Note: An application that runs through Valgrind is orders of magnitude slower than when running natively. This method is recommended for non-production environments.
Install Valgrind with your package manager. Run the application with Valgrind enough to generate a few requests.
The resulting Valgrind trace is printed by default to the standard error, follow the official documentation to print to a different target. The expected output is similar to the example below for a PHP-FPM process:
==322== Conditional jump or move depends on uninitialised value(s)
==322== at 0x41EE82: zend_string_equal_val (zend_string.c:403)
==322== ...
==322== ...
==322==
==322== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==322== at 0x73C8657: kill (syscall-template.S:81)
==322== by 0x1145D0F2: zif_posix_kill (posix.c:468)
==322== by 0x478BFE: ZEND_DO_ICALL_SPEC_RETVAL_UNUSED_HANDLER (zend_vm_execute.h:1269)
==322== by 0x478BFE: execute_ex (zend_vm_execute.h:53869)
==322== by 0x47D9B0: zend_execute (zend_vm_execute.h:57989)
==322== by 0x3F6782: zend_execute_scripts (zend.c:1679)
==322== by 0x394F0F: php_execute_script (main.c:2658)
==322== by 0x1FFE18: main (fpm_main.c:1939)
==322==
==322== Process terminating with default action of signal 11 (SIGSEGV)
==322== ...
==322== ...
==322==
==322== HEAP SUMMARY:
==322== in use at exit: 3,411,619 bytes in 22,428 blocks
==322== total heap usage: 65,090 allocs, 42,662 frees, 23,123,409 bytes allocated
==322==
==322== LEAK SUMMARY:
==322== definitely lost: 216 bytes in 3 blocks
==322== indirectly lost: 951 bytes in 32 blocks
==322== possibly lost: 2,001,304 bytes in 16,840 blocks
==322== still reachable: 1,409,148 bytes in 5,553 blocks
==322== of which reachable via heuristic:
==322== stdstring : 384 bytes in 6 blocks
==322== suppressed: 0 bytes in 0 blocks
==322== Rerun with --leak-check=full to see details of leaked memory
==322==
==322== Use --track-origins=yes to see where uninitialised values come from
==322== For lists of detected and suppressed errors, rerun with: -s
==322== ERROR SUMMARY: 18868 errors from 102 contexts (suppressed: 0 from 0)
Obtaining a strace
Some issues are caused by external factors, so it can be valuable to have a strace.
Note: An application that runs through strace is orders of magnitude slower than when running natively. This method is recommended for non-production environments.
Install strace with your package manager. When generating a strace to send to Datadog Support, ensure you use the -f option to follow child processes.
