Node.js Lambda トレースとバンドラーの互換性
概要
Datadog’s tracing libraries (dd-trace) are known to be not compatible with bundlers, like Webpack or esbuild, due to the use of conditional imports and other issues. While bundlers cannot build dd-trace, your application can still use the dd-trace and datadog-lambda-js libraries provided by the prebuilt Datadog Lambda layer. Follow the instructions below.
Webpack
Follow the installation instructions for Node.js and ensure the Datadog Lambda layer for Node.js is added to your Lambda function.
Exclude datadog-lambda-js and dd-trace, either by removing them from your package.json or by setting an exclude rule. Excluding them tells the bundler to skip building them as dependencies, since they are already available in the Lambda runtime provided by the Datadog Lambda layer.
Mark your dependencies as externals. This tells the bundler to exclude them from the output bundle; instead, they are packaged in node_modules.
webpack.config.js
const nodeExternals = require("webpack-node-externals");
module.exports = {
// Use webpack-node-externals to exclude all node dependencies.
// You can manually set the externals too.
externals: [nodeExternals()],
module: {
rules: [
{
// Provided by the Datadog Lambda layer and the Lambda Runtime.
exclude: [
// AWS SDK v3
/^@aws-sdk.*/,
// AWS SDK v2
/aws-sdk/,
/datadog-lambda-js/,
/dd-trace/
],
}
]
},
}
If you are using the serverless-webpack plugin and have the option includeModules set to any value other than false, the plugin automatically packs external modules under node_modules. Therefore, you must force exclude datadog-lambda-js and dd-trace. Skip this step if you don’t use serverless-webpack or you don’t have the includeModules option in your serverless.yml.
serverless.yml
custom:
webpack:
# You only need the following if you already have the includeModules option configured.
includeModules:
# ... your existing configuration for includeModules
forceExclude:
# @aws-sdk for the AWS SDK v3
- @aws-sdk
# aws-sdk for the AWS SDK v2
- aws-sdk
- datadog-lambda-js
- dd-trace
packagerOptions:
scripts:
# Optional, only needed when they are included as transitive dependencies
- rm -rf node_modules/datadog-lambda-js node_modules/dd-trace
To have more control around what dependencies are included, you could also include your webpack.config.js in your serverless-webpack configuration:
custom:
webpack:
forceExclude:
# @aws-sdk for the AWS SDK v3
- @aws-sdk
# aws-sdk for the AWS SDK v2
- aws-sdk
- datadog-lambda-js
- dd-trace
webpackConfig: 'webpack.config.js'
esbuild
Follow the installation instructions for Node.js and ensure the Datadog Lambda layer for Node.js is added to your Lambda function.
Remove datadog-lambda-js and dd-trace from your package.json and the build process, since they are already available in the Lambda runtime provided by the Datadog Lambda layer.
依存関係を外部としてマークします。これは、出力バンドルからそれらを除外するようバンドラーに指示します。代わりに、それらは node_modules にパッケージされます。
esbuild.config.js
const esbuild = require('esbuild');
esbuild.build({
// ... your existing esbuild configuration
// Same effect as manually passing each dependency to `external`
packages: 'external'
})
serverless-esbuild プラグインを使用している場合、esbuild-node-externals で全ての依存関係を esbuild プラグインとして外部化することが可能です。自動的に外部モジュールを node_modules の下にパックします。
serverless.yml
custom:
esbuild:
exclude:
# @aws-sdk for the AWS SDK v3
- @aws-sdk
# aws-sdk for the AWS SDK v2
- aws-sdk
- datadog-lambda-js
- dd-trace
plugins: plugins.js
# You can also set the specific dependencies to externalize instead of using `plugins`
external: [...]
// plugins.js
const { nodeExternalsPlugin } = require('esbuild-node-externals')
module.exports = [nodeExternalsPlugin()]
AWS CDK
If you deploy Node.js Lambda functions using the NodeJsFunction construct, but aren’t using esbuild or Typescript, you can still use Datadog to observe your serverless applications.
Follow the installation instructions for Node.js and ensure the Datadog Lambda layer for Node.js is added to your Lambda function.
Remove datadog-lambda-js and dd-trace from your package.json and the build process, since they are already available in the Lambda runtime provided by the Datadog Lambda Layer.
Use the NodejsFunction construct in the CDK. Ensure you set the entry property to be the path to the file containing your Lambda function handler, the depsLockFilePath to the path to your lock file for the package manager you are using, and the bundling.commandHooks.beforeBundling to ensure all dependencies are installed.
lambdaFunction.ts
const nodeFunction = new NodejsFunction(this, "test", {
runtime: Runtime.NODEJS_20_X,
entry: './functions/consumer/index.js', // The Javascript file for your Lambda function handler
handler: 'handler',
depsLockFilePath: './package-lock.json', // The path to the lock file for your respective package manager (npm, yarn etc)
bundling: {
commandHooks: {
beforeBundling(inputDir: string, outputDir: string) {
return [
`cd ${inputDir}`,
'npm install', // Ensure all dependencies are installed before your Javascript file is zipped and deployed
]
},
beforeInstall() {
return []
},
afterBundling() {
return []
}
},
externalModules: ['@aws-sdk/client-dynamodb'] // The AWS SDK is included as part of the Node.js Lambda runtime
}
});
AWS CDK & esbuild
The NodeJsFunction construct in the AWS CDK uses esbuild. The default configuration is not compatible with Datadog’s tracing libraries. The CDK allows you to override the default configuration and provide a custom esbuild file to support bundling and the Datadog tracing libraries:
Follow the installation instructions for Node.js and ensure the Datadog Lambda layer for Node.js is added to your Lambda function.
Remove datadog-lambda-js and dd-trace from your package.json and the build process, since they are already available in the Lambda runtime provided by the Datadog Lambda Layer.
Create an esbuild file for each of your Lambda functions. A seperate esbuild file is required per Lambda function so that each entry point can be specified seperately. Notice the entryPoint and outfile properties. For example, if you had a second Lambda function in your project named producer, then the entry point would be ./functions/producer.ts and the outfile would be /out/producer/index.js
buildConsumer.js
const ddPlugin = require('dd-trace/esbuild')
const esbuild = require('esbuild')
esbuild.build({
entryPoints: ['./functions/consumer.ts'],
outfile: 'out/consumer/index.js',
plugins: [ddPlugin],
// Other esbuild configuration
external: [
// esbuild cannot bundle native modules
'@datadog/native-metrics',
// required if you use profiling
'@datadog/pprof',
// required if you use Datadog security features
'@datadog/native-appsec',
'@datadog/native-iast-taint-tracking',
'@datadog/native-iast-rewriter',
// required if you encounter graphql errors during the build step
'graphql/language/visitor',
'graphql/language/printer',
'graphql/utilities',
'@aws-sdk/client-sqs'
// if you are using the package, instead of the layer
'datadog-lambda-js'
]
}).catch((err) => {
console.error(err)
process.exit(1)
})
When defining your NodeJsFunction in the CDK, use the Code.fromCustomCommand function to specify the path to your custom esbuild file and an output folder. For each separate Lambda function, specify the individual esbuild file defined in step three. The output folder should match the folder of the outfile in your esbuild file.
lambdaFunction.ts
// This path will likely be different for each individual Lambda function
const pathToBuildFile = '../functions/buildConsumer.js';
// Ensure the files for each Lambda function are generated into their own directory
const pathToOutputFolder = '../out/consumer/';
const code = Code.fromCustomCommand(
pathToOutputFolder,
['node', pathToBuildFile],
);
const consumerLambdaFunction = new NodejsFunction(this, props.functionName, {
runtime: Runtime.NODEJS_20_X,
code: code,
handler: 'index.handler',
memorySize: 512,
bundling: {
platform: 'node',
esbuildArgs: {
"--bundle": "true"
},
target: 'node20'
}
});
その他の参考資料
