--- title: Amazon SageMaker description: Amazon SageMaker is a fully managed machine learning service. breadcrumbs: Docs > Integrations > Amazon SageMaker --- > For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt). # Amazon SageMaker Integration version1.2.0 {% callout %} # Important note for users on the following Datadog sites: us2.ddog-gov.com {% alert level="info" %} To find out if this integration is available in your organization, see your [Datadog Integrations](https://app.datadoghq.com/integrations) page or ask your organization administrator. To initiate an exception request to enable this integration for your organization, email [support@ddog-gov.com](mailto:support@ddog-gov.com). {% /alert %} {% /callout %} ## Overview{% #overview %} Amazon SageMaker is a fully managed machine learning service. With Amazon SageMaker, data scientists and developers can build and train machine learning models, and then directly deploy them into a production-ready hosted environment. Enable this integration to see all your SageMaker metrics in Datadog. ## Setup{% #setup %} ### Installation{% #installation %} If you haven't already, set up the [Amazon Web Services integration](https://docs.datadoghq.com/integrations/amazon_web_services.md) first. ### Metric collection{% #metric-collection %} 1. In the [AWS integration page](https://app.datadoghq.com/integrations/amazon-web-services), ensure that `SageMaker` is enabled under the `Metric Collection` tab. 1. Install the [Datadog - Amazon SageMaker integration](https://app.datadoghq.com/integrations/amazon-sagemaker). ### Log collection{% #log-collection %} #### Enable logging{% #enable-logging %} Configure Amazon SageMaker to send logs either to a S3 bucket or to CloudWatch. **Note**: If you log to a S3 bucket, make sure that `amazon_sagemaker` is set as *Target prefix*. #### Send logs to Datadog{% #send-logs-to-datadog %} 1. If you haven't already, set up the [Datadog log collection AWS Lambda function](https://docs.datadoghq.com/integrations/amazon_web_services.md?tab=automaticcloudformation#log-collection). 1. Once the lambda function is installed, manually add a trigger on the S3 bucket or CloudWatch log group that contains your Amazon SageMaker logs in the AWS console: - [Add a manual trigger on the S3 bucket](https://docs.datadoghq.com/logs/guide/send-aws-services-logs-with-the-datadog-lambda-function.md#collecting-logs-from-s3-buckets) - [Add a manual trigger on the CloudWatch Log Group](https://docs.datadoghq.com/logs/guide/send-aws-services-logs-with-the-datadog-lambda-function.md#collecting-logs-from-cloudwatch-log-group) ## Data Collected{% #data-collected %} ### Metrics{% #metrics %} | | | | | **aws.sagemaker.approximate_age_of_oldest_request**(gauge) | The age of the oldest request in the queue for an asynchronous inference endpoint.*Shown as second* | | **aws.sagemaker.approximate_backlog_size**(gauge) | The number of items in the queue for an asynchronous inference endpoint that are being processed or waiting to be processed. | | **aws.sagemaker.approximate_backlog_size_per_instance**(gauge) | The number of items in the queue for an asynchronous inference endpoint divided by the number of instances behind the endpoint. | | **aws.sagemaker.concurrent_requests_per_copy**(gauge) | The number of concurrent requests received by an inference component, normalized by each copy of the inference component. | | **aws.sagemaker.concurrent_requests_per_model**(gauge) | The number of concurrent requests received by the model. | | **aws.sagemaker.consumed_read_capacity_units**(count) | The number of provisioned read capacity units consumed by Feature Store runtime operations over the specified time period. | | **aws.sagemaker.consumed_read_requests_units**(count) | The average number of consumed read units over the specified time period. | | **aws.sagemaker.consumed_read_requests_units.maximum**(count) | The maximum number of consumed read units over the specified time period. | | **aws.sagemaker.consumed_read_requests_units.minimum**(count) | The minimum number of consumed read units over the specified time period. | | **aws.sagemaker.consumed_read_requests_units.p90**(count) | The 90th percentile of consumed read units over the specified time period. | | **aws.sagemaker.consumed_read_requests_units.p95**(count) | The 95th percentile of consumed read units over the specified time period. | | **aws.sagemaker.consumed_read_requests_units.p99**(count) | The 99th percentile of consumed read units over the specified time period. | | **aws.sagemaker.consumed_read_requests_units.samplecount**(count) | The sample count of consumed read units over the specified time period. | | **aws.sagemaker.consumed_read_requests_units.sum**(count) | The sum of consumed read units over the specified time period. | | **aws.sagemaker.consumed_write_capacity_units**(count) | The number of provisioned write capacity units consumed by Feature Store runtime operations over the specified time period. | | **aws.sagemaker.consumed_write_requests_units**(count) | The average number of consumed write units over the specified time period. | | **aws.sagemaker.consumed_write_requests_units.maximum**(count) | The maximum number of consumed write units over the specified time period. | | **aws.sagemaker.consumed_write_requests_units.minimum**(count) | The minimum number of consumed write units over the specified time period. | | **aws.sagemaker.consumed_write_requests_units.p90**(count) | The 90th percentile of consumed write units over the specified time period. | | **aws.sagemaker.consumed_write_requests_units.p95**(count) | The 95th percentile of consumed write units over the specified time period. | | **aws.sagemaker.consumed_write_requests_units.p99**(count) | The 99th percentile of consumed write units over the specified time period. | | **aws.sagemaker.consumed_write_requests_units.samplecount**(count) | The sample count of consumed write units over the specified time period. | | **aws.sagemaker.consumed_write_requests_units.sum**(count) | The sum of consumed write units over the specified time period. | | **aws.sagemaker.container_latency**(gauge) | The time it takes for an inference pipeline container to respond, as viewed from SageMaker.*Shown as microsecond* | | **aws.sagemaker.expired_requests**(count) | The number of requests in the queue for an asynchronous inference endpoint that failed because they reached their specified request time to live (TTL). | | **aws.sagemaker.explanations**(count) | The number of records explained by an endpoint with online explainability activated. | | **aws.sagemaker.explanations_per_instance**(count) | The average number of records explained per minute, per instance, for a variant with online explainability activated. | | **aws.sagemaker.first_chunk_latency**(gauge) | The time elapsed from when a bidirectional streaming request arrives at the endpoint until the first chunk of the response is sent to the client.*Shown as microsecond* | | **aws.sagemaker.first_chunk_model_latency**(gauge) | The time taken by the model container to process a bidirectional streaming request and return the first chunk of the response.*Shown as microsecond* | | **aws.sagemaker.first_chunk_overhead_latency**(gauge) | The overhead latency for the first chunk of a bidirectional streaming response, excluding model processing time.*Shown as microsecond* | | **aws.sagemaker.has_backlog_without_capacity**(gauge) | Set to `1` when an asynchronous inference endpoint has requests in the queue but zero instances behind the endpoint. Otherwise, set to `0`. | | **aws.sagemaker.invocation_4xxerrors**(count) | The average number of InvokeEndpoint requests where the model returned a 4xx HTTP response code.*Shown as request* | | **aws.sagemaker.invocation_5xxerrors**(count) | The average number of InvokeEndpoint requests where the model returned a 5xx HTTP response code.*Shown as request* | | **aws.sagemaker.invocation_failures**(count) | The number of invocations to an asynchronous inference endpoint that failed for any reason. | | **aws.sagemaker.invocation_model_errors**(count) | The number of model invocation requests which did not result in 2XX HTTP response. This includes 4XX/5XX status codes, low-level socket errors, malformed HTTP responses, and request timeouts. | | **aws.sagemaker.invocations**(count) | The number of InvokeEndpoint requests sent to a model endpoint.*Shown as request* | | **aws.sagemaker.invocations_per_copy**(count) | The number of invocations normalized by each copy of an inference component. | | **aws.sagemaker.invocations_per_instance**(count) | The number of invocations sent to a model normalized by InstanceCount in each ProductionVariant. | | **aws.sagemaker.invocations_processed**(count) | The number of asynchronous invocations processed by the endpoint. | | **aws.sagemaker.latency**(gauge) | The time interval to process requests made to Feature Store runtime operations.*Shown as microsecond* | | **aws.sagemaker.marketplace_metering_inferences**(count) | The number of successful InvokeEndpoint requests sent to an endpoint that hosts an AWS Marketplace model package, as reported for inference pricing metering. | | **aws.sagemaker.mid_stream_errors**(count) | The number of errors that occur during response streaming after the initial response has been sent to the client. | | **aws.sagemaker.model_cache_hit**(count) | The number of InvokeEndpoint requests sent to the multi-model endpoint for which the model was already loaded.*Shown as request* | | **aws.sagemaker.model_cache_hit.sum**(count) | The sum of InvokeEndpoint requests sent to the multi-model endpoint for which the model was already loaded.*Shown as request* | | **aws.sagemaker.model_downloading_time**(gauge) | The interval of time that it takes to download the model from Amazon Simple Storage Service (Amazon S3).*Shown as microsecond* | | **aws.sagemaker.model_downloading_time.maximum**(gauge) | The maximum interval of time that it takes to download the model from Amazon Simple Storage Service (Amazon S3).*Shown as microsecond* | | **aws.sagemaker.model_downloading_time.minimum**(gauge) | The minimum interval of time that it takes to download the model from Amazon Simple Storage Service (Amazon S3).*Shown as microsecond* | | **aws.sagemaker.model_downloading_time.samplecount**(count) | The sample count interval of time that it takes to download the model from Amazon Simple Storage Service (Amazon S3).*Shown as microsecond* | | **aws.sagemaker.model_downloading_time.sum**(gauge) | The sum interval of time that it takes to download the model from Amazon Simple Storage Service (Amazon S3).*Shown as microsecond* | | **aws.sagemaker.model_latency**(gauge) | The average interval of time taken by a model to respond as viewed from Amazon SageMaker.*Shown as microsecond* | | **aws.sagemaker.model_latency.maximum**(gauge) | The maximum interval of time taken by a model to respond as viewed from Amazon SageMaker.*Shown as microsecond* | | **aws.sagemaker.model_latency.minimum**(gauge) | The minimum interval of time taken by a model to respond as viewed from Amazon SageMaker.*Shown as microsecond* | | **aws.sagemaker.model_latency.samplecount**(count) | The sample count interval of time taken by a model to respond as viewed from Amazon SageMaker.*Shown as microsecond* | | **aws.sagemaker.model_latency.sum**(gauge) | The sum of the interval of time taken by a model to respond as viewed from Amazon SageMaker.*Shown as microsecond* | | **aws.sagemaker.model_loading_time**(gauge) | The interval of time that it takes to load the model through the container's LoadModel API call.*Shown as microsecond* | | **aws.sagemaker.model_loading_time.maximum**(gauge) | The maximum interval of time that it takes to load the model through the container's LoadModel API call.*Shown as microsecond* | | **aws.sagemaker.model_loading_time.minimum**(gauge) | The minimum interval of time that it takes to load the model through the container's LoadModel API call.*Shown as microsecond* | | **aws.sagemaker.model_loading_time.samplecount**(count) | The sample count interval of time that it takes to load the model through the container's LoadModel API call.*Shown as microsecond* | | **aws.sagemaker.model_loading_time.sum**(gauge) | The sum interval of time that it takes to load the model through the container's LoadModel API call.*Shown as microsecond* | | **aws.sagemaker.model_loading_wait_time**(gauge) | The interval of time that an invocation request has waited for the target model to be downloaded, or loaded, or both in order to perform inference.*Shown as microsecond* | | **aws.sagemaker.model_loading_wait_time.maximum**(gauge) | The maximum interval of time that an invocation request has waited for the target model to be downloaded, or loaded, or both in order to perform inference.*Shown as microsecond* | | **aws.sagemaker.model_loading_wait_time.minimum**(gauge) | The minimum interval of time that an invocation request has waited for the target model to be downloaded, or loaded, or both in order to perform inference.*Shown as microsecond* | | **aws.sagemaker.model_loading_wait_time.samplecount**(count) | The sample count interval of time that an invocation request has waited for the target model to be downloaded, or loaded, or both in order to perform inference.*Shown as microsecond* | | **aws.sagemaker.model_loading_wait_time.sum**(gauge) | The sum interval of time that an invocation request has waited for the target model to be downloaded, or loaded, or both in order to perform inference.*Shown as microsecond* | | **aws.sagemaker.model_setup_time**(gauge) | The average time it takes to launch new compute resources for a serverless endpoint.*Shown as microsecond* | | **aws.sagemaker.model_setup_time.maximum**(gauge) | The maximum interval of time it takes to launch new compute resources for a serverless endpoint.*Shown as microsecond* | | **aws.sagemaker.model_setup_time.minimum**(gauge) | The minimum interval of time it takes to launch new compute resources for a serverless endpoint.*Shown as microsecond* | | **aws.sagemaker.model_setup_time.samplecount**(count) | The sample_count of the amount of time it takes to launch new compute resources for a serverless endpoint.*Shown as microsecond* | | **aws.sagemaker.model_setup_time.sum**(gauge) | The total amount of time takes to launch new compute resources for a serverless endpoint.*Shown as microsecond* | | **aws.sagemaker.model_unloading_time**(gauge) | The interval of time that it takes to unload the model through the container's UnloadModel API call.*Shown as microsecond* | | **aws.sagemaker.model_unloading_time.maximum**(gauge) | The maximum interval of time that it takes to unload the model through the container's UnloadModel API call.*Shown as microsecond* | | **aws.sagemaker.model_unloading_time.minimum**(gauge) | The minimum interval of time that it takes to unload the model through the container's UnloadModel API call.*Shown as microsecond* | | **aws.sagemaker.model_unloading_time.samplecount**(count) | The sample count interval of time that it takes to unload the model through the container's UnloadModel API call.*Shown as microsecond* | | **aws.sagemaker.model_unloading_time.sum**(gauge) | The sum interval of time that it takes to unload the model through the container's UnloadModel API call.*Shown as microsecond* | | **aws.sagemaker.no_capacity_invocation_failures**(count) | The number of inference requests that failed because the endpoint had no active instances to serve them. | | **aws.sagemaker.notification_failures**(count) | The number of inference requests to an asynchronous inference endpoint that failed due to an issue publishing notifications. | | **aws.sagemaker.operation_4xxerrors**(count) | The number of requests made to Feature Store runtime operations where the operation returned a 4xx HTTP response code. | | **aws.sagemaker.operation_5xxerrors**(count) | The number of requests made to Feature Store runtime operations where the operation returned a 5xx HTTP response code. | | **aws.sagemaker.overhead_latency**(gauge) | The average interval of time added to the time taken to respond to a client request by Amazon SageMaker overheads.*Shown as microsecond* | | **aws.sagemaker.overhead_latency.maximum**(gauge) | The maximum interval of time added to the time taken to respond to a client request by Amazon SageMaker overheads.*Shown as microsecond* | | **aws.sagemaker.overhead_latency.minimum**(gauge) | The minimum interval of time added to the time taken to respond to a client request by Amazon SageMaker overheads.*Shown as microsecond* | | **aws.sagemaker.overhead_latency.samplecount**(count) | The sample count of the interval of time added to the time taken to respond to a client request by Amazon SageMaker overheads.*Shown as microsecond* | | **aws.sagemaker.overhead_latency.sum**(gauge) | The sum of the interval of time added to the time taken to respond to a client request by Amazon SageMaker overheads.*Shown as microsecond* | | **aws.sagemaker.request_download_failures**(count) | The number of inference failures on an asynchronous inference endpoint that occurred due to an issue downloading the request from Amazon S3. | | **aws.sagemaker.request_download_latency**(gauge) | The total time taken to download the request payload of an asynchronous inference request from Amazon S3.*Shown as microsecond* | | **aws.sagemaker.response_upload_failures**(count) | The number of inference failures on an asynchronous inference endpoint that occurred due to an issue uploading the response to Amazon S3. | | **aws.sagemaker.response_upload_latency**(gauge) | The total time taken to upload the response payload of an asynchronous inference request to Amazon S3.*Shown as microsecond* | | **aws.sagemaker.serverless_concurrent_executions_utilization**(gauge) | The number of concurrent executions for a serverless endpoint divided by its maximum concurrency. | | **aws.sagemaker.serverless_provisioned_concurrency_executions**(gauge) | The number of concurrent executions that use Provisioned Concurrency on a serverless endpoint. | | **aws.sagemaker.serverless_provisioned_concurrency_invocations**(count) | The number of invocations sent to a serverless endpoint that are served by its Provisioned Concurrency. | | **aws.sagemaker.serverless_provisioned_concurrency_spillover_invocations**(count) | The number of invocations sent to a serverless endpoint that exceeded its Provisioned Concurrency and spilled over to on-demand capacity. | | **aws.sagemaker.serverless_provisioned_concurrency_utilization**(gauge) | The number of concurrent executions for a serverless endpoint divided by its allocated Provisioned Concurrency. | | **aws.sagemaker.throttled_requests**(count) | The number of requests made to Feature Store runtime operations that were throttled. | | **aws.sagemaker.time_in_backlog**(gauge) | The total time an asynchronous inference request was queued before being processed, excluding processing time, such as download, upload, and model latency.*Shown as millisecond* | | **aws.sagemaker.total_processing_time**(gauge) | The time from when SageMaker receives an asynchronous inference request until the request finishes processing.*Shown as millisecond* | | **aws.sagemaker.endpoints.cpuutilization**(gauge) | The average percentage of CPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.endpoints.cpuutilization.maximum**(gauge) | The maximum percentage of CPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.endpoints.cpuutilization.minimum**(gauge) | The minimum percentage of CPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.endpoints.disk_utilization**(gauge) | The average percentage of disk space used by the containers on an instance uses.*Shown as percent* | | **aws.sagemaker.endpoints.disk_utilization.maximum**(gauge) | The maximum percentage of disk space used by the containers on an instance uses.*Shown as percent* | | **aws.sagemaker.endpoints.disk_utilization.minimum**(gauge) | The minimum percentage of disk space used by the containers on an instance uses.*Shown as percent* | | **aws.sagemaker.endpoints.gpumemory_utilization**(gauge) | The average percentage of GPU memory used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.endpoints.gpumemory_utilization.maximum**(gauge) | The maximum percentage of GPU memory used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.endpoints.gpumemory_utilization.minimum**(gauge) | The minimum percentage of GPU memory used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.endpoints.gpuutilization**(gauge) | The average percentage of GPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.endpoints.gpuutilization.maximum**(gauge) | The maximum percentage of GPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.endpoints.gpuutilization.minimum**(gauge) | The minimum percentage of GPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.endpoints.loaded_model_count**(count) | The number of models loaded in the containers of the multi-model endpoint. This metric is emitted per instance. | | **aws.sagemaker.endpoints.loaded_model_count.sum**(count) | The sum of models loaded in the containers of the multi-model endpoint. This metric is emitted per instance. | | **aws.sagemaker.endpoints.memory_utilization**(gauge) | The average percentage of memory that is used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.endpoints.memory_utilization.maximum**(gauge) | The maximum percentage of memory that is used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.endpoints.memory_utilization.minimum**(gauge) | The minimum percentage of memory that is used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.processingjobs.cpuutilization**(gauge) | The average percentage of CPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.processingjobs.cpuutilization.maximum**(gauge) | The maximum percentage of CPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.processingjobs.cpuutilization.minimum**(gauge) | The minimum percentage of CPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.processingjobs.disk_utilization**(gauge) | The average percentage of disk space used by the containers on an instance uses.*Shown as percent* | | **aws.sagemaker.processingjobs.disk_utilization.maximum**(gauge) | The maximum percentage of disk space used by the containers on an instance uses.*Shown as percent* | | **aws.sagemaker.processingjobs.disk_utilization.minimum**(gauge) | The minimum percentage of disk space used by the containers on an instance uses.*Shown as percent* | | **aws.sagemaker.processingjobs.gpumemory_utilization**(gauge) | The average percentage of GPU memory used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.processingjobs.gpumemory_utilization.maximum**(gauge) | The maximum percentage of GPU memory used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.processingjobs.gpumemory_utilization.minimum**(gauge) | The minimum percentage of GPU memory used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.processingjobs.gpuutilization**(gauge) | The average percentage of GPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.processingjobs.gpuutilization.maximum**(gauge) | The maximum percentage of GPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.processingjobs.gpuutilization.minimum**(gauge) | The minimum percentage of GPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.processingjobs.memory_utilization**(gauge) | The average percentage of memory that is used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.processingjobs.memory_utilization.maximum**(gauge) | The maximum percentage of memory that is used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.processingjobs.memory_utilization.minimum**(gauge) | The minimum percentage of memory that is used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.trainingjobs.cpuutilization**(gauge) | The average percentage of CPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.trainingjobs.cpuutilization.maximum**(gauge) | The maximum percentage of CPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.trainingjobs.cpuutilization.minimum**(gauge) | The minimum percentage of CPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.trainingjobs.disk_utilization**(gauge) | The average percentage of disk space used by the containers on an instance uses.*Shown as percent* | | **aws.sagemaker.trainingjobs.disk_utilization.maximum**(gauge) | The maximum percentage of disk space used by the containers on an instance uses.*Shown as percent* | | **aws.sagemaker.trainingjobs.disk_utilization.minimum**(gauge) | The minimum percentage of disk space used by the containers on an instance uses.*Shown as percent* | | **aws.sagemaker.trainingjobs.gpumemory_utilization**(gauge) | The average percentage of GPU memory used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.trainingjobs.gpumemory_utilization.maximum**(gauge) | The maximum percentage of GPU memory used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.trainingjobs.gpumemory_utilization.minimum**(gauge) | The minimum percentage of GPU memory used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.trainingjobs.gpuutilization**(gauge) | The average percentage of GPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.trainingjobs.gpuutilization.maximum**(gauge) | The maximum percentage of GPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.trainingjobs.gpuutilization.minimum**(gauge) | The minimum percentage of GPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.trainingjobs.memory_utilization**(gauge) | The average percentage of memory that is used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.trainingjobs.memory_utilization.maximum**(gauge) | The maximum percentage of memory that is used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.trainingjobs.memory_utilization.minimum**(gauge) | The minimum percentage of memory that is used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.transformjobs.cpuutilization**(gauge) | The average percentage of CPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.transformjobs.cpuutilization.maximum**(gauge) | The maximum percentage of CPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.transformjobs.cpuutilization.minimum**(gauge) | The minimum percentage of CPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.transformjobs.disk_utilization**(gauge) | The average percentage of disk space used by the containers on an instance uses.*Shown as percent* | | **aws.sagemaker.transformjobs.disk_utilization.maximum**(gauge) | The maximum percentage of disk space used by the containers on an instance uses.*Shown as percent* | | **aws.sagemaker.transformjobs.disk_utilization.minimum**(gauge) | The minimum percentage of disk space used by the containers on an instance uses.*Shown as percent* | | **aws.sagemaker.transformjobs.gpumemory_utilization**(gauge) | The average percentage of GPU memory used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.transformjobs.gpumemory_utilization.maximum**(gauge) | The maximum percentage of GPU memory used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.transformjobs.gpumemory_utilization.minimum**(gauge) | The minimum percentage of GPU memory used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.transformjobs.gpuutilization**(gauge) | The average percentage of GPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.transformjobs.gpuutilization.maximum**(gauge) | The maximum percentage of GPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.transformjobs.gpuutilization.minimum**(gauge) | The minimum percentage of GPU units that are used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.transformjobs.memory_utilization**(gauge) | The average percentage of memory that is used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.transformjobs.memory_utilization.maximum**(gauge) | The maximum percentage of memory that is used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.transformjobs.memory_utilization.minimum**(gauge) | The minimum percentage of memory that is used by the containers on an instance.*Shown as percent* | | **aws.sagemaker.labelingjobs.dataset_objects_auto_annotated**(count) | The average number of dataset objects auto-annotated in a labeling job. | | **aws.sagemaker.labelingjobs.dataset_objects_human_annotated**(count) | The average number of dataset objects annotated by a human in a labeling job. | | **aws.sagemaker.labelingjobs.dataset_objects_labeling_failed**(count) | The number of dataset objects that failed labeling in a labeling job. | | **aws.sagemaker.labelingjobs.jobs_succeeded.samplecount**(count) | The sample count of occurrences a single labeling job succeeded.*Shown as job* | | **aws.sagemaker.labelingjobs.jobs_succeeded.sum**(count) | The sum of occurrences a single labeling job succeeded.*Shown as job* | | **aws.sagemaker.labelingjobs.total_dataset_objects_labeled**(count) | The average number of dataset objects labeled successfully in a labeling job. | | **aws.sagemaker.workteam.active_workers**(count) | The average number of single active workers on a private work team that submitted, released, or declined a task. | | **aws.sagemaker.workteam.active_workers.samplecount**(count) | The sample count of single active workers on a private work team that submitted, released, or declined a task. | | **aws.sagemaker.workteam.active_workers.sum**(count) | The sum of single active workers on a private work team that submitted, released, or declined a task. | | **aws.sagemaker.workteam.tasks_accepted.samplecount**(count) | The sample count of occurrences a single task was accepted by a worker. | | **aws.sagemaker.workteam.tasks_accepted.sum**(count) | The sum of occurrences a single task was accepted by a worker. | | **aws.sagemaker.workteam.tasks_declined.samplecount**(count) | The sample count of occurrences a single task was declined by a worker. | | **aws.sagemaker.workteam.tasks_declined.sum**(count) | The sum of occurrences a single task was declined by a worker. | | **aws.sagemaker.workteam.tasks_submitted.samplecount**(count) | The average number of occurrences a single task was submitted/completed by a private worker. | | **aws.sagemaker.workteam.tasks_submitted.sum**(count) | The average number of occurrences a single task was submitted/completed by a private worker. | | **aws.sagemaker.modelbuildingpipeline.execution_duration**(gauge) | The average duration in milliseconds that the pipeline execution ran.*Shown as millisecond* | | **aws.sagemaker.modelbuildingpipeline.execution_duration.maximum**(gauge) | The maximum duration in milliseconds that the pipeline execution ran.*Shown as millisecond* | | **aws.sagemaker.modelbuildingpipeline.execution_duration.minimum**(gauge) | The minimum duration in milliseconds that the pipeline execution ran.*Shown as millisecond* | | **aws.sagemaker.modelbuildingpipeline.execution_duration.samplecount**(count) | The sample count duration in milliseconds that the pipeline execution ran.*Shown as millisecond* | | **aws.sagemaker.modelbuildingpipeline.execution_duration.sum**(gauge) | The sum duration in milliseconds that the pipeline execution ran.*Shown as millisecond* | | **aws.sagemaker.modelbuildingpipeline.execution_failed**(count) | The average number of steps that failed. | | **aws.sagemaker.modelbuildingpipeline.execution_failed.sum**(count) | The sum of steps that failed. | | **aws.sagemaker.modelbuildingpipeline.execution_started**(count) | The average number of pipeline executions that started. | | **aws.sagemaker.modelbuildingpipeline.execution_started.sum**(count) | The sum of pipeline executions that started. | | **aws.sagemaker.modelbuildingpipeline.execution_stopped**(count) | The average number of pipeline executions that stopped. | | **aws.sagemaker.modelbuildingpipeline.execution_stopped.sum**(count) | The sum of pipeline executions that stopped. | | **aws.sagemaker.modelbuildingpipeline.execution_succeeded**(count) | The average number of pipeline executions that succeeded. | | **aws.sagemaker.modelbuildingpipeline.execution_succeeded.sum**(count) | The sum of pipeline executions that succeeded. | | **aws.sagemaker.modelbuildingpipeline.step_duration**(gauge) | The average duration in milliseconds that the step ran.*Shown as millisecond* | | **aws.sagemaker.modelbuildingpipeline.step_duration.maximum**(gauge) | The maximum duration in milliseconds that the step ran.*Shown as millisecond* | | **aws.sagemaker.modelbuildingpipeline.step_duration.minimum**(gauge) | The minimum duration in milliseconds that the step ran.*Shown as millisecond* | | **aws.sagemaker.modelbuildingpipeline.step_duration.samplecount**(count) | The sample count duration in milliseconds that the step ran.*Shown as millisecond* | | **aws.sagemaker.modelbuildingpipeline.step_duration.sum**(gauge) | The sum duration in milliseconds that the step ran.*Shown as millisecond* | | **aws.sagemaker.modelbuildingpipeline.step_failed**(count) | The average number of steps that failed. | | **aws.sagemaker.modelbuildingpipeline.step_failed.sum**(count) | The sum of steps that failed. | | **aws.sagemaker.modelbuildingpipeline.step_started**(count) | The average number of steps that started. | | **aws.sagemaker.modelbuildingpipeline.step_started.sum**(count) | The sum of steps that started. | | **aws.sagemaker.modelbuildingpipeline.step_stopped**(count) | The average number of steps that stopped. | | **aws.sagemaker.modelbuildingpipeline.step_stopped.sum**(count) | The sum of steps that stopped. | | **aws.sagemaker.modelbuildingpipeline.step_succeeded**(count) | The average number of steps that succeeded. | | **aws.sagemaker.modelbuildingpipeline.step_succeeded.sum**(count) | The sum of steps that succeeded. | | **aws.sagemaker.inferencecomponents.cpuutilization_normalized**(gauge) | The normalized percentage of CPU units used by each copy of the inference component, relative to its reservation or limit.*Shown as percent* | | **aws.sagemaker.inferencecomponents.gpumemory_utilization_normalized**(gauge) | The normalized percentage of GPU memory used by each copy of the inference component, relative to its reservation or limit.*Shown as percent* | | **aws.sagemaker.inferencecomponents.gpuutilization_normalized**(gauge) | The normalized percentage of GPU units used by each copy of the inference component, relative to its reservation or limit.*Shown as percent* | | **aws.sagemaker.inferencecomponents.memory_utilization_normalized**(gauge) | The normalized percentage of memory used by each copy of the inference component, relative to its reservation or limit.*Shown as percent* | ### Events{% #events %} The Amazon SageMaker integration does not include any events. ### Service Checks{% #service-checks %} The Amazon SageMaker integration does not include any service checks. ## Out-of-the-box monitoring{% #out-of-the-box-monitoring %} Datadog provides out-of-the-box dashboards for your SageMaker endpoints and jobs. ### SageMaker endpoints{% #sagemaker-endpoints %} Use the [SageMaker endpoints dashboard](https://app.datadoghq.com/dash/integration/amazon_sagemaker_endpoints) to help you immediately start monitoring the health and performance of your SageMaker endpoints with no additional configuration. Determine which endpoints have errors, higher-than-expected latency, or traffic spikes. Review and correct your instance type and scaling policy selections using CPU, GPU, memory, and disk utilization metrics. ### SageMaker jobs{% #sagemaker-jobs %} You can use the [SageMaker jobs dashboard](https://app.datadoghq.com/dash/integration/amazon_sagemaker_jobs) to gain insight into the resource utilization (for example, finding CPU, GPU, and storage bottlenecks) of your training, processing, or transform jobs. Use this information to optimize your compute instances. ## Troubleshooting{% #troubleshooting %} Need help? Contact [Datadog support](https://docs.datadoghq.com/help/). ## Further Reading{% #further-reading %} Additional helpful documentation, links, and articles: - [CloudHealth + Datadog: Effectively manage your cloud assets](https://www.datadoghq.com/blog/monitor-cloudhealth-assets-datadog/)