Batch and Memory Settings

Overview

To edit your OpenTelemetry Collector batch and memory settings, configure the batch processor in your Datadog Exporter.

For more information, see the OpenTelemetry project documentation for the batch processor.

Setup

    Add the following lines to your Collector configuration:

    processors:
  batch:
    # Datadog APM Intake limit is 3.2MB.    
    send_batch_max_size: 1000
    send_batch_size: 100
    timeout: 10s
  memory_limiter:
    check_interval: 1s
    limit_mib: 1000

    Add the following lines to values.yaml:

    resources:
  limits:
    cpu: 512m
    memory: 1Gi

    Add the following in the Collector configuration:

    processors:
  batch:
    # Datadog APM Intake limit is 3.2MB.    
    send_batch_max_size: 1000
    send_batch_size: 100
    timeout: 10s

    Data collected

    None.

    Full example configuration

    For a full working example configuration with the Datadog exporter, see batch-memory.yaml.

    Example logging output

    2023-12-05T09:52:58.568Z	warn	memorylimiterprocessor@v0.90.1/memorylimiter.go:276	
Memory usage is above hard limit. Forcing a GC.	
{"kind": "processor", "name": "memory_limiter", "pipeline": "traces", "cur_mem_mib": 44}
2023-12-05T09:52:58.590Z	info	memorylimiterprocessor@v0.90.1/memorylimiter.go:266	
Memory usage after GC.	
{"kind": "processor", "name": "memory_limiter", "pipeline": "traces", "cur_mem_mib": 34}