Overview

The recommended Collector configuration uses the exporter’s sending queue for batching and does not include the batch processor.

For production deployments, configure the memory limiter processor to limit the memory the OpenTelemetry Collector uses.

Setup

Add memory_limiter to the processors list for each pipeline in your configuration. Place it first in the list so that it applies backpressure before other processors allocate memory.

Memory limits depend on the memory available to the Collector, so there is no single recommended YAML configuration. Size the limit for your deployment and keep it below the process or container memory limit. Total process memory can exceed the configured limit, so leave headroom.

For the available options and sizing guidance, see the OpenTelemetry project documentation for the memory limiter processor.

Example logging output

2023-12-05T09:52:58.568Z	warn	memorylimiterprocessor@v0.90.1/memorylimiter.go:276	
Memory usage is above hard limit. Forcing a GC.	
{"kind": "processor", "name": "memory_limiter", "pipeline": "traces", "cur_mem_mib": 44}
2023-12-05T09:52:58.590Z	info	memorylimiterprocessor@v0.90.1/memorylimiter.go:266	
Memory usage after GC.	
{"kind": "processor", "name": "memory_limiter", "pipeline": "traces", "cur_mem_mib": 34}