Datadog-Go Expvar Integration

Go Graph

Overview

Track the memory usage of your Go services and collect metrics instrumented from Go’s expvar package.

If you prefer to instrument your Go code using only dogstats-go, you can still use this integration to collect memory-related metrics.

Setup

Installation

The Go Expvar check is packaged with the Agent, so simply install the Agent anywhere you run Go services whose metrics you want to collect.

Configuration

Prepare your Go service

If your Go service doesn’t use the expvar package already, you’ll need to import it (import "expvar"). If you don’t want to instrument your own metrics with expvar — i.e. you only want to collect your service’s memory metrics — import the package using the blank identifier (import _ "expvar").

If your service doesn’t already listen for HTTP requests (via the http package), make it listen locally, just for the Datadog Agent.

Connect the Agent

Create a file go_expvar.yaml in the Agent’s conf.d directory. See the sample go_expvar.yaml for all available configuration options:

init_config:

instances:
  - expvar_url: http://localhost:<your_apps_port>
    # optionally change the top-level namespace for metrics, e.g. my_go_app.memstats.alloc
    namespace: my_go_app # defaults to go_expvar, e.g. go_expvar.memstats.alloc
    # optionally define the metrics to collect, e.g. a counter var your service exposes with expvar.NewInt("my_func_counter")
    metrics:
      - path: my_func_counter
        # if you don't want it named my_go_app.my_func_counter
        #alias: my_go_app.preferred_counter_name
        type: counter # other valid options: rate, gauge
        #tags:
        #  - "tag_name1:tag_value1"

If you don’t configure a metrics list, the Agent will still collect memstat metrics. Use metrics to tell the Agent which expvar vars to collect.

Restart the Agent to begin sending memstat and expvar metrics to Datadog.

Validation

Run the Agent’s info subcommand and look for go_expvar under the Checks section:

  Checks
  ======
    [...]

    go_expvar
    -------
      - instance #0 [OK]
      - Collected 13 metrics, 0 events & 0 service checks

    [...]

Compatibility

The go_expvar check is compatible with all major platforms.

Data Collected

Metrics

go_expvar.memstats.alloc
(gauge)
Bytes allocated and not yet freed
shown as byte
go_expvar.memstats.frees
(gauge)
Number of frees
shown as operation
go_expvar.memstats.heap_alloc
(gauge)
Bytes allocated and not yet freed
shown as byte
go_expvar.memstats.heap_idle
(gauge)
Bytes in idle spans
shown as byte
go_expvar.memstats.heap_inuse
(gauge)
Bytes in non-idle spans
shown as byte
go_expvar.memstats.heap_objects
(gauge)
Total number of allocated objects
shown as item
go_expvar.memstats.heap_released
(gauge)
Bytes released to the OS
shown as byte
go_expvar.memstats.heap_sys
(gauge)
Bytes obtained from system
shown as byte
go_expvar.memstats.lookups
(gauge)
Number of pointer lookups
shown as operation
go_expvar.memstats.mallocs
(gauge)
Number of mallocs
shown as operation
go_expvar.memstats.num_gc
(gauge)
Number of garbage collections
shown as garbage collection
go_expvar.memstats.pause_ns.95percentile
(gauge)
95th percentile of recent GC pause durations
shown as nanosecond
go_expvar.memstats.pause_ns.avg
(gauge)
Average of recent GC pause durations
shown as nanosecond
go_expvar.memstats.pause_ns.count
(rate)
Number of submitted GC pause durations
shown as sample
go_expvar.memstats.pause_ns.max
(gauge)
Max GC pause duration
shown as nanosecond
go_expvar.memstats.pause_ns.median
(gauge)
Median GC pause duration
shown as nanosecond
go_expvar.memstats.pause_total_ns
(gauge)
Total GC pause duration over lifetime of process
shown as nanosecond
go_expvar.memstats.total_alloc
(gauge)
Bytes allocated (even if freed)
shown as byte

Events

The Go-Expvar check does not include any event at this time.

Service Checks

The Go-Expvar check does not include any service check at this time.

Troubleshooting

Need help? Contact Datadog Support.

Further Reading