Squid

Supported OS Linux Windows Mac OS

Integration version4.0.0

Squid

Overview

Squid is an open-source caching and forwarding web proxy server that operates as an intermediary between clients and servers on a network. It acts as a gateway, enabling clients to access various internet resources such as websites, files, and other content from servers.

This integration provides enrichment and visualization for Squid logs. It helps you visualize detailed insights into Squid log analysis through the out-of-the-box dashboards and detection rules, enhancing detection and response capabilities.

Additionally, it includes pre-configured monitors for proactive notifications on the following:

High rate of server errors
CPU usage exceeded
High latency requests
High rate of client HTTP errors

This check monitors Squid metrics from the Cache Manager through the Datadog Agent.

Setup

Installation

The Agent’s Squid check is included in the Datadog Agent package. No additional installation is needed on your Squid server.

Configuration

Host

To configure this check for an Agent running on a host:

Metric collection

Edit the squid.d/conf.yaml, in the conf.d/ folder at the root of your Agent’s configuration directory. See the sample squid.d/conf.yaml for all available configuration options.
Restart the Agent.

Log collection

Available for Agent versions >6.0

Collecting logs is disabled by default in the Datadog Agent, enable it in your datadog.yaml file:
```
logs_enabled: true
```

Uncomment and edit this configuration block at the bottom of your squid.d/conf.yaml file:

logs:
  - type: file
    path: /var/log/squid/cache.log
    service: "<SERVICE-NAME>"
    source: squid
  - type: file
    path: /var/log/squid/access.log
    service: "<SERVICE-NAME>"
    source: squid

Change the path and service parameter values and configure them for your environment.

Restart the Agent.

Containerized

For containerized environments, see the Autodiscovery Integration Templates for guidance on applying the parameters below.

Metric collection

Parameter	Value
`<INTEGRATION_NAME>`	`squid`
`<INIT_CONFIG>`	blank or `{}`
`<INSTANCE_CONFIG>`	`{"name": "<SQUID_INSTANCE_NAME>", "host": "%%host%%", "port":"3128"}`

Log collection

Available for Agent versions >6.0

Collecting logs is disabled by default in the Datadog Agent. To enable it, see Kubernetes Log Collection.

Parameter	Value
`<LOG_CONFIG>`	`{"source": "squid", "service": "<YOUR_APP_NAME>"}`

Validation

Run the Agent’s status subcommand and look for squid under the Checks section.

Data Collected

Logs

The Squid integration collects access and cache logs.

Supported Access Log Formats

Name	Format Specification
squid	`%ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt`
common	`%>a - %[un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st %Ss:%Sh`
combined	`%>a - %[un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh`

For more information, refer to Squid log formats.

Note: The default logformat type is squid. You can update the supported log format in /etc/squid/squid.conf, then restart Squid.

To use the combined type for logformat, add the following lines to your /etc/squid/squid.conf file:

logformat combined   %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
access_log /var/log/squid/access.log combined

Next, restart the squid service using the following command:

sudo systemctl restart squid

Note:

The Top Avg Request Duration by URL Host panel will be loaded only if the default squid type of logformat is configured.
The Top Browsers and Top HTTP Referrer panels will be loaded only if the combined type of logformat is configured.

Metrics

squid.cachemgr.aborted_requests (gauge)	The number of server-side HTTP requests aborted due to client-side aborts. Shown as request
squid.cachemgr.cd.kbytes_recv (gauge)	The amount of traffic received from neighbors in Cache Digest messages. Shown as kibibyte
squid.cachemgr.cd.kbytes_sent (gauge)	The amount of traffic sent to neighbors in Cache Digest messages. Shown as kibibyte
squid.cachemgr.cd.local_memory (gauge)	The amount of memory used to store Squid's own Cache Digest. Shown as kibibyte
squid.cachemgr.cd.memory (gauge)	The amount of memory used by enabling the Cache Digests' feature. Shown as kibibyte
squid.cachemgr.cd.msgs_recv (gauge)	The number of Cache Digest messages received from neighbors. Shown as message
squid.cachemgr.cd.msgs_sent (gauge)	The number of Cache Digest messages sent to neighbors. Shown as message
squid.cachemgr.cd.times_used (gauge)	The number of times Cache Digests resulted in the selection of a neighbor as the next-hop for a cache miss.
squid.cachemgr.client_http.errors (gauge)	The number of client transactions that resulted in an error. Shown as error
squid.cachemgr.client_http.hit_kbytes_out (gauge)	The amount of traffic sent to clients in responses that are cache hits Shown as kibibyte
squid.cachemgr.client_http.hits (gauge)	The number of cache hits in response to client requests. Shown as hit
squid.cachemgr.client_http.kbytes_in (gauge)	The amount of traffic received from clients in their requests. Shown as kibibyte
squid.cachemgr.client_http.kbytes_out (gauge)	The amount of traffic sent to clients in responses. Shown as kibibyte
squid.cachemgr.client_http.requests (gauge)	The number of HTTP requests received from clients Shown as request
squid.cachemgr.cpu_time (gauge)	The amount of CPU used by squid, as reported by getrusage( ). Shown as percent
squid.cachemgr.icp.kbytes_recv (gauge)	The amount of traffic received in all ICP messages, including both queries and replies. Shown as kibibyte
squid.cachemgr.icp.kbytes_sent (gauge)	The amount of traffic sent in all ICP messages, including both queries and replies. Shown as kibibyte
squid.cachemgr.icp.pkts_recv (gauge)	The number of ICP messages received from neighbors, including both queries and replies Shown as message
squid.cachemgr.icp.pkts_sent (gauge)	The number of ICP messages sent to neighbors. This includes both queries and replies but doesn't include HTCP messages. Shown as message
squid.cachemgr.icp.q_kbytes_recv (gauge)	The amount of traffic received from neighbors in ICP queries. Shown as kibibyte
squid.cachemgr.icp.q_kbytes_sent (gauge)	The amount of traffic sent to neighbors in ICP queries. Shown as kibibyte
squid.cachemgr.icp.queries_recv (gauge)	The number of ICP queries received from neighbors. Shown as query
squid.cachemgr.icp.queries_sent (gauge)	The number of ICP queries sent to neighbors. Shown as query
squid.cachemgr.icp.query_timeouts (gauge)	The number of times that Squid timed out waiting for ICP replies to arrive. Shown as error
squid.cachemgr.icp.r_kbytes_recv (gauge)	The amount of traffic received from neighbors in ICP replies. Shown as kibibyte
squid.cachemgr.icp.r_kbytes_sent (gauge)	The amount of traffic sent to neighbors in ICP replies. Shown as kibibyte
squid.cachemgr.icp.replies_queued (gauge)	The number of times Squid queued an ICP message after the initial attempt to send failed. Shown as message
squid.cachemgr.icp.replies_recv (gauge)	The number of ICP replies received from neighbors. Shown as response
squid.cachemgr.icp.replies_sent (gauge)	The number of ICP replies sent to neighbors. Shown as response
squid.cachemgr.icp.times_used (gauge)	The number of times ICP resulted in the selection of a neighbor as the next-hop for a cache miss.
squid.cachemgr.page_faults (gauge)	The number of (major) page faults as reported by getrusage( ). Shown as fault
squid.cachemgr.select_loops (gauge)	The number of times Squid called select( ) or poll( ) in the main I/O loop. Shown as item
squid.cachemgr.server.all.errors (gauge)	The number of server-side requests (all protocols) that resulted in some kind of error. Shown as error
squid.cachemgr.server.all.kbytes_in (gauge)	The amount of traffic read from the server-side for all protocols. Shown as kibibyte
squid.cachemgr.server.all.kbytes_out (gauge)	The amount of traffic written to origin servers and/or neighbor caches for server-side requests. Shown as kibibyte
squid.cachemgr.server.all.requests (gauge)	The number of requests forwarded to origin servers (or neighbor caches) for all server-side protocols. Shown as request
squid.cachemgr.server.ftp.errors (gauge)	The number of requests sent to FTP servers that resulted in an error. Shown as error
squid.cachemgr.server.ftp.kbytes_in (gauge)	The amount of traffic read from FTP servers, including control channel traffic. Shown as kibibyte
squid.cachemgr.server.ftp.kbytes_out (gauge)	The amount of traffic written to FTP servers, including control channel traffic. Shown as kibibyte
squid.cachemgr.server.ftp.requests (gauge)	The number of requests sent to FTP servers. Shown as request
squid.cachemgr.server.http.errors (gauge)	The number of server-side HTTP requests that resulted in an error. Shown as error
squid.cachemgr.server.http.kbytes_in (gauge)	The amount of traffic read from HTTP origin servers and neighbor caches. Shown as kibibyte
squid.cachemgr.server.http.kbytes_out (gauge)	The amount of traffic written to HTTP origin servers and neighbor caches. Shown as kibibyte
squid.cachemgr.server.http.requests (gauge)	The number of server-side requests to HTTP servers, including neighbor caches. Shown as request
squid.cachemgr.server.other.errors (gauge)	The number of Gopher, WAIS, and SSL requests that resulted in an error. Shown as error
squid.cachemgr.server.other.kbytes_in (gauge)	The amount of traffic read from Gopher, WAIS, and SSL servers. Shown as kibibyte
squid.cachemgr.server.other.kbytes_out (gauge)	The amount of traffic written to Gopher, WAIS, and SSL servers. Shown as kibibyte
squid.cachemgr.server.other.requests (gauge)	The number of "other" server-side requests. Currently, the other protocols are Gopher, WAIS, and SSL. Shown as request
squid.cachemgr.swap.files_cleaned (gauge)	The number of orphaned cache files removed by the periodic cleanup procedure. Shown as file
squid.cachemgr.swap.ins (gauge)	The number of objects (swap files) read from disk. Shown as file
squid.cachemgr.swap.outs (gauge)	The number of objects (swap files) saved to disk. Shown as file
squid.cachemgr.unlink.requests (gauge)	The number of unlink requests given to the (optional) unlinkd process. Shown as request

Events

The Squid check does not include any events.

Service Checks

squid.can_connect
Returns Critical if the Agent cannot connect to the Squid URL, OK otherwise.
Statuses: ok, critical

Troubleshooting

Need help? Contact Datadog support.