Apigee

Overview

Collect Apigee proxy logs to track errors, response time, duration, latency, monitor performance, and proxy issues.

Setup

Log collection

There are two methods for collecting Apigee logs:

  1. Use Apigee’s JavaScript policy to send logs to Datadog.
  2. If you already have a syslog server, use the Apigee MessageLogging policy type to log to a syslog account.

Syslog parameter

Use the MessageLogging policy type with the syslog parameter on your API to log custom messages to syslog. Replace <site_intake_endpoint> with and <site_port> with , in the following example:

<MessageLogging name="LogToSyslog">
    <DisplayName>datadog-logging</DisplayName>
    <Syslog>
        <Message><YOUR API KEY> test</Message>
        <Host><site_intake_endpoint></Host>
        <Port><site_port></Port>
        <Protocol>TCP</Protocol>
    </Syslog>
</MessageLogging>

JavaScript policy

Send Apigee proxy logs to Datadog using Apigee’s JavaScript policy.

The JavaScript has been configured to capture the essential flow variables as log attributes in Datadog. The attributes are named according to the list of standard attributes.

  1. Select the Apigee proxy from which you want to send logs to Datadog.
  2. In the selected proxy overview page, click the DEVELOP tab located in the top-right corner.
Develop
  1. Under Navigator, go to add a new JavaScript policy and edit the JavaScript file created under the Resources –> jsc dropdown menu.
  2. Add the following JavaScript code snippet in it. Make sure to replace <DATADOG_API_KEY> in the dd_api_url variable with your Datadog API KEY.
// Set the Datadog API URL here.
var dd_api_url = "https://http-intake.logs./api/v2/logs?dd-api-key=<DATADOG_API_KEY>&ddsource=apigee";

// Debug
// print(dd_api_url);
// print('Name of the flow: ' + context.flow);

// calculate response times for client, target and total
var request_start_time = context.getVariable('client.received.start.timestamp');
var request_end_time = context.getVariable('client.received.end.timestamp');
var system_timestamp = context.getVariable('system.timestamp');
var target_start_time = context.getVariable('target.sent.start.timestamp');
var target_end_time = context.getVariable('target.received.end.timestamp');
var total_request_time = system_timestamp - request_start_time;
var total_target_time = target_end_time - target_start_time;
var total_client_time = total_request_time - total_target_time;

var timestamp = crypto.dateFormat('YYYY-MM-dd HH:mm:ss.SSS');
var organization = context.getVariable("organization.name");
var networkClientIP = context.getVariable("client.ip");
var httpPort = context.getVariable("client.port");
var environment = context.getVariable("environment.name");
var apiProduct = context.getVariable("apiproduct.name");
var apigeeProxyName = context.getVariable("apiproxy.name");
var apigeeProxyRevision = context.getVariable("apiproxy.revision");
var appName = context.getVariable("developer.app.name");
var httpMethod = context.getVariable("request.verb");
var httpUrl = '' + context.getVariable("client.scheme") + '://' + context.getVariable("request.header.host") + context.getVariable("request.uri");
var httpStatusCode = context.getVariable("message.status.code");
var statusResponse = context.getVariable("message.reason.phrase");
var clientLatency = total_client_time;
var targetLatency = total_target_time;
var totalLatency = total_request_time;
var userAgent = context.getVariable('request.header.User-Agent');
var messageContent = context.getVariable('message.content');


// Datadog log attributes
var logObject = {
    "timestamp": timestamp,
    "organization": organization,
    "network.client.ip": networkClientIP,
    "env": environment,
    "apiProduct": apiProduct,
    "apigee_proxy.name": apigeeProxyName,
    "apigee_proxy.revision": apigeeProxyRevision,
    "service": appName,
    "http.method": httpMethod,
    "http.url": httpUrl,
    "http.status_code": httpStatusCode,
    "http.port": httpPort,
    "status": statusResponse,
    "clientLatency": clientLatency,
    "targetLatency": targetLatency,
    "totalLatency": totalLatency,
    "http.client.start_time_ms": request_start_time,
    "http.client.end_time_ms": request_end_time,
    "http.useragent": userAgent,
    "message": messageContent,
};


var headers = {
    'Content-Type': 'application/json'
};


// Debug
// print('LOGGING OBJECT' + JSON.stringify(logObject));

var myLoggingRequest = new Request(dd_api_url, "POST", headers, JSON.stringify(logObject));

// Send logs to Datadog
httpClient.send(myLoggingRequest);

Note: Add more flow variables into JavaScript from the official Apigee flow variable reference.

Troubleshooting

Need help? Contact Datadog support.

Further Reading

Additional helpful documentation, links, and articles: