Palo Alto Networks Firewall

Supported OS Linux Windows Mac OS

Integration version3.0.0

Overview

Datadog’s Palo Alto Networks Firewall Log integration allows customers to ingest, parse, and analyze Palo Alto Networks firewall logs. This log integration relies on the HTTPS log templating and forwarding capability provided by PAN OS, the operating system that runs in Palo Alto firewalls. PAN-OS allows customers to forward threat, traffic, authentication, and other important log events.

Key use cases

Respond to high severity threat events

Firewall threat logs provide context on threats detected by a firewall, which can be filtered and analyzed by severity, type, origin IPs/countries, and more.

Make informed decisions on Firewall deployment

Firewall traffic logs can be used to measure the traffic and sessions passing through a firewall and also gives you the ability to monitor for anomalous throughput across firewall deployment.

Monitor authentication anomalies

Firewall authentication logs provide detailed information on users as they authenticate with Palo Alto Networks firewall. These logs can be used to monitor anomalous spikes in authentication traffic from specific protocols, users, locations, and more.

Setup

Log collection

  1. Install the Datadog Agent on a machine that is reachable by the firewall and can connect to the internet.

  2. In PanOS, Select Device » Server Profiles » Syslog , add a name for the server profile. Follow the Syslog log forwarding configuration steps. Same steps listed below.

  3. Click Add and provide the following details of the server:

    • Name of the server
    • IP address of the machine with datadog agent
    • Transport as TCP
    • Port as 10518 and format as BSD

  4. Copy and configure custom log format for the required log type. The formats below are mappings from the formats listed in the Palo Alto Networks Syslog Field Descriptions docs.

    NameFormat
    Traffic Log
    View Payload

    timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, time_generated=$time_generated, network.client.ip=$src, network.destination.ip=$dst, natsrc=$natsrc, natdst=$natdst, rule=$rule, usr.id=$srcuser, dstuser=$dstuser, app=$app, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, repeatcnt=$repeatcnt, network.client.port=$sport, network.destination.port=$dport, natsport=$natsport natdport=$natdport, flags=$flags, proto=$proto, evt.name=$action, bytes=$bytes, network.bytes_read=$bytes_sent, network.bytes_written=$bytes_received, start=$start, elapsed=$elapsed, category=$category, seqno=$seqno, actionflags=$actionflags, network.client.geoip.country.name=$srcloc, dstloc=$dstloc, pkts_sent=$pkts_sent, pkts_received=$pkts_received, session_end_reason=$session_end_reason, device_name=$device_name, action_source=$action_source, src_uuid=$src_uuid, dst_uuid=$dst_uuid, tunnelid=$tunnelid, imsi= $imsi, monitortag=$monitortag, imei=$imei, parent_session_id=$parent_session_id, parent_start_time=$parent_start_time, tunnel=$tunnel, assoc_id=$assoc_id, chunks=$chunks chunks_sent=$chunks_sent chunks_received=$chunks_received

    Threat Log (and WildFire Submissions Log)
    View Payload

    timestamp=$receive_time, serial=$serial, type=$type, subtype=$subtype, time_generated=$time_generated, network.client.ip=$src, network.destination.ip=$dst, natsrc=$natsrc, natdst=$natdst, rule=$rule, usr.id=$srcuser, dstuser=$dstuser, app=$app, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, repeatcnt=$repeatcnt, network.client.port=$sport, network.destination.port=$dport, natsport=$natsport, natdport=$natdport, flags=$flags, proto=$proto, evt.name=$action, misc=$misc, threatid=$threatid, category=$category, severity=$severity, direction=$direction, seqno=$seqno, actionflags=$actionflags, network.client.geoip.country.name=$srcloc, dstloc=$dstloc, contenttype=$contenttype, pcap_id=$pcap_id, filedigest=$filedigest, cloud=$cloud, url_idx=$url_idx, http.useragent=$user_agent, filetype=$filetype, xff=$xff referer=$referer, sender=$sender, subject=$subject, recipient=$recipient, reportid=$reportid, vsys_name=$vsys_name, device_name=$device_name, src_uuid=$src_uuid, dst_uuid=$dst_uuid, http_method=$http_method, tunnel_id=$tunnel_id, imsi=$imsi, monitortag=$monitortag, imei=$imei, parent_session_id=$parent_session_id, parent_start_time=$parent_start_time, tunnel=$tunnel, thr_category=$thr_category, contentver=$contentver, assoc_id=$assoc_id, ppid=$ppid, http_headers=$http_headers

    Authentication Log
    View Payload

    timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, vsys=$vsys, network.client.ip=$ip, usr.id=$user, normalize_user=$normalize_user, object=$object, authpolicy=$authpolicy, repeatcnt=$repeatcnt, authid=$authid, vendor=$vendor , logset=$logset, serverprofile=$serverprofile, message=$message ,clienttype=$clienttype, evt.outcome=$event, factorno=$factorno, seqno=$seqno, actionflags=$actionflags, vsys_name=$vsys_name, device_name=$device_name, vsys_id=$vsys_id, evt.name=$authproto

    HIP Match Log
    View Payload

    timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, time_generated=$time_generated, usr.id=$srcuser, vsys=$vsys, machinename=$machinename, os=$os, network.client.ip=$src, matchname=$matchname, repeatcnt=$repeatcnt, matchtype=$matchtype, seqno=$seqno, actionflags=$actionflags, vsys_name=$vsys_name, device_name=$device_name, vsys_id=$vsys_id, srcipv6=$srcipv6, hostid=$hostid

    User-ID Log
    View Payload

    timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, vsys=$vsys, network.client.ip=$ip, usr.id=$user, datasourcename=$datasourcename, evt.name=$eventid, repeatcnt=$repeatcnt, timeout=$timeout, network.client.port=$beginport, network.destination.port=$endport, datasource=$datasource, datasourcetype=$datasourcetype, seqno=$seqno, actionflags=$actionflags, vsys_name=$vsys_name, device_name=$device_name, vsys_id=$vsys_id, factortype=$factortype, factorcompletiontime=$factorcompletiontime,, factorno=$factorno, ugflags=$ugflags, userbysource=$userbysource

    Tunnel Inspection Log
    View Payload

    timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, network.client.ip=$src, network.destination.ip=$dst, natsrc=$natsrc, natdst=$natdst, rule=$rule, usr.id=$srcuser, dstuser=$dstuser, app=$app, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, repeatcnt=$repeatcnt, network.client.port=$sport, network.destination.port=$dport, natsport=$natsport, natdport=$natdport, flags=$flags, proto=$proto, evt.outcome=$action, severity=$severity, seqno=$seqno, actionflags=$actionflags, srcloc=$srcloc, dstloc=$dstloc, vsys_name=$vsys_name, device_name=$device_name, tunnelid=$tunnelid, monitortag=$monitortag, parent_session_id=$parent_session_id, parent_start_time=$parent_start_time, tunnel=$tunnel, bytes=$bytes, network.bytes_read=$bytes_sent, network.bytes_written=$bytes_received, packets=$packets, pkts_sent=$pkts_sent, pkts_received=$pkts_received, max_encap=$max_encap, unknown_proto=$unknown_proto, strict_check=$strict_check, tunnel_fragment=$tunnel_fragment, sessions_created=$sessions_created, sessions_closed=$sessions_closed, session_end_reason=$session_end_reason, evt.name=$action_source, start=$start, elapsed=$elapsed, tunnel_insp_rule=$tunnel_insp_rule

    SCTP Log
    View Payload

    timestamp=$time_generated, serial=$serial, type=$type, network.client.ip=$src, network.destination.ip=$dst, rule=$rule, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, repeatcnt=$repeatcnt, network.client.port=$sport, network.destination.port=$dport, proto=$proto, action=$action, vsys_name=$vsys_name, device_name=$device_name, seqno=$seqno, assoc_id=$assoc_id, ppid=$ppid, severity=$severity, sctp_chunk_type=$sctp_chunk_type, sctp_event_type=$sctp_event_type, verif_tag_1=$verif_tag_1, verif_tag_2=$verif_tag_2, sctp_cause_code=$sctp_cause_code, diam_app_id=$diam_app_id, diam_cmd_code=$diam_cmd_code, diam_avp_code=$diam_avp_code, stream_id=$stream_id, assoc_end_reason=$assoc_end_reason, op_code=$op_code, sccp_calling_ssn=$sccp_calling_ssn, sccp_calling_gt=$sccp_calling_gt, sctp_filter=$sctp_filter, chunks=$chunks, chunks_sent=$chunks_sent, chunks_received=$chunks_received, packets=$packets, pkts_sent=$pkts_sent, pkts_received=$pkts_received

    Config Log
    View Payload

    timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, network.client.ip=$host, vsys=$vsys, evt.name=$cmd, usr.id=$admin, client=$client, evt.outcome=$result, path=$path, before_change_detail=$before_change_detail, after_change_detail=$after_change_detail, seqno=$seqno, actionflags=$actionflags, vsys_name=$vsys_name, device_name=$device_name

    System Log
    View Payload

    timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, vsys=$vsys, evt.name=$eventid, object=$object, module=$module, severity=$severity, opaque=$opaque, seqno=$seqno, actionflags=$actionflags, vsys_name=$vsys_name, device_name=$device_name

    Correlated Events Log
    View Payload

    timestamp=$time_generated, serial=$serial, type=$type, subtype=$subtype, vsys=$vsys, evt.name=$eventid, object=$object, module=$module, severity=$severity, opaque=$opaque, seqno=$seqno, actionflags=$actionflags, vsys_name=$vsys_name, device_name=$device_name

    GTP Log
    View Payload

    timestamp=$start, serial=$serial, type=$type, subtype=$subtype, network.client.ip=$src, network.destination.ip=$dst, rule=$rule, app=$app, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, network.client.port=$sport, network.destination.port=$dport, proto=$proto, evt.name=$action, event_type=$event_type, msisdn=$msisdn, apn=$apn, rat=$rat, msg_type=$msg_type, end_ip_adr=$end_ip_adr, teid1=$teid1, teid2=$teid2, gtp_interface=$gtp_interface, cause_code=$cause_code, severity=$severity, mcc=$mcc, mnc=$mnc, area_code=$area_code, cell_id=$cell_id, event_code=$event_code, srcloc=$srcloc, dstloc=$dstloc, imsi=$imsi, imei=$imei, start=$start, elapsed=$elapsed, tunnel_insp_rule=$tunnel_insp_rule

    Data Filtering Log
    View Payload

    timestamp=$receive_time, serial=$serial, type=$type, subtype=$subtype, time_generated=$time_generated, network.client.ip=$src, network.destination.ip=$dst, natsrc=$natsrc, natdst=$natdst, rule=$rule, usr.id=$srcuser, dstuser=$dstuser, app=$app, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, repeatcnt=$repeatcnt, network.client.port=$sport, network.destination.port=$dport, natsport=$natsport, natdport=$natdport, flags=$flags, proto=$proto, evt.name=$action, misc=$misc, threatid=$threatid, category=$category, severity=$severity, direction=$direction, seqno=$seqno, actionflags=$actionflags, network.client.geoip.country.name=$srcloc, dstloc=$dstloc, contenttype=$contenttype, pcap_id=$pcap_id, filedigest=$filedigest, cloud=$cloud, url_idx=$url_idx, http.useragent=$user_agent, filetype=$filetype, xff=$xff, referer=$referer, sender=$sender, subject=$subject, recipient=$recipient, reportid=$reportid, vsys_name=$vsys_name, device_name=$device_name, src_uuid=$src_uuid, dst_uuid=$dst_uuid, http_method=$http_method, tunnel_id=$tunnel_id, imsi=$imsi, monitortag=$monitortag, imei=$imei, parent_session_id=$parent_session_id, parent_start_time=$parent_start_time, tunnel=$tunnel, thr_category=$thr_category, contentver=$contentver, assoc_id=$assoc_id, ppid=$ppid, http_headers=$http_headers, url_category_list=$url_category_list, rule_uuid=$rule_uuid, http2_connection=$http2_connection, dynusergroup_name=$dynusergroup_name, xff_ip=$xff_ip, src_osfamily=$src_osfamily, src_osversion=$src_osversion, src_host=$src_host, src_mac=$src_mac, dst_osfamily=$dst_osfamily, dst_osversion=$dst_osversion, dst_host=$dst_host, dst_mac=$dst_mac, container_id=$container_id, pod_namespace=$pod_namespace, pod_name=$pod_name, src_edl=$src_edl, dst_edl=$dst_edl, hostid=$hostid, serialnumber=$serialnumber, domain_edl=$domain_edl, src_dag=$src_dag, dst_dag=$dst_dag, partial_hash=$partial_hash, high_res_timestamp=$high_res_timestamp, reason=$reason, justification=$justification

    URL Filtering Log
    View Payload

    timestamp=$receive_time, serial=$serial, type=$type, subtype=$subtype, time_generated=$time_generated, network.client.ip=$src, network.destination.ip=$dst, natsrc=$natsrc, natdst=$natdst, rule=$rule, usr.id=$srcuser, dstuser=$dstuser, app=$app, vsys=$vsys, from=$from, to=$to, inbound_if=$inbound_if, outbound_if=$outbound_if, logset=$logset, sessionid=$sessionid, repeatcnt=$repeatcnt, network.client.port=$sport, network.destination.port=$dport, natsport=$natsport, natdport=$natdport, flags=$flags, proto=$proto, evt.name=$action, misc=$misc, threatid=$threatid, category=$category, severity=$severity, direction=$direction, seqno=$seqno, actionflags=$actionflags, network.client.geoip.country.name=$srcloc, dstloc=$dstloc, contenttype=$contenttype, pcap_id=$pcap_id, filedigest=$filedigest, cloud=$cloud, url_idx=$url_idx, http.useragent=$user_agent, filetype=$filetype, xff=$xff, referer=$referer, sender=$sender, subject=$subject, recipient=$recipient, reportid=$reportid, vsys_name=$vsys_name, device_name=$device_name, src_uuid=$src_uuid, dst_uuid=$dst_uuid, http_method=$http_method, tunnel_id=$tunnel_id, imsi=$imsi, monitortag=$monitortag, imei=$imei, parent_session_id=$parent_session_id, parent_start_time=$parent_start_time, tunnel=$tunnel, thr_category=$thr_category, contentver=$contentver, assoc_id=$assoc_id, ppid=$ppid, http_headers=$http_headers, url_category_list=$url_category_list, rule_uuid=$rule_uuid, http2_connection=$http2_connection, dynusergroup_name=$dynusergroup_name, xff_ip=$xff_ip, src_osfamily=$src_osfamily, src_osversion=$src_osversion, src_host=$src_host, src_mac=$src_mac, dst_osfamily=$dst_osfamily, dst_osversion=$dst_osversion, dst_host=$dst_host, dst_mac=$dst_mac, container_id=$container_id, pod_namespace=$pod_namespace, pod_name=$pod_name, src_edl=$src_edl, dst_edl=$dst_edl, hostid=$hostid, serialnumber=$serialnumber, domain_edl=$domain_edl, src_dag=$src_dag, dst_dag=$dst_dag, partial_hash=$partial_hash, high_res_timestamp=$high_res_timestamp, reason=$reason, justification=$justification

    GlobalProtect Log
    View Payload

    timestamp=$receive_time, serial=$serial, type=$type, subtype=$subtype, time_generated=$time_generated, vsys=$vsys, evt.name=$eventid, stage=$stage, auth_method=$auth_method, tunnel_type=$tunnel_type, usr.id=$srcuser, srcregion=$srcregion, machinename=$machinename, public_ip=$public_ip, public_ipv6=$public_ipv6, private_ip=$private_ip, private_ipv6=$private_ipv6, hostid=$hostid, serialnumber=$serialnumber, client_ver=$client_ver, client_os=$client_os, client_os_ver=$client_os_ver, repeatcnt=$repeatcnt, reason=$reason, error=$error, opaque=$opaque, status=$status, location=$location, login_duration=$login_duration, connect_method=$connect_method, error_code=$error_code, portal=$portal, seqno=$seqno, actionflags=$actionflags, selection_type=$selection_type, response_time=$response_time, priority=$priority, attempted_gateways=$attempted_gateways, gateway=$gateway, dg_hier_level_1=$dg_hier_level_1, dg_hier_level_2=$dg_hier_level_2, dg_hier_level_3=$dg_hier_level_3, dg_hier_level_4=$dg_hier_level_4, vsys_name=$vsys_name, device_name=$device_name, vsys_id=$vsys_id

  5. Click OK, this creates a syslog server profile.

  6. Click on the Objects tab, this opens the log forwarding profile screen.

  7. Create log forwarding profile by providing the name, log type and syslog profile

  8. Create a pan.firewall.d/conf.yaml file at the root of the Agent’s configuration directory with the below content.

    logs:
- type: tcp
  port: 10518
  service: "firewall"
  source: "pan.firewall"

  9. Restart Agent.

Data Collected

Metrics

The metrics collected for this integration are determined by the relevant configured profiles on the network vendors page.

Logs

The PANOS integration collects logs from the Palo Alto Networks firewall integration and forwards them to Datadog.

Events

The PANOS integration does not send any events.

Service Checks

The PANOS integration does not include any service checks.

Further Reading

Additional helpful documentation, links, and articles:

Troubleshooting

Need help? Contact Datadog support.