Definition of a historical job based on a security monitoring rule.
caseIndex
int32
Zero-based index of the rule case to use as the job's signal condition. When omitted, all cases are evaluated. Up to 10 cases are supported, so valid values are 0 to 9.
from [required]
int64
Starting time of data analyzed by the job.
id [required]
string
ID of the detection rule used to create the job.
index [required]
string
Index used to load the data.
notifications
[string]
Notifications sent when the job is completed.
to [required]
int64
Ending time of data analyzed by the job.
jobDefinition
object
Definition of a historical job.
calculatedFields
[object]
Calculated fields.
expression [required]
string
Expression.
name [required]
string
Field name.
cases [required]
[object]
Cases used for generating job results. Up to 10 cases are allowed.
actions
[object]
Action to perform for each rule case.
options
object
Options for the rule action
duration
int64
Duration of the action in seconds. 0 indicates no expiration.
flaggedIPType
enum
Used with the case action of type 'flag_ip'. The value specified in this field is applied as a flag to the IP addresses.
Allowed enum values: SUSPICIOUS,FLAGGED
userBehaviorName
string
Used with the case action of type 'user_behavior'. The value specified in this field is applied as a risk tag to all users affected by the rule.
type
enum
The action type.
Allowed enum values: block_ip,block_user,user_behavior,flag_ip
condition
string
A case contains logical operations (>,>=, &&, ||) to determine if a signal should be generated
based on the event counts in the previously defined queries.
name
string
Name of the case.
notifications
[string]
Notification targets.
status [required]
enum
Severity of the Security Signal.
Allowed enum values: info,low,medium,high,critical
from [required]
int64
Starting time of data analyzed by the job.
groupSignalsBy
[string]
Additional grouping to perform on top of the existing groups in the query section. Must be a subset of the existing groups.
index [required]
string
Index used to load the data.
message [required]
string
Message for generated results.
name [required]
string
Job name.
options
object
Job options.
anomalyDetectionOptions
object
Options on anomaly detection method.
bucketDuration
enum
Duration in seconds of the time buckets used to aggregate events matched by the rule.
Must be greater than or equal to 300.
Allowed enum values: 300,600,900,1800,3600,10800
detectionTolerance
enum
An optional parameter that sets how permissive anomaly detection is.
Higher values require higher deviations before triggering a signal.
Allowed enum values: 1,2,3,4,5
instantaneousBaseline
boolean
When set to true, Datadog uses previous values that fall within the defined learning window to construct the baseline, enabling the system to establish an accurate baseline more rapidly rather than relying solely on gradual learning over time.
learningDuration
enum
Learning duration in hours. Anomaly detection waits for at least this amount of historical data before it starts evaluating.
Allowed enum values: 1,6,12,24,48,168,336
learningPeriodBaseline
int64
An optional override baseline to apply while the rule is in the learning period. Must be greater than or equal to 0.
detectionMethod
enum
The detection method.
Allowed enum values: threshold,new_value,anomaly_detection,impossible_travel,hardcoded,third_party,anomaly_threshold,sequence_detection
evaluationWindow
enum
A time window is specified to match when at least one of the cases matches true. This is a sliding window
and evaluates in real time. For third party detection method, this field is not used.
Allowed enum values: 0,60,300,600,900,1800,3600,7200,10800,21600Show 2 more,43200,86400
impossibleTravelOptions
object
Options on impossible travel detection method.
baselineUserLocations
boolean
If true, signals are suppressed for the first 24 hours. In that time, Datadog learns the user's regular
access locations. This can be helpful to reduce noise and infer VPN usage or credentialed API access.
baselineUserLocationsDuration
int32
The duration in days during which Datadog learns the user's regular access locations. After this period, signals are generated for accesses from unknown locations.
keepAlive
enum
Once a signal is generated, the signal will remain "open" if a case is matched at least once within
this keep alive window. For third party detection method, this field is not used.
Allowed enum values: 0,60,300,600,900,1800,3600,7200,10800,21600Show 2 more,43200,86400
maxSignalDuration
enum
A signal will "close" regardless of the query being matched once the time exceeds the maximum duration.
This time is calculated from the first seen timestamp.
Allowed enum values: 0,60,300,600,900,1800,3600,7200,10800,21600Show 2 more,43200,86400
newValueOptions
object
Options on new value detection method.
forgetAfter
int32
The duration in days after which a learned value is forgotten.
instantaneousBaseline
boolean
When set to true, Datadog uses previous values that fall within the defined learning window to construct the baseline, enabling the system to establish an accurate baseline more rapidly rather than relying solely on gradual learning over time.
learningDuration
int32
The duration in days during which values are learned, and after which signals will be generated for values that
weren't learned. If set to 0, a signal will be generated for all new values after the first value is learned.
learningMethod
enum
The learning method used to determine when signals should be generated for values that weren't learned.
Allowed enum values: duration,threshold
default: duration
learningThreshold
enum
A number of occurrences after which signals will be generated for values that weren't learned.
Allowed enum values: 0,1
sequenceDetectionOptions
object
Options on sequence detection method.
stepTransitions
[object]
Transitions defining the allowed order of steps and their evaluation windows.
child
string
Name of the child step.
evaluationWindow
enum
A time window is specified to match when at least one of the cases matches true. This is a sliding window
and evaluates in real time. For third party detection method, this field is not used.
Allowed enum values: 0,60,300,600,900,1800,3600,7200,10800,21600Show 2 more,43200,86400
parent
string
Name of the parent step.
steps
[object]
Steps that define the conditions to be matched in sequence.
condition
string
Condition referencing rule queries (e.g., a > 0).
evaluationWindow
enum
A time window is specified to match when at least one of the cases matches true. This is a sliding window
and evaluates in real time. For third party detection method, this field is not used.
Allowed enum values: 0,60,300,600,900,1800,3600,7200,10800,21600Show 2 more,43200,86400
name
string
Unique name identifying the step.
thirdPartyRuleOptions
object
Options on third party detection method.
defaultNotifications
[string]
Notification targets for the logs that do not correspond to any of the cases.
defaultStatus
enum
Severity of the Security Signal.
Allowed enum values: info,low,medium,high,critical
rootQueries
[object]
Queries to be combined with third party case queries. Each of them can have different group by fields, to aggregate differently based on the type of alert.
groupByFields
[string]
Fields to group by.
query
string
Query to run on logs.
signalTitleTemplate
string
A template for the signal title; if omitted, the title is generated based on the case name.
queries [required]
[object]
Queries for selecting logs analyzed by the job. Up to 10 queries are allowed.
additionalFilters
string
Additional filters appended to the query at evaluation time.
aggregation
enum
The aggregation type.
Allowed enum values: count,cardinality,sum,max,new_value,geo_data,event_count,none
correlatedByFields
[string]
Fields used to correlate results across queries in sequence detection rules.
correlatedQueryIndex
int64
Zero-based index of the query to correlate with in sequence detection rules. Up to 10 queries are supported, so valid values are 0 to 9.
customQueryExtension
string
Custom query extension used to refine the base query.
dataSource
enum
Source of events, either logs, audit trail, security signals, or Datadog events. app_sec_spans is deprecated in favor of spans.
Allowed enum values: logs,audit,app_sec_spans,spans,security_runtime,network,events,security_signals
default: logs
datasetIds
[string]
IDs of reference datasets used by this query.
distinctFields
[string]
Field for which the cardinality is measured. Sent as an array.
groupByFields
[string]
Fields to group by.
hasOptionalGroupByFields
boolean
When false, events without a group-by value are ignored by the query. When true, events with missing group-by fields are processed with N/A, replacing the missing values.
index
string
Index used to load the data for this query.
indexes
[string]
Indexes used to load the data for this query. Mutually exclusive with index.
metrics
[string]
Group of target fields to aggregate over when using the sum, max, geo data, or new value aggregations. The sum, max, and geo data aggregations only accept one value in this list, whereas the new value aggregation accepts up to five values.
name
string
Name of the query.
query
string
Query to run on logs.
queryLanguage
string
Language used to parse the query string.
referenceTables
[object]
Reference tables used in the queries.
checkPresence
boolean
Whether to include or exclude the matched values.
columnName
string
The name of the column in the reference table.
logFieldPath
string
The field in the log to match against the reference table.
ruleQueryName
string
The name of the query to apply the reference table to.
tableName
string
The name of the reference table.
tags
[string]
Tags for generated signals.
thirdPartyCases
[object]
Cases for generating results from third-party detection method. Only available for third-party detection method. Up to 10 cases are allowed.
name
string
Name of the case.
notifications
[string]
Notification targets for each case.
query
string
A query to map a third party event to this case.
status [required]
enum
Severity of the Security Signal.
Allowed enum values: info,low,medium,high,critical
to [required]
int64
Ending time of data analyzed by the job.
type
string
Job type.
signalOutput
boolean
Whether the job outputs signals when results are converted.
type
enum
Type of data.
Allowed enum values: historicalDetectionsJobCreate
{"data":{"type":"historicalDetectionsJobCreate","attributes":{"jobDefinition":{"type":"log_detection","name":"Excessive number of failed attempts.","queries":[{"query":"source:non_existing_src_weekend","aggregation":"count","groupByFields":[],"distinctFields":[]}],"cases":[{"name":"Condition 1","status":"info","notifications":[],"condition":"a > 1"}],"options":{"keepAlive":3600,"maxSignalDuration":86400,"evaluationWindow":900},"message":"A large number of failed login attempts.","tags":[],"from":1730387522611,"to":1730387532611,"index":"main"}}}}
// Run a historical job returns "Status created" responsepackagemainimport("context""encoding/json""fmt""os""github.com/DataDog/datadog-api-client-go/v2/api/datadog""github.com/DataDog/datadog-api-client-go/v2/api/datadogV2")funcmain(){body:=datadogV2.RunHistoricalJobRequest{Data:&datadogV2.RunHistoricalJobRequestData{Type:datadogV2.RUNHISTORICALJOBREQUESTDATATYPE_HISTORICALDETECTIONSJOBCREATE.Ptr(),Attributes:&datadogV2.RunHistoricalJobRequestAttributes{JobDefinition:&datadogV2.JobDefinition{Type:datadog.PtrString("log_detection"),Name:"Excessive number of failed attempts.",Queries:[]datadogV2.HistoricalJobQuery{{Query:datadog.PtrString("source:non_existing_src_weekend"),Aggregation:datadogV2.SECURITYMONITORINGRULEQUERYAGGREGATION_COUNT.Ptr(),GroupByFields:[]string{},DistinctFields:[]string{},},},Cases:[]datadogV2.SecurityMonitoringRuleCaseCreate{{Name:datadog.PtrString("Condition 1"),Status:datadogV2.SECURITYMONITORINGRULESEVERITY_INFO,Notifications:[]string{},Condition:datadog.PtrString("a > 1"),},},Options:&datadogV2.HistoricalJobOptions{KeepAlive:datadogV2.SECURITYMONITORINGRULEKEEPALIVE_ONE_HOUR.Ptr(),MaxSignalDuration:datadogV2.SECURITYMONITORINGRULEMAXSIGNALDURATION_ONE_DAY.Ptr(),EvaluationWindow:datadogV2.SECURITYMONITORINGRULEEVALUATIONWINDOW_FIFTEEN_MINUTES.Ptr(),},Message:"A large number of failed login attempts.",Tags:[]string{},From:1730387522611,To:1730387532611,Index:"main",},},},}ctx:=datadog.NewDefaultContext(context.Background())configuration:=datadog.NewConfiguration()configuration.SetUnstableOperationEnabled("v2.RunHistoricalJob",true)apiClient:=datadog.NewAPIClient(configuration)api:=datadogV2.NewSecurityMonitoringApi(apiClient)resp,r,err:=api.RunHistoricalJob(ctx,body)iferr!=nil{fmt.Fprintf(os.Stderr,"Error when calling `SecurityMonitoringApi.RunHistoricalJob`: %v\n",err)fmt.Fprintf(os.Stderr,"Full HTTP response: %v\n",r)}responseContent,_:=json.MarshalIndent(resp,""," ")fmt.Fprintf(os.Stdout,"Response from `SecurityMonitoringApi.RunHistoricalJob`:\n%s\n",responseContent)}
// Run a historical job returns "Status created" responseimportcom.datadog.api.client.ApiClient;importcom.datadog.api.client.ApiException;importcom.datadog.api.client.v2.api.SecurityMonitoringApi;importcom.datadog.api.client.v2.model.HistoricalJobOptions;importcom.datadog.api.client.v2.model.HistoricalJobQuery;importcom.datadog.api.client.v2.model.JobCreateResponse;importcom.datadog.api.client.v2.model.JobDefinition;importcom.datadog.api.client.v2.model.RunHistoricalJobRequest;importcom.datadog.api.client.v2.model.RunHistoricalJobRequestAttributes;importcom.datadog.api.client.v2.model.RunHistoricalJobRequestData;importcom.datadog.api.client.v2.model.RunHistoricalJobRequestDataType;importcom.datadog.api.client.v2.model.SecurityMonitoringRuleCaseCreate;importcom.datadog.api.client.v2.model.SecurityMonitoringRuleEvaluationWindow;importcom.datadog.api.client.v2.model.SecurityMonitoringRuleKeepAlive;importcom.datadog.api.client.v2.model.SecurityMonitoringRuleMaxSignalDuration;importcom.datadog.api.client.v2.model.SecurityMonitoringRuleQueryAggregation;importcom.datadog.api.client.v2.model.SecurityMonitoringRuleSeverity;importjava.util.Collections;publicclassExample{publicstaticvoidmain(String[]args){ApiClientdefaultClient=ApiClient.getDefaultApiClient();defaultClient.setUnstableOperationEnabled("v2.runHistoricalJob",true);SecurityMonitoringApiapiInstance=newSecurityMonitoringApi(defaultClient);RunHistoricalJobRequestbody=newRunHistoricalJobRequest().data(newRunHistoricalJobRequestData().type(RunHistoricalJobRequestDataType.HISTORICALDETECTIONSJOBCREATE).attributes(newRunHistoricalJobRequestAttributes().jobDefinition(newJobDefinition().type("log_detection").name("Excessive number of failed attempts.").queries(Collections.singletonList(newHistoricalJobQuery().query("source:non_existing_src_weekend").aggregation(SecurityMonitoringRuleQueryAggregation.COUNT))).cases(Collections.singletonList(newSecurityMonitoringRuleCaseCreate().name("Condition 1").status(SecurityMonitoringRuleSeverity.INFO).condition("a > 1"))).options(newHistoricalJobOptions().keepAlive(SecurityMonitoringRuleKeepAlive.ONE_HOUR).maxSignalDuration(SecurityMonitoringRuleMaxSignalDuration.ONE_DAY).evaluationWindow(SecurityMonitoringRuleEvaluationWindow.FIFTEEN_MINUTES)).message("A large number of failed login attempts.").from(1730387522611L).to(1730387532611L).index("main"))));try{JobCreateResponseresult=apiInstance.runHistoricalJob(body);System.out.println(result);}catch(ApiExceptione){System.err.println("Exception when calling SecurityMonitoringApi#runHistoricalJob");System.err.println("Status code: "+e.getCode());System.err.println("Reason: "+e.getResponseBody());System.err.println("Response headers: "+e.getResponseHeaders());e.printStackTrace();}}}
"""
Run a historical job returns "Status created" response
"""fromdatadog_api_clientimportApiClient,Configurationfromdatadog_api_client.v2.api.security_monitoring_apiimportSecurityMonitoringApifromdatadog_api_client.v2.model.historical_job_optionsimportHistoricalJobOptionsfromdatadog_api_client.v2.model.historical_job_queryimportHistoricalJobQueryfromdatadog_api_client.v2.model.job_definitionimportJobDefinitionfromdatadog_api_client.v2.model.run_historical_job_requestimportRunHistoricalJobRequestfromdatadog_api_client.v2.model.run_historical_job_request_attributesimportRunHistoricalJobRequestAttributesfromdatadog_api_client.v2.model.run_historical_job_request_dataimportRunHistoricalJobRequestDatafromdatadog_api_client.v2.model.run_historical_job_request_data_typeimportRunHistoricalJobRequestDataTypefromdatadog_api_client.v2.model.security_monitoring_rule_case_createimportSecurityMonitoringRuleCaseCreatefromdatadog_api_client.v2.model.security_monitoring_rule_evaluation_windowimport(SecurityMonitoringRuleEvaluationWindow,)fromdatadog_api_client.v2.model.security_monitoring_rule_keep_aliveimportSecurityMonitoringRuleKeepAlivefromdatadog_api_client.v2.model.security_monitoring_rule_max_signal_durationimport(SecurityMonitoringRuleMaxSignalDuration,)fromdatadog_api_client.v2.model.security_monitoring_rule_query_aggregationimport(SecurityMonitoringRuleQueryAggregation,)fromdatadog_api_client.v2.model.security_monitoring_rule_severityimportSecurityMonitoringRuleSeveritybody=RunHistoricalJobRequest(data=RunHistoricalJobRequestData(type=RunHistoricalJobRequestDataType.HISTORICALDETECTIONSJOBCREATE,attributes=RunHistoricalJobRequestAttributes(job_definition=JobDefinition(type="log_detection",name="Excessive number of failed attempts.",queries=[HistoricalJobQuery(query="source:non_existing_src_weekend",aggregation=SecurityMonitoringRuleQueryAggregation.COUNT,group_by_fields=[],distinct_fields=[],),],cases=[SecurityMonitoringRuleCaseCreate(name="Condition 1",status=SecurityMonitoringRuleSeverity.INFO,notifications=[],condition="a > 1",),],options=HistoricalJobOptions(keep_alive=SecurityMonitoringRuleKeepAlive.ONE_HOUR,max_signal_duration=SecurityMonitoringRuleMaxSignalDuration.ONE_DAY,evaluation_window=SecurityMonitoringRuleEvaluationWindow.FIFTEEN_MINUTES,),message="A large number of failed login attempts.",tags=[],_from=1730387522611,to=1730387532611,index="main",),),),)configuration=Configuration()configuration.unstable_operations["run_historical_job"]=TruewithApiClient(configuration)asapi_client:api_instance=SecurityMonitoringApi(api_client)response=api_instance.run_historical_job(body=body)print(response)
# Run a historical job returns "Status created" responserequire"datadog_api_client"DatadogAPIClient.configuredo|config|config.unstable_operations["v2.run_historical_job".to_sym]=trueendapi_instance=DatadogAPIClient::V2::SecurityMonitoringAPI.newbody=DatadogAPIClient::V2::RunHistoricalJobRequest.new({data:DatadogAPIClient::V2::RunHistoricalJobRequestData.new({type:DatadogAPIClient::V2::RunHistoricalJobRequestDataType::HISTORICALDETECTIONSJOBCREATE,attributes:DatadogAPIClient::V2::RunHistoricalJobRequestAttributes.new({job_definition:DatadogAPIClient::V2::JobDefinition.new({type:"log_detection",name:"Excessive number of failed attempts.",queries:[DatadogAPIClient::V2::HistoricalJobQuery.new({query:"source:non_existing_src_weekend",aggregation:DatadogAPIClient::V2::SecurityMonitoringRuleQueryAggregation::COUNT,group_by_fields:[],distinct_fields:[],}),],cases:[DatadogAPIClient::V2::SecurityMonitoringRuleCaseCreate.new({name:"Condition 1",status:DatadogAPIClient::V2::SecurityMonitoringRuleSeverity::INFO,notifications:[],condition:"a > 1",}),],options:DatadogAPIClient::V2::HistoricalJobOptions.new({keep_alive:DatadogAPIClient::V2::SecurityMonitoringRuleKeepAlive::ONE_HOUR,max_signal_duration:DatadogAPIClient::V2::SecurityMonitoringRuleMaxSignalDuration::ONE_DAY,evaluation_window:DatadogAPIClient::V2::SecurityMonitoringRuleEvaluationWindow::FIFTEEN_MINUTES,}),message:"A large number of failed login attempts.",tags:[],from:1730387522611,to:1730387532611,index:"main",}),}),}),})papi_instance.run_historical_job(body)
// Run a historical job returns "Status created" response
usedatadog_api_client::datadog;usedatadog_api_client::datadogV2::api_security_monitoring::SecurityMonitoringAPI;usedatadog_api_client::datadogV2::model::HistoricalJobOptions;usedatadog_api_client::datadogV2::model::HistoricalJobQuery;usedatadog_api_client::datadogV2::model::JobDefinition;usedatadog_api_client::datadogV2::model::RunHistoricalJobRequest;usedatadog_api_client::datadogV2::model::RunHistoricalJobRequestAttributes;usedatadog_api_client::datadogV2::model::RunHistoricalJobRequestData;usedatadog_api_client::datadogV2::model::RunHistoricalJobRequestDataType;usedatadog_api_client::datadogV2::model::SecurityMonitoringRuleCaseCreate;usedatadog_api_client::datadogV2::model::SecurityMonitoringRuleEvaluationWindow;usedatadog_api_client::datadogV2::model::SecurityMonitoringRuleKeepAlive;usedatadog_api_client::datadogV2::model::SecurityMonitoringRuleMaxSignalDuration;usedatadog_api_client::datadogV2::model::SecurityMonitoringRuleQueryAggregation;usedatadog_api_client::datadogV2::model::SecurityMonitoringRuleSeverity;#[tokio::main]asyncfnmain(){letbody=RunHistoricalJobRequest::new().data(RunHistoricalJobRequestData::new().attributes(RunHistoricalJobRequestAttributes::new().job_definition(JobDefinition::new(vec![SecurityMonitoringRuleCaseCreate::new(SecurityMonitoringRuleSeverity::INFO,).condition("a > 1".to_string()).name("Condition 1".to_string()).notifications(vec![])],1730387522611,"main".to_string(),"A large number of failed login attempts.".to_string(),"Excessive number of failed attempts.".to_string(),vec![HistoricalJobQuery::new().aggregation(SecurityMonitoringRuleQueryAggregation::COUNT).distinct_fields(vec![]).group_by_fields(vec![]).query("source:non_existing_src_weekend".to_string())],1730387532611,).options(HistoricalJobOptions::new().evaluation_window(SecurityMonitoringRuleEvaluationWindow::FIFTEEN_MINUTES,).keep_alive(SecurityMonitoringRuleKeepAlive::ONE_HOUR).max_signal_duration(SecurityMonitoringRuleMaxSignalDuration::ONE_DAY),).tags(vec![]).type_("log_detection".to_string()),),).type_(RunHistoricalJobRequestDataType::HISTORICALDETECTIONSJOBCREATE),);letmutconfiguration=datadog::Configuration::new();configuration.set_unstable_operation_enabled("v2.RunHistoricalJob",true);letapi=SecurityMonitoringAPI::with_config(configuration);letresp=api.run_historical_job(body).await;ifletOk(value)=resp{println!("{:#?}",value);}else{println!("{:#?}",resp.unwrap_err());}}
DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com"DD_API_KEY="<DD_API_KEY>"DD_APP_KEY="<DD_APP_KEY>"cargo run
/**
* Run a historical job returns "Status created" response
*/import{client,v2}from"@datadog/datadog-api-client";constconfiguration=client.createConfiguration();configuration.unstableOperations["v2.runHistoricalJob"]=true;constapiInstance=newv2.SecurityMonitoringApi(configuration);constparams: v2.SecurityMonitoringApiRunHistoricalJobRequest={body:{data:{type:"historicalDetectionsJobCreate",attributes:{jobDefinition:{type:"log_detection",name:"Excessive number of failed attempts.",queries:[{query:"source:non_existing_src_weekend",aggregation:"count",groupByFields:[],distinctFields:[],},],cases:[{name:"Condition 1",status:"info",notifications:[],condition:"a > 1",},],options:{keepAlive: 3600,maxSignalDuration: 86400,evaluationWindow: 900,},message:"A large number of failed login attempts.",tags:[],from:1730387522611,to: 1730387532611,index:"main",},},},},};apiInstance.runHistoricalJob(params).then((data: v2.JobCreateResponse)=>{console.log("API called successfully. Returned data: "+JSON.stringify(data));}).catch((error: any)=>console.error(error));