Language

한국어 English Français 日本語 Español

Datadog Site

Site help
US1 US3 US5 EU AP1 AP2 US1-FED
이 페이지는 아직 영어로 제공되지 않습니다. 번역 작업 중입니다.
현재 번역 프로젝트에 대한 질문이나 피드백이 있으신 경우 언제든지 연락주시기 바랍니다.

Test Optimization

Search and manage flaky tests through Test Optimization. See the Test Optimization page for more information.

Search flaky tests

Note: This endpoint is in public beta and may be subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/test/flaky-test-management/testshttps://api.ap2.datadoghq.com/api/v2/test/flaky-test-management/testshttps://api.datadoghq.eu/api/v2/test/flaky-test-management/testshttps://api.ddog-gov.com/api/v2/test/flaky-test-management/testshttps://api.datadoghq.com/api/v2/test/flaky-test-management/testshttps://api.us3.datadoghq.com/api/v2/test/flaky-test-management/testshttps://api.us5.datadoghq.com/api/v2/test/flaky-test-management/tests

개요

List endpoint returning flaky tests from Flaky Test Management. Results are paginated. This endpoint requires the test_optimization_read permission.

OAuth apps require the test_optimization_read authorization scope to access this endpoint.

요청

Body Data

Expand All

항목

유형

설명

data

object

The JSON:API data for flaky tests search request.

attributes

object

Attributes for the flaky tests search request.

filter

object

Search filter settings.

query

string

Search query following log syntax used to filter flaky tests, same as on Flaky Tests Management UI. The supported search keys are:

  • flaky_test_state
  • flaky_test_category
  • @test.name
  • @test.suite
  • @test.module
  • @test.service
  • @git.repository.id_v2
  • @git.branch
  • @test.codeowners
  • env

default: *

page

object

Pagination attributes for listing flaky tests.

cursor

string

List following results with a cursor provided in the previous request.

limit

int64

Maximum number of flaky tests in the response.

default: 10

sort

enum

Parameter for sorting flaky test results. The default sort is by ascending Fully Qualified Name (FQN). The FQN is the concatenation of the test module, suite, and name. Allowed enum values: fqn,-fqn,first_flaked,-first_flaked,last_flaked,-last_flaked,failure_rate,-failure_rate,pipelines_failed,-pipelines_failed,pipelines_duration_lost,-pipelines_duration_lost

type

enum

The definition of FlakyTestsSearchRequestDataType object. Allowed enum values: search_flaky_tests_request

{
  "data": {
    "attributes": {
      "filter": {
        "query": "flaky_test_state:active @git.repository.id_v2:\"github.com/datadog/shopist\""
      },
      "page": {
        "cursor": "eyJzdGFydEF0IjoiQVFBQUFYS2tMS3pPbm40NGV3QUFBQUJCV0V0clRFdDZVbG8zY3pCRmNsbHJiVmxDWlEifQ==",
        "limit": 25
      },
      "sort": "failure_rate"
    },
    "type": "string"
  }
}

응답

OK

Response object with flaky tests matching the search request.

Expand All

항목

유형

설명

data

[object]

Array of flaky tests matching the request.

attributes

object

Attributes of a flaky test.

attempt_to_fix_id

string

Unique identifier for the attempt to fix this flaky test. Use this ID in the Git commit message in order to trigger the attempt to fix workflow.

When the workflow is triggered the test is automatically retried by the tracer a certain number of configurable times. When all retries pass, the test is automatically marked as fixed in Flaky Test Management. Test runs are tagged with @test.test_management.attempt_to_fix_passed and @test.test_management.is_attempt_to_fix when the attempt to fix workflow is triggered.

codeowners

[string]

The name of the test's code owners as inferred from the repository configuration.

envs

[string]

List of environments where this test has been flaky.

first_flaked_branch

string

The branch name where the test exhibited flakiness for the first time.

first_flaked_sha

string

The commit SHA where the test exhibited flakiness for the first time.

first_flaked_ts

int64

Unix timestamp when the test exhibited flakiness for the first time.

flaky_category

string

The category of a flaky test.

flaky_state

enum

The current state of the flaky test. Allowed enum values: active,fixed,quarantined,disabled

last_flaked_branch

string

The branch name where the test exhibited flakiness for the last time.

last_flaked_sha

string

The commit SHA where the test exhibited flakiness for the last time.

last_flaked_ts

int64

Unix timestamp when the test exhibited flakiness for the last time.

module

string

The name of the test module. The definition of module changes slightly per language:

  • In .NET, a test module groups every test that is run under the same unit test project.
  • In Swift, a test module groups every test that is run for a given bundle.
  • In JavaScript, the test modules map one-to-one to test sessions.
  • In Java, a test module groups every test that is run by the same Maven Surefire/Failsafe or Gradle Test task execution.
  • In Python, a test module groups every test that is run under the same .py file as part of a test suite, which is typically managed by a framework like unittest or pytest.
  • In Ruby, a test module groups every test that is run within the same test file, which is typically managed by a framework like RSpec or Minitest.

name

string

The test name. A concise name for a test case. Defined in the test itself.

pipeline_stats

object

CI pipeline related statistics for the flaky test. This information is only available if test runs are associated with CI pipeline events from CI Visibility.

failed_pipelines

int64

The number of pipelines that failed due to this test for the past 7 days. This is computed as the sum of failed CI pipeline events associated with test runs where the flaky test failed.

total_lost_time_ms

int64

The total time lost by CI pipelines due to this flaky test in milliseconds. This is computed as the sum of the duration of failed CI pipeline events associated with test runs where the flaky test failed.

services

[string]

List of test service names where this test has been flaky.

A test service is a group of tests associated with a project or repository. It contains all the individual tests for your code, optionally organized into test suites, which are like folders for your tests.

suite

string

The name of the test suite. A group of tests exercising the same unit of code depending on your language and testing framework.

test_run_metadata

object

Metadata about the latest failed test run of the flaky test.

duration_ms

int64

The duration of the test run in milliseconds.

error_message

string

The error message from the test failure.

error_stack

string

The stack trace from the test failure.

source_end

int64

The line number where the test ends in the source file.

source_file

string

The source file where the test is defined.

source_start

int64

The line number where the test starts in the source file.

test_stats

object

Test statistics for the flaky test.

failure_rate_pct

double

The failure rate percentage of the test for the past 7 days. This is the number of failed test runs divided by the total number of test runs (excluding skipped test runs).

id

string

Test's ID. This ID is the hash of the test's Fully Qualified Name and Git repository ID. On the Test Runs UI it is the same as the test_fingerprint_fqn tag.

type

enum

The type of the flaky test from Flaky Test Management. Allowed enum values: flaky_test

meta

object

Metadata for the flaky tests search response.

pagination

object

Pagination metadata for flaky tests.

next_page

string

Cursor for the next page of results.

{
  "data": [
    {
      "attributes": {
        "attempt_to_fix_id": "I42TEO",
        "codeowners": [
          "@foo",
          "@bar"
        ],
        "envs": "prod",
        "first_flaked_branch": "main",
        "first_flaked_sha": "0c6be03165b7f7ffe96e076ffb29afb2825616c3",
        "first_flaked_ts": 1757688149,
        "flaky_category": "Timeout",
        "flaky_state": "active",
        "last_flaked_branch": "main",
        "last_flaked_sha": "0c6be03165b7f7ffe96e076ffb29afb2825616c3",
        "last_flaked_ts": 1757688149,
        "module": "TestModule",
        "name": "TestName",
        "pipeline_stats": {
          "failed_pipelines": 319,
          "total_lost_time_ms": 1527550000
        },
        "services": [
          "foo",
          "bar"
        ],
        "suite": "TestSuite",
        "test_run_metadata": {
          "duration_ms": 27398,
          "error_message": "Expecting actual not to be empty",
          "error_stack": "Traceback (most recent call last):\n  File \"test_foo.py\", line 10, in test_foo\n    assert actual == expected\nAssertionError: Expecting actual not to be empty",
          "source_end": 20,
          "source_file": "test_foo.py",
          "source_start": 10
        },
        "test_stats": {
          "failure_rate_pct": 0.1
        }
      },
      "id": "string",
      "type": "string"
    }
  ],
  "meta": {
    "pagination": {
      "next_page": "string"
    }
  }
}

Bad Request

API error response.

Expand All

항목

유형

설명

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Not Authorized

API error response.

Expand All

항목

유형

설명

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Too many requests

API error response.

Expand All

항목

유형

설명

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

코드 사례

                  # Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/test/flaky-test-management/tests" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-d @- << EOF
{}
EOF