StreamNative

Información general

StreamNative proporciona una plataforma de mensajería y streaming de eventos de nivel empresarial basada en Apache Pulsar. Ofrece soluciones de streaming de datos escalables y en tiempo real con características como multiinquilino, replicación geográfica e integración continua con servicios de nube.

La integración StreamNative recopila los siguientes tipos de métricas:

  1. Health
  2. Recurso Pulsar
  3. Conector de fuentes
  4. Conector de sumideros
  5. Kafka Connect

Configuración

Generar credenciales de API en StreamNative

  1. Inicia sesión en la cuenta de la consola de StreamNative Cloud.
  2. Haz clic en el icono del perfil y ve a la pestaña Cuentas y accesos.
  3. Busca la cuenta de servicio con permisos de Admin o un enlace al rol metrics-viewer.
    • Si no existe una cuenta de servicio, selecciona New -> Service Account (Nuevo -> Cuenta de servicio) para crear una, y asegúrate de activar la opción Super Admin.
    • Para vincular una cuenta de servicio con el rol metrics-viewer, consulta la documentación metrics-viewer rolebinding.
  4. En la parte derecha de la cuenta de servicio elegida, haz clic en el botón ....
  5. Selecciona Download OAuth2 Key (Descargar clave OAuth2) para obtener el ID de cliente y el Secreto de cliente.

Obtener el Organization ID y el Instance Name

  1. Haz clic en el icono del perfil y selecciona Organizations (Organizaciones).
  2. Selecciona la Organización de la que deben recopilarse datos.
  3. En el menú desplegable Select an Instance (Seleccionar una instancia), obtén el Nombre de la instancia.

Conectar tu cuenta de StreamNative a Datadog

  1. Añade tu ID de organización, nombre de instancia, ID de cliente y secreto de cliente.
    ParámetrosDescripción
    ID de organizaciónID de organización de tu cuenta de StreamNative.
    Nombre de instanciaNombre de instancia de la organización correspondiente.
    ID de clienteID de cliente de tu cuenta de servicio.
    Secreto de clienteSecreto de cliente de tu cuenta de servicio.
  2. Haz clic en el botón Save (Guardar) para guardar la configuración.

Datos recopilados

Logs

La integración StreamNative no incluye logs.

Métricas

La integración StreamNative recopila y envía las siguientes métricas a Datadog.

streamnative.health.pulsar_detector_e2e_latency_ms
(gauge)
The latency distribution from message sending to message consumption
Shown as millisecond
streamnative.health.pulsar_detector_geo_latency_ms
(gauge)
The latency distribution Latency distribution from message sending to message consumption across clusters
Shown as millisecond
streamnative.health.pulsar_detector_publish_latency_ms
(gauge)
The latency distribution of message sending
Shown as millisecond
streamnative.health.pulsar_detector_pulsar_sla_messaging_up
(gauge)
The gauge for indicating the messaging service up or down
streamnative.health.pulsar_detector_pulsar_sla_webservice_up
(gauge)
The gauge for indicating the webservice up or down
streamnative.kafka_connect.jvm_gc_collection_seconds_sum
(gauge)
Time spent in a given JVM garbage collector in seconds.
Shown as second
streamnative.kafka_connect.jvm_memory_committed_bytes
(gauge)
Committed bytes of a given JVM memory area
Shown as byte
streamnative.kafka_connect.jvm_memory_init_bytes
(gauge)
Initial bytes of a given JVM memory area
Shown as byte
streamnative.kafka_connect.jvm_memory_max_bytes
(gauge)
Max bytes of a given JVM memory area
Shown as byte
streamnative.kafka_connect.jvm_memory_used_bytes
(gauge)
Used bytes of a given JVM memory area
Shown as byte
streamnative.kafka_connect.kafka_connect_connector_task_batch_size_avg
(gauge)
The average size of the batches processed by the connector
streamnative.kafka_connect.kafka_connect_connector_task_batch_size_max
(gauge)
The maximum size of the batches processed by the connector
streamnative.kafka_connect.kafka_connect_connector_task_offset_commit_avg_time_ms
(gauge)
The average time in milliseconds taken by this task to commit offsets
Shown as millisecond
streamnative.kafka_connect.kafka_connect_connector_task_offset_commit_failure_percentage
(gauge)
The average percentage of this task’s offset commit attempts that failed
Shown as fraction
streamnative.kafka_connect.kafka_connect_connector_task_offset_commit_max_time_ms
(gauge)
The maximum time in milliseconds taken by this task to commit offsets
Shown as millisecond
streamnative.kafka_connect.kafka_connect_connector_task_offset_commit_success_percentage
(gauge)
The average percentage of this task’s offset commit attempts that succeeded
Shown as fraction
streamnative.kafka_connect.kafka_connect_connector_task_pause_ratio
(gauge)
The fraction of time this task has spent in the pause state
Shown as fraction
streamnative.kafka_connect.kafka_connect_connector_task_running_ratio
(gauge)
The fraction of time this task has spent in the running state
Shown as fraction
streamnative.kafka_connect.kafka_connect_sink_task_offset_commit_completion
(gauge)
The total number of offset commit completions that were completed successfully
Shown as offset
streamnative.kafka_connect.kafka_connect_sink_task_offset_commit_completion_rate
(gauge)
The average per-second number of offset commit completions that were completed successfully
Shown as offset
streamnative.kafka_connect.kafka_connect_sink_task_offset_commit_seq_no
(gauge)
The current sequence number for offset commits
streamnative.kafka_connect.kafka_connect_sink_task_offset_commit_skip
(gauge)
The total number of offset commit completions that were received too late and skipped/ignored
Shown as offset
streamnative.kafka_connect.kafka_connect_sink_task_offset_commit_skip_rate
(gauge)
The average per-second number of offset commit completions that were received too late and skipped/ignored
Shown as offset
streamnative.kafka_connect.kafka_connect_sink_task_partition_count
(gauge)
The number of topic partitions assigned to this task belonging to the named sink connector in this worker
streamnative.kafka_connect.kafka_connect_sink_task_put_batch_avg_time_ms
(gauge)
The average time taken by this task to put a batch of sinks records
Shown as millisecond
streamnative.kafka_connect.kafka_connect_sink_task_put_batch_max_time_ms
(gauge)
The maximum time taken by this task to put a batch of sinks records
Shown as millisecond
streamnative.kafka_connect.kafka_connect_sink_task_sink_record_active_count
(gauge)
The number of records that have been read from Kafka but not yet completely committed/flushed/acknowledged by the sink task
Shown as record
streamnative.kafka_connect.kafka_connect_sink_task_sink_record_active_count_avg
(gauge)
The average number of records that have been read from Kafka but not yet completely committed/flushed/acknowledged by the sink task
Shown as record
streamnative.kafka_connect.kafka_connect_sink_task_sink_record_active_count_max
(gauge)
The maximum number of records that have been read from Kafka but not yet completely committed/flushed/acknowledged by the sink task
Shown as record
streamnative.kafka_connect.kafka_connect_sink_task_sink_record_read
(gauge)
The total number of records read from Kafka by this task belonging to the named sink connector in this worker, since the task was last restarted
Shown as record
streamnative.kafka_connect.kafka_connect_sink_task_sink_record_read_rate
(gauge)
The average per-second number of records read from Kafka for this task belonging to the named sink connector in this worker. This is before transformations are applied
Shown as record
streamnative.kafka_connect.kafka_connect_sink_task_sink_record_send
(gauge)
The total number of records output from the transformations and sent/put to this task belonging to the named sink connector in this worker, since the task was last restarted
Shown as record
streamnative.kafka_connect.kafka_connect_sink_task_sink_record_send_rate
(gauge)
The average per-second number of records output from the transformations and sent/put to this task belonging to the named sink connector in this worker
Shown as record
streamnative.kafka_connect.kafka_connect_source_task_poll_batch_avg_time_ms
(gauge)
The average time in milliseconds taken by this task to poll for a batch of source records
Shown as millisecond
streamnative.kafka_connect.kafka_connect_source_task_poll_batch_max_time_ms
(gauge)
The maximum time in milliseconds taken by this task to poll for a batch of source records
Shown as millisecond
streamnative.kafka_connect.kafka_connect_source_task_source_record_active_count
(gauge)
The number of records that have been produced by this task but not yet completely written to Kafka
Shown as record
streamnative.kafka_connect.kafka_connect_source_task_source_record_active_count_avg
(gauge)
The average number of records that have been produced by this task but not yet completely written to Kafka
Shown as record
streamnative.kafka_connect.kafka_connect_source_task_source_record_active_count_max
(gauge)
The maximum number of records that have been produced by this task but not yet completely written to Kafka
Shown as record
streamnative.kafka_connect.kafka_connect_source_task_source_record_poll
(gauge)
The total number of records produced/polled (before transformation) by this task belonging to the named source connector in this worker
Shown as record
streamnative.kafka_connect.kafka_connect_source_task_source_record_poll_rate
(gauge)
The average per-second number of records produced/polled (before transformation) by this task belonging to the named source connector in this worker
Shown as record
streamnative.kafka_connect.kafka_connect_source_task_source_record_write
(gauge)
The number of records output from the transformations and written to Kafka for this task belonging to the named source connector in this worker, since the task was last restarted
Shown as record
streamnative.kafka_connect.kafka_connect_source_task_source_record_write_rate
(gauge)
The average per-second number of records output from the transformations and written to Kafka for this task belonging to the named source connector in this worker
Shown as record
streamnative.kafka_connect.kafka_connect_task_error_deadletterqueue_produce_failures
(gauge)
The number of failed writes to the dead letter queue
streamnative.kafka_connect.kafka_connect_task_error_deadletterqueue_produce_requests
(gauge)
The number of attempted writes to the dead letter queue
streamnative.kafka_connect.kafka_connect_task_error_last_error_timestamp
(gauge)
The epoch timestamp when this task last encountered an error
Shown as millisecond
streamnative.kafka_connect.kafka_connect_task_error_total_errors_logged
(gauge)
The total number of errors that were logged
Shown as error
streamnative.kafka_connect.kafka_connect_task_error_total_record_errors
(gauge)
The total number of record processing errors in this task
Shown as record
streamnative.kafka_connect.kafka_connect_task_error_total_record_failures
(gauge)
The total number of record processing failures in this task
Shown as record
streamnative.kafka_connect.kafka_connect_task_error_total_records_skipped
(gauge)
The total number of records skipped due to errors
Shown as record
streamnative.kafka_connect.kafka_connect_task_error_total_retries
(gauge)
The total number of operations retried
Shown as operation
streamnative.kafka_connect.kafka_connect_worker_connector_destroyed_task_count
(gauge)
The number of destroyed tasks of the connector on the worker
Shown as task
streamnative.kafka_connect.kafka_connect_worker_connector_failed_task_count
(gauge)
The number of failed tasks of the connector on the worker
Shown as task
streamnative.kafka_connect.kafka_connect_worker_connector_paused_task_count
(gauge)
The number of paused tasks of the connector on the worker
Shown as task
streamnative.kafka_connect.kafka_connect_worker_connector_restarting_task_count
(gauge)
The number of restarting tasks of the connector on the worker
Shown as task
streamnative.kafka_connect.kafka_connect_worker_connector_running_task_count
(gauge)
The number of running tasks of the connector on the worker
Shown as task
streamnative.kafka_connect.kafka_connect_worker_connector_total_task_count
(gauge)
The number of tasks of the connector on the worker
Shown as task
streamnative.kafka_connect.kafka_connect_worker_connector_unassigned_task_count
(gauge)
The number of unassigned tasks of the connector on the worker
Shown as task
streamnative.kafka_connect.process_cpu_seconds_total
(gauge)
Total user and system CPU time spent in seconds
Shown as second
streamnative.pulsar_resource.pulsar_consumers_count
(gauge)
The number of active consumers of the topic connected to this broker.
streamnative.pulsar_resource.pulsar_entry_size_le_100_kb
(gauge)
The entry rate for a topic with entry size smaller than or equal to 100 kilobytes
Shown as fraction
streamnative.pulsar_resource.pulsar_entry_size_le_128
(gauge)
The entry rate for a topic with entry size smaller than or equal to 128 bytes
Shown as fraction
streamnative.pulsar_resource.pulsar_entry_size_le_16_kb
(gauge)
The entry rate for a topic with entry size smaller than or equal to 16 kilobytes
Shown as fraction
streamnative.pulsar_resource.pulsar_entry_size_le_1_kb
(gauge)
The entry rate for a topic with entry size smaller than or equal to 1 kilobyte
Shown as fraction
streamnative.pulsar_resource.pulsar_entry_size_le_1_mb
(gauge)
The entry rate for a topic with entry size smaller than or equal to 1 megabyte
Shown as fraction
streamnative.pulsar_resource.pulsar_entry_size_le_2_kb
(gauge)
The entry rate for a topic with entry size smaller than or equal to 2 kilobytes
Shown as fraction
streamnative.pulsar_resource.pulsar_entry_size_le_4_kb
(gauge)
The entry rate for a topic with entry size smaller than or equal to 4 kilobytes
Shown as fraction
streamnative.pulsar_resource.pulsar_entry_size_le_512
(gauge)
The entry rate for a topic with entry size smaller than or equal to 512 bytes
Shown as fraction
streamnative.pulsar_resource.pulsar_entry_size_le_overflow
(gauge)
The entry rate for a topic with entry size greater than 1 megabyte
Shown as fraction
streamnative.pulsar_resource.pulsar_producers_count
(gauge)
The number of active producers of the topic connected to this broker
streamnative.pulsar_resource.pulsar_rate_in
(gauge)
The total message rate of the namespace coming into this broker
Shown as message
streamnative.pulsar_resource.pulsar_rate_out
(gauge)
The total message rate of the namespace going out from this broker
Shown as message
streamnative.pulsar_resource.pulsar_storage_backlog_size
(gauge)
The total backlog size of the topics of this topic owned by this broker
Shown as byte
streamnative.pulsar_resource.pulsar_storage_offloaded_size
(gauge)
The total amount of the data in this topic offloaded to the tiered storage
Shown as byte
streamnative.pulsar_resource.pulsar_storage_read_rate
(gauge)
The total message batches (entries) read from the storage for this topic
streamnative.pulsar_resource.pulsar_storage_size
(gauge)
The total storage size of the topics in this topic owned by this broker
Shown as byte
streamnative.pulsar_resource.pulsar_storage_write_latency_le_0_5
(gauge)
The entry rate for a topic where the storage write latency is less than or equal to 0.5 milliseconds
Shown as fraction
streamnative.pulsar_resource.pulsar_storage_write_latency_le_1
(gauge)
The entry rate for a topic where the storage write latency is less than or equal to 1 millisecond
Shown as fraction
streamnative.pulsar_resource.pulsar_storage_write_latency_le_10
(gauge)
The entry rate for a topic where the storage write latency is less than or equal to 10 milliseconds
Shown as fraction
streamnative.pulsar_resource.pulsar_storage_write_latency_le_100
(gauge)
The entry rate for a topic where the storage write latency is less than or equal to 100 milliseconds
Shown as fraction
streamnative.pulsar_resource.pulsar_storage_write_latency_le_1000
(gauge)
The entry rate for a topic where the storage write latency is less than or equal to 1000 milliseconds
Shown as fraction
streamnative.pulsar_resource.pulsar_storage_write_latency_le_20
(gauge)
The entry rate for a topic where the storage write latency is less than or equal to 20 milliseconds
Shown as fraction
streamnative.pulsar_resource.pulsar_storage_write_latency_le_200
(gauge)
The entry rate for a topic where the storage write latency is less than or equal to 200 milliseconds
Shown as fraction
streamnative.pulsar_resource.pulsar_storage_write_latency_le_5
(gauge)
The entry rate for a topic where the storage write latency is less than or equal to 5 milliseconds
Shown as fraction
streamnative.pulsar_resource.pulsar_storage_write_latency_le_50
(gauge)
The entry rate for a topic where the storage write latency is less than or equal to 50 milliseconds
Shown as fraction
streamnative.pulsar_resource.pulsar_storage_write_latency_le_overflow
(gauge)
The entry rate for a topic where the storage write latency is greater than 1 second
Shown as fraction
streamnative.pulsar_resource.pulsar_storage_write_rate
(gauge)
The total message batches (entries) written to the storage for this topic
streamnative.pulsar_resource.pulsar_subscription_delayed
(gauge)
The total message batches (entries) are delayed for dispatching
streamnative.pulsar_resource.pulsar_subscriptions_count
(gauge)
The number of Pulsar subscriptions of the topic served by this broker
streamnative.pulsar_resource.pulsar_throughput_in
(gauge)
The total throughput of the topic coming into this broker
Shown as byte
streamnative.pulsar_resource.pulsar_throughput_out
(gauge)
The total throughput of the topic going out from this broker
Shown as byte
streamnative.pulsar_resource.pulsar_topics_count
(gauge)
The number of Pulsar topics of the namespace owned by this broker
streamnative.sink_connector.jvm_gc_collection_seconds_sum
(gauge)
Time spent in a given JVM garbage collector in seconds
Shown as second
streamnative.sink_connector.jvm_memory_bytes_committed
(gauge)
Committed bytes of a given JVM memory area
Shown as byte
streamnative.sink_connector.jvm_memory_bytes_init
(gauge)
Initial bytes of a given JVM memory area
Shown as byte
streamnative.sink_connector.jvm_memory_bytes_max
(gauge)
Max bytes of a given JVM memory area
Shown as byte
streamnative.sink_connector.jvm_memory_direct_bytes_used
(gauge)
Used bytes of a given JVM memory area
Shown as byte
streamnative.sink_connector.process_cpu_seconds_total
(gauge)
Total user and system CPU time spent in seconds
Shown as second
streamnative.sink_connector.pulsar_sink_last_invocation
(gauge)
The timestamp of the last invocation of the sink
Shown as millisecond
streamnative.sink_connector.pulsar_sink_received_1min_total
(gauge)
The total number of records received from sink in the last 1 minute
Shown as record
streamnative.sink_connector.pulsar_sink_received_total
(gauge)
The total number of records received from sink
Shown as record
streamnative.sink_connector.pulsar_sink_sink_exception
(gauge)
The exception from a sink
streamnative.sink_connector.pulsar_sink_sink_exceptions_1min_total
(gauge)
The total number of sink exceptions in the last 1 minute
Shown as exception
streamnative.sink_connector.pulsar_sink_sink_exceptions_total
(gauge)
The total number of sink exceptions
Shown as exception
streamnative.sink_connector.pulsar_sink_system_exception
(gauge)
The exception from system code
streamnative.sink_connector.pulsar_sink_system_exceptions_1min_total
(gauge)
The total number of system exceptions in the last 1 minute
Shown as exception
streamnative.sink_connector.pulsar_sink_system_exceptions_total
(gauge)
The total number of system exceptions
Shown as exception
streamnative.sink_connector.pulsar_sink_written_1min_total
(gauge)
The total number of records written to a Pulsar topic in the last 1 minute
Shown as record
streamnative.sink_connector.pulsar_sink_written_total
(gauge)
The total number of records written to a Pulsar topic
Shown as record
streamnative.source_connector.jvm_gc_collection_seconds_sum
(gauge)
Time spent in a given JVM garbage collector in seconds
Shown as second
streamnative.source_connector.jvm_memory_bytes_committed
(gauge)
Committed bytes of a given JVM memory area
Shown as byte
streamnative.source_connector.jvm_memory_bytes_init
(gauge)
Initial bytes of a given JVM memory area
Shown as byte
streamnative.source_connector.jvm_memory_bytes_max
(gauge)
Max bytes of a given JVM memory area
Shown as byte
streamnative.source_connector.jvm_memory_direct_bytes_used
(gauge)
Used bytes of a given JVM memory area
Shown as byte
streamnative.source_connector.process_cpu_seconds_total
(gauge)
Total user and system CPU time spent in seconds
Shown as second
streamnative.source_connector.pulsar_source_last_invocation
(gauge)
The timestamp of the last invocation of the source
Shown as millisecond
streamnative.source_connector.pulsar_source_received_1min_total
(gauge)
The total number of records received from source in the last 1 minute
Shown as record
streamnative.source_connector.pulsar_source_received_total
(gauge)
The total number of records received from source
Shown as record
streamnative.source_connector.pulsar_source_source_exception
(gauge)
The exception from a source
streamnative.source_connector.pulsar_source_source_exceptions_1min_total
(gauge)
The total number of source exceptions in the last 1 minute
Shown as exception
streamnative.source_connector.pulsar_source_source_exceptions_total
(gauge)
The total number of source exceptions
Shown as exception
streamnative.source_connector.pulsar_source_system_exception
(gauge)
The exception from system code
streamnative.source_connector.pulsar_source_system_exceptions_1min_total
(gauge)
The total number of system exceptions in the last 1 minute
Shown as exception
streamnative.source_connector.pulsar_source_system_exceptions_total
(gauge)
The total number of system exceptions
Shown as exception
streamnative.source_connector.pulsar_source_written_1min_total
(gauge)
The total number of records written to a Pulsar topic in the last 1 minute
Shown as record
streamnative.source_connector.pulsar_source_written_total
(gauge)
The total number of records written to a Pulsar topic
Shown as record

Checks de servicio

La integración StreamNative no incluye checks de servicios.

Eventos

La integración StreamNative no incluye eventos.

Solucionar problemas

¿Necesitas ayuda? Ponte en contacto con el servicio de asistencia de Datadog.