ProphetStor Federator.ai is an AI-based solution that helps enterprise manage, optimize, auto-scale resources for any applications on Kubernetes. Using advanced machine learning algorithms to predict application workload, Federator.ai scales the right amount of resources at the right time for optimized application performance.
With integration of ProphetStor Federator.ai, users can easily track the Kafka message production/consumption rate, as well as the prediction of message production rate from Federator.ai dashboard. Based on the prediction or message production rate, Federator.ai automatically scales Kafka consumer replicas to handle the workload. This can be visualized from Federator.ai dashboard where the recommended consumer replicas and the current number of consumer replicas are shown. Additionally, overall consumer lags as well as the average latency in the queue before a message is received by a consumer are also shown on the dashboard for better performance monitoring.
Install the Federator.ai for OpenShift/Kubernetes by the following command
$ curl https://raw.githubusercontent.com/containers-ai/federatorai-operator/datadog/deploy/install.sh |bash
$ curl https://raw.githubusercontent.com/containers-ai/federatorai-operator/datadog/deploy/install.sh |bash Checking environment version... ...Passed Please input Federator.ai Operator tag: datadog Enter the namespace you want to install Federator.ai [default: federatorai]: ......... (snipped) ......... You can now access GUI through https://federatorai-dashboard-frontend-federatorai.apps.jc-ocp4.172-31-17-84.nip.io Default login credential is admin/admin Also, you can start to apply alamedascaler CR for the namespace you would like to monitor. Review administration guide for further details. ======================================== ......... (snipped) ......... Install Alameda successfully Downloaded YAML files are located under /tmp/install-op
Verify Federator.ai pods are running properly
$ kubectl get pod -n federatorai
Log in to Federator.ai GUI, URL and login credential could be found in the output of Step 2.
A Datadog account is required for connecting and using Datadog. If you don’t have an account, visit the Datadog website and sign up for a free trial account.
Log in to Datadog with your account and get an API key and Application key for using Datadog API.
Configure the Federator.ai Data-Adapter.
Download the Data-Adapter configuration script from Github.
$ curl https://raw.githubusercontent.com/containers-ai/federatorai-operator/datadog/deploy/federatorai-setup-for-datadog.sh -O
Change the execution permission.
$ chomd +x federatorai-setup-for-datadog.sh
Prepare .kubeconfig (sh -c “export KUBECONFIG=.kubeconfig; oc login
$ sh -c "export KUBECONFIG=.kubeconfig; oc login https://api.ocp4.example.com:6443"
Run the configuration script and follow the steps to fill in configuration parameters:
$ ./federatorai-setup-for-datadog.sh -k .kubeconfig
$ ./federatorai-setup-for-datadog.sh -k .kubeconfig You are connecting to cluster: https://api.jc-ocp4.172-31-17-84.nip.io:6443 Getting Datadog info... Input a Datadog API Key :xxxxx9273dxxcbc155xx3a7331xxxxx Input a Datadog Application Key :xxxxx7220db1478xxxxxcb5c323fcb02a11xxxxx Getting the Kafka info... No.1 Input Kafka consumer deployment name : consumer Input Kafka consumer deeployment namespace : myproject Input Kafka consumer minimum replica number : 1 Input Kafka consumer maximum replica number : 30 Input Kafka consumer group name : group0001 Input Kafka consumer group namespace : myproject Input Kafka consumer topic name : topic0001 Input Kafka consumer topic namespace : myproject Do you want to input another set? [default: n]: Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply secret/federatorai-data-adapter-secret configured Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply configmap/federatorai-data-adapter-config configured Setup Federator.ai for datadog successfully
Please refer to Federator.ai and Datadog Integration - Installation and Configuration Guide for more details.
|integration status for showing Federator.ai health status.|
|recommended deployment/statefulset replicas.|
|workload prediction for Kafka metrics.|
|The delta of kafka.broker_offset timeseries in one minute.|
|The delta of kafka.consumer_offset timeseries in one minute.|
Federator.ai does not include any service checks.
Federator.ai does not include any events.