Monitor via Prometheus and Grafana#
First, let’s have some context on the monitoring stack that we will be using during this guide. To leverage the metrics that Jina exposes, we recommend using the Prometheus/Grafana stack. In this setup, Jina will expose different metrics endpoint, and Prometheus will then be in charge of scraping these endpoints, as well as collecting, aggregating, and storing the different metrics. Prometheus will then allow external entities (like Grafana) to access these aggregated metrics via the query language PromQL. Then the role of Grafana here will be to allow users to visualize these metrics by creating dashboards.
Hint
Jina supports exposing the metrics, you are in charge of installing and managing your Prometheus/Grafana instances.
We will show you in this guide how to easily deploy the Prometheus/Grafana stack and used them to monitor a Flow.
Deploying the Flow and the monitoring stack#
Deploying on Kubernetes#
One of the challenges of monitoring a Flow is communicating its different metrics endpoints to Prometheus.
Fortunately, the Prometheus operator for Kubernetes makes the process fairly easy, because it can automatically discover new metrics endpoints to scrape.
Deploying your Jina Flow on Kubernetes is the recommended way to leverage the full potential of the monitoring feature because:
The Prometheus operator can automatically discover new endpoints to scrape
You can extend your monitoring with the rich built-in Kubernetes metrics
Deploying Prometheus and Grafana on your k8s cluster is as easy as executing the following line:
helm install prometheus prometheus-community/kube-prometheus-stack --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false
Hint
setting the serviceMonitorSelectorNilUsesHelmValues to false allows the Prometheus Operator to discover metrics endpoint outside of the helm scope which will be needed to discover the Flow metrics endpoints.
Let’s now deploy the Flow that we want to monitor:
This example shows how to start a Flow with monitoring enabled via yaml:
In a flow.yaml file
jtype: Flow
with:
monitoring: true
executors:
- uses: jinahub://SimpleIndexer
jina export kubernetes flow.yml ./config ```
from jina import Flow
f = Flow(monitoring=True).add(uses='jinahub+docker://SimpleIndexer')
f.to_kubernetes_yaml('config')
This will create a config folder containing the Kubernetes YAML definition of the Flow.
See also
You can see in-depth how to deploy a Flow on kubernetes here
Then deploy the Flow:
kubectl apply -R -f config
Wait for a couple of minutes, and you should see that the Pods are ready:
kubectl get pods
Then you can see that the new metrics endpoints are automatically discovered:
kubectl port-forward svc/prometheus-operated 9090:9090
Before querying the gateway you need to port-forward
kubectl port-forward svc/gateway 8080:8080
Now to access Grafana just do
kb port-forward svc/prometheus-grafana 3000:80
and open http://localhost:3000 in your browser
User is admin, password is prom-operator
You should see the Grafana home page.
Deploying locally#
Let’s first deploy the Flow that we want to monitor:
from jina import Flow
with Flow(monitoring=True, port_monitoring=8000, port=8080).add(
uses='jinahub://SimpleIndexer', port_monitoring=9000
) as f:
f.block()
from jina import Flow
Flow(monitoring=True, port_monitoring=8000, port=8080).add(
uses='jinahub+docker://SimpleIndexer', port_monitoring=9000
).to_docker_compose_yaml('config.yaml')
docker-compose -f config.yaml up
To monitor a flow locally you will need to install Prometheus and Grafana locally. The easiest way to do it is by using Docker Compose
First clone the repo which contains the config file:
git clone https://github.com/jina-ai/example-grafana-prometheus
cd example-grafana-prometheus/prometheus-grafana-local
then
docker-compose up
You can access the Grafana dashboard at http://localhost:3000. the username is admin and the password foobar.
Caution
This example is working locally because Prometheus is configured so that it listens to ports 8000 and 9000. However, in contrast with deploying on Kubernetes, you need to tell Prometheus which port to look at. You can change these ports by modifying this file
Deploying on Jcloud#
In case your Flow is deployed on JCloud, there is no need to provision a monitoring stack yourself. Prometheus and Grafana are
handled by JCloud and you can find a dashboard URL using the command jc status <flow_id>
Using Grafana to visualize metrics#
Once you can access the Grafana homepage then go to Browse then import and copy and paste the JSON file
You should see the following dashboard:
Hint
You should query your Flow generate the first metrics. Othewise the dashboard will look empty.
You can query the flow by doing :
from jina import Client, DocumentArray
client = Client(port=51000)
client.index(inputs=DocumentArray.empty(size=4))
client.search(inputs=DocumentArray.empty(size=4))
