How to deploy and use the monitoring with Jina

First, let’s have some context on the monitoring stack that we will be using during this guide. To leverage the metrics that Jina exposes, we recommend using the Prometheus/Grafana stack. In this setup, Jina will expose different metrics endpoint, and Prometheus will then be in charge of scraping these endpoints, as well as collecting, aggregating, and storing the different metrics. Prometheus will then allow external entities (like Grafana) to access these aggregated metrics via the query language PromQL. Then the role of Grafana here will be to allow users to visualize these metrics by creating dashboards.

Hint

Jina supports exposing the metrics, you are in charge of installing and managing your Prometheus/Grafana instances.

We will show you in this guide how to easily deploy the Prometheus/Grafana stack and used them to monitor a Flow.

1 - Deploying the Flow and the monitoring stack

Deploying on Kubernetes

One of the challenges of monitoring a Flow is communicating its different metrics endpoints to Prometheus. Fortunately, the Prometheus operator for Kubernetes makes the process fairly easy, because it can automatically discover new metrics endpoints to scrape.

Deploying your Jina Flow on Kubernetes is the recommended way to leverage the full potential of the monitoring feature because:

  • The Prometheus operator can automatically discover new endpoints to scrape

  • You can extend your monitoring with the rich built-in Kubernetes metrics

You need to have access to a Kubernetes cluster to follow the rest of this guide.

You can easily have a Kubernetes cluster on your local machine:

Or you can use a managed Kubernetes solution on the cloud:

Deploying Prometheus and Grafana on your k8s cluster is as easy as executing the following line:

helm install prometheus prometheus-community/kube-prometheus-stack --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false

Hint

setting the serviceMonitorSelectorNilUsesHelmValues to false allows the Prometheus Operator to discover metrics endpoint outside of the helm scope which will be needed to discover the Flow metrics endpoints.

Let’s now deploy the Flow that we want to monitor:

from jina import Flow

f = Flow(monitoring=True).add(uses='jinahub+docker://SimpleIndexer')
f.to_k8s_yaml('config')

This will create a config folder containing the Kubernetes YAML definition of the Flow.

See also

You can see in-depth how to deploy a Flow on kubernetes here

Then deploy the Flow:

kubectl apply -R -f config

Wait for a couple of minutes, and you should see that the Pods are ready:

kubectl get pods
../../_images/kubectl_pods.png

Then you can see that the new metrics endpoints are automatically discovered:

kubectl port-forward svc/prometheus-operated 9090:9090
../../_images/prometheus_target.png

Before querying the gateway you need to port-forward

kubectl port-forward svc/gateway 8080:8080

Now to access Grafana just do

kb port-forward svc/prometheus-grafana 3000:80

and open http://localhost:3000 in your browser

User is admin, password is prom-operator

You should see the Grafana home page.

Deploying locally

Let’s first deploy the Flow that we want to monitor:

from jina import Flow

with Flow(monitoring=True, port_monitoring=8000, port=8080).add(
    uses='jinahub://SimpleIndexer', port_monitoring=9000
) as f:
    f.block()
from jina import Flow

Flow(monitoring=True, port_monitoring=8000, port=8080).add(
    uses='jinahub+docker://SimpleIndexer', port_monitoring=9000
).to_docker_compose_yaml('config.yaml')
docker-compose -f config.yaml up

To monitor a flow locally you will need to install Prometheus and Grafana locally. The easiest way to do it is by using Docker Compose

First clone the repo which contains the config file:

git clone https://github.com/jina-ai/example-grafana-prometheus
cd example-grafana-prometheus/prometheus-grafana-local

then

docker-compose up

You can access the Grafana dashboard at http://localhost:3000. the username is admin and the password foobar.

Caution

This example is working locally because Prometheus is configured so that it listens to ports 8000 and 9000. However, in contrast with deploying on Kubernetes, you need to tell Prometheus which port to look at. You can change these ports by modifying this file

2- Using Grafana to visualize metrics

Once you can access the Grafana homepage then go to Browse then import and copy and paste the JSON file

You should see the following dashboard:

../../_images/grafana.png

Hint

You should query your Flow generate the first metrics. Othewise the dashboard will look empty.

You can query the flow by doing :

from jina import Client
from docarray import DocumentArray

client = Client(port=51000)
client.index(inputs=DocumentArray.empty(size=4))
client.search(inputs=DocumentArray.empty(size=4))

See further