Prometheus/Grafana Support (Legacy)#

Deprecated

The Prometheus-only based feature will soon be deprecated in favor of the OpenTelemetry Setup. Refer to OpenTelemetry Setup for the details on OpenTelemetry setup for Jina.

Refer to the OpenTelemetry migration guide for updating your existing Prometheus and Grafana configurations.

We recommend the Prometheus/Grafana stack to leverage the metrics exposed by Jina. In this setup, Jina exposes different metrics, and Prometheus scrapes these endpoints, as well as collecting, aggregating, and storing the metrics.

External entities (like Grafana) can access these aggregated metrics via the query language PromQL and let users visualize the metrics with dashboards.

Hint

Jina supports exposing metrics, but you are in charge of installing and managing your Prometheus/Grafana instances.

In this guide, we deploy the Prometheus/Grafana stack and use it to monitor a Flow.

Deploying the Flow and the monitoring stack#

Deploying on Kubernetes#

One challenge of monitoring a Flow is communicating its different metrics endpoints to Prometheus. Fortunately, the Prometheus operator for Kubernetes makes this fairly easy because it can automatically discover new metrics endpoints to scrape.

We recommend deploying your Jina Flow on Kubernetes to leverage the full potential of the monitoring feature because:

  • The Prometheus operator can automatically discover new endpoints to scrape.

  • You can extend monitoring with the rich built-in Kubernetes metrics.

You can deploy Prometheus and Grafana on your Kubernetes cluster by running:

helm install prometheus prometheus-community/kube-prometheus-stack --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false

Hint

setting the serviceMonitorSelectorNilUsesHelmValues to false allows the Prometheus Operator to discover metrics endpoint outside of the helm scope which is needed to discover the Flow metrics endpoints.

Deploy the Flow that we want to monitor:

For this example we recommend reading how to build and containerize the Executors to be run in Kubernetes.

This example shows how to start a Flow with monitoring enabled via YAML:

In a flow.yaml file

jtype: Flow
with:
  monitoring: true
executors:
- uses: jinaai+docker://<user-id>/EncoderPrivate

jina export kubernetes flow.yml ./config ```

from jina import Flow

f = Flow(monitoring=True).add(uses='jinaai+docker://<user-id>/EncoderPrivate')
f.to_kubernetes_yaml('config')

This creates a config folder containing the Kubernetes YAML definition of the Flow.

See also

You can see in-depth how to deploy a Flow on Kubernetes here

Then deploy the Flow:

kubectl apply -R -f config

Wait for a couple of minutes, and you should see that the Pods are ready:

kubectl get pods
../../_images/kubectl_pods.png

Then you can see that the new metrics endpoints are automatically discovered:

kubectl port-forward svc/prometheus-operated 9090:9090
../../_images/prometheus_target.png

Before querying the gateway you need to port-forward

kubectl port-forward svc/gateway 8080:8080

To access Grafana, run:

kb port-forward svc/prometheus-grafana 3000:80

Then open http://localhost:3000 in your browser. The username is admin and password is prom-operator.

You should see the Grafana home page.

Deploying locally#

Deploy the Flow that we want to monitor:

from jina import Flow

with Flow(monitoring=True, port_monitoring=8000, port=8080).add(
    uses='jinaai+docker://<user-id>/EncoderPrivate', port_monitoring=9000
) as f:
    f.block()

from jina import Flow

Flow(monitoring=True, port_monitoring=8000, port=8080).add(
    uses='jinaai+docker://<user-id>/EncoderPrivate', port_monitoring=9000
).to_docker_compose_yaml('config.yaml')

docker-compose -f config.yaml up

To monitor a Flow locally you need to install Prometheus and Grafana locally. The easiest way to do this is with Docker Compose.

First clone the repo which contains the config file:

git clone https://github.com/jina-ai/example-grafana-prometheus
cd example-grafana-prometheus/prometheus-grafana-local

then

docker-compose up

Access the Grafana dashboard at http://localhost:3000. The username is admin and the password is foobar.

Caution

This example works locally because Prometheus is configured to listen to ports 8000 and 9000. However, in contrast to deploying on Kubernetes, you need to tell Prometheus which port to look at. You can change these ports by modifying prometheus.yml.

Deploying on Jcloud#

If your Flow is deployed on JCloud, you don’t need to provision a monitoring stack yourself. Prometheus and Grafana are handled by JCloud and you can find a dashboard URL with jc status <flow_id>

Using Grafana to visualize metrics#

Access the Grafana homepage, then go to Browse then import and copy and paste the JSON file

You should see the following dashboard:

../../_images/grafana.png

Hint

You should query your Flow to generate the first metrics. Otherwise the dashboard looks empty.

You can query the Flow by running:

from typing import Optional
from docarray import DocList, BaseDoc
from docarray.typing import NdArray
from jina import Client


class MyDoc(BaseDoc):
    text: str
    embedding: Optional[NdArray] = None
client = Client(port=51000)

client.post(on='/', inputs=DocList[MyDoc]([MyDoc(text=f'Text for document {i}') for in range(100)]), return_type=DocList[MyDoc], request_size=10,)

See also#