Monitor Executor with Custom Metrics¶
Jina allows you to monitor every part of a Flow, including Executor, with the Grafana/Prometheus. This section documents the ability to add custom monitoring to the Executor.
Custom metrics are useful when you want to monitor each subpart of your Executors. Jina allows you to leverage
the full power of the Prometheus Client to define useful metrics
for each of your Executors. We provide a convenient wrapper as well, i.e
@monitor(), which let you easily monitor
sub-method of your Executor.
Full detail on monitoring
This section describes how to define and use custom metrics. To use the default metrics exposed by the Executor please refer to this section.
When the monitoring is enabled each Executor will expose its own metrics. This means that in practice each of the Executors will expose a Prometheus endpoint using the Prometheus Client.
By default, every method which is decorated by the
@request decorator will be monitored, it will create a
Prometheus Summary which will keep track of the time of
the execution of the method.
Define custom metrics¶
Sometimes monitoring the
encoding method is not enough, you need to break it up into multiple parts that you want to
monitor one by one.
It could be useful if your encoding phase is composed of two tasks: image processing and image embedding. By using custom metrics on these two tasks you can identify potential bottlenecks.
Overall the ability to add custom metrics allows you to have the full flexibility on the monitoring of your Executor.
Defining custom metrics with
Adding the custom monitoring on a method is as straightforward as decorating the method with
from jina import Executor, monitor class MyExecutor(Executor): @monitor() def my_method(self): ...
This will create a Prometheus summary
jina_my_method_inference_seconds which will keep track of the time of execution of `my_method
By default, the name and the documentation of the metric created by
@monitor are auto-generated based on the name
of the function. However, you can name it by yourself by doing :
@monitor( name='my_custom_metrics_seconds', documentation='This is my custom documentation' ) def method(self): ...
Let’s take an example to illustrate custom metrics:
from jina import requests, Executor from docarray import DocumentArray class MyExecutor(Executor): def preprocessing(self, docs: DocumentArray): ... def model_inference(self, tensor): ... @requests def encode(self, docs: DocumentArray, **kwargs): docs.tensors = self.preprocessing(docs) docs.embedding = self.model_inference(docs.tensors)
The encode function is composed of two sub-functions.
preprocessingwhich takes raw bytes from a DocumentArray and put them into a PyTorch tensor.
model inferencecalls the forward function of a deep learning model.
By default, only the
encode function will be monitored.
from jina import requests,monitor,Executor from docarray import DocumentArray class MyExecutor(Executor): @monitor() def preprocessing(self, docs: DocumentArray): ... @monitor() def model_inference(self, tensor): ... @requests def encode(self, docs: DocumentArray, **kwargs): docs.tensors = self.preprocessing(docs) docs.embedding = self.model_inference(docs.tensors)
Defining custom metrics directly with the Prometheus client¶
Under the hood, the monitoring feature of the Executor is handled by the
Python Prometheus-client. The
@monitor decorator is a convenient tool
to monitor sub-methods of an Executor, but you might need more flexibility and that is why you can access the Prometheus
client directly from the Executor to define every kind of metric supported by Prometheus.
Let’s see it in an example
from jina import requests, Executor from docarray import DocumentArray from prometheus_client import Counter class MyExecutor(Executor): def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.counter = Counter( name='my_count_total', documentation='my count', registry=self.runtime_args.metrics_registry, ) @requests def encode(self, docs: DocumentArray, **kwargs): self.counter.inc(len(docs))
This will create a Prometheus Counter.
Directly using the Prometheus client
You need to pass the metrics registry from the Executor when creating custom metrics directly with the Prometheus client.