Understanding the Request Size parameter

In some use cases, you may wish to vary the number of Documents a single Request will receive. You can achieve this by adjusting the request_size parameter when setting the Flow. This guide covers the different configuration options Jina offers.

Request Size

Jina defines the request_size as the parameter on the client side. By adjusting the request_size in the Flow’s API you can define the number of Documents contained in every Request.

Batch Size

Batch size, commonly used in machine learning, often refers to the number of datapoints that we feed into the model in one iteration.

In Jina batch_size is used by Driver and Executor to guarantee that the Executor processes data in pieces of a specific size.

Before you start

Make sure you install latest version of Jina on your local machine.

pip install -U jina

Implementation

In order to have a better understanding of the influence of request_size and how it is used, let’s take the following code snippets as examples.

We first import the necessary modules.

import time

import numpy as np
from jina import Document
from jina.executors.crafters import BaseCrafter
from jina.flow import Flow

Then we define a SimpleCrafter which will just forward the data.

class SimpleCrafter(BaseCrafter):
    def craft(self, id, *args, **kwargs):
        return {'id': id}

For this example, we will index 100 Documents and use 10 parallel Crafters. The request_size is set to 20. So the 100 Documents will be divided into 5 parts and each Request contains 20 Documents.

def main():

    request_size=20
    start_time = time.time()
    f = Flow(runtime='process').add(
        name='simple_crafter',
        uses='SimpleCrafter',
        parallel=10)
    with f:
        f.index_ndarray(np.random.random([100, 10]), request_size=request_size)
    end_time = time.time()
    seconds_elapsed = end_time - start_time
    print(seconds_elapsed)


if __name__ == '__main__':
    main()

Choosing different request size

Different settings of request_size may influence the running performance. A higher value means a large number Documents will be fed into the Pea and will demand more memory. A lower value will decrease the cost of memory but may increase the running time since we need to send more requests.

A simple extension of the above example generates a box plot showing the relationship between request_size and running time when we have 100 Documents to be indexed. This may help you to get more insights on choosing the request_size.

request_size vs running time