jina.executors.decorators

Decorators and wrappers designed for wrapping BaseExecutor functions.

jina.executors.decorators.as_aggregate_method(func)

Mark a function so that it keeps track of the number of documents evaluated and a running sum to have always access to average value :type func: Callable :param func: the function to decorate :rtype: Callable :return: the wrapped function

jina.executors.decorators.as_update_method(func)

Mark the function as the updating function of this executor, calling this function will change the executor so later you can save the change via save() Will set the is_updated property after function is called. :type func: Callable :param func: the function to decorate :rtype: Callable :return: the wrapped function

jina.executors.decorators.wrap_func(cls, func_lst, wrapper)

Wrapping a class method only once, inherited but not overridden method will not be wrapped again

Parameters

• cls – class

• func_lst – function list to wrap

• wrapper – the wrapper

jina.executors.decorators.as_ndarray(func, dtype=numpy.float32)

Convert an BaseExecutor function returns to a numpy.ndarray, the following type are supported: EagerTensor, Tensor, list

Parameters

• func (Callable) – the function to decorate

• dtype – the converted dtype of the numpy.ndarray

Return type

Callable

Returns

the wrapped function

jina.executors.decorators.store_init_kwargs(func)

Mark the args and kwargs of __init__() later to be stored via save_config() in YAML :type func: Callable :param func: the function to decorate :rtype: Callable :return: the wrapped function

jina.executors.decorators.batching(func=None, batch_size=None, num_batch=None, split_over_axis=0, merge_over_axis=0, slice_on=1, slice_nargs=1, label_on=None, ordinal_idx_arg=None, flatten_output=True)

Split the input of a function into small batches and call func() on each batch , collect the merged result and return. This is useful when the input is too big to fit into memory

Parameters

• func (Optional[Callable[[Any], ndarray]]) – function to decorate

• batch_size (Union[int, Callable, None]) – size of each batch

• num_batch (Optional[int]) – number of batches to take, the rest will be ignored

• split_over_axis (int) – split over which axis into batches

• merge_over_axis (int) – merge over which axis into a single result

• slice_on (int) – the location of the data. When using inside a class, slice_on should take self into consideration.

• slice_nargs (int) – the number of arguments

• label_on (Optional[int]) – the location of the labels. Useful for data with any kind of accompanying labels

• ordinal_idx_arg (Optional[int]) – the location of the ordinal indexes argument. Needed for classes where function decorated needs to know the ordinal indexes of the data in the batch (Not used when label_on is used)

• flatten_output (bool) – If this is set to True, the results from different batches will be chained and the returning value is a list of the results. Otherwise, the returning value is a list of lists, in which each element is a list containing the result from one single batch. Note if there is only one batch returned, the returned result is always flatten.

Return type

Any

Returns

the merged result as if run func() once on the input.

Example:

class MemoryHungryExecutor:

    @batching
    def train(self, batch: 'numpy.ndarray', *args, **kwargs):
        gpu_train(batch)  #: this will respect the ``batch_size`` defined as object attribute

    @batching(batch_size = 64)
    def train(self, batch: 'numpy.ndarray', *args, **kwargs):
        gpu_train(batch)

jina.executors.decorators.single(func=None, merge_over_axis=0, slice_on=1, slice_nargs=1, flatten_output=False)

Guarantee that the inputs of a function with more than one argument is provided as single instances and not in batches

Parameters

• func (Optional[Callable[[Any], ndarray]]) – function to decorate

• merge_over_axis (int) – merge over which axis into a single result

• slice_on (int) – the location of the data. When using inside a class, slice_on should take self into consideration.

• slice_nargs (int) – the number of positional arguments considered as data

• flatten_output (bool) – If this is set to True, the results from different batches will be chained and the returning value is a list of the results. Otherwise, the returning value is a list of lists, in which each element is a list containing the result from one single batch. Note if there is only one batch returned, the returned result is always flatten.

Return type

Any

Returns

the merged result as if run func() once on the input.

..warning:

data arguments will be taken starting from slice_on` to ``slice_on + num_data

Example:

class OneByOneCrafter:

    @single
    def craft(self, text: str, id: str) -> Dict:
...