jina.executors.decorators¶

Decorators and wrappers designed for wrapping BaseExecutor functions.

jina.executors.decorators.as_aggregate_method(func)[source]¶

Mark a function so that it keeps track of the number of documents evaluated and a running sum to have always access to average value

Return type

Callable

jina.executors.decorators.as_update_method(func)[source]¶

Mark the function as the updating function of this executor, calling this function will change the executor so later you can save the change via save() Will set the is_updated property after function is called.

Return type

Callable

jina.executors.decorators.as_train_method(func)[source]¶

Mark a function as the training function of this executor. Will set the is_trained property after function is called.

Return type

Callable

jina.executors.decorators.as_success_fn(func)[source]¶
Return type

Callable

jina.executors.decorators.wrap_func(cls, func_lst, wrapper)[source]¶

Wrapping a class method only once, inherited but not overrided method will not be wrapped again

Parameters
  • cls – class

  • func_lst – function list to wrap

  • wrapper – the wrapper

Returns

jina.executors.decorators.as_ndarray(func, dtype=numpy.float32)[source]¶

Convert an BaseExecutor function returns to a numpy.ndarray, the following type are supported: EagerTensor, Tensor, list

Parameters
  • func (Callable) – the function to decorate

  • dtype – the converted dtype of the numpy.ndarray

Return type

Callable

jina.executors.decorators.require_train(func)[source]¶

Mark an BaseExecutor function as training required, so it can only be called after the function decorated by @as_train_method.

Return type

Callable

jina.executors.decorators.store_init_kwargs(func)[source]¶

Mark the args and kwargs of __init__() later to be stored via save_config() in YAML

Return type

Callable

jina.executors.decorators.batching(func=None, batch_size=None, num_batch=None, split_over_axis=0, merge_over_axis=0, slice_on=1, label_on=None, ordinal_idx_arg=None)[source]¶

Split the input of a function into small batches and call func() on each batch , collect the merged result and return. This is useful when the input is too big to fit into memory

Parameters
  • func (Optional[Callable[[Any], ndarray]]) – function to decorate

  • batch_size (Union[int, Callable, None]) – size of each batch

  • num_batch (Optional[int]) – number of batches to take, the rest will be ignored

  • split_over_axis (int) – split over which axis into batches

  • merge_over_axis (int) – merge over which axis into a single result

  • slice_on (int) – the location of the data. When using inside a class, slice_on should take self into consideration.

  • label_on (Optional[int]) – the location of the labels. Useful for data with any kind of accompanying labels

  • ordinal_idx_arg (Optional[int]) – the location of the ordinal indexes argument. Needed for classes where function decorated needs to know the ordinal indexes of the data in the batch (Not used when label_on is used)

Return type

Any

Returns

the merged result as if run func() once on the input.

Example:
class MemoryHungryExecutor:

    @batching
    def train(self, batch: 'numpy.ndarray', *args, **kwargs):
        gpu_train(batch)  #: this will respect the ``batch_size`` defined as object attribute

    @batching(batch_size = 64)
    def train(self, batch: 'numpy.ndarray', *args, **kwargs):
        gpu_train(batch)
jina.executors.decorators.batching_multi_input(func=None, batch_size=None, num_batch=None, split_over_axis=0, merge_over_axis=0, slice_on=1, num_data=1)[source]¶

Split the input of a function into small batches and call func() on each batch , collect the merged result and return. This is useful when the input is too big to fit into memory

Parameters
  • func (Optional[Callable[[Any], ndarray]]) – function to decorate

  • batch_size (Union[int, Callable, None]) – size of each batch

  • num_batch (Optional[int]) – number of batches to take, the rest will be ignored

  • split_over_axis (int) – split over which axis into batches

  • merge_over_axis (int) – merge over which axis into a single result

  • slice_on (int) – the location of the data. When using inside a class, slice_on should take self into consideration.

  • num_data (int) – the number of data inside the arguments

Return type

Any

Returns

the merged result as if run func() once on the input.

..warning:

data arguments will be taken starting from slice_on` to ``slice_on + num_data

Example:
class MultiModalExecutor:

    @batching_multi_input(batch_size = 64, num_data=2)
    def encode(self, *batches, **kwargs):
        batch_modality0 = batches[0]
        embed0 = _encode_modality(batch_modality0)
        batch_modality1 = batches[1]
        embed1 = _encode_modality(batch_modality0)