jina.executors.decorators¶
Decorators and wrappers designed for wrapping BaseExecutor
functions.
-
jina.executors.decorators.
as_aggregate_method
(func)[source]¶ Mark a function so that it keeps track of the number of documents evaluated and a running sum to have always access to average value :type func:
Callable
:param func: the function to decorate :rtype:Callable
:return: the wrapped function
-
jina.executors.decorators.
as_update_method
(func)[source]¶ Mark the function as the updating function of this executor, calling this function will change the executor so later you can save the change via
save()
Will set the is_updated property after function is called. :type func:Callable
:param func: the function to decorate :rtype:Callable
:return: the wrapped function
-
jina.executors.decorators.
wrap_func
(cls, func_lst, wrapper)[source]¶ Wrapping a class method only once, inherited but not overridden method will not be wrapped again
- Parameters
cls – class
func_lst – function list to wrap
wrapper – the wrapper
-
jina.executors.decorators.
as_ndarray
(func, dtype=numpy.float32)[source]¶ Convert an
BaseExecutor
function returns to anumpy.ndarray
, the following type are supported: EagerTensor, Tensor, list- Parameters
func (
Callable
) – the function to decoratedtype – the converted dtype of the
numpy.ndarray
- Return type
Callable
- Returns
the wrapped function
-
jina.executors.decorators.
store_init_kwargs
(func)[source]¶ Mark the args and kwargs of
__init__()
later to be stored viasave_config()
in YAML :type func:Callable
:param func: the function to decorate :rtype:Callable
:return: the wrapped function
-
jina.executors.decorators.
batching
(func=None, batch_size=None, num_batch=None, split_over_axis=0, merge_over_axis=0, slice_on=1, slice_nargs=1, label_on=None, ordinal_idx_arg=None, flatten_output=True)[source]¶ Split the input of a function into small batches and call
func()
on each batch , collect the merged result and return. This is useful when the input is too big to fit into memory- Parameters
func (
Optional
[Callable
[[Any
],ndarray
]]) – function to decoratebatch_size (
Union
[int
,Callable
,None
]) – size of each batchnum_batch (
Optional
[int
]) – number of batches to take, the rest will be ignoredsplit_over_axis (
int
) – split over which axis into batchesmerge_over_axis (
int
) – merge over which axis into a single resultslice_on (
int
) – the location of the data. When using inside a class,slice_on
should takeself
into consideration.slice_nargs (
int
) – the number of argumentslabel_on (
Optional
[int
]) – the location of the labels. Useful for data with any kind of accompanying labelsordinal_idx_arg (
Optional
[int
]) – the location of the ordinal indexes argument. Needed for classes where function decorated needs to know the ordinal indexes of the data in the batch (Not used when label_on is used)flatten_output (
bool
) – If this is set to True, the results from different batches will be chained and the returning value is a list of the results. Otherwise, the returning value is a list of lists, in which each element is a list containing the result from one single batch. Note if there is only one batch returned, the returned result is always flatten.
- Return type
Any
- Returns
the merged result as if run
func()
once on the input.
- Example:
class MemoryHungryExecutor: @batching def train(self, batch: 'numpy.ndarray', *args, **kwargs): gpu_train(batch) #: this will respect the ``batch_size`` defined as object attribute @batching(batch_size = 64) def train(self, batch: 'numpy.ndarray', *args, **kwargs): gpu_train(batch)
-
jina.executors.decorators.
single
(func=None, merge_over_axis=0, slice_on=1, slice_nargs=1, flatten_output=False)[source]¶ Guarantee that the inputs of a function with more than one argument is provided as single instances and not in batches
- Parameters
func (
Optional
[Callable
[[Any
],ndarray
]]) – function to decoratemerge_over_axis (
int
) – merge over which axis into a single resultslice_on (
int
) – the location of the data. When using inside a class,slice_on
should takeself
into consideration.slice_nargs (
int
) – the number of positional arguments considered as dataflatten_output (
bool
) – If this is set to True, the results from different batches will be chained and the returning value is a list of the results. Otherwise, the returning value is a list of lists, in which each element is a list containing the result from one single batch. Note if there is only one batch returned, the returned result is always flatten.
- Return type
Any
- Returns
the merged result as if run
func()
once on the input.
- ..warning:
data arguments will be taken starting from
slice_on` to ``slice_on + num_data
- Example:
class OneByOneCrafter: @single def craft(self, text: str, id: str) -> Dict: ...