jina.executors

• jina.executors.classifiers
• jina.executors.crafters
• jina.executors.encoders
  • jina.executors.encoders.multimodal
  • jina.executors.encoders.numeric
  • jina.executors.encoders.frameworks
• jina.executors.evaluators
  • jina.executors.evaluators.embedding
    • jina.executors.evaluators.embedding.cosine
    • jina.executors.evaluators.embedding.euclidean
  • jina.executors.evaluators.rank
    • jina.executors.evaluators.rank.precision
    • jina.executors.evaluators.rank.recall
  • jina.executors.evaluators.text
    • jina.executors.evaluators.text.length
  • jina.executors.evaluators.running_stats
• jina.executors.indexers
  • jina.executors.indexers.cache
  • jina.executors.indexers.keyvalue
  • jina.executors.indexers.vector
• jina.executors.rankers
• jina.executors.segmenters

• jina.executors.compound
• jina.executors.decorators
• jina.executors.devices
• jina.executors.metas
• jina.executors.requests

class jina.executors.BaseExecutor(*args, **kwargs)

Bases: jina.jaml.JAMLCompatible

The base class of the executor, can be used to build encoder, indexer, etc.

Any executor inherited from BaseExecutor always has the meta defined in jina.executors.metas.defaults.

All arguments in the __init__() can be specified with a with map in the YAML config. Example:

class MyAwesomeExecutor:
    def __init__(awesomeness = 5):
        pass

is equal to

!MyAwesomeExecutor
with:
    awesomeness: 5

To use an executor in a jina.peapods.runtimes.zmq.zed.ZEDRuntime, a proper jina.drivers.Driver is required. This is because the executor is NOT protobuf-aware and has no access to the key-values in the protobuf message.

Different executor may require different Driver with proper jina.drivers.handlers, jina.drivers.hooks installed.

See also

Methods of the BaseExecutor can be decorated via jina.executors.decorators.

See also

Meta fields jina.executors.metas.defaults.

Constructor.

store_args_kwargs = False

set this to True to save args (in a list) and kwargs (in a map) in YAML config

exec_methods = ['encode', 'add', 'query', 'craft', 'segment', 'score', 'evaluate', 'predict', 'query_by_key', 'delete', 'update']

post_init()

Initialize class attributes/members that can/should not be (de)serialized in standard way.

Examples:

• deep learning models

• index files

• numpy arrays

Warning

All class members created here will NOT be serialized when calling save(). Therefore if you want to store them, please override the __getstate__().

classmethod pre_init()

This function is called before the object initiating (i.e. __call__())

Packages and environment variables can be set and load here.

property save_abspath

Get the file path of the binary serialized object

The file name ends with .bin.

Return type

str

property config_abspath

Get the file path of the YAML config

The file name ends with .yml.

Return type

str

static get_shard_workspace(workspace_folder, workspace_name, pea_id)

Get the path of the current shard.

Parameters

• workspace_folder (str) – Folder of the workspace.

• workspace_name (str) – Name of the workspace.

• pea_id (int) – Id of the pea,

Return type

str

Returns

Return the workspace of the shard of this Executor.

property workspace_name

Get the name of the workspace.

property shard_workspace

Get the path of the current shard.

Return type

str

Returns

returns the workspace of the shard of this Executor

get_file_from_workspace(name)

Get a usable file path under the current workspace

Parameters

name (str) – the name of the file

:return file path

Return type

str

property physical_size

Return the size of the current workspace in bytes

Return type

int

train(*args, **kwargs)

Train this executor, need to be overrided

Return type

None

touch()

Touch the executor and change is_updated to True so that one can call save().

Return type

None

save(filename=None)

Persist data of this executor to the shard_workspace. The data could be a file or collection of files produced/used during an executor run.

These are some of the common data that you might want to persist:

• binary dump/pickle of the executor

• the indexed files

• (pre)trained models

Warning

All class members created here will NOT be serialized when calling save(). Therefore if you want to store them, please implement the __getstate__().

It uses pickle for dumping. For members/attributes that are not valid or not efficient for pickle, you need to implement their own persistence strategy in the __getstate__().

Parameters

filename (Optional[str]) – file path of the serialized file, if not given then save_abspath is used

Returns

successfully persisted or not

classmethod inject_config(raw_config, pea_id=0, read_only=False, *args, **kwargs)

Inject config into the raw_config before loading into an object.

Parameters

• raw_config (Dict) – raw config to work on

• pea_id (int) – the id of the storage of this parallel pea

• read_only (bool) – if the executor should be readonly

Return type

Dict

Returns

an executor object

static load(filename=None)

Build an executor from a binary file

Parameters

filename (Optional[str]) – the file path of the binary serialized file

Return type

~AnyExecutor

Returns

an executor object

It uses pickle for loading.

close()

Release the resources as executor is destroyed, need to be overrided

Return type

None

attach(runtime, *args, **kwargs)

Attach this executor to a jina.peapods.runtime.BasePea.

This is called inside the initializing of a jina.peapods.runtime.BasePea.

class jina.executors.ExecutorType(*args, **kwargs)

Bases: jina.jaml.JAMLCompatibleType, type

The class of Executor type, which is the metaclass of BaseExecutor.

static register_class(cls)

Register a class and wrap update, train, aggregate functions.

Parameters

cls – The class.

Returns

The class, after being registered.