jina.executors

• jina.executors.classifiers
• jina.executors.crafters
• jina.executors.encoders
  • jina.executors.encoders.multimodal
  • jina.executors.encoders.numeric
  • jina.executors.encoders.frameworks
  • jina.executors.encoders.tfserving
• jina.executors.evaluators
  • jina.executors.evaluators.embedding
    • jina.executors.evaluators.embedding.cosine
    • jina.executors.evaluators.embedding.euclidean
  • jina.executors.evaluators.rank
    • jina.executors.evaluators.rank.precision
    • jina.executors.evaluators.rank.recall
  • jina.executors.evaluators.text
    • jina.executors.evaluators.text.length
  • jina.executors.evaluators.running_stats
• jina.executors.indexers
  • jina.executors.indexers.cache
  • jina.executors.indexers.keyvalue
  • jina.executors.indexers.vector
• jina.executors.rankers
• jina.executors.segmenters

• jina.executors.clients
• jina.executors.compound
• jina.executors.decorators
• jina.executors.devices
• jina.executors.metas
• jina.executors.requests

class jina.executors.ExecutorType

Bases: jina.jaml.JAMLCompatibleType, type

static register_class(cls)

class jina.executors.BaseExecutor(*args, **kwargs)

Bases: jina.jaml.JAMLCompatible

The base class of the executor, can be used to build encoder, indexer, etc.

Any executor inherited from BaseExecutor always has the meta defined in jina.executors.metas.defaults.

All arguments in the __init__() can be specified with a with map in the YAML config. Example:

class MyAwesomeExecutor:
    def __init__(awesomeness = 5):
        pass

is equal to

!MyAwesomeExecutor
with:
    awesomeness: 5

To use an executor in a jina.peapods.runtimes.zmq.zed.ZEDRuntime, a proper jina.drivers.Driver is required. This is because the executor is NOT protobuf-aware and has no access to the key-values in the protobuf message.

Different executor may require different Driver with proper jina.drivers.handlers, jina.drivers.hooks installed.

See also

Methods of the BaseExecutor can be decorated via jina.executors.decorators.

See also

Meta fields jina.executors.metas.defaults.

store_args_kwargs = False

set this to True to save args (in a list) and kwargs (in a map) in YAML config

exec_methods = ['encode', 'add', 'query', 'craft', 'segment', 'score', 'evaluate', 'predict', 'query_by_id', 'delete', 'update']

post_init()

Initialize class attributes/members that can/should not be (de)serialized in standard way.

Examples:

• deep learning models

• index files

• numpy arrays

Warning

All class members created here will NOT be serialized when calling save(). Therefore if you want to store them, please override the __getstate__().

classmethod pre_init()

This function is called before the object initiating (i.e. __call__())

Packages and environment variables can be set and load here.

property save_abspath

Get the file path of the binary serialized object

The file name ends with .bin.

Return type

str

property config_abspath

Get the file path of the YAML config

The file name ends with .yml.

Return type

str

property current_workspace

Get the path of the current workspace.

Return type

str

Returns

if separated_workspace is set to False then metas.workspace is returned, otherwise the metas.pea_workspace is returned

get_file_from_workspace(name)

Get a usable file path under the current workspace

Parameters

name (str) – the name of the file

:return depending on metas.separated_workspace the file could be located in metas.workspace or metas.pea_workspace

Return type

str

property physical_size

Return the size of the current workspace in bytes

Return type

int

train(*args, **kwargs)

Train this executor, need to be overrided

Return type

None

touch()

Touch the executor and change is_updated to True so that one can call save().

Return type

None

save(filename=None)

Persist data of this executor to the workspace (or pea_workspace). The data could be a file or collection of files produced/used during an executor run.

These are some of the common data that you might want to persist:

• binary dump/pickle of the executor

• the indexed files

• (pre)trained models

Warning

All class members created here will NOT be serialized when calling save(). Therefore if you want to store them, please implement the __getstate__().

It uses pickle for dumping. For members/attributes that are not valid or not efficient for pickle, you need to implement their own persistence strategy in the __getstate__().

Parameters

filename (Optional[str]) – file path of the serialized file, if not given then save_abspath is used

Returns

successfully persisted or not

classmethod inject_config(raw_config, separated_workspace=False, pea_id=0, read_only=False, *args, **kwargs)

Inject config into the raw_config before loading into an object.

Parameters

• raw_config (Dict) – raw config to work on

• separated_workspace (bool) – the dump and data files associated to this executor will be stored separately for each parallel pea, which will be indexed by the pea_id

• pea_id (int) – the id of the storage of this parallel pea, only effective when separated_workspace=True

• read_only (bool) – if the executor should be readonly

Return type

Dict

Returns

an executor object

static load(filename=None)

Build an executor from a binary file

Parameters

filename (Optional[str]) – the file path of the binary serialized file

Return type

~AnyExecutor

Returns

an executor object

It uses pickle for loading.

close()

Release the resources as executor is destroyed, need to be overrided

Return type

None

attach(runtime, *args, **kwargs)

Attach this executor to a jina.peapods.runtime.BasePea.

This is called inside the initializing of a jina.peapods.runtime.BasePea.