jina.executors.metas

The default meta config that all executors follow, they can be overridden by the YAML config

Warning

When you define your own Executor class, make sure your attributes/methods name do not conflict with the name listed below.

Note

Essentially, the meta config can be set in two places: as part of the YAML file, or as the class attribute via __init__() or in class definition. When multiple meta specification exists, the overwrite priority is:

metas defined in YAML > metas defined as class attribute > metas default values listed below

Any executor inherited from BaseExecutor always has the following meta fields:

is_trained

indicates if the executor is trained or not, if not then methods decorated by @required_train() can not be executed.

Type

bool

Default

False

is_updated

indicates if the executor is updated or changed since last save, if not then save() will do nothing. A forced save is possible to use touch() before save()

Type

bool

Default

False

batch_size

the size of each batch, methods decorated by @batching() will respect this. useful when incoming data is too large to fit into (GPU) memory.

Type

int

Default

None

workspace

the working directory, for persisting the artifacts of the executor. An artifact is a file or collection of files used during a workflow run.

By default it is not set, if you expect your executor to be persisted or to persist any data, remember to set it to the desired value.

When a BaseExecutor is a component of a CompoundExecutor, its workspace value will be overridden by the workspace coming from the CompoundExecutor unless a particular workspace value is set for the component BaseExecutor.

Type

str

Default

None

name

the name of the executor.

Type

str

Default

class name plus a random string

on_gpu

if the executor is set to run on GPU.

Type

bool

Default

False

py_modules

the external python module paths. it is useful when you want to load external python modules using BaseExecutor.load_config() from a YAML file. If a relative path is given then the root path is set to the path of the current YAML file.

Example of py_module usage:

  1. This is a valid structure and it is RECOMMENDED:
    • “my_cust_module” is a python module

    • all core logic of your customized executor goes to __init__.py

    • to import foo.py, you can use relative import, e.g. from .foo import bar

    • helper.py needs to be put BEFORE __init__.py in YAML py_modules

This is also the structure given by jina hub new CLI.

my_cust_module
  |- __init__.py
  |- helper.py
  |- config.yml
       |- py_modules
              |- helper.py
              |- __init__.py
  1. This is a valid structure but not recommended:
    • “my_cust_module” is not a python module (lack of __init__.py under the root)

    • to import foo.py, you must to use from jinahub.foo import bar

    • jinahub is a common namespace for all plugin-modules, not changeable.

    • helper.py needs to be put BEFORE my_cust.py in YAML py_modules

    my_cust_module
      |- my_cust.py
      |- helper.py
      |- config.yml
           |- py_modules
                  |- helper.py
                  |- my_cust.py
    
Type

str/List[str]

Default

None

pea_id

the integer index used for distinguish each parallel pea of this executor, required in shard_workspace

Type

int

Default

'${{root.metas.pea_id}}'

root_workspace

the workspace of the root executor. It will be the same as executor except in the case when an Executor inside a CompoundExecutor is used, or when a BaseNumpyIndexer is used with a ref_indexer.

By default, jina will try to find if a dump of the executor can be found in workspace, otherwise it will try to find it under root_workspace assuming it may be part of a CompoundExecutor.

Type

str

Default

'${{root.metas.workspace}}'

root_name

the name of the root executor. It will be the same as executor except in the case when an Executor inside a CompoundExecutor is used, or when a BaseNumpyIndexer is used with a ref_indexer

Type

str

Default

'${{root.metas.name}}'

read_only

do not allow the pod to modify the model, save calls will be ignored. If set to true no serialization of the executor

Type

bool

Default

False

Warning

name and workspace must be set if you want to serialize/deserialize this executor.

Note

pea_id is set in a way that when the executor A is used as a component of a jina.executors.compound.CompoundExecutor B, then A’s setting will be overridden by B’s counterpart.

These meta fields can be accessed via self.is_trained or loaded from a YAML config via load_config():

!MyAwesomeExecutor
with:
  ...
metas:
  name: my_transformer  # a customized name
  is_trained: true  # indicate the model has been trained
  workspace: ./  # path for serialize/deserialize
jina.executors.metas.get_default_metas()[source]

Get a copy of default meta variables.

Return type

Dict

Returns

default metas

jina.executors.metas.fill_metas_with_defaults(d)[source]

Fill the incomplete metas field with complete default values

Parameters

d (Dict) – the loaded YAML map

Return type

Dict

Returns

dictionary with injected metas