jina.executors¶
- jina.executors.classifiers
- jina.executors.crafters
- jina.executors.encoders
- jina.executors.evaluators
- jina.executors.indexers
- jina.executors.rankers
- jina.executors.segmenters
-
class
jina.executors.
BaseExecutor
(*args, **kwargs)[source]¶ Bases:
jina.jaml.JAMLCompatible
The base class of the executor, can be used to build encoder, indexer, etc.
Any executor inherited from
BaseExecutor
always has the meta defined injina.executors.metas.defaults
.All arguments in the
__init__()
can be specified with awith
map in the YAML config. Example:class MyAwesomeExecutor: def __init__(awesomeness = 5): pass
is equal to
!MyAwesomeExecutor with: awesomeness: 5
To use an executor in a
jina.peapods.runtimes.zmq.zed.ZEDRuntime
, a properjina.drivers.Driver
is required. This is because the executor is NOT protobuf-aware and has no access to the key-values in the protobuf message.Different executor may require different
Driver
with properjina.drivers.handlers
,jina.drivers.hooks
installed.See also
Methods of the
BaseExecutor
can be decorated viajina.executors.decorators
.See also
Meta fields
jina.executors.metas.defaults
.-
store_args_kwargs
= False¶ set this to
True
to saveargs
(in a list) andkwargs
(in a map) in YAML config
-
exec_methods
= ['encode', 'add', 'query', 'craft', 'segment', 'score', 'evaluate', 'predict', 'query_by_id', 'delete', 'update']¶
-
post_init
()[source]¶ Initialize class attributes/members that can/should not be (de)serialized in standard way.
Examples:
deep learning models
index files
numpy arrays
Warning
All class members created here will NOT be serialized when calling
save()
. Therefore if you want to store them, please override the__getstate__()
.
-
classmethod
pre_init
()[source]¶ This function is called before the object initiating (i.e.
__call__()
)Packages and environment variables can be set and load here.
-
property
save_abspath
¶ Get the file path of the binary serialized object
The file name ends with .bin.
- Return type
str
-
property
config_abspath
¶ Get the file path of the YAML config
The file name ends with .yml.
- Return type
str
-
property
current_workspace
¶ Get the path of the current workspace.
- Return type
str
- Returns
if
separated_workspace
is set toFalse
thenmetas.workspace
is returned, otherwise themetas.pea_workspace
is returned
-
get_file_from_workspace
(name)[source]¶ Get a usable file path under the current workspace
- Parameters
name (
str
) – the name of the file
:return depending on
metas.separated_workspace
the file could be located inmetas.workspace
ormetas.pea_workspace
- Return type
str
-
property
physical_size
¶ Return the size of the current workspace in bytes
- Return type
int
-
touch
()[source]¶ Touch the executor and change
is_updated
toTrue
so that one can callsave()
.- Return type
None
-
save
(filename=None)[source]¶ Persist data of this executor to the
workspace
(orpea_workspace
). The data could be a file or collection of files produced/used during an executor run.These are some of the common data that you might want to persist:
binary dump/pickle of the executor
the indexed files
(pre)trained models
Warning
All class members created here will NOT be serialized when calling
save()
. Therefore if you want to store them, please implement the__getstate__()
.It uses
pickle
for dumping. For members/attributes that are not valid or not efficient forpickle
, you need to implement their own persistence strategy in the__getstate__()
.- Parameters
filename (
Optional
[str
]) – file path of the serialized file, if not given thensave_abspath
is used- Returns
successfully persisted or not
-
classmethod
inject_config
(raw_config, separated_workspace=False, pea_id=0, read_only=False, *args, **kwargs)[source]¶ Inject config into the raw_config before loading into an object.
- Parameters
raw_config (
Dict
) – raw config to work onseparated_workspace (
bool
) – the dump and data files associated to this executor will be stored separately for each parallel pea, which will be indexed by thepea_id
pea_id (
int
) – the id of the storage of this parallel pea, only effective whenseparated_workspace=True
read_only (
bool
) – if the executor should be readonly
- Return type
Dict
- Returns
an executor object
-
static
load
(filename=None)[source]¶ Build an executor from a binary file
- Parameters
filename (
Optional
[str
]) – the file path of the binary serialized file- Return type
~AnyExecutor
- Returns
an executor object
It uses
pickle
for loading.
-