jina.jaml

class jina.jaml.JAML[source]

Bases: object

A Jina YAML parser supports loading and dumping and substituting variables.

To use it:

from jina.jaml import JAML

JAML.load(...)
JAML.dump(...)

class DummyClass:
    pass

JAML.register(DummyClass)

You can use expressions to programmatically set variables in YAML files and access contexts. An expression can be any combination of literal values, references to a context, or functions. You can combine literals, context references, and functions using operators.

You need to use specific syntax to tell Jina to evaluate an expression rather than treat it as a string.

${{ <expression> }}

To evaluate (i.e. substitute the value to the real value) the expression when loading, use load(substitute=True)().

To substitute the value based on a dict,

obj = JAML.load(fp, substitute=True,
                    context={'context_var': 3.14,
                            'context_var2': 'hello-world'})

Note

BaseFlow, BaseExecutor, BaseDriver and all their subclasses have already implemented JAML interfaces, to load YAML config into objects, please use Flow.load_config(), BaseExecutor.load_config(), etc.

static load(stream, substitute=False, context=None)[source]

Parse the first YAML document in a stream and produce the corresponding Python object.

Parameters
  • substitute (bool) – substitute environment, internal reference and context variables.

  • context (Optional[Dict[str, Any]]) – context replacement variables in a dict, the value of the dict is the replacement.

Note

BaseFlow, BaseExecutor, BaseDriver and all their subclasses have already implemented JAML interfaces, to load YAML config into objects, please use Flow.load_config(), BaseExecutor.load_config(), etc.

static load_no_tags(stream, **kwargs)[source]

Load yaml object but ignore all customized tags, e.g. !Executor, !Driver, !Flow

static expand_dict(d, context=None, resolve_cycle_ref=True, resolve_passes=3)[source]
Return type

Dict[str, Any]

static dump(data, stream=None, **kwargs)[source]

Serialize a Python object into a YAML stream. If stream is None, return the produced string instead.

static register(cls)[source]

register a class for dumping loading - if it has attribute yaml_tag use that to register, else use class name - if it has methods to_yaml/from_yaml use those to dump/load else dump attributes

as mapping

class jina.jaml.JAMLCompatible[source]

Bases: object

JAMLCompatible is a mixin class designed to be used with multiple inheritance. It will add to_yaml() and from_yaml() to the target class, making that class JAML-friendly.

Warning

For the sake of cooperative multiple inheritance, do NOT implement __init__() for this class

save_config(filename=None)[source]

Save the object’s config into a YAML file

Parameters

filename (Optional[str]) – file path of the yaml file, if not given then config_abspath is used

Returns

successfully dumped or not

classmethod load_config(source, *, allow_py_modules=True, substitute=True, context=None, **kwargs)[source]

A high-level interface for loading configuration with features of loading extra py_modules, substitute env & context variables. Any class that implements JAMLCompatible mixin can enjoy this feature, e.g. BaseFlow, BaseExecutor, BaseDriver and all their subclasses.

Parameters
  • source (Union[str, TextIO, Dict]) – the multi-kind source of the configs.

  • allow_py_modules (bool) – allow importing plugins specified by py_modules in YAML at any levels

  • substitute (bool) – substitute environment, internal reference and context variables.

  • context (Optional[Dict[str, Any]]) – context replacement variables in a dict, the value of the dict is the replacement.

Return type

JAMLCompatible

Returns

JAMLCompatible object

Support substitutions in YAML:
  • Environment variables: ${{ENV.VAR}} (recommended), ${{VAR}}, $VAR.

  • Context dict (context): ${{VAR}}``(recommended), ``$VAR.

  • Internal reference via this and root: ${{this.same_level_key}}, ${{root.root_level_key}}

Substitutions are carried in the order and multiple passes to resolve variables with best effort.

!BaseEncoder
metas:
    name: ${{VAR_A}}  # env or context variables
    workspace: my-${{this.name}}  # internal reference
# load Executor from yaml file
BaseExecutor.load_config('a.yml')

# load Executor from yaml file and substitute environment variables
os.environ['VAR_A'] = 'hello-world'
b = BaseExecutor.load_config('a.yml')
assert b.name == hello-world

# load Executor from yaml file and substitute variables from a dict
b = BaseExecutor.load_config('a.yml', context={'VAR_A': 'hello-world'})
assert b.name == hello-world

# disable substitute
b = BaseExecutor.load_config('a.yml', substitute=False)
classmethod inject_config(raw_config, *args, **kwargs)[source]

Inject/modify the config before loading it into an object.

Parameters

raw_config (Dict) –

raw config to work on

Note

This function is most likely to be overridden by its subclass.

Return type

Dict