jina.jaml.helper

class jina.jaml.helper.JinaConstructor[source]

Bases: yaml.constructor.FullConstructor

Convert List into tuple when doing hashing.

get_hashable_key(key)[source]

Get the hash value of key.

Parameters

key – key value to be hashed.

Returns

Hash value of key.

construct_mapping(node, deep=True)[source]

Build the mapping from node.

Parameters
  • node – the node to traverse

  • deep – required param from YAML constructor

Returns

Mapped data

class jina.jaml.helper.JinaResolver[source]

Bases: yaml.resolver.Resolver

Remove on|On|ON as bool resolver.

class jina.jaml.helper.JinaLoader(stream)[source]

Bases: yaml.reader.Reader, yaml.scanner.Scanner, yaml.parser.Parser, yaml.composer.Composer, jina.jaml.helper.JinaConstructor, jina.jaml.helper.JinaResolver

The Jina loader which should be able to load YAML safely.

Parameters

stream – the stream to load.

Initialize the scanner.

yaml_constructors = {'tag:yaml.org,2002:null': <function SafeConstructor.construct_yaml_null>, 'tag:yaml.org,2002:bool': <function SafeConstructor.construct_yaml_bool>, 'tag:yaml.org,2002:int': <function SafeConstructor.construct_yaml_int>, 'tag:yaml.org,2002:float': <function SafeConstructor.construct_yaml_float>, 'tag:yaml.org,2002:binary': <function SafeConstructor.construct_yaml_binary>, 'tag:yaml.org,2002:timestamp': <function SafeConstructor.construct_yaml_timestamp>, 'tag:yaml.org,2002:omap': <function SafeConstructor.construct_yaml_omap>, 'tag:yaml.org,2002:pairs': <function SafeConstructor.construct_yaml_pairs>, 'tag:yaml.org,2002:set': <function SafeConstructor.construct_yaml_set>, 'tag:yaml.org,2002:str': <function SafeConstructor.construct_yaml_str>, 'tag:yaml.org,2002:seq': <function SafeConstructor.construct_yaml_seq>, 'tag:yaml.org,2002:map': <function SafeConstructor.construct_yaml_map>, None: <function SafeConstructor.construct_undefined>, 'tag:yaml.org,2002:python/none': <function SafeConstructor.construct_yaml_null>, 'tag:yaml.org,2002:python/bool': <function SafeConstructor.construct_yaml_bool>, 'tag:yaml.org,2002:python/str': <function FullConstructor.construct_python_str>, 'tag:yaml.org,2002:python/unicode': <function FullConstructor.construct_python_unicode>, 'tag:yaml.org,2002:python/bytes': <function FullConstructor.construct_python_bytes>, 'tag:yaml.org,2002:python/int': <function SafeConstructor.construct_yaml_int>, 'tag:yaml.org,2002:python/long': <function FullConstructor.construct_python_long>, 'tag:yaml.org,2002:python/float': <function SafeConstructor.construct_yaml_float>, 'tag:yaml.org,2002:python/complex': <function FullConstructor.construct_python_complex>, 'tag:yaml.org,2002:python/list': <function SafeConstructor.construct_yaml_seq>, 'tag:yaml.org,2002:python/tuple': <function FullConstructor.construct_python_tuple>, 'tag:yaml.org,2002:python/dict': <function SafeConstructor.construct_yaml_map>, '!JAMLCompatible': <bound method JAMLCompatible._from_yaml of <class 'jina.jaml.JAMLCompatible'>>, '!BetterEnum': <bound method BetterEnum._from_yaml of <enum 'BetterEnum'>>, '!SchedulerType': <bound method BetterEnum._from_yaml of <enum 'SchedulerType'>>, '!PollingType': <bound method BetterEnum._from_yaml of <enum 'PollingType'>>, '!LogVerbosity': <bound method BetterEnum._from_yaml of <enum 'LogVerbosity'>>, '!SocketType': <bound method BetterEnum._from_yaml of <enum 'SocketType'>>, '!FlowOutputType': <bound method BetterEnum._from_yaml of <enum 'FlowOutputType'>>, '!FlowBuildLevel': <bound method BetterEnum._from_yaml of <enum 'FlowBuildLevel'>>, '!PeaRoleType': <bound method BetterEnum._from_yaml of <enum 'PeaRoleType'>>, '!PodRoleType': <bound method BetterEnum._from_yaml of <enum 'PodRoleType'>>, '!RequestType': <bound method BetterEnum._from_yaml of <enum 'RequestType'>>, '!CompressAlgo': <bound method BetterEnum._from_yaml of <enum 'CompressAlgo'>>, '!OnErrorStrategy': <bound method BetterEnum._from_yaml of <enum 'OnErrorStrategy'>>, '!FlowInspectType': <bound method BetterEnum._from_yaml of <enum 'FlowInspectType'>>, '!RemoteAccessType': <bound method BetterEnum._from_yaml of <enum 'RemoteAccessType'>>, '!BuildTestLevel': <bound method BetterEnum._from_yaml of <enum 'BuildTestLevel'>>, '!DataInputType': <bound method BetterEnum._from_yaml of <enum 'DataInputType'>>, '!RuntimeBackendType': <bound method BetterEnum._from_yaml of <enum 'RuntimeBackendType'>>, '!BaseExecutor': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.BaseExecutor'>>, '!CompoundExecutor': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.compound.CompoundExecutor'>>, '!BaseDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.BaseDriver'>>, '!BaseRecursiveDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.BaseRecursiveDriver'>>, '!BaseExecutableDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.BaseExecutableDriver'>>, '!BaseEncodeDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.encode.BaseEncodeDriver'>>, '!EncodeDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.encode.EncodeDriver'>>, '!ScipySparseEncodeDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.encode.ScipySparseEncodeDriver'>>, '!MultiModalDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.multimodal.MultiModalDriver'>>, '!FilterQL': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.querylang.filter.FilterQL'>>, '!ReduceAllDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.reduce.ReduceAllDriver'>>, '!CollectEvaluationDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.reduce.CollectEvaluationDriver'>>, '!ConcatEmbedDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.reduce.ConcatEmbedDriver'>>, '!ReverseQL': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.querylang.reverse.ReverseQL'>>, '!ConvertDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.convert.ConvertDriver'>>, '!URI2Buffer': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.convert.URI2Buffer'>>, '!URI2DataURI': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.convert.URI2DataURI'>>, '!Buffer2URI': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.convert.Buffer2URI'>>, '!BufferImage2Blob': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.convert.BufferImage2Blob'>>, '!URI2Blob': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.convert.URI2Blob'>>, '!DataURI2Blob': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.convert.DataURI2Blob'>>, '!Text2URI': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.convert.Text2URI'>>, '!URI2Text': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.convert.URI2Text'>>, '!Blob2PngURI': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.convert.Blob2PngURI'>>, '!BaseIndexDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.index.BaseIndexDriver'>>, '!VectorIndexDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.index.VectorIndexDriver'>>, '!KVIndexDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.index.KVIndexDriver'>>, '!DBMSIndexDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.index.DBMSIndexDriver'>>, '!SegmentDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.segment.SegmentDriver'>>, '!SliceQL': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.querylang.slice.SliceQL'>>, '!SortQL': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.querylang.sort.SortQL'>>, '!ExcludeQL': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.querylang.select.ExcludeQL'>>, '!SelectQL': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.querylang.select.SelectQL'>>, '!ExcludeReqQL': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.querylang.select.ExcludeReqQL'>>, '!SelectReqQL': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.querylang.select.SelectReqQL'>>, '!BaseCacheDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.cache.BaseCacheDriver'>>, '!TaggingCacheDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.cache.TaggingCacheDriver'>>, '!BaseSearchDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.search.BaseSearchDriver'>>, '!KVSearchDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.search.KVSearchDriver'>>, '!VectorFillDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.search.VectorFillDriver'>>, '!VectorSearchDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.search.VectorSearchDriver'>>, '!BaseEvaluateDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.evaluate.BaseEvaluateDriver'>>, '!FieldEvaluateDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.evaluate.FieldEvaluateDriver'>>, '!RankEvaluateDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.evaluate.RankEvaluateDriver'>>, '!NDArrayEvaluateDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.evaluate.NDArrayEvaluateDriver'>>, '!TextEvaluateDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.evaluate.TextEvaluateDriver'>>, '!LoadGroundTruthDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.evaluate.LoadGroundTruthDriver'>>, '!DeleteDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.delete.DeleteDriver'>>, '!BaseRankDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.rank.BaseRankDriver'>>, '!Matches2DocRankDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.rank.Matches2DocRankDriver'>>, '!CraftDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.craft.CraftDriver'>>, '!BasePredictDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.predict.BasePredictDriver'>>, '!BaseLabelPredictDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.predict.BaseLabelPredictDriver'>>, '!BinaryPredictDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.predict.BinaryPredictDriver'>>, '!OneHotPredictDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.predict.OneHotPredictDriver'>>, '!MultiLabelPredictDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.predict.MultiLabelPredictDriver'>>, '!Prediction2DocBlobDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.predict.Prediction2DocBlobDriver'>>, '!BaseControlDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.control.BaseControlDriver'>>, '!LogInfoDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.control.LogInfoDriver'>>, '!WaitDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.control.WaitDriver'>>, '!ControlReqDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.control.ControlReqDriver'>>, '!RouteDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.control.RouteDriver'>>, '!ForwardDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.control.ForwardDriver'>>, '!WhooshDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.control.WhooshDriver'>>, '!BaseRanker': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.rankers.BaseRanker'>>, '!Chunk2DocRanker': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.rankers.Chunk2DocRanker'>>, '!Match2DocRanker': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.rankers.Match2DocRanker'>>, '!BaseAggregateMatchesRankerDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.rank.aggregate.BaseAggregateMatchesRankerDriver'>>, '!Chunk2DocRankDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.rank.aggregate.Chunk2DocRankDriver'>>, '!AggregateMatches2DocRankDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.rank.aggregate.AggregateMatches2DocRankDriver'>>, '!DumpDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.dump.DumpDriver'>>, '!PngToDiskDriver': <bound method JAMLCompatible._from_yaml of <class 'jina.drivers.debug.PngToDiskDriver'>>, '!BaseClassifier': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.classifiers.BaseClassifier'>>, '!BaseFlow': <bound method JAMLCompatible._from_yaml of <class 'jina.flow.base.BaseFlow'>>, '!Flow': <bound method JAMLCompatible._from_yaml of <class 'jina.flow.Flow'>>, '!BaseEvaluator': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.evaluators.BaseEvaluator'>>, '!FileBasedEvaluator': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.evaluators.FileBasedEvaluator'>>, '!BaseRankingEvaluator': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.evaluators.rank.BaseRankingEvaluator'>>, '!PrecisionEvaluator': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.evaluators.rank.precision.PrecisionEvaluator'>>, '!BaseIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.BaseIndexer'>>, '!BaseVectorIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.BaseVectorIndexer'>>, '!BaseKVIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.BaseKVIndexer'>>, '!UniqueVectorIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.UniqueVectorIndexer'>>, '!CompoundIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.CompoundIndexer'>>, '!BaseQueryIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.query.BaseQueryIndexer'>>, '!BaseEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.BaseEncoder'>>, '!BaseNumericEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.BaseNumericEncoder'>>, '!BaseImageEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.BaseImageEncoder'>>, '!BaseVideoEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.BaseVideoEncoder'>>, '!BaseAudioEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.BaseAudioEncoder'>>, '!BaseTextEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.BaseTextEncoder'>>, '!TransformEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.numeric.TransformEncoder'>>, '!BaseEmbeddingEvaluator': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.evaluators.embedding.BaseEmbeddingEvaluator'>>, '!EuclideanEvaluator': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.evaluators.embedding.euclidean.EuclideanEvaluator'>>, '!BaseOnnxEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.frameworks.BaseOnnxEncoder'>>, '!BaseTFEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.frameworks.BaseTFEncoder'>>, '!BaseTorchEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.frameworks.BaseTorchEncoder'>>, '!BasePaddleEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.frameworks.BasePaddleEncoder'>>, '!BaseMindsporeEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.frameworks.BaseMindsporeEncoder'>>, '!BinaryPbIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.keyvalue.BinaryPbIndexer'>>, '!KeyValueIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.keyvalue.KeyValueIndexer'>>, '!DataURIPbIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.keyvalue.DataURIPbIndexer'>>, '!UniquePbIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.keyvalue.UniquePbIndexer'>>, '!BaseCrafter': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.crafters.BaseCrafter'>>, '!BaseTextEvaluator': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.evaluators.text.BaseTextEvaluator'>>, '!BaseNumpyIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.vector.BaseNumpyIndexer'>>, '!NumpyIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.vector.NumpyIndexer'>>, '!VectorIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.vector.VectorIndexer'>>, '!NumpyQueryIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.query.vector.NumpyQueryIndexer'>>, '!BaseDBMSIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.dbms.BaseDBMSIndexer'>>, '!BaseMultiModalEncoder': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.encoders.multimodal.BaseMultiModalEncoder'>>, '!BinaryPbQueryIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.query.keyvalue.BinaryPbQueryIndexer'>>, '!KeyValueQueryIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.query.keyvalue.KeyValueQueryIndexer'>>, '!CompoundQueryExecutor': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.query.compound.CompoundQueryExecutor'>>, '!CompoundQueryIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.query.compound.CompoundQueryIndexer'>>, '!CosineEvaluator': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.evaluators.embedding.cosine.CosineEvaluator'>>, '!TextLengthEvaluator': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.evaluators.text.length.TextLengthEvaluator'>>, '!RecallEvaluator': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.evaluators.rank.recall.RecallEvaluator'>>, '!BaseSegmenter': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.segmenters.BaseSegmenter'>>, '!BaseCache': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.cache.BaseCache'>>, '!DocCache': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.cache.DocCache'>>, '!BinaryPbDBMSIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.dbms.keyvalue.BinaryPbDBMSIndexer'>>, '!KeyValueDBMSIndexer': <bound method JAMLCompatible._from_yaml of <class 'jina.executors.indexers.dbms.keyvalue.KeyValueDBMSIndexer'>>, '!AsyncFlow': <bound method JAMLCompatible._from_yaml of <class 'jina.flow.asyncio.AsyncFlow'>>}
jina.jaml.helper.parse_config_source(path, allow_stream=True, allow_yaml_file=True, allow_builtin_resource=True, allow_raw_yaml_content=True, allow_raw_driver_yaml_content=True, allow_class_type=True, allow_dict=True, allow_json=True, *args, **kwargs)[source]

Check if the text or text stream is valid.

Parameters
  • path (Union[str, TextIO, Dict]) – the multi-kind source of the configs.

  • allow_stream (bool) – flag

  • allow_yaml_file (bool) – flag

  • allow_builtin_resource (bool) – flag

  • allow_raw_yaml_content (bool) – flag

  • allow_raw_driver_yaml_content (bool) – flag

  • allow_class_type (bool) – flag

  • allow_dict (bool) – flag

  • allow_json (bool) – flag

  • args – unused

  • kwargs – unused

Return type

Tuple[TextIO, Optional[str]]

Returns

a tuple, the first element is the text stream, the second element is the file path associate to it if available.

jina.jaml.helper.complete_path(path, extra_search_paths=None)[source]

Complete the path of file via searching in abs and relative paths.

Parameters
  • path (str) – path of file.

  • extra_search_paths (Optional[Tuple[str]]) – extra paths to conduct search

Return type

str

Returns

Completed file path.

jina.jaml.helper.load_py_modules(d, extra_search_paths=None)[source]

Find ‘py_modules’ in the dict recursively and then load them.

Parameters
  • d (Dict) – the dictionary to traverse

  • extra_search_paths (Optional[Tuple[str]]) – any extra paths to search

Return type

None