jina.executors.indexers.dump

jina.executors.indexers.dump.export_dump_streaming(path, shards, size, data)[source]

Export the data to a path, based on sharding,

Parameters
  • path (str) – path to dump

  • shards (int) – the nr of shards this pea is part of

  • size (int) – total amount of entries

  • data (Generator[Tuple[str, array, bytes], None, None]) – the generator of the data (ids, vectors, metadata)

jina.executors.indexers.dump.import_vectors(path, pea_id)[source]

Import id and vectors

Parameters
  • path (str) – the path to the dump

  • pea_id (str) – the id of the pea (as part of the shards)

Returns

the generators for the ids and for the vectors

jina.executors.indexers.dump.import_metas(path, pea_id)[source]

Import id and metadata

Parameters
  • path (str) – the path of the dump

  • pea_id (str) – the id of the pea (as part of the shards)

Returns

the generators for the ids and for the metadata