jina.drivers.multimodal

class jina.drivers.multimodal.MultiModalDriver(traversal_paths=('r', ), *args, **kwargs)[source]

Bases: jina.drivers.encode.BaseEncodeDriver

Extract multimodal embeddings from different modalities. Input-Output

Input:
document:
        |- chunk: {modality: mode1}
        |
        |- chunk: {modality: mode2}
Output:
document: (embedding: multimodal encoding)
        |- chunk: {modality: mode1}
        |
        |- chunk: {modality: mode2}

Note

  • It traverses on the documents for which we want to apply the multimodal embedding. This way

we can use the batching capabilities for the executor.

Warning

  • It assumes that every chunk of a document belongs to a different modality.

Initialize a BaseExecutableDriver

Parameters
property positional_modality

Get position per modality. :rtype: List[str] :return: the list of strings representing the name and order of the modality.