Protocol Documentation

jina.proto

Document

Represents a Document

Field Type Label Description
id string A hexdigest that represents a unique document ID
granularity uint32 the depth of the recursive chunk structure
adjacency uint32 the width of the recursive match structure
level_name string the name of the level
parent_id string the parent id from the previous level
buffer bytes the raw binary content of this document, which often represents the original document when comes into jina
blob NdArray the ndarray of the image/audio/video document
text string a text document
chunks Document repeated list of the chunks of this document (recursive structure)
weight float the weight of this document
length uint32 total number of siblings of this document (docs that are in the same granularity and parent_id)
meta_info bytes some binary meta information of this Doc in bytes
matches Document repeated the top-k matched Docs on the same level (recursive structure)
mime_type string mime type of this document, for buffer content, this is required; for other contents, this can be guessed
uri string a uri of the document could be: a local file path, a remote url starts with http or https or data URI scheme
tags google.protobuf.Struct a structured data value, consisting of field which map to dynamically typed values.
location uint32 repeated the position of the doc, could be start and end index of a string; could be x,y (top, left) coordinate of an image crop; could be timestamp of an audio clip
offset uint32 the offset of this doc in the previous level document
embedding NdArray the embedding array of this doc
score NamedScore TODO: List of matching scores performed on the document, each element corresponds to a metric
modality string modality, an identifier to the modality this document belongs to. In the scope of multi/cross modal search
evaluations NamedScore repeated List of evaluations performed on the document, each element corresponds to a metric

Envelope

Represents a Envelope, a part of the Message.

Field Type Label Description
sender_id string unique id of the sender of the message
receiver_id string unique id of the receiver of the message, only used in router-dealer pattern
request_id string unique id of the request
timeout uint32 timeout in second until this message is dropped
routes Envelope.Route repeated a list of routes this message goes through
version Envelope.Version version info
status Status status info
num_part uint32 repeated the number of partial messages

Envelope.Route

Represents a the route paths of this message

Field Type Label Description
pod string the name of the BasePod
pod_id string the id of the BasePod
start_time google.protobuf.Timestamp receiving time
end_time google.protobuf.Timestamp sending (out) time

Envelope.Version

Represents a the version information

Field Type Label Description
jina string jina's version
proto string protobuf's version
vcs string vcs's version

Message

Represents a Message

Field Type Label Description
envelope Envelope the envelope of the message, used internally in jina, dropped when returning to client
request Request the request body

NamedScore

Represents the relevance model to ref_id

Field Type Label Description
value float value
op_name string the name of the operator/score function
description string text description of the score
operands NamedScore repeated the score can be nested
ref_id string the score is computed between doc id and ref_id

NdArray

Represents the a (quantized) numpy ndarray

Field Type Label Description
buffer bytes the actual array data, in bytes
shape uint32 repeated the shape (dimensions) of the array
dtype string the data type of the array
quantization NdArray.QuantizationMode quantization mode
max_val float the max value of the ndarray
min_val float the min value of the ndarray
scale float the scale of the ndarray
original_dtype string the original dtype of the array

QueryLang

Field Type Label Description
name string the name of the query lang, map to the corresponding driver
parameters google.protobuf.Struct the parameter of the query lang
disabled bool when set, then is query lang is disabled
priority int32 when query lang is set on both driver and request levels, higher priority query lang is applied, lower priority is dropped

Request

Represents a Request

Field Type Label Description
request_id string the unique ID of this request. Multiple requests with the same ID will be gathered
train Request.TrainRequest a train request
index Request.IndexRequest an index request
search Request.SearchRequest a search request
control Request.ControlRequest a control request
status Status status info
queryset QueryLang repeated the query set

Request.ControlRequest

Represents a control request used to control the BasePod

Field Type Label Description
command Request.ControlRequest.Command the control command
args Request.ControlRequest.ArgsEntry repeated the pea arguments, useful in STATUS

Request.ControlRequest.ArgsEntry

Field Type Label Description
key string
value string

Request.IndexRequest

Represents an index request

Field Type Label Description
docs Document repeated a list of Documents to index
groundtruths Document repeated the groundtruth document you want to evaluate it with

Request.SearchRequest

Represents a search request

Field Type Label Description
docs Document repeated a list of Documents to query
groundtruths Document repeated the groundtruth document you want to evaluate it with

Request.TrainRequest

Represents a train request

Field Type Label Description
docs Document repeated a list of Documents to train
groundtruths Document repeated the groundtruth document you want to evaluate it with
flush bool if True then do actual training, otherwise only collect all documents but not do training.

SpawnRequest

Field Type Label Description
pea SpawnRequest.PeaSpawnRequest
pod SpawnRequest.PodSpawnRequest send a PodSpawnRequest and it will switch to PodDictSpawnRequest as returns
mutable_pod SpawnRequest.MutablepodSpawnRequest
log_record string one line of the log record
status Status status info

SpawnRequest.MutablepodSpawnRequest

Spawn a parsed pod remotely, requires parsed pod structure

Field Type Label Description
head SpawnRequest.PeaSpawnRequest a list of args string for the head BasePea
tail SpawnRequest.PeaSpawnRequest a list of args string for the tail BasePea
peas SpawnRequest.PeaSpawnRequest repeated a list of args string for the Peas

SpawnRequest.PeaSpawnRequest

Spawn a pea remotely, requires args of a pea

Field Type Label Description
args string repeated a list of args string for BasePea

SpawnRequest.PodSpawnRequest

Spawn a pod remotely, requires args of a pea

Field Type Label Description
args string repeated a list of args string for BasePod

Status

Field Type Label Description
code Status.StatusCode status code
description string error description of the very first exception
details Status.Details repeated the details of the error

Status.Details

Field Type Label Description
pod string the name of that problematic peapod
pod_id string the id of that problematic peapod
executor string the name of the executor binded to that peapod (if applicable)
exception string the class name of the exception
traceback string the reason of the exception
time google.protobuf.Timestamp the timestamp when error occurs

NdArray.QuantizationMode

Name Number Description
NONE 0 no quantization is performed, stored in the original dtype
FP16 1 2x smaller if dtype is set to FP32
UINT8 2 4x smaller but lossy when dtype is FP32

Request.ControlRequest.Command

Name Number Description
TERMINATE 0 shutdown the BasePod
STATUS 1 check the status of the BasePod
IDLE 3 used in ROUTER-DEALER pattern, tells the router that the dealer is idle

Status.StatusCode

Name Number Description
SUCCESS 0 success
PENDING 1 there are pending messages, more messages are followed
READY 2 ready to use
ERROR 3 error
ERROR_DUPLICATE 4 already a existing pea/pod running
ERROR_NOTALLOWED 5 not allowed to open pea/pod remotely

JinaRPC

jina gRPC service.

Method Name Request Type Response Type Description
Call Request stream Request stream Pass in a Request and a filled Request with matches will be returned.
CallUnary Request Request
Spawn SpawnRequest SpawnRequest stream Pass in a Request and a filled Request will be returned.

Scalar Value Types

.proto Type Notes C++ Java Python Go C# PHP Ruby
double double double float float64 double float Float
float float float float float32 float float Float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int int32 int integer Bignum or Fixnum (as required)
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long int64 long integer/string Bignum
uint32 Uses variable-length encoding. uint32 int int/long uint32 uint integer Bignum or Fixnum (as required)
uint64 Uses variable-length encoding. uint64 long int/long uint64 ulong integer/string Bignum or Fixnum (as required)
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 int integer Bignum or Fixnum (as required)
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long int64 long integer/string Bignum
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int uint32 uint integer Bignum or Fixnum (as required)
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long uint64 ulong integer/string Bignum
sfixed32 Always four bytes. int32 int int int32 int integer Bignum or Fixnum (as required)
sfixed64 Always eight bytes. int64 long int/long int64 long integer/string Bignum
bool bool boolean boolean bool bool boolean TrueClass/FalseClass
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode string string string String (UTF-8)
bytes May contain any arbitrary sequence of bytes. string ByteString str []byte ByteString string String (ASCII-8BIT)