Protocol Documentation

jina.proto

DenseNdArrayProto

Represents a (quantized) dense n-dim array

Field Type Label Description
buffer bytes the actual array data, in bytes
shape uint32 repeated the shape (dimensions) of the array
dtype string the data type of the array
quantization DenseNdArrayProto.QuantizationMode quantization mode
max_val float the max value of the ndarray
min_val float the min value of the ndarray
scale float the scale of the ndarray
original_dtype string the original dtype of the array

DocumentProto

Represents a Document

Field Type Label Description
id string A hexdigest that represents a unique document ID
content_hash string A hexdigest that represents the hash of the content of the document
granularity uint32 the depth of the recursive chunk structure
adjacency uint32 the width of the recursive match structure
level_name string the name of the level
parent_id string the parent id from the previous level
buffer bytes the raw binary content of this document, which often represents the original document when comes into jina
blob NdArrayProto the ndarray of the image/audio/video document
text string a text document
chunks DocumentProto repeated list of the chunks of this document (recursive structure)
weight float the weight of this document
length uint32 total number of siblings of this document (docs that are in the same granularity and parent_id)
meta_info bytes some binary meta information of this Doc in bytes
matches DocumentProto repeated the top-k matched Docs on the same level (recursive structure)
mime_type string mime type of this document, for buffer content, this is required; for other contents, this can be guessed
uri string a uri of the document could be: a local file path, a remote url starts with http or https or data URI scheme
tags google.protobuf.Struct a structured data value, consisting of field which map to dynamically typed values.
location uint32 repeated the position of the doc, could be start and end index of a string; could be x,y (top, left) coordinate of an image crop; could be timestamp of an audio clip
offset uint32 the offset of this doc in the previous level document
embedding NdArrayProto the embedding array of this doc
score NamedScoreProto TODO: List of matching scores performed on the document, each element corresponds to a metric
modality string modality, an identifier to the modality this document belongs to. In the scope of multi/cross modal search
evaluations NamedScoreProto repeated List of evaluations performed on the document, each element corresponds to a metric

EnvelopeProto

Represents a Envelope, a part of the Message.

Field Type Label Description
sender_id string unique id of the sender of the message
receiver_id string unique id of the receiver of the message, only used in router-dealer pattern
request_id string unique id of the request
timeout uint32 timeout in second until this message is dropped
routes RouteProto repeated a list of routes this message goes through
version EnvelopeProto.VersionProto version info
status StatusProto status info, when present, it is the first exception that routes carry
request_type string type of the request: TrainRequest, IndexRequest, SearchRequest, ControlRequest
check_version bool check local Protobuf version on every Pod that this message flows to
compression EnvelopeProto.CompressConfigProto compress configuration used for request

EnvelopeProto.CompressConfigProto

Represents a config for the compression algorithm

Field Type Label Description
algorithm string compress algorithm used for request
high_watermark uint64 the high watermark that triggers the message compression. message bigger than this HWM (in bytes) will be compressed by the algorithm.
low_watermark float the low watermark that enables the sending of a compressed message. compression rate (after_size/before_size) lower than this LWM will be considered as successeful compression, and will be sent. Otherwise, it will send the original message without compression
parameters google.protobuf.Struct other parameters that can be accepted by the algorithm

EnvelopeProto.VersionProto

Represents a the version information

Field Type Label Description
jina string jina's version
proto string protobuf's version
vcs string vcs's version

MessageProto

Represents a Message

Field Type Label Description
envelope EnvelopeProto the envelope of the message, used internally in jina, dropped when returning to client
request RequestProto the request body

NamedScoreProto

Represents the relevance model to ref_id

Field Type Label Description
value float value
op_name string the name of the operator/score function
description string text description of the score
operands NamedScoreProto repeated the score can be nested
ref_id string the score is computed between doc id and ref_id

NdArrayProto

Represents a general n-dim array, can be either dense or sparse

Field Type Label Description
dense DenseNdArrayProto dense representation of the ndarray
sparse SparseNdArrayProto sparse representation of the ndarray

QueryLangProto

Field Type Label Description
name string the name of the query lang, map to the corresponding driver
parameters google.protobuf.Struct the parameter of the query lang
disabled bool when set, then is query lang is disabled
priority int32 when query lang is set on both driver and request levels, higher priority query lang is applied, lower priority is dropped

RequestProto

Represents a Request

Field Type Label Description
request_id string the unique ID of this request. Multiple requests with the same ID will be gathered
train RequestProto.TrainRequestProto a train request
index RequestProto.IndexRequestProto an index request
search RequestProto.SearchRequestProto a search request
control RequestProto.ControlRequestProto a control request
delete RequestProto.DeleteRequestProto a delete request
update RequestProto.UpdateRequestProto an update request
status StatusProto status info
routes RouteProto repeated status info on every routes
queryset QueryLangProto repeated the query set

RequestProto.ControlRequestProto

Represents a control request used to control the BasePod

Field Type Label Description
command RequestProto.ControlRequestProto.Command the control command
args google.protobuf.Struct the pea arguments, useful in STATUS

RequestProto.DeleteRequestProto

Represents a delete request

Field Type Label Description
docs DocumentProto repeated a list of Documents to delete
groundtruths DocumentProto repeated the groundtruth document you want to evaluate it with

RequestProto.IndexRequestProto

Represents an index request

Field Type Label Description
docs DocumentProto repeated a list of Documents to index
groundtruths DocumentProto repeated the groundtruth document you want to evaluate it with

RequestProto.SearchRequestProto

Represents a search request

Field Type Label Description
docs DocumentProto repeated a list of Documents to query
groundtruths DocumentProto repeated the groundtruth document you want to evaluate it with

RequestProto.TrainRequestProto

Represents a train request

Field Type Label Description
docs DocumentProto repeated a list of Documents to train
groundtruths DocumentProto repeated the groundtruth document you want to evaluate it with
flush bool if True then do actual training, otherwise only collect all documents but not do training.

RequestProto.UpdateRequestProto

Represents an update request

Field Type Label Description
docs DocumentProto repeated a list of Documents to update
groundtruths DocumentProto repeated the groundtruth document you want to evaluate it with

RouteProto

Represents a the route paths of this message

Field Type Label Description
pod string the name of the BasePod
pod_id string the id of the BasePod
start_time google.protobuf.Timestamp receiving time
end_time google.protobuf.Timestamp sending (out) time
status StatusProto the status of the execution

SparseNdArrayProto

Represents a sparse ndarray

Field Type Label Description
indices DenseNdArrayProto A 2-D int64 tensor of shape [N, ndims], which specifies the indices of the elements in the sparse tensor that contain nonzero values (elements are zero-indexed)
values DenseNdArrayProto A 1-D tensor of any type and shape [N], which supplies the values for each element in indices.
dense_shape int64 repeated A 1-D int64 tensor of shape [ndims], which specifies the dense_shape of the sparse tensor.

StatusProto

Field Type Label Description
code StatusProto.StatusCode status code
description string error description of the very first exception
exception StatusProto.ExceptionProto the details of the error

StatusProto.ExceptionProto

Field Type Label Description
name string the class name of the exception
args string repeated the list of arguments given to the exception constructor.
stacks string repeated the exception traceback stacks
executor string the name of the executor bind to that peapod (if applicable)
driver string the name of the driver bind to that peapod (if applicable)

DenseNdArrayProto.QuantizationMode

Name Number Description
NONE 0 no quantization is performed, stored in the original dtype
FP16 1 2x smaller if dtype is set to FP32
UINT8 2 4x smaller but lossy when dtype is FP32

RequestProto.ControlRequestProto.Command

Name Number Description
TERMINATE 0 shutdown the BasePod
STATUS 1 check the status of the BasePod
IDLE 3 used in ROUTER-DEALER pattern, tells the router that the dealer is idle

StatusProto.StatusCode

Name Number Description
SUCCESS 0 success
PENDING 1 there are pending messages, more messages are followed
READY 2 ready to use
ERROR 3 error
ERROR_DUPLICATE 4 already a existing pod running
ERROR_NOTALLOWED 5 not allowed to open pod remotely
ERROR_CHAINED 6 chained from the previous error

JinaRPC

jina gRPC service.

Method Name Request Type Response Type Description
Call RequestProto stream RequestProto stream Pass in a Request and a filled Request with matches will be returned.
CallUnary RequestProto RequestProto

Scalar Value Types

.proto Type Notes C++ Java Python Go C# PHP Ruby
double double double float float64 double float Float
float float float float float32 float float Float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int int32 int integer Bignum or Fixnum (as required)
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long int64 long integer/string Bignum
uint32 Uses variable-length encoding. uint32 int int/long uint32 uint integer Bignum or Fixnum (as required)
uint64 Uses variable-length encoding. uint64 long int/long uint64 ulong integer/string Bignum or Fixnum (as required)
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 int integer Bignum or Fixnum (as required)
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long int64 long integer/string Bignum
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int uint32 uint integer Bignum or Fixnum (as required)
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long uint64 ulong integer/string Bignum
sfixed32 Always four bytes. int32 int int int32 int integer Bignum or Fixnum (as required)
sfixed64 Always eight bytes. int64 long int/long int64 long integer/string Bignum
bool bool boolean boolean bool bool boolean TrueClass/FalseClass
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode string string string String (UTF-8)
bytes May contain any arbitrary sequence of bytes. string ByteString str []byte ByteString string String (ASCII-8BIT)