Protocol Documentation

jina.proto

Chunk

Represents a Chunk

Field Type Label Description
doc_id uint32 the document ID of this chunk, universally unique
chunk_id uint32 the chunk ID, universally unique
text string the original text of the chunk (only apply to the text document)
blob NdArray the original ndarray of the chunk (apply to the image/audio/video document)
buffer bytes raw bytes of chunk
embedding NdArray the embedding array of this chunk
offset uint32 the offset of this chunk in the current document
weight float the weight of this chunk
length uint32 the total number of chunks in the current document
meta_info bytes some binary meta information of this chunk in bytes
topk_results ScoredResult repeated the top-k matched chunks
mime_type string mime type of this chunk, by default it is inherit from the document
field_name string the field name
tags Chunk.TagsEntry repeated a map of tags
location uint32 repeated the position of the chunk, could be start and end index of a string; could be x,y coordinate of a image; could be timestamp of an audio clip

Chunk.TagsEntry

Field Type Label Description
key string
value string

Document

Represents a Document

Field Type Label Description
doc_id uint32 The unique document ID
buffer bytes the raw binary content of this document, which often represents the original document when comes into jina
blob NdArray the ndarray of the image/audio/video document
text string a text document
chunks Chunk repeated list of the chunks of this document
weight float the weight of this document
length uint32 total number of chunks in this document
meta_info bytes some binary meta information of this chunk in bytes
topk_results ScoredResult repeated the top-k matched chunks
mime_type string mime type of this document, for buffer content, this is required; for other contents, this can be guessed
uri string a uri of the document could be: a local file path, a remote url starts with http or https or data URI scheme
tags Document.TagsEntry repeated a map of tags

Document.TagsEntry

Field Type Label Description
key string
value string

Envelope

Represents a Envelope, a part of the Message.

Field Type Label Description
sender_id string unique id of the sender of the message
receiver_id string unique id of the receiver of the message, only used in router-dealer pattern
request_id uint32 unique id of the request
timeout uint32 timeout in second until this message is droped
routes Envelope.Route repeated a list of routes this message goes through
version Envelope.Version version info
status Status status info
num_part uint32 repeated the number of partial messages

Envelope.Route

Represents a the route paths of this message

Field Type Label Description
pod string the name of the BasePod
pod_id string the id of the BasePod
start_time google.protobuf.Timestamp receiving time
end_time google.protobuf.Timestamp sending (out) time

Envelope.Version

Represents a the version information

Field Type Label Description
jina string jina's version
proto string protobuf's version
vcs string vcs's version

Message

Represents a Message

Field Type Label Description
envelope Envelope the envelope of the message, used internally in jina, dropped when returning to client
request Request the request body

NdArray

Represents the a (quantized) numpy ndarray

Field Type Label Description
buffer bytes the actual array data, in bytes
shape uint32 repeated the shape (dimensions) of the array
dtype string the data type of the array
quantization NdArray.QuantizationMode quantization mode
max_val float the max value of the ndarray
min_val float the min value of the ndarray
scale float the scale of the ndarray
original_dtype string the original dtype of the array

Request

Represents a Request

Field Type Label Description
request_id uint32 the unique ID of this request. Multiple requests with the same ID will be gathered
train Request.TrainRequest a train request
index Request.IndexRequest an index request
search Request.SearchRequest a search request
control Request.ControlRequest a control request
status Status status info

Request.ControlRequest

Represents a control request used to control the BasePod

Field Type Label Description
command Request.ControlRequest.Command the control command
args Request.ControlRequest.ArgsEntry repeated the pea arguments, useful in STATUS

Request.ControlRequest.ArgsEntry

Field Type Label Description
key string
value string

Request.IndexRequest

Represents an index request

Field Type Label Description
docs Document repeated a list of Documents to index
filter_by string repeated a list of the selected field names to be queried on

Request.SearchRequest

Represents a search request

Field Type Label Description
docs Document repeated a list of Documents to query
top_k uint32 the number of most related results to return
filter_by string repeated a list of the selected field names to be queried on

Request.TrainRequest

Represents a train request

Field Type Label Description
docs Document repeated a list of Documents to train
flush bool if True then do actual training, otherwise only collect all documents but not do training.
filter_by string repeated a list of the selected field names to be queried on

ScoredResult

Represents an unary match result

Field Type Label Description
match_chunk Chunk the matched chunk
match_doc Document the matched document
score ScoredResult.Score the score of this match

ScoredResult.Score

Represents the score of a match

Field Type Label Description
value float value
op_name string the name of the operator/score function
description string text description of the score
operands ScoredResult.Score repeated the score can be nested

SpawnRequest

Field Type Label Description
pea SpawnRequest.PeaSpawnRequest
pod SpawnRequest.PodSpawnRequest send a PodSpawnRequest and it will switch to PodDictSpawnRequest as returns
mutable_pod SpawnRequest.MutablepodSpawnRequest
log_record string one line of the log record
status Status status info

SpawnRequest.MutablepodSpawnRequest

Spawn a parsed pod remotely, requires parsed pod structure

Field Type Label Description
head SpawnRequest.PeaSpawnRequest a list of args string for the head BasePea
tail SpawnRequest.PeaSpawnRequest a list of args string for the tail BasePea
peas SpawnRequest.PeaSpawnRequest repeated a list of args string for the Peas

SpawnRequest.PeaSpawnRequest

Spawn a pea remotely, requires args of a pea

Field Type Label Description
args string repeated a list of args string for BasePea

SpawnRequest.PodSpawnRequest

Spawn a pod remotely, requires args of a pea

Field Type Label Description
args string repeated a list of args string for BasePod

Status

Field Type Label Description
code Status.StatusCode status code
description string error description of the very first exception
details Status.Details repeated the details of the error

Status.Details

Field Type Label Description
pod string the name of that problematic peapod
pod_id string the id of that problematic peapod
executor string the name of the executor binded to that peapod (if applicable)
exception string the class name of the exception
traceback string the reason of the exception
time google.protobuf.Timestamp the timestamp when error occurs

NdArray.QuantizationMode

Name Number Description
NONE 0 no quantization is performed, stored in the original dtype
FP16 1 2x smaller if dtype is set to FP32
UINT8 2 4x smaller but lossy when dtype is FP32

Request.ControlRequest.Command

Name Number Description
TERMINATE 0 shutdown the BasePod
STATUS 1 check the status of the BasePod
IDLE 3 used in ROUTER-DEALER pattern, tells the router that the dealer is idle

Status.StatusCode

Name Number Description
SUCCESS 0 success
PENDING 1 there are pending messages, more messages are followed
READY 2 ready to use
ERROR 3 error
ERROR_DUPLICATE 4 already a existing pea/pod running
ERROR_NOTALLOWED 5 not allowed to open pea/pod remotely

JinaRPC

jina gRPC service.

Method Name Request Type Response Type Description
Call Request stream Request stream Pass in a Request and a filled Request with topk_results will be returned.
CallUnary Request Request
Spawn SpawnRequest SpawnRequest stream Pass in a Request and a filled Request will be returned.

Scalar Value Types

.proto Type Notes C++ Java Python Go C# PHP Ruby
double double double float float64 double float Float
float float float float float32 float float Float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int int32 int integer Bignum or Fixnum (as required)
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long int64 long integer/string Bignum
uint32 Uses variable-length encoding. uint32 int int/long uint32 uint integer Bignum or Fixnum (as required)
uint64 Uses variable-length encoding. uint64 long int/long uint64 ulong integer/string Bignum or Fixnum (as required)
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 int integer Bignum or Fixnum (as required)
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long int64 long integer/string Bignum
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int uint32 uint integer Bignum or Fixnum (as required)
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long uint64 ulong integer/string Bignum
sfixed32 Always four bytes. int32 int int int32 int integer Bignum or Fixnum (as required)
sfixed64 Always eight bytes. int64 long int/long int64 long integer/string Bignum
bool bool boolean boolean bool bool boolean TrueClass/FalseClass
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode string string string String (UTF-8)
bytes May contain any arbitrary sequence of bytes. string ByteString str []byte ByteString string String (ASCII-8BIT)