Tip

To update docarray Protobuf:

cd docarrat
docker run -v $(pwd)/proto:/jina/proto jinaai/protogen

To update jina Protobuf:

docker run -v $(pwd)/jina/:/jina/ -v $(pwd)/docarray/:/docarray/ jinaai/protogen

Protocol Documentation

Table of Contents

Top

docarray.proto

DenseNdArrayProto

Represents a (quantized) dense n-dim array

Field

Type

Label

Description

buffer

bytes

the actual array data, in bytes

shape

uint32

repeated

the shape (dimensions) of the array

dtype

string

the data type of the array

DocumentArrayProto

Field

Type

Label

Description

docs

DocumentProto

repeated

a list of Documents

DocumentProto

Represents a Document

Field

Type

Label

Description

id

string

A hexdigest that represents a unique document ID

granularity

uint32

the depth of the recursive chunk structure

adjacency

uint32

the width of the recursive match structure

parent_id

string

the parent id from the previous granularity

buffer

bytes

the raw binary content of this document, which often represents the original document when comes into jina

blob

NdArrayProto

the ndarray of the image/audio/video document

text

string

a text document

chunks

DocumentProto

repeated

list of the sub-documents of this document (recursive structure)

weight

float

The weight of this document

matches

DocumentProto

repeated

the matched documents on the same level (recursive structure)

uri

string

a uri of the document could be: a local file path, a remote url starts with http or https or data URI scheme

mime_type

string

mime type of this document, for buffer content, this is required; for other contents, this can be guessed

tags

google.protobuf.Struct

a structured data value, consisting of field which map to dynamically typed values.

location

float

repeated

the position of the doc, could be start and end index of a string; could be x,y (top, left) coordinate of an image crop; could be timestamp of an audio clip

offset

float

the offset of this doc in the previous granularity document

embedding

NdArrayProto

the embedding ndarray of this document

scores

DocumentProto.ScoresEntry

repeated

Scores performed on the document, each element corresponds to a metric

modality

string

modality, an identifier to the modality this document belongs to. In the scope of multi/cross modal search

evaluations

DocumentProto.EvaluationsEntry

repeated

Evaluations performed on the document, each element corresponds to a metric

DocumentProto.EvaluationsEntry

Field

Type

Label

Description

key

string

value

NamedScoreProto

DocumentProto.ScoresEntry

Field

Type

Label

Description

key

string

value

NamedScoreProto

GraphProto

Represents a Graph

Field

Type

Label

Description

adjacency

NdArrayProto

adjacency list

edge_features

google.protobuf.Struct

Container structure to store edge features

undirected

bool

Flag indicating if the graph is to be interpreted as an undirected graph

NamedScoreProto

Represents the relevance model to ref_id

Field

Type

Label

Description

value

float

value

op_name

string

the name of the operator/score function

description

string

text description of the score

operands

NamedScoreProto

repeated

the score can be nested

ref_id

string

the score is computed between doc id and ref_id

NdArrayProto

Represents a general n-dim array, can be either dense or sparse

Field

Type

Label

Description

dense

DenseNdArrayProto

dense representation of the ndarray

sparse

SparseNdArrayProto

sparse representation of the ndarray

cls_name

string

the name of the ndarray class

parameters

google.protobuf.Struct

SparseNdArrayProto

Represents a sparse ndarray

Field

Type

Label

Description

indices

DenseNdArrayProto

A 2-D int64 tensor of shape [N, ndims], which specifies the indices of the elements in the sparse tensor that contain nonzero values (elements are zero-indexed)

values

DenseNdArrayProto

A 1-D tensor of any type and shape [N], which supplies the values for each element in indices.

shape

uint32

repeated

A 1-D int64 tensor of shape [ndims], which specifies the shape of the sparse tensor.

Top

jina.proto

EnvelopeProto

Represents a Envelope, a part of the Message.

Field

Type

Label

Description

sender_id

string

unique id of the sender of the message

receiver_id

string

unique id of the receiver of the message, only used in router-dealer pattern

request_id

string

unique id of the request

timeout

uint32

timeout in second until this message is dropped

version

EnvelopeProto.VersionProto

version info

request_type

string

type of the request: DataRequest, ControlRequest

check_version

bool

check local Protobuf version on every Pod that this message flows to

compression

EnvelopeProto.CompressConfigProto

compress configuration used for request

routes

RouteProto

repeated

status info on every routes

routing_table

RoutingTableProto

the routing table contains information to the next pods

status

StatusProto

status info

header

HeaderProto

header contains meta info defined by the user, copied from Request, for lazy serialization

EnvelopeProto.CompressConfigProto

Represents a config for the compression algorithm

Field

Type

Label

Description

algorithm

string

compress algorithm used for request

min_bytes

uint64

the high watermark that triggers the message compression. message bigger than this HWM (in bytes) will be compressed by the algorithm.

min_ratio

float

the low watermark that enables the sending of a compressed message. compression rate (after_size/before_size) lower than this LWM will be considered as successeful compression, and will be sent. Otherwise, it will send the original message without compression

parameters

google.protobuf.Struct

other parameters that can be accepted by the algorithm

EnvelopeProto.VersionProto

Represents a the version information

Field

Type

Label

Description

jina

string

jina’s version

proto

string

protobuf’s version

vcs

string

vcs’s version

HeaderProto

Represents a Header.

  • The header’s content will be defined by the user request.

  • It will be copied to the envelope.header

  • In-flow operations will modify the envelope.header

  • While returning, copy envelope.header back to request.header

Field

Type

Label

Description

exec_endpoint

string

the endpoint specified by @requests(on='/abc')

target_peapod

string

if set, the request is targeted to certain peas/pods, regex strings

no_propagate

bool

if set, then this request is not propagate over the Flow topology

MessageProto

Represents a Message

Field

Type

Label

Description

envelope

EnvelopeProto

the envelope of the message, used internally in jina, dropped when returning to client

request

RequestProto

the request body

RequestProto

Represents a Request

Field

Type

Label

Description

request_id

string

the unique ID of this request. Multiple requests with the same ID will be gathered

control

RequestProto.ControlRequestProto

a control request

data

RequestProto.DataRequestProto

a data request

header

HeaderProto

header contains meta info defined by the user

parameters

google.protobuf.Struct

extra kwargs that will be used in executor

routes

RouteProto

repeated

status info on every routes

status

StatusProto

status info

RequestProto.ControlRequestProto

Represents a control request used to control the BasePod

Field

Type

Label

Description

command

RequestProto.ControlRequestProto.Command

the control command

RequestProto.DataRequestProto

Represents a general data request

Field

Type

Label

Description

docs

docarray.DocumentProto

repeated

a list of Documents to query

groundtruths

docarray.DocumentProto

repeated

a list of groundtruth Document you want to evaluate it with

RouteProto

Represents a the route paths of this message

Field

Type

Label

Description

pod

string

the name of the BasePod

pod_id

string

the id of the BasePod

start_time

google.protobuf.Timestamp

receiving time

end_time

google.protobuf.Timestamp

sending (out) time

status

StatusProto

the status of the execution

RoutingEdgeProto

Field

Type

Label

Description

pod

string

send_as_bind

bool

RoutingTableProto

Field

Type

Label

Description

pods

RoutingTableProto.PodsEntry

repeated

Pods that get visited during a Flow. Gateway should be both the first and the last entry.

active_pod

string

The currently active Pod. Needed for ZMQ.

RoutingTableProto.PodsEntry

Field

Type

Label

Description

key

string

value

TargetPodProto

StatusProto

Represents a Status

Field

Type

Label

Description

code

StatusProto.StatusCode

status code

description

string

error description of the very first exception

exception

StatusProto.ExceptionProto

the details of the error

StatusProto.ExceptionProto

Field

Type

Label

Description

name

string

the class name of the exception

args

string

repeated

the list of arguments given to the exception constructor.

stacks

string

repeated

the exception traceback stacks

executor

string

the name of the executor bind to that peapod (if applicable)

TargetPodProto

Field

Type

Label

Description

host

string

the host HeadPea of the BasePod

port

uint32

the port HeadPea of the BasePod

port_out

uint32

the port TailPea of the BasePod

expected_parts

uint32

the number of parts the pod should expect

out_edges

RoutingEdgeProto

repeated

pod_name of Pods, the TailPea should send traffic to

target_identity

string

RequestProto.ControlRequestProto.Command

Name

Number

Description

TERMINATE

0

shutdown the BasePod

STATUS

1

check the status of the BasePod

IDLE

2

used in ROUTER-DEALER pattern, tells the router that the dealer is idle

CANCEL

3

used in ROUTER-DEALER pattern, tells the router that the dealer is busy (or closed)

SCALE

4

scale up/down a Pod

ACTIVATE

5

used in ROUTER-DEALER pattern, Indicate a Pea that it can activate itself and send the IDLE command to their router

DEACTIVATE

6

used in ROUTER-DEALER pattern, Indicate a Pea that it can deactivate itself and send the CANCEL command to their router

StatusProto.StatusCode

Name

Number

Description

SUCCESS

0

success

PENDING

1

there are pending messages, more messages are followed

READY

2

ready to use

ERROR

3

error

ERROR_DUPLICATE

4

already a existing pod running

ERROR_NOTALLOWED

5

not allowed to open pod remotely

ERROR_CHAINED

6

chained from the previous error

JinaDataRequestRPC

jina gRPC service for DataRequests.

Method Name

Request Type

Response Type

Description

Call

MessageProto

.google.protobuf.Empty

Pass in a Message, wrapping a DataRequest

JinaRPC

jina gRPC service.

Method Name

Request Type

Response Type

Description

Call

RequestProto stream

RequestProto stream

Pass in a Request and a filled Request with matches will be returned.

Scalar Value Types

.proto Type

Notes

C++

Java

Python

Go

C#

PHP

Ruby

double

double

double

float

float64

double

float

Float

float

float

float

float

float32

float

float

Float

int32

Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.

int32

int

int

int32

int

integer

Bignum or Fixnum (as required)

int64

Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.

int64

long

int/long

int64

long

integer/string

Bignum

uint32

Uses variable-length encoding.

uint32

int

int/long

uint32

uint

integer

Bignum or Fixnum (as required)

uint64

Uses variable-length encoding.

uint64

long

int/long

uint64

ulong

integer/string

Bignum or Fixnum (as required)

sint32

Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.

int32

int

int

int32

int

integer

Bignum or Fixnum (as required)

sint64

Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.

int64

long

int/long

int64

long

integer/string

Bignum

fixed32

Always four bytes. More efficient than uint32 if values are often greater than 2^28.

uint32

int

int

uint32

uint

integer

Bignum or Fixnum (as required)

fixed64

Always eight bytes. More efficient than uint64 if values are often greater than 2^56.

uint64

long

int/long

uint64

ulong

integer/string

Bignum

sfixed32

Always four bytes.

int32

int

int

int32

int

integer

Bignum or Fixnum (as required)

sfixed64

Always eight bytes.

int64

long

int/long

int64

long

integer/string

Bignum

bool

bool

boolean

boolean

bool

bool

boolean

TrueClass/FalseClass

string

A string must always contain UTF-8 encoded or 7-bit ASCII text.

string

String

str/unicode

string

string

string

String (UTF-8)

bytes

May contain any arbitrary sequence of bytes.

string

ByteString

str

[]byte

ByteString

string

String (ASCII-8BIT)