Protocol Documentation¶

Table of Contents¶

jina.proto¶

DenseNdArrayProto¶

Represents a (quantized) dense n-dim array

Field	Type	Label	Description
buffer	bytes		the actual array data, in bytes
shape	uint32	repeated	the shape (dimensions) of the array
dtype	string		the data type of the array
quantization	DenseNdArrayProto.QuantizationMode		quantization mode
max_val	float		the max value of the ndarray
min_val	float		the min value of the ndarray
scale	float		the scale of the ndarray
original_dtype	string		the original dtype of the array

DocumentArrayProto¶

Field	Type	Label	Description
docs	DocumentProto	repeated	a list of Documents

DocumentProto¶

Represents a Document

Field	Type	Label	Description
id	string		A hexdigest that represents a unique document ID
granularity	uint32		the depth of the recursive chunk structure
adjacency	uint32		the width of the recursive match structure
parent_id	string		the parent id from the previous granularity
buffer	bytes		the raw binary content of this document, which often represents the original document when comes into jina
blob	NdArrayProto		the ndarray of the image/audio/video document
text	string		a text document
graph	GraphProto		Related information to be used when a Document represents a graph with its chunks as nodes
chunks	DocumentProto	repeated	list of the sub-documents of this document (recursive structure)
weight	float		The weight of this document
matches	DocumentProto	repeated	the matched documents on the same level (recursive structure)
uri	string		a uri of the document could be: a local file path, a remote url starts with http or https or data URI scheme
mime_type	string		mime type of this document, for buffer content, this is required; for other contents, this can be guessed
tags	google.protobuf.Struct		a structured data value, consisting of field which map to dynamically typed values.
location	uint32	repeated	the position of the doc, could be start and end index of a string; could be x,y (top, left) coordinate of an image crop; could be timestamp of an audio clip
offset	uint32		the offset of this doc in the previous granularity document
embedding	NdArrayProto		the embedding `ndarray` of this document
scores	DocumentProto.ScoresEntry	repeated	Scores performed on the document, each element corresponds to a metric
modality	string		modality, an identifier to the modality this document belongs to. In the scope of multi/cross modal search
evaluations	DocumentProto.EvaluationsEntry	repeated	Evaluations performed on the document, each element corresponds to a metric

DocumentProto.EvaluationsEntry¶

Field	Type	Label	Description
key	string
value	NamedScoreProto

DocumentProto.ScoresEntry¶

Field	Type	Label	Description
key	string
value	NamedScoreProto

EnvelopeProto¶

Represents a Envelope, a part of the Message.

Field	Type	Label	Description
sender_id	string		unique id of the sender of the message
receiver_id	string		unique id of the receiver of the message, only used in router-dealer pattern
request_id	string		unique id of the request
timeout	uint32		timeout in second until this message is dropped
version	EnvelopeProto.VersionProto		version info
request_type	string		type of the request: DataRequest, ControlRequest
check_version	bool		check local Protobuf version on every Pod that this message flows to
compression	EnvelopeProto.CompressConfigProto		compress configuration used for request
routes	RouteProto	repeated	status info on every routes
routing_table	RoutingTableProto		the routing table contains information to the next pods
status	StatusProto		status info
header	HeaderProto		header contains meta info defined by the user, copied from Request, for lazy serialization

EnvelopeProto.CompressConfigProto¶

Represents a config for the compression algorithm

Field	Type	Description
algorithm	string	compress algorithm used for request
min_bytes	uint64	the high watermark that triggers the message compression. message bigger than this HWM (in bytes) will be compressed by the algorithm.
min_ratio	float	the low watermark that enables the sending of a compressed message. compression rate (after_size/before_size) lower than this LWM will be considered as successeful compression, and will be sent. Otherwise, it will send the original message without compression
parameters	google.protobuf.Struct	other parameters that can be accepted by the algorithm

EnvelopeProto.VersionProto¶

Represents a the version information

Field	Type	Description
jina	string	jina's version
proto	string	protobuf's version
vcs	string	vcs's version

GraphProto¶

Represents a Graph

Field	Type	Description
adjacency	SparseNdArrayProto	adjacency list
edge_features	google.protobuf.Struct	Container structure to store edge features
undirected	bool	Flag indicating if the graph is to be interpreted as an undirected graph

HeaderProto¶

Represents a Header.

The header’s content will be defined by the user request.
It will be copied to the envelope.header
In-flow operations will modify the envelope.header
While returning, copy envelope.header back to request.header

Field	Type	Description
exec_endpoint	string	the endpoint specified by `@requests(on='/abc')`
target_peapod	string	if set, the request is targeted to certain peas/pods, regex strings
no_propagate	bool	if set, then this request is not propagate over the Flow topology

MessageProto¶

Represents a Message

Field	Type	Label	Description
envelope	EnvelopeProto		the envelope of the message, used internally in jina, dropped when returning to client
request	RequestProto		the request body

NamedScoreProto¶

Represents the relevance model to ref_id

Field	Type	Label	Description
value	float		value
op_name	string		the name of the operator/score function
description	string		text description of the score
operands	NamedScoreProto	repeated	the score can be nested
ref_id	string		the score is computed between doc `id` and `ref_id`

NdArrayProto¶

Represents a general n-dim array, can be either dense or sparse

Field	Type	Label	Description
dense	DenseNdArrayProto		dense representation of the ndarray
sparse	SparseNdArrayProto		sparse representation of the ndarray

RequestProto¶

Represents a Request

Field	Type	Label	Description
request_id	string		the unique ID of this request. Multiple requests with the same ID will be gathered
control	RequestProto.ControlRequestProto		a control request
data	RequestProto.DataRequestProto		a data request
header	HeaderProto		header contains meta info defined by the user
parameters	google.protobuf.Struct		extra kwargs that will be used in executor
routes	RouteProto	repeated	status info on every routes
status	StatusProto		status info

RequestProto.ControlRequestProto¶

Represents a control request used to control the BasePod

Field	Type	Label	Description
command	RequestProto.ControlRequestProto.Command		the control command

RequestProto.DataRequestProto¶

Represents a general data request

Field	Type	Label	Description
docs	DocumentProto	repeated	a list of Documents to query
groundtruths	DocumentProto	repeated	a list of groundtruth Document you want to evaluate it with

RouteProto¶

Represents a the route paths of this message

Field	Type	Description
pod	string	the name of the BasePod
pod_id	string	the id of the BasePod
start_time	google.protobuf.Timestamp	receiving time
end_time	google.protobuf.Timestamp	sending (out) time
status	StatusProto	the status of the execution

RoutingEdgeProto¶

Field	Type	Label	Description
pod	string
send_as_bind	bool

RoutingTableProto¶

Field	Type	Label	Description
pods	RoutingTableProto.PodsEntry	repeated	Pods that get visited during a Flow. Gateway should be both the first and the last entry.
active_pod	string		The currently active Pod. Needed for ZMQ.

RoutingTableProto.PodsEntry¶

Field	Type	Label	Description
key	string
value	TargetPodProto

SparseNdArrayProto¶

Represents a sparse ndarray

Field	Type	Label	Description
indices	DenseNdArrayProto		A 2-D int64 tensor of shape [N, ndims], which specifies the indices of the elements in the sparse tensor that contain nonzero values (elements are zero-indexed)
values	DenseNdArrayProto		A 1-D tensor of any type and shape [N], which supplies the values for each element in indices.
shape	uint32	repeated	A 1-D int64 tensor of shape [ndims], which specifies the shape of the sparse tensor.

StatusProto¶

Represents a Status

Field	Type	Description
code	StatusProto.StatusCode	status code
description	string	error description of the very first exception
exception	StatusProto.ExceptionProto	the details of the error

StatusProto.ExceptionProto¶

Field	Type	Label	Description
name	string		the class name of the exception
args	string	repeated	the list of arguments given to the exception constructor.
stacks	string	repeated	the exception traceback stacks
executor	string		the name of the executor bind to that peapod (if applicable)

TargetPodProto¶

Field	Type	Label	Description
host	string		the host HeadPea of the BasePod
port	uint32		the port HeadPea of the BasePod
port_out	uint32		the port TailPea of the BasePod
expected_parts	uint32		the number of parts the pod should expect
out_edges	RoutingEdgeProto	repeated	pod_name of Pods, the TailPea should send traffic to
target_identity	string

DenseNdArrayProto.QuantizationMode¶

Name	Number	Description
NONE	0	no quantization is performed, stored in the original `dtype`
FP16	1	4x smaller if original dtype FP64, lossy
UINT8	2	4x smaller if original dtype FP32, lossy
FP32	3	2x smaller if dtype is set to FP64

RequestProto.ControlRequestProto.Command¶

Name	Number	Description
TERMINATE	0	shutdown the BasePod
STATUS	1	check the status of the BasePod
IDLE	2	used in ROUTER-DEALER pattern, tells the router that the dealer is idle
CANCEL	3	used in ROUTER-DEALER pattern, tells the router that the dealer is busy (or closed)
SCALE	4	scale up/down a Pod
ACTIVATE	5	used in ROUTER-DEALER pattern, Indicate a Pea that it can activate itself and send the IDLE command to their router
DEACTIVATE	6	used in ROUTER-DEALER pattern, Indicate a Pea that it can deactivate itself and send the CANCEL command to their router

StatusProto.StatusCode¶

Name	Number	Description
SUCCESS	0	success
PENDING	1	there are pending messages, more messages are followed
READY	2	ready to use
ERROR	3	error
ERROR_DUPLICATE	4	already a existing pod running
ERROR_NOTALLOWED	5	not allowed to open pod remotely
ERROR_CHAINED	6	chained from the previous error

JinaDataRequestRPC¶

jina gRPC service for DataRequests.

Method Name	Request Type	Response Type	Description
Call	MessageProto	.google.protobuf.Empty	Pass in a Message, wrapping a DataRequest

JinaRPC¶

jina gRPC service.

Method Name	Request Type	Response Type	Description
Call	RequestProto stream	RequestProto stream	Pass in a Request and a filled Request with matches will be returned.

Scalar Value Types¶

.proto Type	Notes	C++	Java	Python	Go	C#	PHP	Ruby
double		double	double	float	float64	double	float	Float
float		float	float	float	float32	float	float	Float
int32	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
int64	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.	int64	long	int/long	int64	long	integer/string	Bignum
uint32	Uses variable-length encoding.	uint32	int	int/long	uint32	uint	integer	Bignum or Fixnum (as required)
uint64	Uses variable-length encoding.	uint64	long	int/long	uint64	ulong	integer/string	Bignum or Fixnum (as required)
sint32	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sint64	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.	int64	long	int/long	int64	long	integer/string	Bignum
fixed32	Always four bytes. More efficient than uint32 if values are often greater than 2^28.	uint32	int	int	uint32	uint	integer	Bignum or Fixnum (as required)
fixed64	Always eight bytes. More efficient than uint64 if values are often greater than 2^56.	uint64	long	int/long	uint64	ulong	integer/string	Bignum
sfixed32	Always four bytes.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sfixed64	Always eight bytes.	int64	long	int/long	int64	long	integer/string	Bignum
bool		bool	boolean	boolean	bool	bool	boolean	TrueClass/FalseClass
string	A string must always contain UTF-8 encoded or 7-bit ASCII text.	string	String	str/unicode	string	string	string	String (UTF-8)
bytes	May contain any arbitrary sequence of bytes.	string	ByteString	str	[]byte	ByteString	string	String (ASCII-8BIT)