Protocol Documentation#

Table of Contents#

Top

docarray.proto#

DocListProto#

this file is just a placeholder for the DA coming from jina._docarray dependency

DocProto#

Top

jina.proto#

DataRequestListProto#

Represents a list of data requests This should be replaced by streaming

Field

Type

Label

Description

requests

DataRequestProto

repeated

requests in this list

DataRequestProto#

Represents a DataRequest

Field

Type

Label

Description

header

HeaderProto

header contains meta info defined by the user

parameters

google.protobuf.Struct

extra kwargs that will be used in executor

routes

RouteProto

repeated

status info on every routes

data

DataRequestProto.DataContentProto

container for docs and groundtruths

DataRequestProto.DataContentProto#

Field

Type

Label

Description

docs

docarray.DocListProto

the docs in this request

docs_bytes

bytes

the docs in this request as bytes

DataRequestProtoWoData#

Field

Type

Label

Description

header

HeaderProto

header contains meta info defined by the user

parameters

google.protobuf.Struct

extra kwargs that will be used in executor

routes

RouteProto

repeated

status info on every routes

EndpointsProto#

Represents the set of Endpoints exposed by an Executor

Field

Type

Label

Description

endpoints

string

repeated

list of endpoints exposed by an Executor

write_endpoints

string

repeated

schemas

google.protobuf.Struct

Dictionary containing input and output schema per endpoint

extra kwargs that will be used in executor |

HeaderProto#

Represents a Header.

  • The header’s content will be defined by the user request.

  • It will be copied to the envelope.header

  • In-flow operations will modify the envelope.header

  • While returning, copy envelope.header back to request.header

Field

Type

Label

Description

request_id

string

the unique ID of this request. Multiple requests with the same ID will be gathered

status

StatusProto

status info

exec_endpoint

string

optional

the endpoint specified by @requests(on='/abc')

target_executor

string

optional

if set, the request is targeted to certain executor, regex strings

timeout

uint32

optional

epoch time in seconds after which the request should be dropped

JinaInfoProto#

Field

Type

Label

Description

jina

JinaInfoProto.JinaEntry

repeated

information about the system running and package version information including jina

envs

JinaInfoProto.EnvsEntry

repeated

the environment variable setting

JinaInfoProto.EnvsEntry#

Field

Type

Label

Description

key

string

value

string

JinaInfoProto.JinaEntry#

Field

Type

Label

Description

key

string

value

string

RelatedEntity#

Represents an entity (like an ExecutorRuntime)

Field

Type

Label

Description

id

string

unique id of the entity, like the name of a pod

address

string

address of the entity, could be an IP address, domain name etc, does not include port

port

uint32

port this entity is listening on

shard_id

uint32

optional

the id of the shard it belongs to, if it is a shard

RestoreId#

An ID representation for the Restore Job

Field

Type

Label

Description

value

string

RestoreSnapshotCommand#

Commands to restore an Executor from a snapshot file

Field

Type

Label

Description

snapshot_file

string

snapshot file

RestoreSnapshotStatusProto#

Represents the status of a Restore.

Field

Type

Label

Description

id

RestoreId

An ID for the Restore Job.

status

RestoreSnapshotStatusProto.Status

status code

RouteProto#

Represents a the route paths of this message as perceived by the Gateway start_time is set when the Gateway sends a message to a Pod end_time is set when the Gateway receives a message from a Pod thus end_time - start_time includes Executor computation, runtime overhead, serialization and network

Field

Type

Label

Description

executor

string

the name of the BasePod

start_time

google.protobuf.Timestamp

time when the Gateway starts sending to the Pod

end_time

google.protobuf.Timestamp

time when the Gateway received it from the Pod

status

StatusProto

the status of the execution

SingleDocumentRequestProto#

Represents a Single Document DataRequest

Field

Type

Label

Description

header

HeaderProto

header contains meta info defined by the user

parameters

google.protobuf.Struct

extra kwargs that will be used in executor

routes

RouteProto

repeated

status info on every routes

document

docarray.DocProto

the document in this request

SnapshotId#

An ID representation for the Snapshot Job

Field

Type

Label

Description

value

string

SnapshotStatusProto#

Represents the status of a Snapshot.

Field

Type

Label

Description

id

SnapshotId

An ID for the Snapshot Job.

status

SnapshotStatusProto.Status

status code

snapshot_file

string

snapshot file

StatusProto#

Represents a Status

Field

Type

Label

Description

code

StatusProto.StatusCode

status code

description

string

error description of the very first exception

exception

StatusProto.ExceptionProto

the details of the error

StatusProto.ExceptionProto#

Field

Type

Label

Description

name

string

the class name of the exception

args

string

repeated

the list of arguments given to the exception constructor.

stacks

string

repeated

the exception traceback stacks

executor

string

the name of the executor bind to that Executor (if applicable)

RestoreSnapshotStatusProto.Status#

Name

Number

Description

STATE_UNSPECIFIED

0

RUNNING

1

Resource allocation has been successful. Restore Job is RUNNING.

SUCCEEDED

2

Resource Job has finished successfully.

FAILED

3

Resource Job has failed.

NOT_FOUND

6

If the specific restore id is unknown or not found

SnapshotStatusProto.Status#

Name

Number

Description

STATE_UNSPECIFIED

0

QUEUED

1

Snapshot Job is admitted (validated and persisted) and waiting for resources.

SCHEDULED

2

Snapshot Job is scheduled to run as soon as resource allocation is ready.

RUNNING

3

Resource allocation has been successful. Snapshot Job is RUNNING.

SUCCEEDED

4

Snapshot Job has finished successfully.

FAILED

5

Snapshot Job has failed.

NOT_FOUND

6

If the specific snapshot id is unknown or not found

StatusProto.StatusCode#

Name

Number

Description

SUCCESS

0

success

ERROR

1

error

JinaDataRequestRPC#

jina gRPC service for DataRequests.

Method Name

Request Type

Response Type

Description

process_data

DataRequestListProto

DataRequestProto

Used for passing DataRequests to the Executors

JinaDiscoverEndpointsRPC#

jina gRPC service to expose Endpoints from Executors.

Method Name

Request Type

Response Type

Description

endpoint_discovery

.google.protobuf.Empty

EndpointsProto

JinaExecutorRestore#

jina gRPC service to trigger a restore at the Executor Runtime.

Method Name

Request Type

Response Type

Description

restore

RestoreSnapshotCommand

RestoreSnapshotStatusProto

JinaExecutorRestoreProgress#

jina gRPC service to trigger a snapshot at the Executor Runtime.

Method Name

Request Type

Response Type

Description

restore_status

RestoreId

RestoreSnapshotStatusProto

JinaExecutorSnapshot#

jina gRPC service to trigger a snapshot at the Executor Runtime.

Method Name

Request Type

Response Type

Description

snapshot

.google.protobuf.Empty

SnapshotStatusProto

JinaExecutorSnapshotProgress#

jina gRPC service to trigger a snapshot at the Executor Runtime.

Method Name

Request Type

Response Type

Description

snapshot_status

SnapshotId

SnapshotStatusProto

JinaGatewayDryRunRPC#

jina gRPC service to expose Endpoints from Executors.

Method Name

Request Type

Response Type

Description

dry_run

.google.protobuf.Empty

StatusProto

JinaInfoRPC#

jina gRPC service to expose information about running jina version and environment.

Method Name

Request Type

Response Type

Description

_status

.google.protobuf.Empty

JinaInfoProto

JinaRPC#

jina streaming gRPC service.

Method Name

Request Type

Response Type

Description

Call

DataRequestProto stream

DataRequestProto stream

Pass in a Request and a filled Request with matches will be returned.

JinaSingleDataRequestRPC#

jina gRPC service for DataRequests. This is used to send requests to Executors when a list of requests is not needed

Method Name

Request Type

Response Type

Description

process_single_data

DataRequestProto

DataRequestProto

Used for passing DataRequests to the Executors

JinaSingleDocumentRequestRPC#

jina gRPC service for DataRequests. This is used to send requests to Executors when a list of requests is not needed

Method Name

Request Type

Response Type

Description

stream_doc

SingleDocumentRequestProto

SingleDocumentRequestProto stream

Used for streaming one document to the Executors

Scalar Value Types#

.proto Type

Notes

C++

Java

Python

Go

C#

PHP

Ruby

double

double

double

float

float64

double

float

Float

float

float

float

float

float32

float

float

Float

int32

Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.

int32

int

int

int32

int

integer

Bignum or Fixnum (as required)

int64

Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.

int64

long

int/long

int64

long

integer/string

Bignum

uint32

Uses variable-length encoding.

uint32

int

int/long

uint32

uint

integer

Bignum or Fixnum (as required)

uint64

Uses variable-length encoding.

uint64

long

int/long

uint64

ulong

integer/string

Bignum or Fixnum (as required)

sint32

Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.

int32

int

int

int32

int

integer

Bignum or Fixnum (as required)

sint64

Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.

int64

long

int/long

int64

long

integer/string

Bignum

fixed32

Always four bytes. More efficient than uint32 if values are often greater than 2^28.

uint32

int

int

uint32

uint

integer

Bignum or Fixnum (as required)

fixed64

Always eight bytes. More efficient than uint64 if values are often greater than 2^56.

uint64

long

int/long

uint64

ulong

integer/string

Bignum

sfixed32

Always four bytes.

int32

int

int

int32

int

integer

Bignum or Fixnum (as required)

sfixed64

Always eight bytes.

int64

long

int/long

int64

long

integer/string

Bignum

bool

bool

boolean

boolean

bool

bool

boolean

TrueClass/FalseClass

string

A string must always contain UTF-8 encoded or 7-bit ASCII text.

string

String

str/unicode

string

string

string

String (UTF-8)

bytes

May contain any arbitrary sequence of bytes.

string

ByteString

str

[]byte

ByteString

string

String (ASCII-8BIT)