jina gateway

Start a Jina gateway that receives client remote requests via gRPC

usage: jina gateway [-h] [--name] [--identity] [--yaml-path]
                    [--py-modules [[...]]] [--image] [--entrypoint]
                    [--pull-latest] [--volumes [[...]]] [--port-in]
                    [--port-out] [--host-in] [--host-out]
                    [--socket-in {PULL_BIND, PULL_CONNECT, PUSH_BIND, PUSH_CONNECT ... 8 more choices}]
                    [--socket-out {PULL_BIND, PULL_CONNECT, PUSH_BIND, PUSH_CONNECT ... 8 more choices}]
                    [--port-ctrl] [--ctrl-with-ipc] [--timeout]
                    [--timeout-ctrl] [--timeout-ready] [--dump-interval]
                    [--exit-no-dump] [--read-only] [--separated-workspace]
                    [--replica-id] [--check-version] [--array-in-pb]
                    [--compress-hwm] [--compress-lwm]
                    [--role {REPLICA, HEAD, TAIL, SHARD ... 1 more choices}]
                    [--skip-on-error {NONE, EXECUTOR, DRIVER, HANDLE ... 1 more choices}]
                    [--memory-hwm] [--runtime {thread, process}]
                    [--max-idle-time] [--log-sse] [--log-remote]
                    [--log-profile] [--log-with-own-name] [--host]
                    [--port-expose] [--max-message-size] [--proxy]
                    [--prefetch] [--prefetch-on-recv] [--allow-spawn]
                    [--rest-api]

pea basic arguments

--name

the name of this pea, used to identify the pod and its logs.

Default: “gateway”

--identity

the identity of the sockets, default a random string

Default: “f532e55ac7”

--yaml-path

the yaml config of the executor, it could be > a YAML file path, > a supported executor’s class name, > one of “_clear”, “_route”, “_forward”, “_logforward”, “_merge” > the content of YAML config (must starts with “!”)

Default: “BaseExecutor”

--py-modules

the customized python modules need to be imported before loading the executor

pea container arguments

--image

the name of the docker image that this pea runs with. it is also served as an indicator of containerization. when this and –yaml-path are both given then the docker image is used and its original yaml configuration is replaced by the given –yaml-path

--entrypoint

the entrypoint command overrides the ENTRYPOINT in docker image. when not set then the docker image ENTRYPOINT takes effective.

--pull-latest

pull the latest image before running

Default: False

--volumes

the path on the host to be mounted inside the container. they will be mounted to the root path, i.e. /user/test/my-workspace will be mounted to /my-workspace inside the container. all volumes are mounted with read-write mode.

pea network arguments

--port-in

port for input data, default a random port between [49152, 65535]

Default: 59327

--port-out

port for output data, default a random port between [49152, 65535]

Default: 51641

--host-in

host address for input, by default it is 0.0.0.0

Default: “0.0.0.0”

--host-out

host address for output, by default it is 0.0.0.0

Default: “0.0.0.0”

--socket-in

Possible choices: PULL_BIND, PULL_CONNECT, PUSH_BIND, PUSH_CONNECT, SUB_BIND, SUB_CONNECT, PUB_BIND, PUB_CONNECT, PAIR_BIND, PAIR_CONNECT, ROUTER_BIND, DEALER_CONNECT

socket type for input port

Default: 1

--socket-out

Possible choices: PULL_BIND, PULL_CONNECT, PUSH_BIND, PUSH_CONNECT, SUB_BIND, SUB_CONNECT, PUB_BIND, PUB_CONNECT, PAIR_BIND, PAIR_CONNECT, ROUTER_BIND, DEALER_CONNECT

socket type for output port

Default: 3

--port-ctrl

port for controlling the pod, default a random port between [49152, 65535]

Default: 44473

--ctrl-with-ipc

use ipc protocol for control socket

Default: True

--timeout

timeout (ms) of all requests, -1 for waiting forever

Default: -1

--timeout-ctrl

timeout (ms) of the control request, -1 for waiting forever

Default: 5000

--timeout-ready

timeout (ms) of a pea is ready for request, -1 for waiting forever

Default: 10000

pea IO arguments

--dump-interval

serialize the model in the pod every n seconds if model changes. -1 means –read-only.

Default: 240

--exit-no-dump

do not serialize the model when the pod exits

Default: False

--read-only

do not allow the pod to modify the model, dump_interval will be ignored

Default: True

--separated-workspace

the data and config files are separated for each pea in this pod, only effective when BasePod’s replicas > 1

Default: False

--replica-id

the id of the storage of this replica, only effective when separated_workspace=True

Default: -1

pea messaging arguments

--check-version

comparing the jina and proto version of incoming message with local setup, mismatch raise an exception

Default: False

--array-in-pb

sending buffer and numpy ndarray together within or separately from the protobuf message, the latter often yields a better network efficiency

Default: False

--compress-hwm

the high watermark that triggers the message compression. message bigger than this HWM (in bytes) will be compressed by lz4 algorithm.set this to -1 to disable this feature.

Default: -1

--compress-lwm

the low watermark that enables the sending of a compressed message. compression rate (after_size/before_size) lower than this LWM will be considered as successeful compression, and will be sent. Otherwise, it will send the original message without compression

Default: 1.0

--role

Possible choices: REPLICA, HEAD, TAIL, SHARD, SINGLETON

the role of this pea in a pod

--skip-on-error

Possible choices: NONE, EXECUTOR, DRIVER, HANDLE, CALLBACK

skip strategy on error message.

Default: 0

pea EXPERIMENTAL arguments

--memory-hwm

memory high watermark of this pod in Gigabytes, pod will restart when this is reached. -1 means no restriction

Default: -1

--runtime

Possible choices: thread, process

the parallel runtime of the pod

Default: “process”

--max-idle-time

label this pea as inactive when it does not process any request after certain time (in second)

Default: 60

logging arguments

--log-sse

turn on server-side event logging

Default: False

--log-remote

turn on remote logging

Default: False

--log-profile

turn on the profiling logger

Default: False

--log-with-own-name

turn on to let each logger outputs in its own name (i.e. parent class name as the context), by default it is off so all logs from the same pod will have the same prefix. turn on to help debugging, turn off to have more clear logs and better grouping in dashboard

Default: False

grpc and remote arguments

--host

host address of the pea/gateway, by default it is 0.0.0.0.

Default: “0.0.0.0”

--port-expose, --port-grpc

host port of the gateway, “port-grpc” alias will be removed in future versions

Default: 54183

--max-message-size

maximum send and receive size for grpc server in bytes, -1 means unlimited

Default: -1

--proxy

respect the http_proxy and https_proxy environment variables. otherwise, it will unset these proxy variables before start. gRPC seems to prefer –no-proxy

Default: False

gateway arguments

--prefetch

the number of pre-fetched requests from the client

Default: 50

--prefetch-on-recv

the number of additional requests to fetch on every receive

Default: 1

--allow-spawn

accept the spawn requests sent from other remote Jina

Default: False

--rest-api

use REST-API as the interface instead of gRPC with port number set to the value of “port-expose”

Default: False