Gateway#

Every Flow has a Gateway component that receives requests over the network, allowing clients to send data to the Flow for processing.

The Gateway is the first destination of a client request and its final destination, meaning that all incoming requests are routed to the Gateway and the Gateway is responsible for handling and responding to those requests. The Gateway supports multiple protocols and endpoints, such as gRPC, HTTP, WebSocket, and GraphQL, allowing clients to communicate with the Flow using the protocol of their choice.

In most cases, the Gateway is automatically configured when you initialize a Flow object, so you do not need to configure it yourself.

However, you can always explicitly configure the Gateway in Python using the config_gateway() method, or in YAML. The full YAML specification for configuring the Gateway can be found here.

Set protocol in Python#

You can use three different protocols to serve the Flow: gRPC, HTTP and Websocket.

from docarray import Document, DocumentArray
from jina import Client, Executor, Flow, requests


class FooExecutor(Executor):
    @requests
    def foo(self, docs: DocumentArray, **kwargs):
        docs.append(Document(text='foo was called'))


f = Flow().config_gateway(protocol='grpc', port=12345).add(uses=FooExecutor)
with f:
    client = Client(port=12345)
    docs = client.post(on='/')
    print(docs.texts)

['foo was called']

from docarray import Document, DocumentArray
from jina import Client, Executor, Flow, requests


class FooExecutor(Executor):
    @requests
    def foo(self, docs: DocumentArray, **kwargs):
        docs.append(Document(text='foo was called'))


f = Flow().config_gateway(protocol='http', port=12345).add(uses=FooExecutor)
with f:
    client = Client(port=12345, protocol='http')
    docs = client.post(on='/')
    print(docs.texts)

['foo was called']

from docarray import Document, DocumentArray
from jina import Client, Executor, Flow, requests


class FooExecutor(Executor):
    @requests
    def foo(self, docs: DocumentArray, **kwargs):
        docs.append(Document(text='foo was called'))


f = Flow().config_gateway(protocol='websocket', port=12345).add(uses=FooExecutor)
with f:
    client = Client(port=12345, protocol='websocket')
    docs = client.post(on='/')
    print(docs.texts)

['foo was called']

Set protocol in YAML#

To configure the protocol in a YAML file:

Note that gRPC is the default protocol, so you can just omit it.

jtype: Flow
gateway:
  protocol: 'grpc'

jtype: Flow
gateway:
  protocol: 'http'

jtype: Flow
gateway:
  protocol: 'websocket'

Enable multiple protocols#

You can enable multiple protocols on the Gateway. This allows polyglot clients connect to your Flow with different protocols.

from jina import Flow
flow = Flow().config_gateway(protocol=['grpc', 'http', 'websocket'])
with flow:
    flow.block()

jtype: Flow
gateway:
  protocol:
    - 'grpc'
    - 'http'
    - 'websocket'
../../_images/multi-protocol-flow.png

Important

In case you want to serve a Flow using multiple protocols, make sure to specify as much ports as protocols used.

Enable TLS for client traffics#

You can enable TLS encryption between your Gateway and Clients, for any of the protocols supported by Jina (HTTP, gRPC, and Websocket).

Caution

Enabling TLS will encrypt the data that is transferred between the Flow and the Client. Data that is passed between the microservices configured by the Flow, such as Executors, will not be encrypted.

To enable TLS encryption, you need to pass a valid keyfile and certfile to the Flow, using the ssl_keyfile ssl_certfile parameters:

from jina import Flow

Flow().config_gateway(
    port=12345,
    ssl_certfile='path/to/certfile.crt',
    ssl_keyfile='path/to/keyfile.crt',
)

If both of these are provided, the Flow will automatically configure itself to use TLS encryption for its communication with any Client.

Enable in-Flow compression#

The communication between Executors inside a Flow is done via gRPC. To optimize the performance and the bandwidth of these connections, you can enable compression by specifying compression argument to the Gateway.

The supported methods are: none, gzip and deflate.

from jina import Flow

f = Flow().config_gateway(compression='gzip').add(...)

Note that this setting is only effective the internal communication of the Flow. One can also specify the compression between client and gateway as described here.

Get environment information#

Gateway provides an endpoint that exposes environment information where it runs.

It is a dict-like structure with the following keys:

  • jina: A dictionary containing information about the system and the versions of several packages including jina package itself

  • envs: A dictionary containing all the values if set of the environment variables used in Jina

Use gRPC#

To see how this works, first instantiate a Flow with an Executor exposed to a specific port and block it for serving:

from jina import Flow

with Flow().config_gateway(protocol=['grpc'], port=12345) as f:
    f.block()

Then, you can use grpcurl sending status check request to the Gateway.

docker pull fullstorydev/grpcurl:latest
docker run --network='host' fullstorydev/grpcurl -plaintext 127.0.0.1:12345 jina.JinaInfoRPC/_status

The error-free output below signifies a correctly running Gateway:

{
  "jina": {
    "architecture": "######",
    "ci-vendor": "######",
    "docarray": "######",
    "grpcio": "######",
    "jina": "######",
    "jina-proto": "######",
    "jina-vcs-tag": "######",
    "platform": "######",
    "platform-release": "######",
    "platform-version": "######",
    "processor": "######",
    "proto-backend": "######",
    "protobuf": "######",
    "python": "######",
    "pyyaml": "######",
    "session-id": "######",
    "uid": "######",
    "uptime": "######"
  },
  "envs": {
    "JINA_AUTH_TOKEN": "(unset)",
    "JINA_DEFAULT_HOST": "(unset)",
    "JINA_DEFAULT_TIMEOUT_CTRL": "(unset)",
    "JINA_DEPLOYMENT_NAME": "(unset)",
    "JINA_DISABLE_HEALTHCHECK_LOGS": "(unset)",
    "JINA_DISABLE_UVLOOP": "(unset)",
    "JINA_EARLY_STOP": "(unset)",
    "JINA_FULL_CLI": "(unset)",
    "JINA_GATEWAY_IMAGE": "(unset)",
    "JINA_GRPC_RECV_BYTES": "(unset)",
    "JINA_GRPC_SEND_BYTES": "(unset)",
    "JINA_HUBBLE_REGISTRY": "(unset)",
    "JINA_HUB_NO_IMAGE_REBUILD": "(unset)",
    "JINA_LOCKS_ROOT": "(unset)",
    "JINA_LOG_CONFIG": "(unset)",
    "JINA_LOG_LEVEL": "(unset)",
    "JINA_LOG_NO_COLOR": "(unset)",
    "JINA_MP_START_METHOD": "(unset)",
    "JINA_RANDOM_PORT_MAX": "(unset)",
    "JINA_RANDOM_PORT_MIN": "(unset)"
  }
}

Tip

You can also use it to check Executor status, as Executor’s communication protocol is gRPC.

Use HTTP/Websocket#

When using HTTP or Websocket as the Gateway protocol, you can use curl to target the /status endpoint and get the Jina info.

curl http://localhost:12345/status

{
  "jina": {
    "jina": "######",
    "docarray": "######",
    "jina-proto": "######",
    "jina-vcs-tag": "(unset)",
    "protobuf": "######",
    "proto-backend": "######",
    "grpcio": "######",
    "pyyaml": "######",
    "python": "######",
    "platform": "######",
    "platform-release": "######",
    "platform-version": "######",
    "architecture": "######",
    "processor": "######",
    "uid": "######",
    "session-id": "######",
    "uptime": "######",
    "ci-vendor": "(unset)"
  },
  "envs": {
    "JINA_AUTH_TOKEN": "(unset)",
    "JINA_DEFAULT_HOST": "(unset)",
    "JINA_DEFAULT_TIMEOUT_CTRL": "(unset)",
    "JINA_DEPLOYMENT_NAME": "(unset)",
    "JINA_DISABLE_UVLOOP": "(unset)",
    "JINA_EARLY_STOP": "(unset)",
    "JINA_FULL_CLI": "(unset)",
    "JINA_GATEWAY_IMAGE": "(unset)",
    "JINA_GRPC_RECV_BYTES": "(unset)",
    "JINA_GRPC_SEND_BYTES": "(unset)",
    "JINA_HUBBLE_REGISTRY": "(unset)",
    "JINA_HUB_NO_IMAGE_REBUILD": "(unset)",
    "JINA_LOG_CONFIG": "(unset)",
    "JINA_LOG_LEVEL": "(unset)",
    "JINA_LOG_NO_COLOR": "(unset)",
    "JINA_MP_START_METHOD": "(unset)",
    "JINA_RANDOM_PORT_MAX": "(unset)",
    "JINA_RANDOM_PORT_MIN": "(unset)",
    "JINA_DISABLE_HEALTHCHECK_LOGS": "(unset)",
    "JINA_LOCKS_ROOT": "(unset)"
  }
}

See further#