jina.serve.runtimes.gateway.grpc.gateway module#

class jina.serve.runtimes.gateway.grpc.gateway.GRPCGateway(grpc_server_options=None, ssl_keyfile=None, ssl_certfile=None, **kwargs)[source]#

Bases: BaseGateway

GRPC Gateway implementation

Initialize the gateway :type grpc_server_options: Optional[dict] :param grpc_server_options: Dictionary of kwargs arguments that will be passed to the grpc server as options when starting the server, example : {‘grpc.max_send_message_length’: -1} :type ssl_keyfile: Optional[str] :param ssl_keyfile: the path to the key file :type ssl_certfile: Optional[str] :param ssl_certfile: the path to the certificate file :param kwargs: keyword args

async setup_server()[source]#

setup GRPC server

async shutdown()[source]#

Free other resources allocated with the server, e.g, gateway object, …

async run_server()[source]#

Run GRPC server forever

async dry_run(empty, context)[source]#

Process the call requested by having a dry run call to every Executor in the graph

Parameters:

  • empty – The service expects an empty protobuf message

  • context – grpc context

Return type:

StatusProto

Returns:

the response request

async stream(request_iterator, context=None, *args, **kwargs)[source]#

stream requests from client iterator and stream responses back.

Parameters:

  • request_iterator – iterator of requests

  • context – context of the grpc call

  • args – positional arguments

  • kwargs – keyword arguments

Yield:

responses to the request after streaming to Executors in Flow

Return type:

AsyncIterator[Request]

async process_single_data(request, context=None)[source]#

Implements request and response handling of a single DataRequest :type request: DataRequest :param request: DataRequest from Client :param context: grpc context :rtype: DataRequest :return: response DataRequest

async Call(request_iterator, context=None, *args, **kwargs)#

stream requests from client iterator and stream responses back.

Parameters:

  • request_iterator – iterator of requests

  • context – context of the grpc call

  • args – positional arguments

  • kwargs – keyword arguments

Yield:

responses to the request after streaming to Executors in Flow

Return type:

AsyncIterator[Request]

property host#

Gets the host from the runtime_args :return: The host where to bind the gateway

static is_valid_jaml(obj)#

Verifies the yaml syntax of a given object by first serializing it and attempting to deserialize and catch parser errors :type obj: Dict :param obj: yaml object :rtype: bool :return: whether the syntax is valid or not

classmethod load_config(source, *, allow_py_modules=True, substitute=True, context=None, uses_with=None, uses_metas=None, uses_requests=None, extra_search_paths=None, py_modules=None, runtime_args=None, uses_dynamic_batching=None, **kwargs)#

A high-level interface for loading configuration with features of loading extra py_modules, substitute env & context variables. Any class that implements JAMLCompatible mixin can enjoy this feature, e.g. BaseFlow, BaseExecutor, BaseGateway and all their subclasses.

Support substitutions in YAML:

  • Environment variables: ${{ ENV.VAR }} (recommended), $VAR (deprecated).

  • Context dict (context): ${{ CONTEXT.VAR }}``(recommended), ``${{ VAR }}.

  • Internal reference via this and root: ${{this.same_level_key}}, ${{root.root_level_key}}

Substitutions are carried in the order and multiple passes to resolve variables with best effort.

!BaseEncoder
metas:
    name: ${{VAR_A}}  # env or context variables
    workspace: my-${{this.name}}  # internal reference

# load Executor from yaml file
BaseExecutor.load_config('a.yml')

# load Executor from yaml file and substitute environment variables
os.environ['VAR_A'] = 'hello-world'
b = BaseExecutor.load_config('a.yml')
assert b.name == 'hello-world'

# load Executor from yaml file and substitute variables from a dict
b = BaseExecutor.load_config('a.yml', context={'VAR_A': 'hello-world'})
assert b.name == 'hello-world'

# disable substitute
b = BaseExecutor.load_config('a.yml', substitute=False)
Parameters:

  • source (Union[str, TextIO, Dict]) – the multi-kind source of the configs.

  • allow_py_modules (bool) – allow importing plugins specified by py_modules in YAML at any levels

  • substitute (bool) – substitute environment, internal reference and context variables.

  • context (Optional[Dict[str, Any]]) – context replacement variables in a dict, the value of the dict is the replacement.

  • uses_with (Optional[Dict]) – dictionary of parameters to overwrite from the default config’s with field

  • uses_metas (Optional[Dict]) – dictionary of parameters to overwrite from the default config’s metas field

  • uses_requests (Optional[Dict]) – dictionary of parameters to overwrite from the default config’s requests field

  • extra_search_paths (Optional[List[str]]) – extra paths used when looking for executor yaml files

  • py_modules (Optional[str]) – Optional py_module from which the object need to be loaded

  • runtime_args (Optional[Dict[str, Any]]) – Optional dictionary of parameters runtime_args to be directly passed without being parsed into a yaml config

  • uses_dynamic_batching (Optional[Dict]) – dictionary of parameters to overwrite from the default config’s dynamic_batching field

  • kwargs – kwargs for parse_config_source

Return type:

JAMLCompatible

Returns:

JAMLCompatible object

property port#

Gets the first port of the port list argument. To be used in the regular case where a Gateway exposes a single port :return: The first port to be exposed

property ports#

Gets all the list of ports from the runtime_args as a list. :return: The lists of ports to be exposed

property protocols#

Gets all the list of protocols from the runtime_args as a list. :return: The lists of protocols to be exposed

save_config(filename=None)#

Save the object’s config into a YAML file.

Parameters:

filename (Optional[str]) – file path of the yaml file, if not given then config_abspath is used