The timeout in milliseconds of the control request, -1 for waiting forever
Default: 60
--polling
The polling strategy of the Deployment and its endpoints (when shards>1).
Can be defined for all endpoints of a Deployment or by endpoint.
Define per Deployment:
- ANY: only one (whoever is idle) Pod polls the message
- ALL: all Pods poll the message (like a broadcast)
Define per Endpoint:
JSON dict, {endpoint: PollingType}
{‘/custom’: ‘ALL’, ‘/search’: ‘ANY’, ‘*’: ‘ANY’}
The config of the executor, it could be one of the followings:
* the string literal of an Executor class name
* an Executor YAML file (.yml, .yaml, .jaml)
* a Jina Hub Executor (must start with jinahub:// or jinahub+docker://)
* a docker image (must start with docker://)
* the string literal of a YAML config (must start with ! or `jtype: `)
* the string literal of a JSON config
When use it under Python, one can use the following values additionally:
- a Python dict that represents the config
- a text file stream has .read() interface
Default: “BaseExecutor”
--uses-with
Dictionary of keyword arguments that will override the with configuration in uses
--uses-metas
Dictionary of keyword arguments that will override the metas configuration in uses
--uses-requests
Dictionary of keyword arguments that will override the requests configuration in uses
--uses-dynamic-batching
Dictionary of keyword arguments that will override the dynamic_batching configuration in uses
--py-modules
The customized python modules need to be imported before loading the executor
Note that the recommended way is to only import a single module - a simple python file, if your
executor can be defined in a single file, or an __init__.py file if you have multiple files,
which should be structured as a python package. For more details, please see the
Executor cookbook
--output-array-type
The type of array tensor and embedding will be serialized to.
Supports the same types as docarray.to_protobuf(.., ndarray_type=…), which can be found
here <https://docarray.jina.ai/fundamentals/document/serialization/#from-to-protobuf>.
Defaults to retaining whatever type is returned by the Executor.
--exit-on-exceptions
List of exceptions that will cause the Executor to shut down.
Default: []
--no-reduce, --disable-reduce
Disable the built-in reduction mechanism. Set this if the reduction is to be handled by the Executor itself by operating on a docs_matrix or docs_map
Default: False
--grpc-server-options
Dictionary of kwargs arguments that will be passed to the grpc server as options when starting the server, example : {‘grpc.max_send_message_length’: -1}
--grpc-channel-options
Dictionary of kwargs arguments that will be passed to the grpc channel as options when creating a channel, example : {‘grpc.max_send_message_length’: -1}. When max_attempts > 1, the ‘grpc.service_config’ option will not be applicable.
The path on the host to be mounted inside the container.
Note,
- If separated by :, then the first part will be considered as the local host path and the second part is the path in the container system.
- If no split provided, then the basename of that directory will be mounted into container’s root path, e.g. –volumes=”/user/test/my-workspace” will be mounted into /my-workspace inside the container.
- All volumes are mounted with read-write mode.
--gpus
This argument allows dockerized Jina Executors to discover local gpu devices.
Note,
- To access all gpus, use –gpus all.
- To access multiple gpus, e.g. make use of 2 gpus, use –gpus 2.
- To access specified gpus based on device id, use –gpus device=[YOUR-GPU-DEVICE-ID]
- To access specified gpus based on multiple device id, use –gpus device=[YOUR-GPU-DEVICE-ID1],device=[YOUR-GPU-DEVICE-ID2]
- To specify more parameters, use `–gpus device=[YOUR-GPU-DEVICE-ID],runtime=nvidia,capabilities=display
--disable-auto-volume
Do not automatically mount a volume for dockerized Executors.
The host of the Gateway, which the client should connect to, by default it is 0.0.0.0. In the case of an external Executor (–external or external=True) this can be a list of hosts. Then, every resulting address will be considered as one replica of the Executor.
The timeout in milliseconds of a Pod waits for the runtime to be ready, -1 for waiting forever
Default: 600000
--env
The map of environment variables that are available inside runtime
--env-from-secret
The map of environment variables that are read from kubernetes cluster secrets
--floating
If set, the current Pod/Deployment can not be further chained, and the next .add() will chain after the last Pod/Deployment not this current one.
Default: False
--reload
If set, the Executor will restart while serving if YAML configuration source or Executor modules are changed. If YAML configuration is changed, the whole deployment is reloaded and new processes will be restarted. If only Python modules of the Executor have changed, they will be reloaded to the interpreter without restarting process.
Default: False
--install-requirements
If set, try to install requirements.txt from the local Executor if exists in the Executor folder. If using Hub, install requirements.txt in the Hub Executor bundle to local.
Default: False
--port, --ports
The port for input data to bind to, default is a random port between [49152, 65535]. In the case of an external Executor (–external or external=True) this can be a list of ports. Then, every resulting address will be considered as one replica of the Executor.
Default: [57500]
--monitoring
If set, spawn an http server with a prometheus endpoint to expose metrics
Default: False
--port-monitoring
The port on which the prometheus server is exposed, default is a random port between [49152, 65535]
Default: [55507]
--retries
Number of retries per gRPC call. If <0 it defaults to max(3, num_replicas)
Default: -1
--tracing
If set, the sdk implementation of the OpenTelemetry tracer will be available and will be enabled for automatic tracing of requests and customer span creation. Otherwise a no-op implementation will be provided.
Default: False
--traces-exporter-host
If tracing is enabled, this hostname will be used to configure the trace exporter agent.
--traces-exporter-port
If tracing is enabled, this port will be used to configure the trace exporter agent.
--metrics
If set, the sdk implementation of the OpenTelemetry metrics will be available for default monitoring and custom measurements. Otherwise a no-op implementation will be provided.
Default: False
--metrics-exporter-host
If tracing is enabled, this hostname will be used to configure the metrics exporter agent.
--metrics-exporter-port
If tracing is enabled, this port will be used to configure the metrics exporter agent.
The YAML path represents a flow. It can be either a local file path or a URL.
--reload
If set, auto-reloading on file changes is enabled: the Flow will restart while blocked if YAML configuration source is changed. This also applies apply to underlying Executors, if their source code or YAML configuration has changed.
Default: False
--env
The map of environment variables that are available inside runtime
--inspect
Possible choices: HANG, REMOVE, COLLECT
The strategy on those inspect deployments in the flow.
If REMOVE is given then all inspect deployments are removed when building the flow.
The target type to ping. For executor and gateway, checks the readiness of the individual service. For flow it checks the connectivity of the complete microservice architecture.
Default: “executor”
host
The host address with port of a target Executor, Gateway or a Flow, e.g. 0.0.0.0:8000. For Flow or Gateway, host can also indicate the protocol, grpc will be used if not provided, e.g http://0.0.0.0:8000
Communication protocol of the server exposed by the Gateway. This can be a single value or a list of protocols, depending on your chosen Gateway. Choose the convenient protocols from: [‘GRPC’, ‘HTTP’, ‘WEBSOCKET’].
The host address of the runtime, by default it is 0.0.0.0.
Default: “0.0.0.0”
--proxy
If set, respect the http_proxy and https_proxy environment variables. otherwise, it will unset these proxy variables before start. gRPC seems to prefer no proxy
Default: False
--uses
The config of the gateway, it could be one of the followings:
* the string literal of an Gateway class name
* a Gateway YAML file (.yml, .yaml, .jaml)
* a docker image (must start with docker://)
* the string literal of a YAML config (must start with ! or `jtype: `)
* the string literal of a JSON config
When use it under Python, one can use the following values additionally:
- a Python dict that represents the config
- a text file stream has .read() interface
--uses-with
Dictionary of keyword arguments that will override the with configuration in uses
--py-modules
The customized python modules need to be imported before loading the gateway
Note that the recommended way is to only import a single module - a simple python file, if your
gateway can be defined in a single file, or an __init__.py file if you have multiple files,
which should be structured as a python package.
--replicas
The number of replicas of the Gateway. This replicas will only be applied when converted into Kubernetes YAML
Default: 1
--grpc-server-options
Dictionary of kwargs arguments that will be passed to the grpc server as options when starting the server, example : {‘grpc.max_send_message_length’: -1}
--grpc-channel-options
Dictionary of kwargs arguments that will be passed to the grpc channel as options when creating a channel, example : {‘grpc.max_send_message_length’: -1}. When max_attempts > 1, the ‘grpc.service_config’ option will not be applicable.
--graph-description
Routing graph for the gateway
Default: “{}”
--graph-conditions
Dictionary stating which filtering conditions each Executor in the graph requires to receive Documents.
Default: “{}”
--deployments-addresses
JSON dictionary with the input addresses of each Deployment
Default: “{}”
--deployments-metadata
JSON dictionary with the request metadata for each Deployment
The timeout in milliseconds of a Pod waits for the runtime to be ready, -1 for waiting forever
Default: 600000
--env
The map of environment variables that are available inside runtime
--env-from-secret
The map of environment variables that are read from kubernetes cluster secrets
--floating
If set, the current Pod/Deployment can not be further chained, and the next .add() will chain after the last Pod/Deployment not this current one.
Default: False
--reload
If set, the Gateway will restart while serving if YAML configuration source is changed.
Default: False
--port, --ports, --port-expose, --port-in
The port for input data to bind the gateway server to, by default, random ports between range [49152, 65535] will be assigned. The port argument can be either 1 single value in case only 1 protocol is used or multiple values when many protocols are used.
Default: [61101]
--monitoring
If set, spawn an http server with a prometheus endpoint to expose metrics
Default: False
--port-monitoring
The port on which the prometheus server is exposed, default is a random port between [49152, 65535]
Default: [51478]
--retries
Number of retries per gRPC call. If <0 it defaults to max(3, num_replicas)
Default: -1
--tracing
If set, the sdk implementation of the OpenTelemetry tracer will be available and will be enabled for automatic tracing of requests and customer span creation. Otherwise a no-op implementation will be provided.
Default: False
--traces-exporter-host
If tracing is enabled, this hostname will be used to configure the trace exporter agent.
--traces-exporter-port
If tracing is enabled, this port will be used to configure the trace exporter agent.
--metrics
If set, the sdk implementation of the OpenTelemetry metrics will be available for default monitoring and custom measurements. Otherwise a no-op implementation will be provided.
Default: False
--metrics-exporter-host
If tracing is enabled, this hostname will be used to configure the metrics exporter agent.
--metrics-exporter-port
If tracing is enabled, this port will be used to configure the metrics exporter agent.
The string ID of a flow for single removal, or a list of space seperated string IDs for multiple removal, or string ‘all’ for deleting ALL SERVING flows.
The timeout in milliseconds of the control request, -1 for waiting forever
Default: 60
--polling
The polling strategy of the Deployment and its endpoints (when shards>1).
Can be defined for all endpoints of a Deployment or by endpoint.
Define per Deployment:
- ANY: only one (whoever is idle) Pod polls the message
- ALL: all Pods poll the message (like a broadcast)
Define per Endpoint:
JSON dict, {endpoint: PollingType}
{‘/custom’: ‘ALL’, ‘/search’: ‘ANY’, ‘*’: ‘ANY’}
The config of the executor, it could be one of the followings:
* the string literal of an Executor class name
* an Executor YAML file (.yml, .yaml, .jaml)
* a Jina Hub Executor (must start with jinahub:// or jinahub+docker://)
* a docker image (must start with docker://)
* the string literal of a YAML config (must start with ! or `jtype: `)
* the string literal of a JSON config
When use it under Python, one can use the following values additionally:
- a Python dict that represents the config
- a text file stream has .read() interface
Default: “BaseExecutor”
--uses-with
Dictionary of keyword arguments that will override the with configuration in uses
--uses-metas
Dictionary of keyword arguments that will override the metas configuration in uses
--uses-requests
Dictionary of keyword arguments that will override the requests configuration in uses
--uses-dynamic-batching
Dictionary of keyword arguments that will override the dynamic_batching configuration in uses
--py-modules
The customized python modules need to be imported before loading the executor
Note that the recommended way is to only import a single module - a simple python file, if your
executor can be defined in a single file, or an __init__.py file if you have multiple files,
which should be structured as a python package. For more details, please see the
Executor cookbook
--output-array-type
The type of array tensor and embedding will be serialized to.
Supports the same types as docarray.to_protobuf(.., ndarray_type=…), which can be found
here <https://docarray.jina.ai/fundamentals/document/serialization/#from-to-protobuf>.
Defaults to retaining whatever type is returned by the Executor.
--exit-on-exceptions
List of exceptions that will cause the Executor to shut down.
Default: []
--no-reduce, --disable-reduce
Disable the built-in reduction mechanism. Set this if the reduction is to be handled by the Executor itself by operating on a docs_matrix or docs_map
Default: False
--grpc-server-options
Dictionary of kwargs arguments that will be passed to the grpc server as options when starting the server, example : {‘grpc.max_send_message_length’: -1}
--grpc-channel-options
Dictionary of kwargs arguments that will be passed to the grpc channel as options when creating a channel, example : {‘grpc.max_send_message_length’: -1}. When max_attempts > 1, the ‘grpc.service_config’ option will not be applicable.
The path on the host to be mounted inside the container.
Note,
- If separated by :, then the first part will be considered as the local host path and the second part is the path in the container system.
- If no split provided, then the basename of that directory will be mounted into container’s root path, e.g. –volumes=”/user/test/my-workspace” will be mounted into /my-workspace inside the container.
- All volumes are mounted with read-write mode.
--gpus
This argument allows dockerized Jina Executors to discover local gpu devices.
Note,
- To access all gpus, use –gpus all.
- To access multiple gpus, e.g. make use of 2 gpus, use –gpus 2.
- To access specified gpus based on device id, use –gpus device=[YOUR-GPU-DEVICE-ID]
- To access specified gpus based on multiple device id, use –gpus device=[YOUR-GPU-DEVICE-ID1],device=[YOUR-GPU-DEVICE-ID2]
- To specify more parameters, use `–gpus device=[YOUR-GPU-DEVICE-ID],runtime=nvidia,capabilities=display
--disable-auto-volume
Do not automatically mount a volume for dockerized Executors.
The host of the Gateway, which the client should connect to, by default it is 0.0.0.0. In the case of an external Executor (–external or external=True) this can be a list of hosts. Then, every resulting address will be considered as one replica of the Executor.
The timeout in milliseconds of a Pod waits for the runtime to be ready, -1 for waiting forever
Default: 600000
--env
The map of environment variables that are available inside runtime
--env-from-secret
The map of environment variables that are read from kubernetes cluster secrets
--floating
If set, the current Pod/Deployment can not be further chained, and the next .add() will chain after the last Pod/Deployment not this current one.
Default: False
--reload
If set, the Executor will restart while serving if YAML configuration source or Executor modules are changed. If YAML configuration is changed, the whole deployment is reloaded and new processes will be restarted. If only Python modules of the Executor have changed, they will be reloaded to the interpreter without restarting process.
Default: False
--install-requirements
If set, try to install requirements.txt from the local Executor if exists in the Executor folder. If using Hub, install requirements.txt in the Hub Executor bundle to local.
Default: False
--port, --ports
The port for input data to bind to, default is a random port between [49152, 65535]. In the case of an external Executor (–external or external=True) this can be a list of ports. Then, every resulting address will be considered as one replica of the Executor.
Default: [59049]
--monitoring
If set, spawn an http server with a prometheus endpoint to expose metrics
Default: False
--port-monitoring
The port on which the prometheus server is exposed, default is a random port between [49152, 65535]
Default: [62214]
--retries
Number of retries per gRPC call. If <0 it defaults to max(3, num_replicas)
Default: -1
--tracing
If set, the sdk implementation of the OpenTelemetry tracer will be available and will be enabled for automatic tracing of requests and customer span creation. Otherwise a no-op implementation will be provided.
Default: False
--traces-exporter-host
If tracing is enabled, this hostname will be used to configure the trace exporter agent.
--traces-exporter-port
If tracing is enabled, this port will be used to configure the trace exporter agent.
--metrics
If set, the sdk implementation of the OpenTelemetry metrics will be available for default monitoring and custom measurements. Otherwise a no-op implementation will be provided.
Default: False
--metrics-exporter-host
If tracing is enabled, this hostname will be used to configure the metrics exporter agent.
--metrics-exporter-port
If tracing is enabled, this port will be used to configure the metrics exporter agent.
The timeout in milliseconds of the control request, -1 for waiting forever
Default: 60
--polling
The polling strategy of the Deployment and its endpoints (when shards>1).
Can be defined for all endpoints of a Deployment or by endpoint.
Define per Deployment:
- ANY: only one (whoever is idle) Pod polls the message
- ALL: all Pods poll the message (like a broadcast)
Define per Endpoint:
JSON dict, {endpoint: PollingType}
{‘/custom’: ‘ALL’, ‘/search’: ‘ANY’, ‘*’: ‘ANY’}
The config of the executor, it could be one of the followings:
* the string literal of an Executor class name
* an Executor YAML file (.yml, .yaml, .jaml)
* a Jina Hub Executor (must start with jinahub:// or jinahub+docker://)
* a docker image (must start with docker://)
* the string literal of a YAML config (must start with ! or `jtype: `)
* the string literal of a JSON config
When use it under Python, one can use the following values additionally:
- a Python dict that represents the config
- a text file stream has .read() interface
Default: “BaseExecutor”
--uses-with
Dictionary of keyword arguments that will override the with configuration in uses
--uses-metas
Dictionary of keyword arguments that will override the metas configuration in uses
--uses-requests
Dictionary of keyword arguments that will override the requests configuration in uses
--uses-dynamic-batching
Dictionary of keyword arguments that will override the dynamic_batching configuration in uses
--py-modules
The customized python modules need to be imported before loading the executor
Note that the recommended way is to only import a single module - a simple python file, if your
executor can be defined in a single file, or an __init__.py file if you have multiple files,
which should be structured as a python package. For more details, please see the
Executor cookbook
--output-array-type
The type of array tensor and embedding will be serialized to.
Supports the same types as docarray.to_protobuf(.., ndarray_type=…), which can be found
here <https://docarray.jina.ai/fundamentals/document/serialization/#from-to-protobuf>.
Defaults to retaining whatever type is returned by the Executor.
--exit-on-exceptions
List of exceptions that will cause the Executor to shut down.
Default: []
--no-reduce, --disable-reduce
Disable the built-in reduction mechanism. Set this if the reduction is to be handled by the Executor itself by operating on a docs_matrix or docs_map
Default: False
--grpc-server-options
Dictionary of kwargs arguments that will be passed to the grpc server as options when starting the server, example : {‘grpc.max_send_message_length’: -1}
--grpc-channel-options
Dictionary of kwargs arguments that will be passed to the grpc channel as options when creating a channel, example : {‘grpc.max_send_message_length’: -1}. When max_attempts > 1, the ‘grpc.service_config’ option will not be applicable.
The path on the host to be mounted inside the container.
Note,
- If separated by :, then the first part will be considered as the local host path and the second part is the path in the container system.
- If no split provided, then the basename of that directory will be mounted into container’s root path, e.g. –volumes=”/user/test/my-workspace” will be mounted into /my-workspace inside the container.
- All volumes are mounted with read-write mode.
--gpus
This argument allows dockerized Jina Executors to discover local gpu devices.
Note,
- To access all gpus, use –gpus all.
- To access multiple gpus, e.g. make use of 2 gpus, use –gpus 2.
- To access specified gpus based on device id, use –gpus device=[YOUR-GPU-DEVICE-ID]
- To access specified gpus based on multiple device id, use –gpus device=[YOUR-GPU-DEVICE-ID1],device=[YOUR-GPU-DEVICE-ID2]
- To specify more parameters, use `–gpus device=[YOUR-GPU-DEVICE-ID],runtime=nvidia,capabilities=display
--disable-auto-volume
Do not automatically mount a volume for dockerized Executors.
The host of the Gateway, which the client should connect to, by default it is 0.0.0.0. In the case of an external Executor (–external or external=True) this can be a list of hosts. Then, every resulting address will be considered as one replica of the Executor.
The timeout in milliseconds of a Pod waits for the runtime to be ready, -1 for waiting forever
Default: 600000
--env
The map of environment variables that are available inside runtime
--env-from-secret
The map of environment variables that are read from kubernetes cluster secrets
--floating
If set, the current Pod/Deployment can not be further chained, and the next .add() will chain after the last Pod/Deployment not this current one.
Default: False
--reload
If set, the Executor will restart while serving if YAML configuration source or Executor modules are changed. If YAML configuration is changed, the whole deployment is reloaded and new processes will be restarted. If only Python modules of the Executor have changed, they will be reloaded to the interpreter without restarting process.
Default: False
--install-requirements
If set, try to install requirements.txt from the local Executor if exists in the Executor folder. If using Hub, install requirements.txt in the Hub Executor bundle to local.
Default: False
--port, --ports
The port for input data to bind to, default is a random port between [49152, 65535]. In the case of an external Executor (–external or external=True) this can be a list of ports. Then, every resulting address will be considered as one replica of the Executor.
Default: [56892]
--monitoring
If set, spawn an http server with a prometheus endpoint to expose metrics
Default: False
--port-monitoring
The port on which the prometheus server is exposed, default is a random port between [49152, 65535]
Default: [51637]
--retries
Number of retries per gRPC call. If <0 it defaults to max(3, num_replicas)
Default: -1
--tracing
If set, the sdk implementation of the OpenTelemetry tracer will be available and will be enabled for automatic tracing of requests and customer span creation. Otherwise a no-op implementation will be provided.
Default: False
--traces-exporter-host
If tracing is enabled, this hostname will be used to configure the trace exporter agent.
--traces-exporter-port
If tracing is enabled, this port will be used to configure the trace exporter agent.
--metrics
If set, the sdk implementation of the OpenTelemetry metrics will be available for default monitoring and custom measurements. Otherwise a no-op implementation will be provided.
Default: False
--metrics-exporter-host
If tracing is enabled, this hostname will be used to configure the metrics exporter agent.
--metrics-exporter-port
If tracing is enabled, this port will be used to configure the metrics exporter agent.
The executor attached before the Pods described by –uses, typically before sending to all shards, accepted type follows –uses. This argument only applies for sharded Deployments (shards > 1).
--uses-after
The executor attached after the Pods described by –uses, typically used for receiving from all shards, accepted type follows –uses. This argument only applies for sharded Deployments (shards > 1).
--when
The condition that the documents need to fulfill before reaching the Executor.The condition can be defined in the form of a DocArray query condition <https://docarray.jina.ai/fundamentals/documentarray/find/#query-by-conditions>
--external
The Deployment will be considered an external Deployment that has been started independently from the Flow.This Deployment will not be context managed by the Flow.
Default: False
--grpc-metadata
The metadata to be passed to the gRPC request.
--tls
If set, connect to deployment using tls encryption
If set, then the input and output of this Client work in an asynchronous manner.
Default: False
--tracing
If set, the sdk implementation of the OpenTelemetry tracer will be available and will be enabled for automatic tracing of requests and customer span creation. Otherwise a no-op implementation will be provided.
Default: False
--traces-exporter-host
If tracing is enabled, this hostname will be used to configure the trace exporter agent.
--traces-exporter-port
If tracing is enabled, this port will be used to configure the trace exporter agent.
--metrics
If set, the sdk implementation of the OpenTelemetry metrics will be available for default monitoring and custom measurements. Otherwise a no-op implementation will be provided.
Default: False
--metrics-exporter-host
If tracing is enabled, this hostname will be used to configure the metrics exporter agent.
--metrics-exporter-port
If tracing is enabled, this port will be used to configure the metrics exporter agent.
--log-config
The config name or the absolute path to the YAML config file of the logger used in this object.
Default: “default”
--protocol
Possible choices: GRPC, HTTP, WEBSOCKET
Communication protocol between server and client.
Default: GRPC
--grpc-channel-options
Dictionary of kwargs arguments that will be passed to the grpc channel as options when creating a channel, example : {‘grpc.max_send_message_length’: -1}. When max_attempts > 1, the ‘grpc.service_config’ option will not be applicable.
--suppress-root-logging
If set, then no root handlers will be suppressed from logging.
If set, respect the http_proxy and https_proxy environment variables. otherwise, it will unset these proxy variables before start. gRPC seems to prefer no proxy
Default: False
--host, --host-in
The host of the Gateway, which the client should connect to, by default it is 0.0.0.0.
Default: “0.0.0.0”
--port
The port of the Gateway, which the client should connect to.