Preliminaries

This chapter introduces the basic terminology and concepts you will encounter in the docs. But first, look at the code below:

In this code, we are going to use Jina to serve simple logic with one Deployment, or a combination of two services with a Flow. We are also going to see how we can query these services with Jina’s client.

Deployment

from jina import Executor, Flow, requests
from docarray import DocList
from docarray.documents import TextDoc


class FooExec(Executor):
    @requests
    async def add_text(self, docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        for d in docs:
            d.text += 'hello, world!'

dep = Deployment(port=12345, uses=FooExec, replicas=3)

with dep:
    dep.block()

Flow

from jina import Executor, Flow, requests
from docarray import DocList
from docarray.documents import TextDoc


class FooExec(Executor):
    @requests
    async def add_text(self, docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        for d in docs:
            d.text += 'hello, world!'

class BarExec(Executor):
    @requests
    async def add_text(self, docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        for d in docs:
            d.text += 'goodbye!'

f = Flow(port=12345).add(uses=FooExec, replicas=3).add(uses=BarExec, replicas=2)

with f:
    f.block()

Client

from jina import Client
from docarray import DocList
from docarray.documents import TextDoc

c = Client(port=12345)

r = c.post(on='/', inputs=DocList[TextDoc]([TextDoc(text='')]), return_type=DocList[TextDoc])
print([d.text for d in r])

Running it gives you:

Deployment

['hello, world!', 'hello, world!']

Flow

['hello, world!goodbye!', 'hello, world!goodbye!']

Architecture

This animation shows what’s happening behind the scenes when running the previous examples:

Deployment

Flow

Hint

gRPC, WebSocket and HTTP are network protocols for transmitting data. gRPC is always used for communication between the Gateway and Executors inside a Flow.

Hint

TLS is a security protocol to facilitate privacy and data security for communications over the Internet. The communication between Client and Gateway is protected by TLS.

Jina is an MLOPs serving framework that is structured in two main layers. These layers work with DocArray’s data structure and Jina’s Python Client to complete the framework. All of these are covered in the user guide and contains the following concepts:

DocArray data structure

Data structures coming from docarray are the basic fundamental data structure in Jina.

- BaseDoc

Document is the basic object for representing multimodal data. It can be extended to represent any data you want. More information can be found in DocArray’s Docs.

- DocList

DocList is a list-like container of multiple Documents. More information can be found in DocArray’s Docs.

All the components in Jina use BaseDoc and/or DocList as the main data format for communication, making use of the different

serialization capabilities of these structures.

Serving

This layer contains all the objects and concepts that are used to actually serve the logic and receive and respond to queries. These components are designed to be used as microservices ready to be containerized.

These components can be orchestrated by Jina’s orchestration layer or by other container orchestration frameworks such as Kubernetes or Docker Compose.

- Executor

A Executor is a Python class that serves logic using Documents. Loosely speaking, each Executor is a service wrapping a model or application.

- Gateway

A Gateway is the entrypoint of a Flow. It exposes multiple protocols for external communications; routing all internal traffic to different Executors that work together to provide a more complex service.

Orchestration

This layer contains the components making sure that the objects (especially the Executor) are deployed and scaled for serving.

They wrap them to provide them the scalability and serving capabilities. They also provide easy translation to other orchestration

frameworks (Kubernetes, Docker compose) to provide more advanced and production-ready settings. They can also be directly deployed to Jina AI Cloud

with a single command line.

- Deployment

Deployment is a layer that orchestrates Executor. It can be used to serve an Executor as a standalone service or as part of a Flow. It encapsulates and abstracts internal replication and serving details.

- Flow

Flow ties multiple Deploymentss together into a logic pipeline to achieve a more complex task. It orchestrates both Executors and the Gateway.

Client

The Client connects to a Gateway or Executor and sends/receives/streams data from them.

Deployments on JCloud

At present, JCloud is only available for Flows. We are currently working on supporting Deployments.