Welcome to Jina!#

Survey

Take our user experience survey to let us know your thoughts and help shape the future of Jina!

Jina is an MLOps framework to build multimodal AI services and pipelines then serve, scale and deploy them to a production-ready environment like Kubernetes or Jina AI Cloud. Jina handles the infrastructure complexity, making advanced solution engineering and cloud-native technologies accessible to every developer.

Build and deploy a gRPC microserviceBuild and deploy a pipeline

Applications built with Jina enjoy the following features out of the box:

🌌 Universal

  • Build applications that deliver fresh insights from multiple data types such as text, image, audio, video, 3D mesh, PDF with LF’s DocArray.

  • Support for all mainstream deep learning frameworks.

  • Polyglot gateway that supports gRPC, Websockets, HTTP, GraphQL protocols with TLS.

Performance

  • Intuitive design pattern for high-performance microservices.

  • Easy scaling: set replicas, sharding in one line.

  • Duplex streaming between client and server.

  • Async and non-blocking data processing over dynamic flows.

☁️ Cloud native

  • Seamless Docker container integration: sharing, exploring, sandboxing, versioning and dependency control via Executor Hub.

  • Full observability via OpenTelemetry, Prometheus and Grafana.

  • Fast deployment to Kubernetes and Docker Compose.

🍱 Ecosystem

  • Improved engineering efficiency thanks to the Jina AI ecosystem, so you can focus on innovating with the data applications you build.

  • Free CPU/GPU hosting via Jina AI Cloud.

Install#

Make sure that you have Python 3.7+ installed on Linux/macOS/Windows.

pip install -U jina
conda install jina -c conda-forge

Getting Started#

Jina supports developers in building AI services and pipelines:

Open In Colab

Let’s build a fast, reliable and scalable gRPC-based AI service. In Jina we call this an Executor. Our simple Executor will use Facebook’s mBART-50 model to translate French to English. We’ll then use a Deployment to serve it.

Note A Deployment serves just one Executor. To combine multiple Executors into a pipeline and serve that, use a Flow.

Note Run the code in Colab to install all dependencies.

Let’s implement the service’s logic:

translate_executor.py
from docarray import DocumentArray
from jina import Executor, requests
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM


class Translator(Executor):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        self.tokenizer = AutoTokenizer.from_pretrained(
            "facebook/mbart-large-50-many-to-many-mmt", src_lang="fr_XX"
        )
        self.model = AutoModelForSeq2SeqLM.from_pretrained(
            "facebook/mbart-large-50-many-to-many-mmt"
        )

    @requests
    def translate(self, docs: DocumentArray, **kwargs):
        for doc in docs:
            doc.text = self._translate(doc.text)

    def _translate(self, text):
        encoded_en = self.tokenizer(text, return_tensors="pt")
        generated_tokens = self.model.generate(
            **encoded_en, forced_bos_token_id=self.tokenizer.lang_code_to_id["en_XX"]
        )
        return self.tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)[
            0
        ]

Then we deploy it with either the Python API or YAML:

Python API: deployment.py YAML: deployment.yml
from jina import Deployment

with Deployment(uses=Translator, timeout_ready=-1) as dep:
    dep.block()
jtype: Deployment
with:
  uses: Translator
  py_modules:
    - translate_executor.py # name of the module containing Translator
  timeout_ready: -1

And run the YAML Deployment with the CLI: jina deployment --uses deployment.yml

──────────────────────────────────────── 🎉 Deployment is ready to serve! ─────────────────────────────────────────
╭────────────── 🔗 Endpoint ───────────────╮
│  ⛓      Protocol                   GRPC │
│  🏠        Local          0.0.0.0:12345  │
│  🔒      Private      172.28.0.12:12345  │
│  🌍       Public    35.230.97.208:12345  │
╰──────────────────────────────────────────╯

Use Jina Client to make requests to the service:

from docarray import Document
from jina import Client

french_text = Document(
    text='un astronaut est en train de faire une promenade dans un parc'
)

client = Client(port=12345)  # use port from output above
response = client.post(on='/', inputs=[french_text])

print(response[0].text)
an astronaut is walking in a park

Open In Colab

Sometimes you want to chain microservices together into a pipeline. That’s where a Flow comes in.

A Flow is a DAG pipeline, composed of a set of steps, It orchestrates a set of Executors and a Gateway to offer an end-to-end service.

Note If you just want to serve a single Executor, you can use a Deployment.

For instance, let’s combine our French translation service with a Stable Diffusion image generation service from Jina AI’s Executor Hub. Chaining these services together into a Flow will give us a multilingual image generation service.

Build the Flow with either Python or YAML:

Python API: flow.py YAML: flow.yml
from jina import Flow

flow = (
    Flow()
    .add(uses=Translator, timeout_ready=-1)
    .add(
        uses='jinaai://jina-ai/TextToImage',
        timeout_ready=-1,
        install_requirements=True,
    )
)  # use the Executor from Executor hub

with flow:
    flow.block()
jtype: Flow
executors:
  - uses: Translator
    timeout_ready: -1
    py_modules:
      - translate_executor.py
  - uses: jinaai://jina-ai/TextToImage
    timeout_ready: -1
    install_requirements: true

Then run the YAML Flow with the CLI: jina flow --uses flow.yml

─────────────────────────────────────────── 🎉 Flow is ready to serve! ────────────────────────────────────────────
╭────────────── 🔗 Endpoint ───────────────╮
│  ⛓      Protocol                   GRPC  │
│  🏠        Local          0.0.0.0:12345  │
│  🔒      Private      172.28.0.12:12345  │
│  🌍       Public    35.240.201.66:12345  │
╰──────────────────────────────────────────╯

Then, use Jina Client to make requests to the Flow:

from jina import Client, Document

client = Client(port=12345)  # use port from output above

french_text = Document(
    text='un astronaut est en train de faire une promenade dans un parc'
)

response = client.post(on='/', inputs=[french_text])

response[0].display()

stable-diffusion-output.png

You can also deploy a Flow to JCloud.

First, turn the flow.yml file into a JCloud-compatible YAML by specifying resource requirements and using containerized Hub Executors.

Then, use jina cloud deploy command to deploy to the cloud:

wget https://raw.githubusercontent.com/jina-ai/jina/master/.github/getting-started/jcloud-flow.yml
jina cloud deploy jcloud-flow.yml

⚠️ Caution: Make sure to delete/clean up the Flow once you are done with this tutorial to save resources and credits.

Read more about deploying Flows to JCloud.

Next steps#

Learn DocArray API

DocArray is the foundational data structure of Jina. Before starting Jina, first learn DocArray to quickly build a PoC.

Learn Executor

Executor is a Python class that can serve logic using Documents.

Learn Flow

Flow orchestrates Executors into a processing pipeline to accomplish a task.

Explore Executor Hub

Executor Hub is a marketplace that allows you to share, explore and test Executors.

Deploy a Flow to Cloud

Jina AI Cloud is the MLOps platform for hosting Jina projects.

Support#

Join Us#

Jina is backed by Jina AI and licensed under Apache-2.0.


Index | Module Index