Welcome to Jina!#

Survey

Take our user experience survey to let us know your thoughts and help shape the future of Jina!

Jina is an MLOps framework to build multimodal AI microservice-based applications written in Python that can communicate via gRPC, HTTP and WebSocket protocols. It allows developers to build and serve services and pipelines while scaling and deploying them to a production while removing the complexity, letting them focus on the logic/algorithmic part, saving valuable time and resources for engineering teams.

Jina aims to provide a smooth Pythonic experience transitioning from local deployment to deploying to advanced orchestration frameworks such as Docker-Compose, Kubernetes, or Jina AI Cloud. It handles the infrastructure complexity, making advanced solution engineering and cloud-native technologies accessible to every developer.

Build and deploy a gRPC microserviceBuild and deploy a pipeline

Applications built with Jina enjoy the following features out of the box:

🌌 Universal

  • Build applications that deliver fresh insights from multiple data types such as text, image, audio, video, 3D mesh, PDF with LF’s DocArray.

  • Support for all mainstream deep learning frameworks.

  • Polyglot gateway that supports gRPC, Websockets, HTTP, GraphQL protocols with TLS.

Performance

  • Intuitive design pattern for high-performance microservices.

  • Easy scaling: set replicas, sharding in one line.

  • Duplex streaming between client and server.

  • Async and non-blocking data processing over dynamic flows.

☁️ Cloud native

  • Seamless Docker container integration: sharing, exploring, sandboxing, versioning and dependency control via Executor Hub.

  • Full observability via OpenTelemetry, Prometheus and Grafana.

  • Fast deployment to Kubernetes and Docker Compose.

🍱 Ecosystem

  • Improved engineering efficiency thanks to the Jina AI ecosystem, so you can focus on innovating with the data applications you build.

  • Free CPU/GPU hosting via Jina AI Cloud.

Jina’s value proposition may seem quite similar to that of FastAPI. However, there are several fundamental differences:

Data structure and communication protocols

  • FastAPI communication relies on Pydantic and Jina relies on DocArray allowing Jina to support multiple protocols to expose its services.

Advanced orchestration and scaling capabilities

  • Jina lets you deploy applications formed from multiple microservices that can be containerized and scaled independently.

  • Jina allows you to easily containerize and orchestrate your services, providing concurrency and scalability.

Journey to the cloud

  • Jina provides a smooth transition from local development (using DocArray) to local serving using (Jina’s orchestration layer) to having production-ready services by using Kubernetes capacity to orchestrate the lifetime of containers.

  • By using Jina AI Cloud you have access to scalable and serverless deployments of your applications in one command.

Install#

Make sure that you have Python 3.7+ installed on Linux/macOS/Windows.

pip install -U jina
conda install jina -c conda-forge

Getting Started#

Jina supports developers in building AI services and pipelines:

Open In Colab

Let’s build a fast, reliable and scalable gRPC-based AI service. In Jina we call this an Executor. Our simple Executor will use Facebook’s mBART-50 model to translate French to English. We’ll then use a Deployment to serve it.

Note A Deployment serves just one Executor. To combine multiple Executors into a pipeline and serve that, use a Flow.

Note Run the code in Colab to install all dependencies.

Let’s implement the service’s logic:

translate_executor.py
from docarray import DocumentArray
from jina import Executor, requests
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM


class Translator(Executor):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        self.tokenizer = AutoTokenizer.from_pretrained(
            "facebook/mbart-large-50-many-to-many-mmt", src_lang="fr_XX"
        )
        self.model = AutoModelForSeq2SeqLM.from_pretrained(
            "facebook/mbart-large-50-many-to-many-mmt"
        )

    @requests
    def translate(self, docs: DocumentArray, **kwargs):
        for doc in docs:
            doc.text = self._translate(doc.text)

    def _translate(self, text):
        encoded_en = self.tokenizer(text, return_tensors="pt")
        generated_tokens = self.model.generate(
            **encoded_en, forced_bos_token_id=self.tokenizer.lang_code_to_id["en_XX"]
        )
        return self.tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)[
            0
        ]

Then we deploy it with either the Python API or YAML:

Python API: deployment.py YAML: deployment.yml
from jina import Deployment
from translate_executor import Translator

with Deployment(uses=Translator, timeout_ready=-1) as dep:
    dep.block()
jtype: Deployment
with:
  uses: Translator
  py_modules:
    - translate_executor.py # name of the module containing Translator
  timeout_ready: -1

And run the YAML Deployment with the CLI: jina deployment --uses deployment.yml

──────────────────────────────────────── 🎉 Deployment is ready to serve! ─────────────────────────────────────────
╭────────────── 🔗 Endpoint ───────────────╮
│  ⛓      Protocol                   GRPC │
│  🏠        Local          0.0.0.0:12345  │
│  🔒      Private      172.28.0.12:12345  │
│  🌍       Public    35.230.97.208:12345  │
╰──────────────────────────────────────────╯

Use Jina Client to make requests to the service:

from docarray import Document
from jina import Client

french_text = Document(
    text='un astronaut est en train de faire une promenade dans un parc'
)

client = Client(port=12345)  # use port from output above
response = client.post(on='/', inputs=[french_text])

print(response[0].text)
an astronaut is walking in a park

Open In Colab

Sometimes you want to chain microservices together into a pipeline. That’s where a Flow comes in.

A Flow is a DAG pipeline, composed of a set of steps, It orchestrates a set of Executors and a Gateway to offer an end-to-end service.

Note If you just want to serve a single Executor, you can use a Deployment.

For instance, let’s combine our French translation service with a Stable Diffusion image generation service from Jina AI’s Executor Hub. Chaining these services together into a Flow will give us a multilingual image generation service.

Build the Flow with either Python or YAML:

Python API: flow.py YAML: flow.yml
from jina import Flow

flow = (
    Flow()
    .add(uses=Translator, timeout_ready=-1)
    .add(
        uses='jinaai://jina-ai/TextToImage',
        timeout_ready=-1,
        install_requirements=True,
    )
)  # use the Executor from Executor hub

with flow:
    flow.block()
jtype: Flow
executors:
  - uses: Translator
    timeout_ready: -1
    py_modules:
      - translate_executor.py
  - uses: jinaai://jina-ai/TextToImage
    timeout_ready: -1
    install_requirements: true

Then run the YAML Flow with the CLI: jina flow --uses flow.yml

─────────────────────────────────────────── 🎉 Flow is ready to serve! ────────────────────────────────────────────
╭────────────── 🔗 Endpoint ───────────────╮
│  ⛓      Protocol                   GRPC  │
│  🏠        Local          0.0.0.0:12345  │
│  🔒      Private      172.28.0.12:12345  │
│  🌍       Public    35.240.201.66:12345  │
╰──────────────────────────────────────────╯

Then, use Jina Client to make requests to the Flow:

from jina import Client, Document

client = Client(port=12345)  # use port from output above

french_text = Document(
    text='un astronaut est en train de faire une promenade dans un parc'
)

response = client.post(on='/', inputs=[french_text])

response[0].display()

stable-diffusion-output.png

You can also deploy a Flow to JCloud.

First, turn the flow.yml file into a JCloud-compatible YAML by specifying resource requirements and using containerized Hub Executors.

Then, use jina cloud deploy command to deploy to the cloud:

wget https://raw.githubusercontent.com/jina-ai/jina/master/.github/getting-started/jcloud-flow.yml
jina cloud deploy jcloud-flow.yml

⚠️ Caution: Make sure to delete/clean up the Flow once you are done with this tutorial to save resources and credits.

Read more about deploying Flows to JCloud.

Next steps#

Learn DocArray API

DocArray is the foundational data structure of Jina. Before starting Jina, first learn DocArray to quickly build a PoC.

Learn Executor

Executor is a Python class that can serve logic using Documents.

Learn Deployment

Deployment serves an Executor as a scalable service making it available to receive Documents using gRPC or HTTP.

Learn Flow

Flow orchestrates Executors using different Deployments into a processing pipeline to accomplish a task.

Learn Gateway

The Gateway is a microservice that serves as the entrypoint of a Flow. It exposes multiple protocols for external communications and routes all internal traffic.

Explore Executor Hub

Executor Hub is a marketplace that allows you to share, explore and test Executors.

Deploy a Flow to Cloud

Jina AI Cloud is the MLOps platform for hosting Jina projects.

Support#

Join Us#

Jina is backed by Jina AI and licensed under Apache-2.0.


Index | Module Index