Jina AI Cloud Hosting#
After building a Jina project, the next step is to deploy and host it on the cloud. Jina AI Cloud is Jina’s reliable, scalable and production-ready cloud-hosting solution that manages your project lifecycle without surprises or hidden development costs.
At present, Jina AI Cloud hosts all your Jina projects and offers computational/storage resources for free!
Jina AI Cloud provides a CLI that you can use via
jina cloud from the terminal (or
jcloud or simply
jc for minimalists.)
You can also install just the JCloud CLI without installing the Jina package.
pip install jcloud jc -h
If you installed the JCloud CLI individually, all of its commands fall under the
In case the command
jc is already occupied by another tool, use
jcloud instead. If your pip install doesn’t register bash commands for you, you can run
python -m jcloud -h.
For the rest of this section, we use
jcloud. But again they are interchangable with
In Jina’s idiom, a project is a Flow, which represents an end-to-end task such as indexing, searching or recommending. In this document, we use “project” and “Flow” interchangeably.
Flows have a maximum lifetime after which they are automatically deleted.
A Flow can have two types of file structure: a single YAML file or a project folder.
Single YAML file#
usesmust follow the format
jinahub+docker://MyExecutor(from Executor Hub) to avoid any local file dependencies:
# flow.yml jtype: Flow executors: - name: sentencizer uses: jinahub+docker://Sentencizer
jc deploy flow.yml
We recommend testing locally before deployment:
jina flow --uses flow.yml
The best practice of creating a JCloud project is to use:
This ensures the correct project structure accepted by JCloud.
Just like a regular Python project, you can have sub-folders of Executor implementations and a
flow.yml on the top-level to connect all Executors together.
You can create an example local project using
jc new hello. The default structure looks like:
hello/ ├── .env ├── executor1 │ ├── config.yml │ ├── executor.py │ └── requirements.txt └── flow.yml
hello/is your top-level project folder.
executor1directory has all Executor related code/configuration. You can read the best practices for file structures. Multiple Executor directories can be created.
flow.ymlYour Flow YAML.
.envAll environment variables used during deployment.
jc deploy hello
The Flow is successfully deployed when you see:
You will get a Flow ID, say
173503c192. This ID is required to manage, view logs and remove the Flow.
As this Flow is deployed with the default gRPC gateway (feel free to change it to
websocket), you can use
jina.Client to access it:
from jina import Client, Document c = Client(host='https://173503c192.wolf.jina.ai') print(c.post('/', Document(text='hello')))
To watch the logs in real time:
jc logs 173503c192
You can also stream logs for a particular Executor by passing its name:
jc logs 173503c192 --executor sentencizer
You can remove a single Flow, multiple Flows or even all Flows by passing different identifiers.
To remove a single Flow:
jc remove 173503c192
To remove multiple Flows:
jc remove 173503c192 887f6313e5 ddb8a2c4ef
To remove all Flows:
jc remove all
By default, removing multiple or all Flows is an interactive process where you must give confirmation before each Flow is deleted. To make it non-interactive, set the below environment variable before running the command:
To get the status of a Flow:
jc status 15937a10bd
Basic monitoring is provided to Flows deployed on Jina AI Cloud.
To access the Grafana-powered dashboard, first get the status of the Flow. The
dashboards link is displayed at the bottom of the pane. Visit the URL to find basic metrics like ‘Number of Request Gateway Received’ and ‘Time elapsed between receiving a request and sending back the response’:
To list all of your “ALIVE” Flows:
You can also filter your Flows by passing a status:
jc list --status FAILED
Or see all Flows:
jc list --status ALL
Pass environment variables#
jc deploy flow.yml --env-file flow.env
You can include your environment variables in the
.envfile in the local project and Jina AI Cloud manages them.
You can optionally pass a
jc deploy ./hello --env-file ./hello/custom.env
If your deployment failed, enable verbose logging and redeploy it. You can add
--loglevel DEBUG before each CLI subcommand:
jc --loglevel DEBUG deploy flow.yml
Alternatively, you can configure it by using environment variable
JCLOUD_LOGLEVEL=DEBUG jc deploy flow.yml
If you don’t see any obvious errors, please raise an issue in the JCloud repository.
JCloud scales according to your needs. You can demand different resources (GPU/RAM/CPU/storage/instance-capacity) based on the needs of your Flows and Executors. If you have specific resource requirements, please contact us on Slack or raise a GitHub issue.
Deployments are only supported in the
Each Executor is allocated a maximum of 4GB RAM, 2 CPU cores & 10GB of block storage.
Three Flows can be deployed at a time, out of which one Flow can use a GPU.
A maximum of two GPUs are allocated per Flow.
Flows with Executors using GPU are removed after 12 hours, whereas other Flows are removed after 72 hours.