jina.serve.runtimes.base module#
- class jina.serve.runtimes.base.BaseRuntime(args, **kwargs)[source]#
Bases:
object
A Jina Runtime is a procedure that blocks the main process once running (i.e.
run_forever()
), therefore should be put into a separated thread/process, or inside the main process of a docker container.Any program/library/package/module that blocks the main process, can be formulated into a
BaseRuntime
class and then be started from aPod
.In the sequel, we call the main process/thread as
M
, the process/thread blockedRuntime
asS
.In Jina, a
Pod
object is used to manage aRuntime
object’s lifecycle. APod
acts as amultiprocessing.Process
orthreading.Thread
, it starts fromM
and once theS
is spawned, it usesRuntime
as a context manager:__init__()
- meth
__enter__
2.
run_forever()
. Note that this will blockS
, step 3 won’t be reached until it is unblocked bycancel()
.3. When an error occurs during run_forever or cancel signal is reached by the runtime. The run_forever method is cancelled and the managed context is closed. The __exit__ of Runtime guarantees that the Runtime is properly shut by calling teardown.
The
__init__()
andteardown()
pair together, which defines instructions that will be executed before and after. In subclasses, teardown is optional.In order to cancel the run_forever method of a Runtime, you can use their static cancel method that will make sure that the runtime is properly cancelled.
Use
threading.Event
or multiprocessing.Event, whilerun_forever()
polls for this eventUse
GrpcConnectionPool
to send a TERMINATE message, whilerun_forever()
polls for this message
Note, another way to jump out from
run_forever()
is raise exceptions from it. This will immediately move toteardown()
.Note
Rule of thumb on exception handling: if you are not sure if you should handle exception inside
run_forever()
,cancel()
,teardown()
, then DO NOT catch exception in them. Exception is MUCH better handled byPod
.See also
Pod
for managing aRuntime
object’s lifecycle.- run_forever()[source]#
Running the blocking procedure inside
S
. Note, once this method is called,S
is blocked.Note
If this method raises any exception,
teardown()
will be called.See also
cancel()
for cancelling the forever loop.
- teardown()[source]#
Method called immediately after
run_forever()
is unblocked. You can tidy up things here. Optional in subclasses. The default implementation does nothing.