jina.helper module

jina.helper.batch_iterator(data, batch_size, axis=0)[source]

Get an iterator of batches of data.

For example: .. highlight:: python .. code-block:: python

for req in batch_iterator(data, batch_size, split_over_axis):

# Do something with batch

Parameters
  • data (Iterable[Any]) – Data source.

  • batch_size (int) – Size of one batch.

  • axis (int) – Determine which axis to iterate for np.ndarray data.

Yield

data

Return type

Iterator[Any]

Returns

An Iterator of batch data.

jina.helper.parse_arg(v)[source]

Parse the arguments from string to Union[bool, int, str, list, float].

Parameters

v (str) – The string of arguments

Return type

Union[bool, int, str, list, float, None]

Returns

The parsed arguments list.

jina.helper.random_port()[source]

Get a random available port number from ‘49153’ to ‘65535’.

Return type

Optional[int]

Returns

A random port.

jina.helper.random_identity(use_uuid1=False)[source]

Generate random UUID.

..note::

A MAC address or time-based ordering (UUID1) can afford increased database performance, since it’s less work to sort numbers closer-together than those distributed randomly (UUID4) (see here).

A second related issue, is that using UUID1 can be useful in debugging, even if origin data is lost or not explicitly stored.

Parameters

use_uuid1 (bool) – use UUID1 instead of UUID4. This is the default Document ID generator.

Return type

str

Returns

A random UUID.

jina.helper.random_uuid(use_uuid1=False)[source]

Get a random UUID.

Parameters

use_uuid1 (bool) – Use UUID1 if True, else use UUID4.

Return type

UUID

Returns

A random UUID.

jina.helper.expand_env_var(v)[source]

Expand the environment variables.

Parameters

v (str) – String of environment variables.

Return type

Union[bool, int, str, list, float, None]

Returns

Parsed environment variables.

jina.helper.colored(text, color=None, on_color=None, attrs=None)[source]

Give the text with color.

Parameters
  • text (str) – The target text.

  • color (Optional[str]) –

    The color of text. Chosen from the following. {

    ’grey’: 30, ‘red’: 31, ‘green’: 32, ‘yellow’: 33, ‘blue’: 34, ‘magenta’: 35, ‘cyan’: 36, ‘white’: 37

    }

  • on_color (Optional[str]) –

    The on_color of text. Chosen from the following. {

    ’on_grey’: 40, ‘on_red’: 41, ‘on_green’: 42, ‘on_yellow’: 43, ‘on_blue’: 44, ‘on_magenta’: 45, ‘on_cyan’: 46, ‘on_white’: 47

    }

  • attrs (Union[str, list, None]) –

    Attributes of color. Chosen from the following. {

    ’bold’: 1, ‘dark’: 2, ‘underline’: 4, ‘blink’: 5, ‘reverse’: 7, ‘concealed’: 8

    }

Return type

str

Returns

Colored text.

class jina.helper.ArgNamespace[source]

Bases: object

Helper function for argparse.Namespace object.

static kwargs2list(kwargs)[source]

Convert dict to an argparse-friendly list.

Parameters

kwargs (Dict) – dictionary of key-values to be converted

Return type

List[str]

Returns

argument list

static kwargs2namespace(kwargs, parser, warn_unknown=False, fallback_parsers=None)[source]

Convert dict to a namespace.

Parameters
  • kwargs (Dict[str, Union[str, int, bool]]) – dictionary of key-values to be converted

  • parser (ArgumentParser) – the parser for building kwargs into a namespace

  • warn_unknown (bool) – True, if unknown arguments should be logged

  • fallback_parsers (Optional[List[ArgumentParser]]) – a list of parsers to help resolving the args

Return type

Namespace

Returns

argument list

static get_non_defaults_args(args, parser, taboo=None)[source]

Get non-default args in a dict.

Parameters
  • args (Namespace) – the namespace to parse

  • parser (ArgumentParser) – the parser for referring the default values

  • taboo (Optional[Set[str]]) – exclude keys in the final result

Return type

Dict

Returns

non defaults

static flatten_to_dict(args)[source]

Convert argparse.Namespace to dict to be uploaded via REST.

Parameters

args (Union[Dict[str, Namespace], Namespace]) – namespace or dict or namespace to dict.

Return type

Dict[str, Any]

Returns

pea args

jina.helper.is_valid_local_config_source(path)[source]

Check if the path is valid.

Parameters

path (str) – Local file path.

Return type

bool

Returns

True if the path is valid else False.

class jina.helper.cached_property(func)[source]

Bases: object

The decorator to cache property of a class.

Create the cached_property.

Parameters

func – Cached function.

jina.helper.typename(obj)[source]

Get the typename of object.

Parameters

obj – Target object.

Returns

Typename of the obj.

jina.helper.get_public_ip(timeout=0.3)[source]

Return the public IP address of the gateway for connecting from other machine in the public network.

Parameters

timeout (float) – the seconds to wait until return None.

Returns

Public IP address.

jina.helper.get_internal_ip()[source]

Return the private IP address of the gateway for connecting from other machine in the same network.

Returns

Private IP address.

jina.helper.convert_tuple_to_list(d)[source]

Convert all the tuple type values from a dict to list.

Parameters

d (Dict) – Dict type of data.

jina.helper.run_async(func, *args, **kwargs)[source]

Generalized asyncio.run for jupyter notebook.

When running inside jupyter, an eventloop is already exist, can’t be stopped, can’t be killed. Directly calling asyncio.run will fail, as This function cannot be called when another asyncio event loop is running in the same thread.

call run_async(my_function, any_event_loop=True, *args, **kwargs) to enable run with any eventloop

Parameters
  • func – function to run

  • args – parameters

  • kwargs – key-value parameters

Returns

asyncio.run(func)

jina.helper.deprecated_alias(**aliases)[source]

Usage, kwargs with key as the deprecated arg name and value be a tuple, (new_name, deprecate_level).

With level 0 means warning, level 1 means exception.

For example:
Parameters

aliases – maps aliases to new arguments

Returns

wrapper

jina.helper.countdown(t, reason='I am blocking this thread')[source]

Display the countdown in console.

For example:
Parameters
  • t (int) – Countdown time.

  • reason (str) – A string message of reason for this Countdown.

Return type

None

class jina.helper.CatchAllCleanupContextManager(sub_context)[source]

Bases: object

This context manager guarantees, that the :method:__exit__ of the sub context is called, even when there is an Exception in the :method:__enter__.

Parameters

sub_context – The context, that should be taken care of.

jina.helper.download_mermaid_url(mermaid_url, output)[source]

Download the jpg image from mermaid_url.

Parameters
  • mermaid_url – The URL of the image.

  • output – A filename specifying the name of the image to be created, the suffix svg/jpg determines the file type of the output image.

Return type

None

jina.helper.get_readable_size(num_bytes)[source]

Transform the bytes into readable value with different units (e.g. 1 KB, 20 MB, 30.1 GB).

Parameters

num_bytes (Union[int, float]) – Number of bytes.

Return type

str

Returns

Human readable string representation.

jina.helper.get_or_reuse_loop()[source]

Get a new eventloop or reuse the current opened eventloop.

Returns

A new eventloop or reuse the current opened eventloop.