jina.types.ndarray.generic module

class jina.types.ndarray.generic.NdArray(proto=None, is_sparse=False, dense_cls=<class 'jina.types.ndarray.dense.numpy.DenseNdArray'>, sparse_cls=<class 'jina.types.ndarray.sparse.scipy.SparseNdArray'>, *args, **kwargs)[source]

Bases: jina.types.ndarray.BaseNdArray

NdArray is one of the primitive data type in Jina.

It offers a Pythonic interface to allow users access and manipulate jina.jina_pb2.NdArrayProto object without working with Protobuf itself.

A generic view of the Protobuf NdArray, unifying the view of DenseNdArray and SparseNdArray

This class should be used in nearly all the Jina context.

Simple usage:

# start from empty proto
a = NdArray()

# start from an existig proto
a = NdArray(doc.embedding)

# set value
a.value = np.random.random([10, 5])

# get value
print(a.value)

# set value to a TF sparse tensor
a.is_sparse = True
a.value = SparseTensor(...)
print(a.value)

Advanced usage:

NdArray also takes a dense NdArray and a sparse NdArray constructor as arguments. You can consider them as the backend for dense and sparse NdArray. The combination is your choice, it could be:

# numpy (dense) + scipy (sparse)
from .dense.numpy import DenseNdArray
from .sparse.scipy import SparseNdArray
NdArray(dense_cls=DenseNdArray, sparse_cls=SparseNdArray)

# numpy (dense) + pytorch (sparse)
from .dense.numpy import DenseNdArray
from .sparse.pytorch import SparseNdArray
NdArray(dense_cls=DenseNdArray, sparse_cls=SparseNdArray)

# numpy (dense) + tensorflow (sparse)
from .dense.numpy import DenseNdArray
from .sparse.tensorflow import SparseNdArray
NdArray(dense_cls=DenseNdArray, sparse_cls=SparseNdArray)

Once you set sparse_cls, it will only accept the data type in that particular type. That is, you can not use a NdArray equipped with Tensorflow sparse to set/get Pytorch or Scipy sparse matrices.

Parameters

  • proto (Optional[NdArrayProto]) – the protobuf message, when not given then create a new one via get_null_proto()

  • is_sparse (bool) – if the ndarray is sparse, can be changed later

  • dense_cls (Type[BaseDenseNdArray]) – the to-be-used class for DenseNdArray when is_sparse=False

  • sparse_cls (Type[BaseSparseNdArray]) – the to-be-used class for SparseNdArray when is_sparse=True

  • args – additional positional arguments stored as member and used for the parent initialization

  • kwargs – additional key value arguments stored as member and used for the parent initialization

Set the constructor method.

null_proto()[source]

Get the new protobuf representation.

Returns

ndarray proto instance

property value

Get the value of protobuf and return in corresponding type.

Returns

value