ndarray#

Extension of nptyping NDArray for pydantic that allows for JSON-Schema serialization

  • Order to store data in (row first)

COMPRESSION_THRESHOLD = 16384#

Arrays larger than this size (in bytes) will be compressed and b64 encoded when serializing to JSON.

list_of_lists_schema(shape: ~nptyping.shape.Shape[*, ...], array_type_handler: dict) → ListSchema#

Make a pydantic JSON schema for an array as a list of lists.

jsonize_array(array: ndarray | dask.array.core.Array | NDArrayProxy) → list | dict#

Render an array to base python types that can be serialized to JSON

For small arrays, returns a list of lists.

If the array is over COMPRESSION_THRESHOLD bytes, use compress_array() to return a compressed b64 encoded string.

Parameters:

array (np.ndarray, dask.DaskArray) – Array to render as a list!

get_validate_shape(shape: ~nptyping.shape.Shape[*, ...]) → Callable#

Get a closure around a shape validation function that includes the shape definition

get_validate_dtype(dtype: dtype) → Callable#

Get a closure around a dtype validation function that includes the dtype definition

coerce_list(value: Any) → ndarray#

If a value is passed as a list or list of lists, try and coerce it into an array rather than failing validation.

class NDArrayMeta(name: str, *args: Any, **kwargs: Any)#

Kept here to allow for hooking into metaclass, which has been necessary on and off as we work this class into a stable state

class NDArray(*_: Any, **__: Any)#

Constrained array type allowing npytyping syntax for dtype and shape validation and serialization.

Integrates with pydantic such that - JSON schema for list of list encoding - Serialized as LoL, with automatic compression for large arrays - Automatic coercion from lists on instantiation

Also supports validation on NDArrayProxy types for lazy loading.

References