Interfaces

Base Interface metaclass

pydantic model InterfaceMark[source]

JSON-able mark to be able to round-trip json dumps

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Fields:
field cls: str [Required]
field module: str [Required]
field name: str [Required]
field version: str [Required]
is_valid(cls: Type[Interface], raise_on_error: bool = False) → bool[source]

Check that a given interface matches the mark.

Parameters:

  • cls (Type) – Interface type to check

  • raise_on_error (bool) – Raise an MarkMismatchError when the match is incorrect

Returns:

bool

Raises:

  • .MarkMismatchError –

  • for an invalid match –

match_by_name() → Type[Interface] | None[source]

Try to find a matching interface by its name, returning it if found, or None if not found.

pydantic model JsonDict[source]

Representation of array when dumped with round_trip == True.

Developer’s Note

Any JsonDict that contains an actual array should be named value rather than array (or any other name), and nothing but the array data should be named value .

During JSON serialization, it becomes ambiguous what contains an array of data vs. an array of metadata. For the moment we would like to reserve the ability to have lists of metadata, so until we rule that out, we would like to be able to avoid iterating over every element of an array in any context parameter transformation like relativizing/absolutizing paths. To avoid that, it’s good to agree on a single value name – value – and avoid using it for anything else.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Fields:
field type: str [Required]
classmethod handle_input(value: dict | U | W) → V | W[source]

Handle input that is the json serialized roundtrip version (from model_dump() with round_trip=True) converting it to the input format with JsonDict.to_array_input() or passing it through if not applicable

classmethod is_valid(val: dict, raise_on_error: bool = False) → bool[source]

Check whether a given dictionary matches this JsonDict specification

Parameters:

  • val (dict) – The dictionary to check for validity

  • raise_on_error (bool) – If True, raise the validation error rather than returning a bool. (default: False)

Returns:

bool - true if valid, false if not

static reshape_input(value: T, shape: tuple[int, ...]) → T[source]

If a reshape value is present on the array, and the array shape doesn’t match, attempt to reshape it.

abstract to_array_input() → V[source]

Convert this roundtrip specifier to the relevant input class (one of the input_types of an interface).

pydantic model MarkedJson[source]

Model of JSON dumped with an additional interface mark with model_dump_json({'mark_interface': True})

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Fields:
field interface: InterfaceMark [Required]
field value: list | dict [Required]

Inner value of the array, we don’t validate for JsonDict here, that should be downstream from us for performance reasons

classmethod try_cast(value: V | dict) → V | MarkedJson[source]

Try to cast to MarkedJson if applicable, otherwise return input

class Interface(shape: Tuple[int, ...] | Any = typing.Any, dtype: str | type | Any | generic = typing.Any)[source]

Abstract parent class for interfaces to different array formats

validate(array: Any) → T[source]

Validate input, returning final array type

Calls the methods, in order:

  • array = deserialize() (array)

  • array = before_validation() (array)

  • dtype = get_dtype() (array) - get the dtype from the array,

    override if eg. the dtype is not contained in array.dtype

  • valid = validate_dtype() (dtype) - check that the dtype matches

    the one in the NDArray specification. Override if special validation logic is needed for a given format

  • raise_for_dtype() (valid, dtype) - after checking dtype validity,

    raise an exception if it was invalid. Override to implement custom exceptions or error conditions, or make validation errors conditional.

  • array = after_validate_dtype() (array) - hook for additional

    validation or array modification mid-validation

  • shape = get_shape() (array) - get the shape from the array,

    override if eg. the shape is not contained in array.shape

  • valid = validate_shape() (shape) - check that the shape matches

    the one in the NDArray specification. Override if special validation logic is needed.

  • raise_for_shape() (valid, shape) - after checking shape validity,

    raise an exception if it was invalid. You know the deal bc it’s the same as raise for dtype.

  • after_validation() - hook after validation for modifying the array

    that is set as the model field value

Follow the method signatures and return types to override.

Implementing an interface subclass largely consists of overriding these methods as needed.

Raises:

  • If validation fails, rather than eg. returning False, exceptions will –

  • be raised (to halt the rest of the pydantic validation process). –

  • When using interfaces outside of pydantic, you must catch both –

  • .DtypeError –

:raises of InterfaceError ):

deserialize(array: Any) → V | Any[source]

If given a JSON serialized version of the array, deserialize it first.

If a roundtrip-serialized JsonDict, pass to JsonDict.handle_input().

If a roundtrip-serialized MarkedJson, unpack mark, check for validity, warn if not, and try to continue with validation

before_validation(array: Any) → NDArrayType[source]

Optional step pre-validation that coerces the input into a type that can be validated for shape and dtype

Default method is a no-op

get_dtype(array: NDArrayType) → str | type | Any | generic[source]

Get the dtype from the input array.

get_object_dtype(array: NDArrayType) → str | type | Any | generic[source]

When an array contains an object, get the dtype of the object contained by the array.

If this method returns Any, the dtype validation passes - used for e.g. empty arrays for which the dtype of the array can’t be determined (since there are no objects).

validate_dtype(dtype: str | type | Any | generic) → bool[source]

Validate the dtype of the given array, returning True if valid, False if not.

raise_for_dtype(valid: bool, dtype: str | type | Any | generic) → None[source]

After validating, raise an exception if invalid :raises ~numpydantic.exceptions.DtypeError:

after_validate_dtype(array: NDArrayType) → NDArrayType[source]

Hook to modify array after validating dtype. Default is a no-op.

get_shape(array: NDArrayType) → Tuple[int, ...][source]

Get the shape from the array as a tuple of integers

validate_shape(shape: Tuple[int, ...]) → bool[source]

Validate the shape of the given array against the shape specifier, returning True if valid, False if not.

raise_for_shape(valid: bool, shape: Tuple[int, ...]) → None[source]

Raise a ShapeError if the shape is invalid.

Raises:

ShapeError –

after_validation(array: NDArrayType) → T[source]

Optional step post-validation that coerces the intermediate array type into the return type

Default method is a no-op

abstract classmethod check(array: Any) → bool[source]

Method to check whether a given input applies to this interface

abstract classmethod enabled() → bool[source]

Check whether this array interface can be used (eg. its dependent packages are installed, etc.)

abstract property name: str

Short name for this interface

abstract property json_model: JsonDict

The JsonDict model used for roundtripping JSON serialization

abstract classmethod to_json(array: Type[T], info: SerializationInfo) → list | JsonDict[source]

Convert an array of return_type to a JSON-compatible format using base python types

classmethod mark_json(array: list | dict) → dict[source]

When using model_dump_json with mark_interface: True in the context, add additional annotations that would allow the serialized array to be roundtripped.

Default is just to add an InterfaceMark

Examples

>>> from pprint import pprint
>>> pprint(Interface.mark_json([1.0, 2.0]))
{'interface': {'cls': 'Interface',
               'module': 'numpydantic.interface.interface',
               'version': '1.2.2'},
 'value': [1.0, 2.0]}
classmethod interfaces(with_disabled: bool = False, sort: bool = True) → Tuple[Type[Interface], ...][source]

Enabled interface subclasses

Parameters:

  • with_disabled (bool) – If True , get every known interface. If False (default), get only enabled interfaces.

  • sort (bool) – If True (default), sort interfaces by priority. If False , sorted by definition order. Used for recursion: we only want to sort once at the top level.

classmethod return_types() → Tuple[NDArrayType, ...][source]

Return types for all enabled interfaces

classmethod input_types() → Tuple[Any, ...][source]

Input types for all enabled interfaces

classmethod match_mark(array: Any) → Type[Interface] | None[source]

Match a marked JSON dump of this array to the interface that it indicates.

First find an interface that matches by name, and then run its check method, because arrays can be dumped with a mark but without round_trip == True (and thus can’t necessarily use the same interface that they were dumped with)

Returns:

Interface if match found, None otherwise

classmethod match(array: Any, fast: bool = False) → Type[Interface][source]

Find the interface that should be used for this array based on its input type

First runs the check method for all interfaces returned by Interface.interfaces() except for NumpyInterface , and if no match is found then try the numpy interface. This is because NumpyInterface.check() can be expensive, as we could potentially try to

Parameters:

fast (bool) – if False , check all interfaces and raise exceptions for having multiple matching interfaces (default). If True , check each interface (as ordered by its priority , decreasing), and return on the first match.

classmethod match_output(array: Any) → Type[Interface][source]

Find the interface that should be used based on the output type - in the case that the output type differs from the input type, eg. the HDF5 interface, match an instantiated array for purposes of serialization to json, etc.

classmethod mark_interface() → InterfaceMark[source]

Create an interface mark indicating this interface for validation after JSON serialization with round_trip==True