TODO¶
v2¶
We will be moving away from using nptyping in v2.0.0.
It was written for an older era in python before the dramatic changes in the Python type system and is no longer actively maintained. We will be reimplementing a syntax that extends its array specification syntax to include things like ranges and extensible dtypes with varying precision (and is much less finnicky to deal with).
Type Checker Integration¶
The .pyi stubfile generation (numpydantic.meta) works for
keeping type checkers from complaining about various array formats
not literally being NDArray objects, but it doesn’t do the kind of
validation we would want to be able to use NDArray objects as full-fledged
python types, including validation propagation through scopes and
IDE type checking for invalid literals.
We want to hook into the type checking process to satisfy these type checkers:
mypy - has hooks, can be done with an extension
pyright - unclear if has hooks, might nee to monkeypatch
pycharm - unlikely this is possible, extensions need to be in Java and installed separately
Validation¶
Todo
Support pydantic value/range constraints - less than, greater than, etc.
Todo
Support different precision modes - eg. exact precision, or minimum precision where specifying a float32 would also accept a float64, etc.
Metadata¶
Todo
Use names in nptyping annotations in generated JSON schema metadata
All TODOs¶
Todo
To better support static type hinting and inspection (ie. so the type checker
is not only aware of the union of all return_types, but the specific array
type that was passed on model instantiation, as well as potentially
do shape and dtype checks during type checking (eg. so a wrongly shaped or dtyped
array assignment will be highlighted as wrong), we will be exploring adding
mypy/pylance/pyright hooks for dynamic type evaluation.
Todo
Document dev environment
Really it’s very simple, you just clone a fork and install
the dev environment like pip install '.[dev]'
Todo
Document pull requests if we ever receive one
Todo
Sorry this is unreadable, need to recall how to change the theme for generated mermaid diagrams but it is very late and i want to push this.
Todo
Currently, the version of the package the interface is from (usually numpydantic)
will be stored, but there is no means of resolving it on the fly.
If there is a mismatch between the marked interface description and the interface
that was matched on revalidation, a warning is emitted, but validation
attempts to proceed as normal.
This feature is for extra-verbose provenance, rather than airtight serialization and deserialization, but PRs welcome if you would like to make it be that way.
Todo
We will also add a separate mark_version parameter for marking
the specific version of the relevant interface package, like zarr, or numpy,
patience.
Todo
Future versions will support interfaces providing type maps for declaring equality between dtypes that may be specific to that library but should be considered equivalent to numpy or other library’s dtypes.
Todo
Future versions will also support declaring minimum or maximum precisions, so one might say “at least a 16-bit float” and also accept a 32-bit float.
Todo
numpydantic currently does not support structured dtypes or numpy.recarray
specifications like nptyping does. It will in future versions.
Todo
numpydantic also does not support the variable shape definition form like
Shape['Dim, Dim']
where there are two dimensions of any shape as long as they are equal
because at the moment it appears impossible to express dynamic constraints
(ie. minItems/maxItems that depend on the shape of another array)
in JSON Schema. A future minor version will allow them by generating a JSON
schema with a warning that the equal shape constraint will not be represented.
See: https://github.com/orgs/json-schema-org/discussions/730
Todo
Support pydantic value/range constraints - less than, greater than, etc.
Todo
Support different precision modes - eg. exact precision, or minimum precision where specifying a float32 would also accept a float64, etc.
Todo
Use names in nptyping annotations in generated JSON schema metadata