Using Annotated in attrs

[Dr-ZeeD]

I have been playing with Annotated recently, and was wondering whether this is something that might be used within attrs. My first instinct was "maybe this could be used for validators?". Maybe even field()?

[euresti]

Full disclosure I haven't had a chance to use Annotated yet, though I'm excited about it for my own projects.

Are you thinking of something like:

@attr.define
class MyClass:
   x: int
   y: Annotated[int, Field(validator=...)] = 15

My big concerns of doing it for validators are:

1. Type checking those validators. I don't know that the validator can actually be type-checked by mypy against the int. Though maybe that doesn't matter too much.
2. In 3.10 those will be "string annotations" and so you're more likely to have the get_type_hints issues.

We'd probably not just want to stick attr.ib as the annotation since we probably don't want people sticking default values etc in there.

Reading the Pep a little more it looks like one idea is to have a series of classes rather than one class for the Annotations.

So you could do something like this

But it could be nice to stick the bool arguments in there oh you can even do it as separate "Annotations"

from attr import NoInit, Comparable, define
@attr.define
class MyClass:
   x: Annotated[int, NoInit(), Comparable()]
   y: Annotated[str, Comparable()]  = "wheee"

I can't decide if that's worse or better though.

[uSpike]

Just wanted to chime-in that I have a use-case for adding field configuration in Annotated types. Currently I'm using Annotated to denote parental relationship between objects:

from typing import Annotated, TypeVar
import attr

T = TypeVar("T", bound="Driver")

Parent = Annotated[T, "parent"]

@attr.s
class Driver:
    @property
    def parents(self) -> list[Driver]:
        def _is_parent_type(field: attr.Attribute, value: Any) -> bool:
            return type(field.type) is type(Parent) and "parent" in getattr(field.type, "__metadata__")
        return list(attr.asdict(self, recurse=False, filter=_is_parent_type).values())

@attr.s
class Bus(Driver):
   pass
   
@attr.s
class Device(Driver):
   bus: Parent[Bus] = attr.ib()

bus = Bus()
dev = Device(bus)
assert dev.parents == [bus]

What would be nice is to be able to add extra metadata into Parent so that things like repr=False could be passed into the attr.ib() field:

Parent = Annotated[T, "parent", Field(repr=False)]

[uSpike]

That said, it could even be improved to allow insertion of field metadata via Annotated:

Parent = Annotated[T, Field(repr=False, metadata={"parent": True})]

@attr.s
class Driver:
    @property
    def parents(self) -> list[Driver]:
        def _is_parent_type(field: attr.Attribute, value: Any) -> bool:
            return field.metadata.get("parent", False)
        return list(attr.asdict(self, recurse=False, filter=_is_parent_type).values())

[AdrianSosic]

I just also wanted to briefly express my interest in the possibility of using Annotated. As mentioned in #1275, I oftentimes find myself in situations where I have several attributes that have the exact same type hint + field definition, i.e. they more or less represent the same semantic meaning. In these cases, it would be nice if one could simply define this "meaning" as a type upfront and reuse the type across the different contexts where it is needed. AFAIK that's the primary use case for which pydantic uses Annotated.

[hynek]

Skimming the comments, do I see it correctly that there's somewhat a consensus to get rid of the special-cased x: int = field(validator=le_validator) / = attr.ib and use x: Annotated[int, Field(validator=le_validator))]?

Generally speaking, that seems like a good way to make the typing situation less about lying to the type checker.

Open questions for me are:

• is Annotated meant for that? I've seen some rescinding of feature in Pyright around that
• is it possible to express any of this using dataclass transforms?

cc @Tinche

[AdrianSosic]

Hi @hynek, just to make my point clear: I'm not suggesting to get rid of the good old field assignment approach. But using typing.Annotated in addition would allow to not only assign a "concept" to an attribute of one particular class but rather encapsulate this "concept" in the form of its own reusable semantic object.

Concrete example

Ideally, I would like to capture the semantics of a PrimeNumber in its own type and then outsource the parts of the field definition that are necessary to describe that specific meaning (i.e., the corresponding validators, converters, ...) to Annotated.

So instead if having to repeat the following field assignment

@define
class NeedsPrimeNumber:
    prime: int = field(default=3, validator=is_prime, converter=le_converter)


@define
class AlsoNeedsPrimeNumber:
    prime: int = field(default=5, validator=is_prime, converter=le_converter)

I'd rather want to write

PrimeNumber = Annotated[int, Field(validator=is_prime, converter=le_converter)]


@define
class UsingAnnotated:
    prime: PrimeNumber = field(default=3)

@define
class AlsoUsingAnnotated:
    prime: PrimeNumber = field(default=5)

where in this case only the default (which is class specific) remains part of the regular assignment.

Perhaps this was already clear to you, just thought a specific example could help 🙃

[dlax]

@AdrianSosic, in your example, I believe the right-side field(default=...) is superfluous and the example should rather lean towards:

PrimeNumber = Annotated[int, Field(validator=is_prime, converter=le_converter)]

@define
class UsingAnnotated:
    prime: PrimeNumber = 3

because, otherwise, the library has to tell type checkers that field(default=...) means the field has a default value; whereas, as written above, there's nothing to do on our side.

---

@hynek

do I see it correctly that there's somewhat a consensus to get rid of the special-cased x: int = field(validator=le_validator) / = attr.ib and use x: Annotated[int, Field(validator=le_validator))]?

IMHO, yes!

[AdrianSosic]

Hi @dlax 👋🏼 Probably, setting a default was not the best example from my side. But what I meant is that there could be other field-related setting that do not belong the the concept of a PrimeNumber and which I would like to set outside the PrimeNumber annotation.

For example:

PrimeNumber = Annotated[int, Field(validator=is_prime, converter=le_converter)]

@define
class RequiresPrimeLargerThanFive:
    prime: PrimeNumber = field(default=7, validator=gt(5))

So the specific class here needs a prime larger than five, but primes in general don't have this requirement.

Or are you suggesting you use a second layer of Annotated in this case, a la:

PrimeNumber = Annotated[int, Field(validator=is_prime, converter=le_converter)]

@define
class RequiresPrimeLargerThanFive:
    prime: Annotated[PrimeNumber, validator=gt(5)] = 7

with the general idea of moving every functionality of field to Annotated, i.e. also stuff like kw_only, eq, on_setattr, etc?

[dlax]

Yes, one key aspect of Annotated is that you can combine multiple pieces of metadata. But in your example, I might be strange to allow several Field values in Annotated, so perhaps we'd need dedicated metadata for validation/conversion:

PrimeNumber = Annotated[int, Validator(is_prime), Converter(le_converter)]

@define
class RequiresPrimeLargerThanFive:
    prime: Annotated[PrimeNumber, Validator(gt(5))] = 7

and keep the Field for things that are meant to be specified only once (like eq, kw_only, etc.)?