canonical way to get python type from C++ type

[ghorn]

I'm trying to customize signatures programatically, which requires a reliable way of getting python types from C++ types.

The low-level interface docs suggest for this use case to use nb::type<MyType>, which works for bound types. However, this doesn't show types that are converted by type casters. If I use make_caster<MyType>::Name, that doesn't show bound types. Right now I'm trying one and falling back on the other, and that seems to work, but it seems brittle.

From reading the source code, it seems that the true type signature is actually made in nb_func_render_signature. Does that mean that the canonical way to get the python type from C++ type is to create a function (perhaps with cpp_function on a throwaway lambda), and then get its signature with getattr/__nb_signature__ and then parse it? That would make sense, since C++ types can have different python types depending on if it's an argument or return value, etc.

Is this the simplest way to canonically get a python type string from a C++ type? Something like:

// parse "T" from "def f() -> T"
std::string parse_ret_val(std::string) {...};

template <typename T>
object c_to_py_name_retval() {
  auto fun = nb::cpp_function([]() -> T {
    return {};
  });
  return parse_ret_val(nb::getattr(fun.release(), "__nb_signature__")));
}

// parse "T" from "def f(arg: T) -> None"
std::string parse_arg_val(std::string) {...};

template <typename T>
object c_to_py_name_argval() {
  auto fun = nb::cpp_function([](const T&) -> void {
    return;
  });
  return parse_arg_val(nb::getattr(fun.release(), "__nb_signature__")));
}

Is there a reason not to expose such functionality as part of the API?

edit - for reference, my current method is something like:

template <typename T>
std::string lookup_bound_python_type_name()
{
  nb::handle type_handle = nb::type<T>();
  if (!type_handle.is_valid())  {
    // No bound types, use a type caster.
    return std::string(nb::detail::make_caster<T>::Name.text);
  }

  / sanity check
  if (!nb::type_check(type_handle))  {
    throw std::runtime_error("Failed type checking");
  }

  // extract the bound type name
  nb::str type_name_str = nb::type_name(type_handle);
  if (type_name_str.c_str() == nullptr)  {
    throw std::runtime_error("Type name fail");
  }
  return std::string(type_name_str.c_str());
}

[wjakob]

Right now I'm trying one and falling back on the other, and that seems to work, but it seems brittle.

No, that is the way to go. You should be able to avoid the first call using something like (untested)

using Caster = nanobind::detail::make_caster<T>;

if constexpr (nanobind::detail::is_base_caster_v<Caster>)
     // case A
else
     // case B

[wjakob]

That's a good point. nb::detail::make_caster<T>::Name is an implementation detail and not really expected to be parseable in user code.

It would be useful if you could explain in more detail what you wish to do. Is the stock stubgen.py script not expressive enough to accomplish something?

The reason why I was negative about __nb_signature__ is because it targets the entire function. Your request was more specifically about the name of an individual type.

[ghorn]

My use case is almost supported but not quite. I'm using bind_vector (and a custom bind_array) to bind some in-house containers. These containers are fields in my structs and I'm using def_prop_rw. This is a rabbit hole, sorry:

by default:

bar.my_vector = FooVector([Foo(), Foo()])  # works fine
bar.my_vector = [Foo(), Foo()] # works at runtime but mypy is not happy

This is because the conversion from iterator to FooVector happens in __init__ and not on assignment, so mypy can't see past the setter property signature.

I tried many things to get around this. The simplest would be

.def_prop_rw(
  "my_vector",
  // getter
  [](Bar&) -> vector<Foo>& {...},
  // setter
  [](Bar&, vector<Foo>&) -> void {...},
  nb::for_setter(nb::sig("def my_vector(self, arg: FooVector | list[Foo], /) -> None"))
)

but I ran into this 7 year old megathread about how mypy only looks at the getter, not the setter python/mypy#3004

I tried the nanobind-suggested white lie where in bind_vector you pretend to inherit from collections.abc.Sequence[Foo]. That had its own issues - MutableSequence is compatible with std::vector, but there is no abc container that's compatible with std::array. Also there was some interaction with __eq__ that I didn't get to the bottom of, maybe a non-issue.

The mypy thread python/mypy#3004 suggests that custom getter/setter decorators would work around the mypy issue and allow different getter/setter types, but that would require a nanobind change and I'm not sure where the custom decorator would live - maybe in the type stub itself?

Currently I want to just get past this, so I'll probably do:

.def_prop_rw(
  "my_vector",
  // getter
  [](Bar&) -> vector<Foo>& {...},
  // setter
  [](Bar&, vector<Foo>&) -> void {...},
  nb::for_getter(nb::sig("def my_vector(self) -> FooVector | list[Foo]")),
  nb::for_setter(nb::sig("def my_vector(self, arg: FooVector | list[Foo], /) -> None")),
)

so that the setter is compatible with lists. I don't think users will mind the default getter maybe returning a list (it never actually would, it'd always return FooVector). I think most of our code is agnostic to the setter part, we just __getitem__ or __iter__ usually and list and bind_vector both support that.

Anyway, the thing these approaches mostly have in common is that it's useful to be able to get the string representation of the container FooVector and it's element type Foo (or int or int | None), so that I can modify the signatures with nb::sig().

[ghorn]

I ended up writing custom protocols for my container types. List and other containers are compatible with these protocols. When I bind my containers I fake the inheritance from the protocols. When I bind field types setting/getting the containers, I override the signature to show the protocol type instead. I got the cpp_function based type lookup function working and it's very robust.

[ghorn]

do you have any ideas how to get around the mypy getter/setter type issue? python/mypy#3004
Do you grok the custom decorator workaround?

[wjakob]

I got around it by giving up on MyPy and switching to Pyright. It seems to me that MyPy does not see that much active development these days.