Type stubs for rosbag2_py

[MichaelFishmanBDAII]

Description

I'd like type stubs for rosbag2_py, so that I can get autocomplete/static analysis.

Completion Criteria

** What needs to be true before we can call this "Done"? Bullet lists are appropriate. **

• The maximal version: All classes/methods/functions exposed by rosbag2_py have full type annotations. A static analysis tool like mypy/pylance can tell parse these type annotations.

Implementation Notes / Suggestions

Stubs can be generated with pybind11-stubgen:

pip install pip install pybind11-stubgen

pybind11-stubgen rosbag2_py

After doing this, making a setup.py, and installing the newly created rosbag2_py-stubs library, we get some type information, though the function argument and return types are incomplete:

The output from the pybind11-stubgen command is

pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialCompressionReader : Invalid expression 'rosbag2_storage::TopicMetadata'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialCompressionReader : Invalid expression 'rosbag2_storage::StorageOptions'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialCompressionReader : Invalid expression 'rosbag2_cpp::ConverterOptions'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialCompressionReader : Invalid expression 'rosbag2_storage::StorageFilter'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialReader : Invalid expression 'rosbag2_storage::TopicMetadata'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialReader : Invalid expression 'rosbag2_storage::StorageOptions'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialReader : Invalid expression 'rosbag2_cpp::ConverterOptions'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialReader : Invalid expression 'rosbag2_storage::StorageFilter'
pybind11_stubgen - [WARNING] Raw C++ types/values were found in signatures extracted from docstrings.
Please check the corresponding sections of pybind11 documentation to avoid common mistakes in binding code:
 - https://pybind11.readthedocs.io/en/latest/advanced/misc.html#avoiding-cpp-types-in-docstrings
 - https://pybind11.readthedocs.io/en/latest/advanced/functions.html#default-arguments-revisited

I expect that addressing the issues in this output would get us the missing type information.

Testing Notes / Suggestions

** All features in this project need tests. Please give some input on cases that will need to be tested - and how the testing might be implemented. **

A test could run mypy on an example python script using rosbag2_py. I'm not sure what mypy options would need to be set to verify that all the rosbag2_py objects are properly typed.

[MichaelOrlov]

@emersonknapp Do you have thoughts about this proposal?

[MichaelFishmanBDAII]

Currently, the type annotations for methods are generic. Eg. SequentialReader.read_next has the return type of tuple:

pybind11::tuple read_next()

Ideally, it would have the return type tuple[str, bytes, int].

pybind's test_python_types.cpp has examples showing how to specify these more specific types.

    /* pybind automatically translates between C++11 and Python tuples */
    std::tuple<int, std::string, bool> tuple_passthrough(std::tuple<bool, std::string, int> input) {
        return std::make_tuple(std::get<2>(input), std::get<1>(input), std::get<0>(input));
    }

[r7vme]

@MichaelOrlov should we reopen this issue or create a new one for addressing issues that pybind11-stubgen found?

pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialCompressionReader : Invalid expression 'rosbag2_storage::TopicMetadata'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialCompressionReader : Invalid expression 'rosbag2_storage::StorageOptions'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialCompressionReader : Invalid expression 'rosbag2_cpp::ConverterOptions'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialCompressionReader : Invalid expression 'rosbag2_storage::StorageFilter'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialReader : Invalid expression 'rosbag2_storage::TopicMetadata'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialReader : Invalid expression 'rosbag2_storage::StorageOptions'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialReader : Invalid expression 'rosbag2_cpp::ConverterOptions'
pybind11_stubgen - [  ERROR] In rosbag2_py._reader.SequentialReader : Invalid expression 'rosbag2_storage::StorageFilter'
pybind11_stubgen - [WARNING] Raw C++ types/values were found in signatures extracted from docstrings.
Please check the corresponding sections of pybind11 documentation to avoid common mistakes in binding code:
 - https://pybind11.readthedocs.io/en/latest/advanced/misc.html#avoiding-cpp-types-in-docstrings
 - https://pybind11.readthedocs.io/en/latest/advanced/functions.html#default-arguments-revisited

[MichaelOrlov]

@r7vme Hold on. But how the pybind11-stubgen found these issues if we didn't change those files from the last run?

Certainly need to open a new issue or PR.

[r7vme]

@MichaelOrlov I just copy pasted output from the above. We did not break anyting yet :) In any case, I think we can try to address those issues.

[r7vme]

Can we reopen this issue?

[MichaelOrlov]

Ok. I reopen this issue.

[MichaelOrlov]

@r7vme Am I understand correctly that we already added py stubs in the scope of the #1569 and can close this issue now?

[r7vme]

Yeah, let's close this issue, major issue is fixed.