Session close deadlock

[yellowhatter]

@fuzzypixelz : I finally understand the deadlock. It's between a TaskController::wait in zenoh (Rust) and a C++ std::recursive_mutex in rmw_zenoh.

• some other peer shuts down
• zenoh rust receives a token undeclaration from a peer
• the callback reaches into rmw_zenoh while on the RX runtime
• the subscriber tries to get the context lock but the rmw_shutdown function gets the lock first and then tries to close the session
• the session close hangs because it's waiting on the RX task to complete but it can't because it's waiting on the lock we just took

@yellowhatter : Okay, let me take care of this
I think we need to solve it on Rust side
to make our core more safe in terms of user-dependent deadlocks like this

[YuanYuYuan]

To reproduce, compile the test_communication and run the test with the following.

colcon test --event-handlers console_cohesion+ console_direct+ --packages-select test_rclcpp --ctest-args -R test_n_nodes__rmw_zenoh_cpp

The deadlock happens upon acquiring the lock here

rmw_zenoh/rmw_zenoh_cpp/src/detail/rmw_context_impl_s.cpp

Line 62 in 28d917e

std::lock_guard<std::recursive_mutex> lock(data_ptr->mutex_);