diff --git a/.github/ISSUE_TEMPLATE/BUG.yml b/.github/ISSUE_TEMPLATE/BUG.yml index ab864385..aee5410c 100644 --- a/.github/ISSUE_TEMPLATE/BUG.yml +++ b/.github/ISSUE_TEMPLATE/BUG.yml @@ -1,24 +1,74 @@ name: 🐞 Bug title: "[Bug]: " -description: Something doesn't behave as it should. +description: A bug that requires attention. labels: ["bug"] body: - type: textarea id: current attributes: label: Current Behavior - placeholder: The car doesnt see a traffic light on Map1. + description: Describe the current, unexpected behavior. + placeholder: The car doesn’t detect the traffic light on Map1. - type: textarea id: expected attributes: label: Expected Behavior - placeholder: The car does see the traffic light on Map1. + description: Describe the expected, correct behavior. + placeholder: The car should detect the traffic light on Map1. - type: textarea id: reproduce attributes: - label: How to reproduce the issue + label: Steps to Reproduce + description: Provide steps to reproduce the issue. placeholder: | - * Start the simulation with Map1 - * after 600m the car won't stop at the traffic light. \ No newline at end of file + * Start simulation on Map1. + * After 600m, the car should stop at the traffic light. + + - type: textarea + id: definition_of_done + attributes: + label: Definition of Done + description: Criteria to consider this issue resolved. + placeholder: | + - The car detects traffic lights on Map1 correctly. + - The issue no longer occurs in similar scenarios. + + - type: input + id: effort_estimate + attributes: + label: Effort Estimate + description: Approximate effort required (e.g., hours). + placeholder: Enter effort estimate. + + - type: textarea + id: testability + attributes: + label: Testability + description: Describe how to test the fix. + placeholder: Describe testing steps for verification. + + - type: textarea + id: dependencies + attributes: + label: Dependencies + description: List any dependent tasks or issues. + placeholder: Link dependencies here (e.g., \#123). + + - type: markdown + attributes: + value: | + **Add Priority Label**: + - p1: Immediate attention + - p2: High priority + - p3: Standard priority + - p4: Low priority + + **Add Group Label**: + - perception: Related to sensor processing and scene understanding + - planning: Related to path planning and decision making + - acting: Related to vehicle control and actuation + - system: Related to the general behavior of the system + - research: Related to research and experimentation + - infrastructure: Related to system infrastructure and setup diff --git a/.github/ISSUE_TEMPLATE/FEATURE.yml b/.github/ISSUE_TEMPLATE/FEATURE.yml index 65b5c9a6..366bab21 100644 --- a/.github/ISSUE_TEMPLATE/FEATURE.yml +++ b/.github/ISSUE_TEMPLATE/FEATURE.yml @@ -1,20 +1,58 @@ name: 💡 Feature title: "[Feature]: " -description: Something we should implement. +description: A new feature to be implemented. labels: ["feature"] body: - type: textarea - id: desc + id: description attributes: - label: Description - description: Short description about what to do. - placeholder: The vehicle should detect traffic lights and it's states. + label: Feature Description + description: Provide a summary of the feature. + placeholder: The vehicle should detect traffic lights and their states. - type: textarea - id: dod + id: definition_of_done attributes: label: Definition of Done - description: What is required to mark the issue as done? + description: Completion criteria for the feature. placeholder: | - - 90% of the traffic lights are detected - - 90% of the traffic light states are detected correctly" \ No newline at end of file + - Detects 90% of traffic lights. + - Correctly identifies 90% of traffic light states. + + - type: input + id: effort_estimate + attributes: + label: Effort Estimate + description: Approximate effort required (e.g., hours). + placeholder: Enter effort estimate. + + - type: textarea + id: testability + attributes: + label: Testability + description: How will the feature be tested? + placeholder: Describe test cases and success criteria. + + - type: textarea + id: dependencies + attributes: + label: Dependencies + description: List any dependencies on other issues. + placeholder: Link dependencies here (e.g., \#456). + + - type: markdown + attributes: + value: | + **Add Priority Label**: + - p1: Immediate attention + - p2: High priority + - p3: Standard priority + - p4: Low priority + + **Add Group Label**: + - perception: Related to sensor processing and scene understanding + - planning: Related to path planning and decision making + - acting: Related to vehicle control and actuation + - system: Related to the general behavior of the system + - research: Related to research and experimentation + - infrastructure: Related to system infrastructure and setup diff --git a/.github/ISSUE_TEMPLATE/ISSUE.yml b/.github/ISSUE_TEMPLATE/ISSUE.yml index 0ebbfe5d..6ad88fbf 100644 --- a/.github/ISSUE_TEMPLATE/ISSUE.yml +++ b/.github/ISSUE_TEMPLATE/ISSUE.yml @@ -1,78 +1,55 @@ -name: "Issue Template" -description: "Template for creating issues" -labels: [] -assignees: [] +name: "General Issue" +title: "[General]: " +description: Template for general issues. +labels: ["general"] body: - - type: markdown - attributes: - value: | - ## Please fill out all relevant fields: - - type: textarea id: description attributes: - label: "Detailed Description" - description: "Describe the issue in detail" + label: Detailed Description + description: Describe the issue in detail. placeholder: | - Enter description here + Enter description here: - What is the problem? - - What is the expected behavior? - - What is the actual behavior? + - What is the expected outcome? - type: textarea id: definition_of_done attributes: - label: "Definition of Done" - description: "What needs to be completed for the task to be considered done?" + label: Definition of Done + description: Criteria for considering the task complete. placeholder: | - Example criteria: - - [ ] Implementation completed and tested - - [ ] Documentation updated - - [ ] Code review passed - - [ ] All tests passing + - Implementation completed and tested + - Documentation updated + - Code review passed + - All tests passing - type: input id: effort_estimate attributes: - label: "Effort Estimate" - description: "Estimate the effort required (e.g., in hours)" - placeholder: "Enter effort estimate here" + label: Effort Estimate + description: Approximate effort required (e.g., hours). + placeholder: Enter effort estimate. - type: textarea id: testability attributes: - label: "Testability" - description: "How can the implementation be tested?" - placeholder: "Enter testing methods here" + label: Testability + description: How to verify the issue's resolution. + placeholder: Enter test methods and validation criteria. - type: textarea id: dependencies attributes: - label: "Dependencies" - description: "Are there dependencies on other tasks or systems?" - placeholder: | - Enter dependencies here - - Task A must be completed before this task can start - - Link issues here with # + label: Dependencies + description: List dependencies, if any. + placeholder: Link dependencies here (e.g., the issue \#789). - type: markdown - id: prioritization attributes: value: | - Add a priority label to this issue (p1 to p4) based on the following criteria: - - p1: Critical issue that needs immediate attention - - p2: Important issue that should be addressed soon - - p3: Issue that should be addressed in the near future - - p4: Issue that can be addressed later - - - type: markdown - id: labels - attributes: - value: | - Add any relevant labels to this issue: - - standard labels: + **Add Standard Label**: - bug: Something isn't working - enhancement: New feature or request - help wanted: Extra attention is needed @@ -81,9 +58,16 @@ body: - question: Further information is requested - wontfix: This will not be worked on - project-specific labels: + **Add Priority Label**: + - p1: Immediate attention + - p2: High priority + - p3: Standard priority + - p4: Low priority + + **Add Group Label**: - perception: Related to sensor processing and scene understanding - planning: Related to path planning and decision making - acting: Related to vehicle control and actuation + - system: Related to the general behavior of the system - research: Related to research and experimentation - infrastructure: Related to system infrastructure and setup diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index b6682121..04e74f47 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -123,5 +123,5 @@ jobs: rm -rf /tmp/.buildx-cache/cache/latest mv /tmp/.buildx-cache/cache-new/latest /tmp/.buildx-cache/cache/latest - - name: Prune all images older than 1 days from self-hosted runner + - name: Prune all images older than 1 day from self-hosted runner run: docker image prune -a -f --filter "until=24h" \ No newline at end of file diff --git a/.vscode/extensions.json b/.vscode/extensions.json index c27bbad5..596e223e 100644 --- a/.vscode/extensions.json +++ b/.vscode/extensions.json @@ -12,6 +12,7 @@ "richardkotze.git-mob", "ms-vscode-remote.remote-containers", "valentjn.vscode-ltex", - "ms-python.black-formatter" + "ms-python.black-formatter", + "augustocdias.tasks-shell-input" ] } \ No newline at end of file diff --git a/.vscode/launch.json b/.vscode/launch.json new file mode 100644 index 00000000..68420ca7 --- /dev/null +++ b/.vscode/launch.json @@ -0,0 +1,23 @@ +{ + // Use IntelliSense to learn about possible attributes. + // Hover to view descriptions of existing attributes. + // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 + "version": "0.2.0", + "configurations": [ + { + "name": "53000 leaderboard attach", + "type": "debugpy", + "request": "attach", + "connect": { + "host": "localhost", + "port": 53000 + }, + "pathMappings": [ + { + "localRoot": "${workspaceFolder}/code", + "remoteRoot": "${env:PAF_CATKIN_CODE_ROOT}" + } + ] + } + ], +} \ No newline at end of file diff --git a/.vscode/tasks.json b/.vscode/tasks.json new file mode 100644 index 00000000..9f845638 --- /dev/null +++ b/.vscode/tasks.json @@ -0,0 +1,22 @@ +{ + "version": "2.0.0", + "tasks": [ + { + "label": "Run 'catkin_make' in docker container. Choose container from list of running containers.", + "type": "shell", + "command": "docker exec -it ${input:container_name} bash -c 'cd ../../catkin_ws && source /opt/ros/noetic/setup.bash && bash devel/setup.bash && catkin_make'", + "problemMatcher": [], + "detail": "Executes 'catkin_make' selected container. Requires Tasks Shell Input Extension, in order to generate the list of containers.", + } + ], + "inputs": [ + { + "id": "container_name", + "type": "command", + "command": "shellCommand.execute", + "args": { + "command": "docker ps --format '{{.Names}}'", + } + } + ] +} \ No newline at end of file diff --git a/build/agent_service.yaml b/build/agent_service.yaml index 530eb208..5255aae4 100644 --- a/build/agent_service.yaml +++ b/build/agent_service.yaml @@ -30,6 +30,7 @@ services: - ROS_HOSTNAME=agent - XDG_RUNTIME_DIR=/tmp/runtime-carla - ROUTE=/opt/leaderboard/data/routes_devtest.xml + - DEBUG_WRAPPER_DEFAULT_HOST=0.0.0.0 # Simple route without special scenarios # - ROUTE=/workspace/code/routes/routes_simple.xml volumes: diff --git a/build/docker-compose.leaderboard.yaml b/build/docker-compose.leaderboard.yaml index 32fc98fc..4c43bd87 100644 --- a/build/docker-compose.leaderboard.yaml +++ b/build/docker-compose.leaderboard.yaml @@ -8,4 +8,11 @@ services: extends: file: agent_service.yaml service: agent - command: bash -c "sleep 10 && sudo chown -R ${USER_UID}:${USER_GID} ../ && sudo chmod -R a+w ../ && python3 /opt/leaderboard/leaderboard/leaderboard_evaluator.py --debug=0 --routes=$${ROUTE} --agent=/workspace/code/agent/src/agent/agent.py --host=$${CARLA_SIM_HOST} --track=MAP" + ports: + # Reserved ports for the debugger + - "53000-53100:53000-53100" + command: |- + bash -c "sleep 10 && sudo chown -R ${USER_UID}:${USER_GID} ../ && \ + sudo chmod -R a+w ../ && sudo mkdir -p $${XDG_RUNTIME_DIR} && sudo chmod 0700 $${XDG_RUNTIME_DIR} && sudo chown -R ${USER_UID}:${USER_GID} $${XDG_RUNTIME_DIR} && \ + (rqt_console &) && disown -a && \ + python3 /opt/leaderboard/leaderboard/leaderboard_evaluator.py --debug=0 --routes=$${ROUTE} --agent=/workspace/code/agent/src/agent/agent.py --host=$${CARLA_SIM_HOST} --track=MAP" diff --git a/build/docker/agent/Dockerfile b/build/docker/agent/Dockerfile index 408fae6c..e2176159 100644 --- a/build/docker/agent/Dockerfile +++ b/build/docker/agent/Dockerfile @@ -162,8 +162,10 @@ RUN source ~/.bashrc && pip install -r /workspace/requirements.txt # Add agent code COPY --chown=$USERNAME:$USERNAME ./code /workspace/code/ +# For debugger +ENV PAF_CATKIN_CODE_ROOT=/catkin_ws/src/code # Link code into catkin workspace -RUN ln -s /workspace/code /catkin_ws/src +RUN ln -s -T /workspace/code ${PAF_CATKIN_CODE_ROOT} # re-make the catkin workspace RUN source /opt/ros/noetic/setup.bash && catkin_make diff --git a/code/acting/launch/acting_debug.launch b/code/acting/launch/acting_debug.launch new file mode 100644 index 00000000..e0353b7c --- /dev/null +++ b/code/acting/launch/acting_debug.launch @@ -0,0 +1,40 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/code/acting/src/acting/Acting_Debug_Node.py b/code/acting/src/acting/Acting_Debug_Node.py index b3289747..da6104c8 100755 --- a/code/acting/src/acting/Acting_Debug_Node.py +++ b/code/acting/src/acting/Acting_Debug_Node.py @@ -178,6 +178,17 @@ def __init__(self): self.__vehicle_steers = [] self.positions = [] + # Initialize agent position + self.x = 0 + self.y = 0 + self.z = 0 + + # For visual purposes we set the trajectory height to vehicle heigt. + # The trajectory z coordinated do not affect steering or else. + # There is quite some confusion about visual height an actual height + # This parameter might needs adustment + self.z_visual = 0 + # Generate Trajectory as selected in TRAJECTORY_TYPE self.path_msg = Path() self.path_msg.header.stamp = rospy.Time.now() @@ -286,7 +297,7 @@ def updated_trajectory(self, target_trajectory): pos.header.frame_id = "global" pos.pose.position.x = wp[0] pos.pose.position.y = wp[1] - pos.pose.position.z = 704 # needed for visuals + pos.pose.position.z = self.z_visual # currently not used therefore zeros pos.pose.orientation.x = 0 pos.pose.orientation.y = 0 diff --git a/code/acting/src/debug_wrapper.py b/code/acting/src/debug_wrapper.py new file mode 120000 index 00000000..e4e2c378 --- /dev/null +++ b/code/acting/src/debug_wrapper.py @@ -0,0 +1 @@ +../../debug_wrapper.py \ No newline at end of file diff --git a/code/agent/launch/agent.launch b/code/agent/launch/agent.launch index fb9ef983..7861a38f 100644 --- a/code/agent/launch/agent.launch +++ b/code/agent/launch/agent.launch @@ -21,5 +21,9 @@ + + + + diff --git a/code/debug_wrapper.py b/code/debug_wrapper.py new file mode 100755 index 00000000..cafd493f --- /dev/null +++ b/code/debug_wrapper.py @@ -0,0 +1,201 @@ +#!/usr/bin/env python3 +"""Debug wrapper node + +Node that wraps a python ros node +and is able to open a debugpy remote debugger instance. + +Logs any exceptions from the node and other information +into the ros console via the debug_logger node. + +Always tries to at least start the node, +even if dependencies like debugpy are missing. + +Usage: + Already done for existing ros packages: symlink this file into the package. Example: + `cd code/perception/src && ln -s ../../debug_wrapper.py debug_wrapper.py` + + Adjust the launch configuration to use the debug_wrapper.py + instead of the node file and set the required args + + More info in [debugging.md](doc/development/debugging.md) + +Arguments: + --debug_node: Required: The filename of the node to debug + --debug_port: The port the debugger listens on. If not set, + --debug_host: The host the debugger binds to. + Defaults to the environment variable `DEBUG_WRAPPER_DEFAULT_HOST` if set, + otherwise `localhost` + --debug_wait: If True, the wrapper waits until a client (VS Code) is connected + and only then starts node execution + +Raises: + ArgumentParserError: Missing required parameters + error: If the started debug_node raises an exception, + it is logged and then raised again +""" + + +import importlib.util +import os +import runpy +import sys +import argparse +import time +from multiprocessing.connection import Client + +import rospy + +NODE_NAME = "NAMEERR" + + +def eprint(msg: str): + """Log msg into stderr. + + Used instead of print, because only stderr seems to reliably land in agent.log + + Args: + msg (str): Log message + """ + print(f"[debug_wrapper]: {msg}", file=sys.stderr) + + +def log(msg: str, level: str): + """Log msg via the debug_logger node + + Args: + msg (str): Log message + level (str): Log level. One of (debug), info, warn, error, fatal. + debug level not recommended, because these are not published to /rosout + """ + error = None + success = False + start_time = time.monotonic() + while not success and start_time + 5.0 > time.monotonic(): + try: + address = ("localhost", 52999) + conn = Client(address, authkey=b"debug_logger") + conn.send({"name": NODE_NAME, "msg": msg, "level": level}) + conn.close() + success = True + except Exception as e: + error = e + if not success: + eprint(msg) + if error is not None: + eprint(f"Failed to send to logger: {error}") + + +def logfatal(msg: str): + log(f"FAILED TO START NODE - NODE WILL NOT SHOW UP: {msg}", "fatal") + + +def logerr(msg: str): + log(msg, "error") + + +def logwarn(msg: str): + log(msg, "warn") + + +def loginfo(msg: str): + log(msg, "info") + + +def run_module_at(path: str): + """Runs a python module based on its file path + + Args: + path (str): python file path to run + """ + basename = os.path.basename(path) + module_dir = os.path.dirname(path) + module_name = os.path.splitext(basename)[0] + sys.path.append(module_dir) + runpy.run_module(module_name, run_name="__main__") + + +def start_debugger( + node_module_name: str, host: str, port: int, wait_for_client: bool = False +): + """_summary_ + + Args: + node_module_name (str): Name of the underlying node. Only used for logging + host (str): host the debugger binds to + port (int): debugger port + wait_for_client (bool, optional): If the debugger should wait + for a client to attach. Defaults to False. + """ + debugger_spec = importlib.util.find_spec("debugpy") + if debugger_spec is not None: + try: + import debugpy + + debugpy.listen((host, port)) + logwarn(f"Started debugger on {host}:{port} for {node_module_name}") + if wait_for_client: + logwarn("Waiting until debugging client is attached...") + debugpy.wait_for_client() + except Exception as error: + # Yes, all exceptions should be catched and sent into rosconsole + logerr(f"Failed to start debugger: {error}") + else: + logerr("debugpy module not found. Unable to start debugger") + + +class ArgumentParserError(Exception): + pass + + +class ThrowingArgumentParser(argparse.ArgumentParser): + def error(self, message): + logfatal(f"Wrong node arguments. Check launch config. : {message}") + raise ArgumentParserError(message) + + +def main(argv): + default_host = "localhost" + if "DEBUG_WRAPPER_DEFAULT_HOST" in os.environ: + default_host = os.environ["DEBUG_WRAPPER_DEFAULT_HOST"] + + node_args = rospy.myargv(argv=argv) + parser = ThrowingArgumentParser( + prog="debug wrapper", + ) + parser.add_argument("--debug_node", required=True, type=str) + parser.add_argument("--debug_port", required=False, type=int) + parser.add_argument("--debug_host", default=default_host, type=str) + parser.add_argument("--debug_wait", action="store_true") + args, unknown_args = parser.parse_known_args(node_args) + + debug_node = args.debug_node + global NODE_NAME + NODE_NAME = debug_node + base_dir = os.path.abspath(os.path.dirname(__file__)) + + if args.debug_port is not None: + start_debugger( + args.debug_node, + args.debug_host, + args.debug_port, + wait_for_client=args.debug_wait, + ) + else: + logerr( + """Missing parameter to start debugger: --debug_port + Add it in the launch configuration""" + ) + + target_type_path = os.path.join(base_dir, debug_node) + loginfo(f"Node {args.debug_node} starting at {base_dir}") + try: + run_module_at(target_type_path) + except BaseException as error: + # Yes, all exceptions including SystemExit should be catched. + # We want to always know when a node exits + logfatal(f"Failed to run node {debug_node}: {error}") + raise error + + +if __name__ == "__main__": + main(argv=sys.argv) diff --git a/code/debugging/CMakeLists.txt b/code/debugging/CMakeLists.txt new file mode 100644 index 00000000..a857d84f --- /dev/null +++ b/code/debugging/CMakeLists.txt @@ -0,0 +1,204 @@ +cmake_minimum_required(VERSION 3.0.2) +project(debugging) + +## Compile as C++11, supported in ROS Kinetic and newer +# add_compile_options(-std=c++11) + +## Find catkin macros and libraries +## if COMPONENTS list like find_package(catkin REQUIRED COMPONENTS xyz) +## is used, also find other catkin packages +find_package(catkin REQUIRED COMPONENTS + rospy +) + +## System dependencies are found with CMake's conventions +# find_package(Boost REQUIRED COMPONENTS system) + + +## Uncomment this if the package has a setup.py. This macro ensures +## modules and global scripts declared therein get installed +## See http://ros.org/doc/api/catkin/html/user_guide/setup_dot_py.html +catkin_python_setup() + +################################################ +## Declare ROS messages, services and actions ## +################################################ + +## To declare and build messages, services or actions from within this +## package, follow these steps: +## * Let MSG_DEP_SET be the set of packages whose message types you use in +## your messages/services/actions (e.g. std_msgs, actionlib_msgs, ...). +## * In the file package.xml: +## * add a build_depend tag for "message_generation" +## * add a build_depend and a exec_depend tag for each package in MSG_DEP_SET +## * If MSG_DEP_SET isn't empty the following dependency has been pulled in +## but can be declared for certainty nonetheless: +## * add a exec_depend tag for "message_runtime" +## * In this file (CMakeLists.txt): +## * add "message_generation" and every package in MSG_DEP_SET to +## find_package(catkin REQUIRED COMPONENTS ...) +## * add "message_runtime" and every package in MSG_DEP_SET to +## catkin_package(CATKIN_DEPENDS ...) +## * uncomment the add_*_files sections below as needed +## and list every .msg/.srv/.action file to be processed +## * uncomment the generate_messages entry below +## * add every package in MSG_DEP_SET to generate_messages(DEPENDENCIES ...) + +## Generate messages in the 'msg' folder +# add_message_files( +# FILES +# Message1.msg +# Message2.msg +# ) + +## Generate services in the 'srv' folder +# add_service_files( +# FILES +# Service1.srv +# Service2.srv +# ) + +## Generate actions in the 'action' folder +# add_action_files( +# FILES +# Action1.action +# Action2.action +# ) + +## Generate added messages and services with any dependencies listed here +# generate_messages( +# DEPENDENCIES +# std_msgs # Or other packages containing msgs +# ) + +################################################ +## Declare ROS dynamic reconfigure parameters ## +################################################ + +## To declare and build dynamic reconfigure parameters within this +## package, follow these steps: +## * In the file package.xml: +## * add a build_depend and a exec_depend tag for "dynamic_reconfigure" +## * In this file (CMakeLists.txt): +## * add "dynamic_reconfigure" to +## find_package(catkin REQUIRED COMPONENTS ...) +## * uncomment the "generate_dynamic_reconfigure_options" section below +## and list every .cfg file to be processed + +## Generate dynamic reconfigure parameters in the 'cfg' folder +# generate_dynamic_reconfigure_options( +# cfg/DynReconf1.cfg +# cfg/DynReconf2.cfg +# ) + +################################### +## catkin specific configuration ## +################################### +## The catkin_package macro generates cmake config files for your package +## Declare things to be passed to dependent projects +## INCLUDE_DIRS: uncomment this if your package contains header files +## LIBRARIES: libraries you create in this project that dependent projects also need +## CATKIN_DEPENDS: catkin_packages dependent projects also need +## DEPENDS: system dependencies of this project that dependent projects also need +catkin_package( +# INCLUDE_DIRS include +# LIBRARIES debugging +# CATKIN_DEPENDS rospy +# DEPENDS system_lib +) + +########### +## Build ## +########### + +## Specify additional locations of header files +## Your package locations should be listed before other locations +include_directories( +# include + ${catkin_INCLUDE_DIRS} +) + +## Declare a C++ library +# add_library(${PROJECT_NAME} +# src/${PROJECT_NAME}/debugging.cpp +# ) + +## Add cmake target dependencies of the library +## as an example, code may need to be generated before libraries +## either from message generation or dynamic reconfigure +# add_dependencies(${PROJECT_NAME} ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS}) + +## Declare a C++ executable +## With catkin_make all packages are built within a single CMake context +## The recommended prefix ensures that target names across packages don't collide +# add_executable(${PROJECT_NAME}_node src/debugging_node.cpp) + +## Rename C++ executable without prefix +## The above recommended prefix causes long target names, the following renames the +## target back to the shorter version for ease of user use +## e.g. "rosrun someones_pkg node" instead of "rosrun someones_pkg someones_pkg_node" +# set_target_properties(${PROJECT_NAME}_node PROPERTIES OUTPUT_NAME node PREFIX "") + +## Add cmake target dependencies of the executable +## same as for the library above +# add_dependencies(${PROJECT_NAME}_node ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS}) + +## Specify libraries to link a library or executable target against +# target_link_libraries(${PROJECT_NAME}_node +# ${catkin_LIBRARIES} +# ) + +############# +## Install ## +############# + +# all install targets should use catkin DESTINATION variables +# See http://ros.org/doc/api/catkin/html/adv_user_guide/variables.html + +## Mark executable scripts (Python etc.) for installation +## in contrast to setup.py, you can choose the destination +# catkin_install_python(PROGRAMS +# scripts/my_python_script +# DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION} +# ) + +## Mark executables for installation +## See http://docs.ros.org/melodic/api/catkin/html/howto/format1/building_executables.html +# install(TARGETS ${PROJECT_NAME}_node +# RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION} +# ) + +## Mark libraries for installation +## See http://docs.ros.org/melodic/api/catkin/html/howto/format1/building_libraries.html +# install(TARGETS ${PROJECT_NAME} +# ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION} +# LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION} +# RUNTIME DESTINATION ${CATKIN_GLOBAL_BIN_DESTINATION} +# ) + +## Mark cpp header files for installation +# install(DIRECTORY include/${PROJECT_NAME}/ +# DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION} +# FILES_MATCHING PATTERN "*.h" +# PATTERN ".svn" EXCLUDE +# ) + +## Mark other files for installation (e.g. launch and bag files, etc.) +# install(FILES +# # myfile1 +# # myfile2 +# DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION} +# ) + +############# +## Testing ## +############# + +## Add gtest based cpp test target and link libraries +# catkin_add_gtest(${PROJECT_NAME}-test test/test_debugging.cpp) +# if(TARGET ${PROJECT_NAME}-test) +# target_link_libraries(${PROJECT_NAME}-test ${PROJECT_NAME}) +# endif() + +## Add folders to be run by python nosetests +# catkin_add_nosetests(test) diff --git a/code/debugging/launch/debugging.launch b/code/debugging/launch/debugging.launch new file mode 100644 index 00000000..330263e8 --- /dev/null +++ b/code/debugging/launch/debugging.launch @@ -0,0 +1,4 @@ + + + + diff --git a/code/debugging/package.xml b/code/debugging/package.xml new file mode 100644 index 00000000..5598d5ed --- /dev/null +++ b/code/debugging/package.xml @@ -0,0 +1,62 @@ + + + debugging + 0.0.0 + The debugging package + + + + + peter + + + + + + TODO + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + catkin + rospy + rospy + rospy + + + + + + + + diff --git a/code/debugging/setup.py b/code/debugging/setup.py new file mode 100644 index 00000000..d320ac00 --- /dev/null +++ b/code/debugging/setup.py @@ -0,0 +1,6 @@ +#!/usr/bin/env python +from distutils.core import setup +from catkin_pkg.python_setup import generate_distutils_setup + +setup_args = generate_distutils_setup(packages=["debugging"], package_dir={"": "src"}) +setup(**setup_args) diff --git a/code/debugging/src/debug_logger.py b/code/debugging/src/debug_logger.py new file mode 100755 index 00000000..62f7eb4b --- /dev/null +++ b/code/debugging/src/debug_logger.py @@ -0,0 +1,160 @@ +#!/usr/bin/env python3 +"""Dedicated debug logger node. + +It can receive messages via a multiprocessing ipc socket on localhost:52999. +Message format is a python dict with keys {name, msg, level}. +It will then send those messages into the ros console via rospy.log*() + +Main usecase: it enables not initialized python nodes to publish log messages +to the /rosout topic. + +Without a fully initialized node, +log messages from python files only show up in the ros logfile, +but not in the /rosout topic and thus not in the rqt_console + +This is why this node is used by the debug_wrapper for logging +before it has initialized its own node +""" + +import time +import sys +from multiprocessing.connection import Listener, Client +import threading +import signal + +import rospy + +ADDRESS = ("localhost", 52999) +AUTHKEY = b"debug_logger" +CLOSE_MSG = "close" +LISTENER_THREAD = None +NODE_RUNNING = True + +MESSAGES_MUTEX = threading.Lock() +MESSAGES = [] + + +def eprint(msg: str): + """Log msg into stderr. + + Used instead of print, because only stderr seems to reliably land in agent.log + + Args: + msg (str): Log message + """ + print(f"[debug_logger]: {msg}", file=sys.stderr) + + +def log(name: str, msg: str, level: str): + """Log to the ros console + + Args: + name (str): Node name this message should be associated with + msg (str): Log message + level (str): Log level. One of (debug), info, warn, error, fatal. + debug level not recommended, because these are not published to /rosout + """ + msg = f"[debug_logger for {name}]: {msg}" + + level = level.lower() + if level == "debug": + rospy.logdebug(msg) + elif level == "info": + rospy.loginfo(msg) + elif level == "warn": + rospy.logwarn(msg) + elif level == "error": + rospy.logerr(msg) + else: + rospy.logfatal(msg) + + +def run_listener(listener: Listener): + """Run the multiprocessing listener + + The listener exits when it receives CLOSE_MSG + + Args: + listener (Listener): Listener to run + """ + running = True + while running: + try: + conn = listener.accept() + print(f"[debug_logger]: connection accepted from {listener.last_accepted}") + msg = None + if conn.poll(timeout=2.0): + msg = conn.recv() + if isinstance(msg, str): + if msg.lower() == CLOSE_MSG: + running = False + msg = None + conn.close() + except Exception as error: + eprint(f"Failed to receive message: {error}") + continue + + if msg is not None: + with MESSAGES_MUTEX: + MESSAGES.append(msg) + listener.close() + + +def close_listener(): + try: + conn = Client(ADDRESS, authkey=AUTHKEY) + conn.send(CLOSE_MSG) + conn.close() + except Exception: + pass + + +def exit_cleanup(signum=None, frame=None): + close_listener() + if LISTENER_THREAD is not None: + LISTENER_THREAD.join() + global NODE_RUNNING + NODE_RUNNING = False + + +def main(): + try: + # Based on + # https://stackoverflow.com/questions/6920858/interprocess-communication-in-python + # Only one listener can be active at once + listener = Listener(ADDRESS, authkey=AUTHKEY) + eprint("Debug logger node started") + except OSError as error: + eprint(f"Failed to run listener: {error}. Exiting...") + exit(0) + signal.signal(signal.SIGINT, exit_cleanup) + signal.signal(signal.SIGTERM, exit_cleanup) + global LISTENER_THREAD + LISTENER_THREAD = threading.Thread(target=run_listener, args=(listener,)) + LISTENER_THREAD.start() + + rospy.init_node(name="debug_logger") + # We need to wait a bit until the node is fully initialized to send log messages + time.sleep(5.0) + rospy.on_shutdown(exit_cleanup) + + while NODE_RUNNING: + with MESSAGES_MUTEX: + while len(MESSAGES) > 0: + msg = MESSAGES.pop(0) + log_msg = "Wrong log message format" + log_name = "NAMERR" + log_level = "fatal" + if isinstance(msg, dict): + if "name" in msg: + log_name = msg["name"] + if "msg" in msg: + log_msg = msg["msg"] + if "level" in msg: + log_level = msg["level"] + log(log_name, log_msg, log_level) + time.sleep(0.5) + + +if __name__ == "__main__": + main() diff --git a/code/mock/src/debug_wrapper.py b/code/mock/src/debug_wrapper.py new file mode 120000 index 00000000..e4e2c378 --- /dev/null +++ b/code/mock/src/debug_wrapper.py @@ -0,0 +1 @@ +../../debug_wrapper.py \ No newline at end of file diff --git a/code/perception/launch/perception.launch b/code/perception/launch/perception.launch index 8d6072e7..10f79a80 100644 --- a/code/perception/launch/perception.launch +++ b/code/perception/launch/perception.launch @@ -75,7 +75,7 @@ - + diff --git a/code/perception/src/debug_wrapper.py b/code/perception/src/debug_wrapper.py new file mode 120000 index 00000000..e4e2c378 --- /dev/null +++ b/code/perception/src/debug_wrapper.py @@ -0,0 +1 @@ +../../debug_wrapper.py \ No newline at end of file diff --git a/code/planning/src/debug_wrapper.py b/code/planning/src/debug_wrapper.py new file mode 120000 index 00000000..e4e2c378 --- /dev/null +++ b/code/planning/src/debug_wrapper.py @@ -0,0 +1 @@ +../../debug_wrapper.py \ No newline at end of file diff --git a/code/requirements.txt b/code/requirements.txt index b9b6155d..55de87f0 100644 --- a/code/requirements.txt +++ b/code/requirements.txt @@ -16,3 +16,4 @@ numpy==1.23.5 ultralytics==8.1.11 scikit-learn>=0.18 pandas==2.0.3 +debugpy==1.8.7 \ No newline at end of file diff --git a/doc/acting/acting_testing.md b/doc/acting/acting_testing.md index a5a67791..49bd88e7 100644 --- a/doc/acting/acting_testing.md +++ b/doc/acting/acting_testing.md @@ -2,26 +2,27 @@ **Summary:** This page shows ways to test and tune acting components and to verify that they work as intended. -- [How to test/tune acting components independedly](#how-to-testtune-acting-components-independedly) - - [Acting\_Debug\_Node](#acting_debug_node) - - [Setup for Testing with the Debug-Node](#setup-for-testing-with-the-debug-node) - - [Operating the Debug-Node](#operating-the-debug-node) +- [Acting\_Debug\_Node](#acting_debug_node) + - [Setup for Testing with the Debug-Node](#setup-for-testing-with-the-debug-node) + - [Operating the Debug-Node](#operating-the-debug-node) ## Acting_Debug_Node The [Acting_Debug_Node](../../code/acting/src/acting/Acting_Debug_Node.py) allows you to tune/test/verify all acting components independently of Planning-Inputs and also lets you easily output the data needed to plot/verify the tuning/testing you have done. +There is a dedicated [acting_debug.launch](../../code/acting/launch/acting_debug.launch) file which starts the acting component **as well as the necesarry perception components** automatically. It is recommended to first test the [Acting_Debug_Node](../../code/acting/src/acting/Acting_Debug_Node.py) +with an empty road scenario. The following guide provides instructions on how to launch this setup. + ### Setup for Testing with the Debug-Node -To use the [Acting_Debug_Node](../../code/acting/src/acting/Acting_Debug_Node.py) you first have to edit a few files in your current branch: +To use the [Acting_Debug_Node](../../code/acting/src/acting/Acting_Debug_Node.py) you first have to edit a few files in your current branch: \\ +**! Make sure to revert these changes when pushing your branch!** -- In the [acting.launch](../../code/acting/launch/acting.launch) file, you need to un-comment the ```Acting_Debug_Node```-node. -- Disable Planning from sending *trajectory*, *target_velocity*, *unstuck* or *emergency* Messages. -The easiest way to achieve this is to go to the [planning.launch](../../code/planning/launch/planning.launch) and just comment-out every active node. -- As the Perception-nodes use alot of processing power and ressources and you most likely do not need them, you can disable them, again commenting-out unwanted nodes in the [perception.launch](../../code/perception/launch/perception.launch) file. -**IMPORTANT:** As you need the **position_heading_publisher_node** and most likely the **kalman_filter** for the GNSS/IMU data, do keep those 2 active! -- There is a developer-enviroment ready to use, if you want to test your components on an empty road. -To switch to this, go to [docker_compose.yml](../../build/docker-compose.yml), where you will find which enviroment is currently chosen in lines 64-66. Uncomment the ```dev_launch``` and comment-out the ```leaderboard_evaluator``` if you want to use this developer-enviroment. +- In [dev.launch](../../code/agent/launch/dev.launch) the [agent.launch](../../code/agent/launch/agent.launch) file is included. Change this to include [acting_debug.launch](../../code/acting/launch/acting_debug.launch) from the **acting** package. +As mentioned above this includes everything we need. +- In [docker-compose.devroute.yaml](../../build/docker-compose.devroute.yaml) change the command to 'bash -c "sleep 10 && roslaunch agent/launch/dev.launch"' +- You can now 'docker compose up' the [docker-compose.devroute.yaml](../../build/docker-compose.devroute.yaml). + This should result in an empty road scenario with rviz and rqt. To now decide whether to work on noisy sensor data, with or without the kalman-filter, here is how to change this: @@ -30,6 +31,8 @@ To now decide whether to work on noisy sensor data, with or without the kalman-f If you want to test a component on ideal sensor data (position/heading/both), go to the [dev_objects.json](../../code/agent/config/dev_objects.json) and find the ```GNSS```(position) and the ```IMU```(heading) sensors (lines 142-173) and set their noise-attributes to 0. It is then also recommended to set the ```pos_filter``` and ```heading_filter``` of the **position_heading_publisher_node** to None (as filtering ideal sensor data will make them inaccurate again). +In [acting_debug.launch](../../code/acting/launch/acting_debug.launch) you can also set up different plot configurations for the rqt-window. + ### Operating the Debug-Node When you open the [Acting_Debug_Node](../../code/acting/src/acting/Acting_Debug_Node.py) you will see, that alot of Testing-Options are selectable via Global Variables. diff --git a/doc/assets/research_assets/drawio_legend/Loesungs_Vorschlag.jpg b/doc/assets/research_assets/drawio_legend/Loesungs_Vorschlag.jpg new file mode 100644 index 00000000..2146e7dc Binary files /dev/null and b/doc/assets/research_assets/drawio_legend/Loesungs_Vorschlag.jpg differ diff --git a/doc/assets/research_assets/drawio_legend/Problem.jpg b/doc/assets/research_assets/drawio_legend/Problem.jpg new file mode 100644 index 00000000..d05f15cb Binary files /dev/null and b/doc/assets/research_assets/drawio_legend/Problem.jpg differ diff --git a/doc/assets/research_assets/drawio_legend/Team.jpg b/doc/assets/research_assets/drawio_legend/Team.jpg new file mode 100644 index 00000000..92743423 Binary files /dev/null and b/doc/assets/research_assets/drawio_legend/Team.jpg differ diff --git a/doc/assets/research_assets/drawio_legend/Ziel.jpg b/doc/assets/research_assets/drawio_legend/Ziel.jpg new file mode 100644 index 00000000..48b73172 Binary files /dev/null and b/doc/assets/research_assets/drawio_legend/Ziel.jpg differ diff --git a/doc/assets/research_assets/drawio_legend/gemeinsames_Problem.jpg b/doc/assets/research_assets/drawio_legend/gemeinsames_Problem.jpg new file mode 100644 index 00000000..01ec28e0 Binary files /dev/null and b/doc/assets/research_assets/drawio_legend/gemeinsames_Problem.jpg differ diff --git a/doc/assets/research_assets/drawio_legend/generell.jpg b/doc/assets/research_assets/drawio_legend/generell.jpg new file mode 100644 index 00000000..9dd47446 Binary files /dev/null and b/doc/assets/research_assets/drawio_legend/generell.jpg differ diff --git a/doc/assets/research_assets/drawio_legend/two_teams_problem.jpg b/doc/assets/research_assets/drawio_legend/two_teams_problem.jpg new file mode 100644 index 00000000..79b917c0 Binary files /dev/null and b/doc/assets/research_assets/drawio_legend/two_teams_problem.jpg differ diff --git "a/doc/assets/research_assets/drawio_legend/\303\234bergeordnetes_Problem.jpg" "b/doc/assets/research_assets/drawio_legend/\303\234bergeordnetes_Problem.jpg" new file mode 100644 index 00000000..88862627 Binary files /dev/null and "b/doc/assets/research_assets/drawio_legend/\303\234bergeordnetes_Problem.jpg" differ diff --git a/doc/dev_talks/paf24/README.md b/doc/dev_talks/paf24/README.md index 4a2ada4b..70503b4d 100644 --- a/doc/dev_talks/paf24/README.md +++ b/doc/dev_talks/paf24/README.md @@ -11,6 +11,7 @@ - [Student Roles](./student_roles.md) - [Joker Rules](./joker_rules_paf24.md) - [Sprint Review Meeting Guidelines](./sprint_review_meeting_guidelines.md) + - [Sprint Summary Template](./sprint_summary_template.md) ## Notes for Sprints diff --git a/doc/dev_talks/paf24/sprint_review_meeting_guidelines.md b/doc/dev_talks/paf24/sprint_review_meeting_guidelines.md index 3dc2228d..0c41eeef 100644 --- a/doc/dev_talks/paf24/sprint_review_meeting_guidelines.md +++ b/doc/dev_talks/paf24/sprint_review_meeting_guidelines.md @@ -17,7 +17,7 @@ ### 1.1. Pre-Meeting Agenda - **Summary Submission**: By **Monday at 11:30 AM** (before the meeting), submit a summary of your work on Digicampus. - - TODO A template will be provided. + - A template can be found here: [Summary of Work Template](./sprint_summary_template.md). - Upload your document to DigiCampus at [this link](https://digicampus.uni-augsburg.de/dispatch.php/course/files?cid=5b0c38206c78cc03880bc2e71997220f). - Inform us of your group composition and presentation order for our note-taking. - **12:00 - 13:30**: Sample Review diff --git a/doc/dev_talks/paf24/sprint_summary_template.md b/doc/dev_talks/paf24/sprint_summary_template.md new file mode 100644 index 00000000..27785587 --- /dev/null +++ b/doc/dev_talks/paf24/sprint_summary_template.md @@ -0,0 +1,35 @@ +# Sprint Summary by *FirstName LastName* + +- Sprint Review Date: YYYY-MM-DD +- My team members: +- Area of contribution: + +## 1. My work + +### 1.1. Issues and PRs I worked on and my contribution + +- Link + - My contribution + +### 1.2. PRs I reviewed + +- Link + +### 1.3. Work I contributed but is not directly linked to a PR or an issue + +- For example draw.io board updates. + +### 1.4. Effort estimate (optional) + +- Estimated hours I worked for the project during the last sprint: + +## 2. Things I would like to mention (optional) + +- Lessons learned +- Issues faced +- Feedback +- Personal reflections + +--- + +Submit your summary to our DigiCampus course page by Monday at 11:30 AM. diff --git a/doc/development/build_action.md b/doc/development/build_action.md index 191f8ae2..ec534253 100644 --- a/doc/development/build_action.md +++ b/doc/development/build_action.md @@ -8,9 +8,16 @@ - [The `build-and-push-image` job](#the-build-and-push-image-job) - [1. Checkout repository (`actions/checkout@v3`)](#1-checkout-repository-actionscheckoutv3) - [2. Set up Docker Buildx (`docker/setup-buildx-action@v2`)](#2-set-up-docker-buildx-dockersetup-buildx-actionv2) - - [3. Log in to the Container registry (`docker/login-action@v2`)](#3-log-in-to-the-container-registry-dockerlogin-actionv2) - - [4. Test building the image (`docker/build-push-action@v3`)](#4-test-building-the-image-dockerbuild-push-actionv3) - - [5. Build and push the image (`docker/build-push-action@v3`)](#5-build-and-push-the-image-dockerbuild-push-actionv3) + - [3. Cache Docker layers](#3-cache-docker-layers) + - [4. Log in to the Container registry (`docker/login-action@v2`)](#4-log-in-to-the-container-registry-dockerlogin-actionv2) + - [5. Test building the image (`docker/build-push-action@v3`)](#5-test-building-the-image-dockerbuild-push-actionv3) + - [6. Build and push the image (`docker/build-push-action@v3`)](#6-build-and-push-the-image-dockerbuild-push-actionv3) + - [7. Save pull request artifact](#7-save-pull-request-artifact) + - [8. Save merge artifact](#8-save-merge-artifact) + - [9. Upload artifact](#9-upload-artifact) + - [10. Clean up PR Cache](#10-clean-up-pr-cache) + - [11. Clean up merge Cache](#11-clean-up-merge-cache) + - [12. Prune all images older than one day](#12-prune-all-images-older-than-one-day) ## General @@ -21,8 +28,7 @@ to [GitHub Packages](ghcr.io). The image can then be pulled with `docker pull ghcr.io/una-auxme/paf:latest` to get the latest version or `docker pull ghcr.io/una-auxme/paf:` to get a specific version. -If action is triggered by a pull request the created image is then used to execute a test run in the leaderboard, using -the devtest routes. The results of this simulation are then added as a comment to the pull request. +After the action is finished the `drive` action is triggered. ## The `build-and-push-image` job @@ -37,19 +43,47 @@ Set's up Buildx. This is needed to set up the correct driver to allow caching in Detailed description why this is needed can be found [here](https://github.com/docker/build-push-action/issues/163#issuecomment-1053657228). -### 3. Log in to the Container registry ([`docker/login-action@v2`](https://github.com/docker/login-action)) +### 3. Cache Docker layers + +Creates (if not done previously) and sets up the cache for the Docker layers. + +### 4. Log in to the Container registry ([`docker/login-action@v2`](https://github.com/docker/login-action)) Logs in with `GITHUB_TOKEN` into the registry (ghcr.io). Example taken from [here](https://docs.github.com/en/actions/publishing-packages/publishing-docker-images) -### 4. Test building the image ([`docker/build-push-action@v3`](https://github.com/docker/build-push-action/)) +### 5. Test building the image ([`docker/build-push-action@v3`](https://github.com/docker/build-push-action/)) Tries to build the image without pushing it to the repository packages if the workflow was triggered with a pull request. -### 5. Build and push the image ([`docker/build-push-action@v3`](https://github.com/docker/build-push-action/)) +### 6. Build and push the image ([`docker/build-push-action@v3`](https://github.com/docker/build-push-action/)) This action builds the image and pushes it to repository under the `latest` tag if the workflow was triggered with a merge to the `main` branch. To avoid large downloads of the base image the [GitHub Actions cache](https://docs.docker.com/build/building/cache/backends/gha/) is used to cache the image after build. + +### 7. Save pull request artifact + +If the action was triggered by a pull request an artifact is created with the corresponding PR ID (used for commenting on the PR in the `drive` action). + +### 8. Save merge artifact + +If the action was triggered by a merge an artifact is created with an invalid PR ID to signalise the `drive` action that it was not triggered by a PR. + +### 9. Upload artifact + +Uploads the artifact to the given path with a retention of the given number of days. + +### 10. Clean up PR Cache + +Removes the previous obsolete PR cache. + +### 11. Clean up merge Cache + +Removes the previous obsolete test cache. + +### 12. Prune all images older than one day + +Removes all images from the runner that are older than one day to free disk space. diff --git a/doc/development/debugging.md b/doc/development/debugging.md new file mode 100644 index 00000000..3e26453f --- /dev/null +++ b/doc/development/debugging.md @@ -0,0 +1,144 @@ +# Debugging + +**Summary:** This page explains multiple debugging methods for ROS nodes. + +- [Debugging possibilities](#debugging-possibilities) + - [Message based debugging](#message-based-debugging) + - [Viewing the messages](#viewing-the-messages) + - [Problems of message based debugging](#problems-of-message-based-debugging) + - [VS Code debugger](#vs-code-debugger) + - [Setup steps required once](#setup-steps-required-once) + - [Required once for the node you want to debug](#required-once-for-the-node-you-want-to-debug) + - [Debugging workflow after setup](#debugging-workflow-after-setup) + - [Known problems of debugging with VS Code](#known-problems-of-debugging-with-vs-code) + - [Debug Components Architecture](#debug-components-architecture) +- [Rebuild docker containers](#rebuild-docker-containers) +- [Sources](#sources) + +## Debugging possibilities + +There are two main debugging possibilities: Message based and VS Code based + +### Message based debugging + +Messages can be logged into the ROS console vie the `rospy.logdebug`, `rospy.loginfo`, `rospy.logwarn`, `rospy.logerror` and `rospy.logfatal` functions. +Most messages are then published to the /rosout topic. + +`rospy.logdebug` is not recommended, because these messages are not published to /rosout by default. + +Note that the node has to be initialized with `rospy.init_node()` before logging, otherwise the messages are not published to /rosout and just end up in stdout/stderr. + +#### Viewing the messages + +There are several ways to view the ROS log messages + +- The most convenient way to view logs is via the **rqt_console** GUI, which starts when the leaderboard is started. + It allows filtering for debug levels, nodes and message content. + + Caveat: It only includes messages published to the /rosout topic. + +- Execute `rosconsole echo` inside the *build-agent* container. This shows the /rosout messages from this point on. + +- The leaderboard logs usually end up in [code/log/ros](../../code/log/ros). + + This includes the [agent.log](../../code/log/ros/agent.log) file where most of all the nodes output is captured. + It seems to exclude stdout of the nodes and excludes the debug log level. stderr and all other levels are included. + Exceptions that occur at node initialization can also be found here. + +- More accurate "per-node" logs including the debug level and stdout end up in the individual node log files. + These can be found in the directory returned by running `roslaunch-logs` inside the *build-agent* container. (Usually `~/.ros/log`) + +- Manually starting a node with `rosrun ` inside the *build-agent* container helps to view stdout, stderr and exceptions. + +More information can be found in the [Wiki](https://wiki.ros.org/rospy/Overview/Logging) + +#### Problems of message based debugging + +- Time intensive and cumbersome: leaderboard has to be restarted to add a new message +- Frequent log messages "spam" the log +- Exceptions on node initialization do not show up in the /rosout topic +- If messages are sent right after `rospy.init_node()`, they might not be published to the /rosout topic + +### VS Code debugger + +Debug individual nodes with the VS Code debugger. This allows usage of breakpoints and code stepping. + +#### Setup steps required once + +1. Make sure the docker images are up-to-date: [Rebuild docker containers](#rebuild-docker-containers) +2. Start the `build/docker-compose.dev.yaml` containers and attach VS Code to the **build-agent-dev** container. Always use the VS Code remote for debugging +3. Make sure the recommended extensions are installed in the VS Code remote. + +#### Required once for the node you want to debug + +Adjust the launch configuration of the node to use the [debug_wrapper.py](../../code/debug_wrapper.py) node. + +Change the type to **debug_wrapper.py** and set `args="--debug_node= --debug_port="`. +*debug_port* can be a port in range **53000-53100**, because those are exposed from the [leaderboard docker configuration](../../build/docker-compose.leaderboard.yaml). + +Configuration example for the [lidar_distance.py](../../code/perception/src/lidar_distance.py) node: [launch configuration](../../code/perception/launch/perception.launch) + +- Original entry: + + ```launch + + + + + ``` + +- Entry modified for debugging: + + ```launch + + + + + ``` + +By default, the affected node will start up as usual. When you attach the debugger later, you are then only able to debug the callbacks the node receives. +But if you set `--debug_wait`, it will block the node from starting until VS Code attaches and allow you to debug the initialization of the node. + +If the leaderboard hangs on `Setting up the agent` after the configuration has been adjusted, there most likely is a mistake in the launch configuration. + +#### Debugging workflow after setup + +1. Start/Restart the [leaderboard](../../build/docker-compose.leaderboard.yaml) as usual +2. Wait for the application to start up +3. Add breakpoints to your node. +4. Start the VS Code debugger (debugging tab). A default launch configuration named *53000 leaderboard attach* is available. It uses port **53000**. + +Any errors/exceptions the debug_wrapper encounters are published to /rosout via the [debug_logger](../../code/debugging/src/debug_logger.py). +With the **rqt_console** GUI you can filter by node *debug_logger* to see all messages related to the wrapper. + +#### Known problems of debugging with VS Code + +- Adjusting the launch configurations is a bit cumbersome +- The side effects of just "stopping" a node with the debugger are unknown + +#### Debug Components Architecture + +More technical usage information can be found in the: + +- [debug_wrapper](../../code/debug_wrapper.py): Acts as a proxy that wraps the target node and enables remote debugging capabilities. +- [debug_logger](../../code/debugging/src/debug_logger.py): Handles the logging of debugging-related events and exceptions, ensuring they are properly published to the ROS ecosystem. + +The wrapper uses the logger to ensure that any debugging-related issues (connection problems, initialization errors, etc.) are visible through the /rosout topic. + +## Rebuild docker containers + +The docker images on your pc might not match the latest Dockerfile in the repository + +To update them, open a terminal, change into the *build* directory and execute: + +```bash +export USER_UID=$(id -u) +export USER_GID=$(id -g) +export USERNAME=$(id -u -n) +docker compose -f ./docker-compose.leaderboard.yaml up -d --build +docker compose -f ./docker-compose.dev.yaml up -d --build +``` + +## Sources + + diff --git a/doc/development/drive_action.md b/doc/development/drive_action.md index b5570002..b734105b 100644 --- a/doc/development/drive_action.md +++ b/doc/development/drive_action.md @@ -4,11 +4,17 @@ - [The drive job](#the-drive-job) - [1. Checkout repository (`actions/checkout@v3`)](#1-checkout-repository-actionscheckoutv3) - - [2. Run agent with docker-compose](#2-run-agent-with-docker-compose) - - [3. Copy simulation results file out of container](#3-copy-simulation-results-file-out-of-container) - - [4. Stop docker-compose stack](#4-stop-docker-compose-stack) - - [5. Comment result in pull request `actions/github-script@v6`](#5-comment-result-in-pull-request-actionsgithub-scriptv6) + - [2. Download artifact](#2-download-artifact) + - [3. Unzip artifact](#3-unzip-artifact) + - [4. Return artifact JSON](#4-return-artifact-json) + - [5. Run agent with docker-compose](#5-run-agent-with-docker-compose) + - [6. Copy simulation results file out of container](#6-copy-simulation-results-file-out-of-container) + - [7. Stop docker-compose stack](#7-stop-docker-compose-stack) + - [8. Create simulation results table](#8-create-simulation-results-table) + - [9. Print simulation results](#9-print-simulation-results) + - [10. Comment result in pull request `actions/github-script@v6`](#10-comment-result-in-pull-request-actionsgithub-scriptv6) - [Simulation results](#simulation-results) + - [11. Prune all images older than one day](#11-prune-all-images-older-than-one-day) ## The drive job @@ -20,7 +26,19 @@ The `drive` job is executed conditionally on `pull_request`, after the build suc Same step as in the [build job](#1-checkout-repository--actionscheckoutv3-) -### 2. Run agent with docker-compose +### 2. Download artifact + +Downloads the artifact that was uploaded during the preceeding `build` action. + +### 3. Unzip artifact + +Extracts the files of the downloaded artifact. + +### 4. Return artifact JSON + +Parses the extracted file in the JSON format to read the information inside the file. + +### 5. Run agent with docker-compose Runs the agent with the [`build/docker-compose.cicd.yaml`](../../build/docker-compose.cicd.yaml) that only contains the bare minimum components for test execution: @@ -30,11 +48,11 @@ bare minimum components for test execution: - Agent container, run through the Carla [`leaderboard_evaluator`](https://github.com/carla-simulator/leaderboard/blob/leaderboard-2.0/leaderboard/leaderboard_evaluator.py). -### 3. Copy simulation results file out of container +### 6. Copy simulation results file out of container Copies the created `simulation_results.json` file out of the agent container into the current container -### 4. Stop docker-compose stack +### 7. Stop docker-compose stack Stops the remaining containers (Carla, roscore) and removes the volumes with: `$ docker-compose down -v`. @@ -42,7 +60,15 @@ Stops the remaining containers (Carla, roscore) and removes the volumes with: This step is important to clean up the remaining containers to have a clean run everytime. This is also the reason for the `if: always()`, that ensures step execution. -### 5. Comment result in pull request [`actions/github-script@v6`](https://github.com/marketplace/actions/github-script) +### 8. Create simulation results table + +Reads the simulation results an creates a table for better readability. + +### 9. Print simulation results + +Prints the simulation results table to the action. + +### 10. Comment result in pull request [`actions/github-script@v6`](https://github.com/marketplace/actions/github-script) This steps uses a JS script to parse the simulation results and add a comment with a results table to the corresponding pull request. @@ -68,3 +94,7 @@ An example comment for this would be: | Yield emergency vehicles infractions | 0.0 | | Scenario timeouts | 62.046 | | Min speed infractions | 0.0 | + +### 11. Prune all images older than one day + +Removes all images from the runner that are older than one day to free disk space. diff --git a/doc/general/drawio_board.md b/doc/general/drawio_board.md new file mode 100644 index 00000000..1b866821 --- /dev/null +++ b/doc/general/drawio_board.md @@ -0,0 +1,53 @@ +# Drawio Board for planning future work + +**Summary:** This page provides an overview of the draw.io board usage and provides links to videos with general tips on how to work with draw.io + +- [Drawio Board for planning future work](#drawio-board-for-planning-future-work) + - [Link](#link) + - [Legend](#legend) + - [Tips](#tips) + +## Link + +To access the draw.io board, please click [here](https://drive.google.com/drive/folders/1dNyrnDdsj0m7kymDQUUqu2WinR4lQJpe) + +## Legend + +Please use the predefined nodes when working with the board. If you think something is missing, please contact Marcin Kuhnert (Discord: RoyaLxPole) for the PAF24. +Explanation: + +![Ziele](../assets/research_assets/drawio_legend/Ziel.jpg) + +This node is primarily for the software engineers who define the goals or sub-goals to be achieved for all teams. The goals should be defined as precisely as possible.These goals are added either in a separate graph or directly above the problems. + +![Team](../assets/research_assets/drawio_legend/Team.jpg) + +This node is for assigning the respective teams or a person from a team. Each team is assigned its own color. Green for Perception, Red for Acting, and Blue for Planning. + +![Uebergeordnetes Problem](../assets/research_assets/drawio_legend/Übergeordnetes_Problem.jpg) + +This node is used to describe a higher-level problem that cannot currently be defined in detail or can be broken down into smaller problems. As described in the previous node, each team has its own color. +If there is a problem that affects two teams, the color is determined by adding the colors in hexadecimal notation. + +![two teams prolbem](../assets/research_assets/drawio_legend/two_teams_problem.jpg) + +Example for problem that affects two teams (HEX: 00FFFF): + +![Problem](../assets/research_assets/drawio_legend/Problem.jpg)![gemeinsam](../assets/research_assets/drawio_legend/gemeinsames_Problem.jpg) + +This node is used to provide a description for an atomic problem that cannot be further simplified. The color coding for this node applies as described above. + +![Loesungs Vorschlag](../assets/research_assets/drawio_legend/Loesungs_Vorschlag.jpg) + +This node is used to describe the approach to the problem. If an issue has already been created on GitHub, the issue number should be included here. + +![generell](../assets/research_assets/drawio_legend/generell.jpg) +These nodes are used to describe problems that have not yet been assigned to a team or to describe problems that affect all teams equally. + +## Tips + +For effective work on the board, please watch the following video and read the blog posting. + +[Video](https://www.google.com/search?q=draw.io+mind+map+tutorial&rlz=1C1VDKB_deDE929DE929&oq=draw+io+mind&gs_lcrp=EgZjaHJvbWUqCAgDEAAYFhgeMggIABBFGCcYOzIGCAEQRRg5MgcIAhAAGIAEMggIAxAAGBYYHjIICAQQABgWGB4yCAgFEAAYFhgeMgYIBhBFGDwyBggHEEUYPdIBCDgyNzBqMGo3qAIAsAIA&sourceid=chrome&ie=UTF-8#fpstate=ive&vld=cid:6448d106,vid:mooBFL_jhxg,st:0) + +[Blog](https://drawio-app.com/blog/organization-charts-and-mind-maps-in-draw-io/)