From d4812730d19e7938b328a25dbd5a7182f0f377da Mon Sep 17 00:00:00 2001 From: fedor98 Date: Mon, 4 Nov 2024 02:19:21 +0100 Subject: [PATCH 1/3] Add doc/research/paf24/planning/decision_making_rationale.md --- .../planning/decision_making_rationale.md | 70 +++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100644 doc/research/paf24/planning/decision_making_rationale.md diff --git a/doc/research/paf24/planning/decision_making_rationale.md b/doc/research/paf24/planning/decision_making_rationale.md new file mode 100644 index 00000000..7557b1f3 --- /dev/null +++ b/doc/research/paf24/planning/decision_making_rationale.md @@ -0,0 +1,70 @@ +# Decision Makaing Approaches + +Responsible for making decisions based on information provided by other system components. It should account for all possible traffic scenarios to ensure comprehensive coverage and safe driving behavior + +## Decision Tree - current implementations + +- a tool to organizes decisions and their possible outcomes in a tree-like structure +- a branch represents a choice/condition which may lead to further options. + - starting at the root: checking surrounding conditions (traffic lights, obstacles, speed limits etc) branching out based on yes/no questions at each node +- in our case each node is deterministic (as no ML is used) + +### Pros and Cons + + + +### Changes in Previous Semesters + +- removing redundant behaviours + + - reducing to `Intersection`, `Lane Switch` and `Cruise` + +![decision_tree](doc/assets/behaviour_tree.png) + +## Finite State Machine - previous implementations + +- model with an discrete number of possible states with rule-based control + - can only inherit one state at a time +- each state represents a specific driving mode, such as "Lane Switching" "Stopping," or "Overtaking" + - triggered by inputs like traffic signals, obstacles, or speed limits + +### Pros and Cons + + + +## Combining Planning and Reinforcement Learning + +[C.-J. Hoel, K. Driggs-Campbell, K. Wolff, L. Laine, and M. J. Kochenderfer, "Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving," arXiv preprint arXiv:1905.02680, 2019.](https://arxiv.org/abs/1905.02680) + +- Monte Carlo Tree Search (MCTS): Used for planning by creating a search tree that simulates possible actions, selecting those that maximize expected rewards (maintaining desired speed, reaching end points, avoinding collision etc.) + - considers all possible actions → Each action creates a new branch that represents what the new situation might look like if the agent took that action +- deep neural network guides the MCTS by predicting action probabilities (which actions are most likely to lead to good outcomes: changing lanes, speeding up, breaking etc.) and state values (scores that tell the agent whether its current situation is favorable or not based on expected future rewards) + +### Pros and Cons + + From a32d6c62c642ec29b4ab7280b28ba4247330732d Mon Sep 17 00:00:00 2001 From: fedor98 Date: Mon, 4 Nov 2024 10:02:57 +0100 Subject: [PATCH 2/3] Update: resolve linting errors due to use of HTML --- .../planning/decision_making_rationale.md | 46 +++++++------------ 1 file changed, 17 insertions(+), 29 deletions(-) diff --git a/doc/research/paf24/planning/decision_making_rationale.md b/doc/research/paf24/planning/decision_making_rationale.md index 7557b1f3..b051b1d0 100644 --- a/doc/research/paf24/planning/decision_making_rationale.md +++ b/doc/research/paf24/planning/decision_making_rationale.md @@ -11,20 +11,14 @@ Responsible for making decisions based on information provided by other system c ### Pros and Cons - +- `+` are easy to understand and interpret +- `+` decision paths are traceable → suitable for debugging +- `+` deterministic outputs beneficial when predictability is essential +- `+` compared to NN, low computational cost +- `-` Adding too many branches for every possible situation can make the tree unwieldy and hard to manage + - can easily overfit with too many conditions +- `-` no reasoning and no probabilistic capabilities; trees tend to make suboptimal decisions in new, unclear situations + - hard to account for every situation ### Changes in Previous Semesters @@ -32,7 +26,7 @@ Responsible for making decisions based on information provided by other system c - reducing to `Intersection`, `Lane Switch` and `Cruise` -![decision_tree](doc/assets/behaviour_tree.png) +![decision_tree](/doc/assets/behaviour_tree.png) ## Finite State Machine - previous implementations @@ -43,12 +37,10 @@ Responsible for making decisions based on information provided by other system c ### Pros and Cons - +- `+` each state and transition is explicitly defined → suitable for debugging and testing +- `+` easier implementations than a decision tree +- `-` scalability issues with increasing number of states +- `-` less flexible than decision trees ## Combining Planning and Reinforcement Learning @@ -60,11 +52,7 @@ Responsible for making decisions based on information provided by other system c ### Pros and Cons - +- `+` generalizes better to complex scenarios + - takes advantage of continuous and high-dimensional state spaces +- `-` computational complexity and real-time feasibility +- `-` extensive training data and tuning required to handle a broad range of driving situations From 4a1ba66b6535ac9377aa726b7f968593634d9645 Mon Sep 17 00:00:00 2001 From: fedor98 Date: Mon, 4 Nov 2024 10:30:17 +0100 Subject: [PATCH 3/3] Update: correcting typos --- doc/research/paf24/planning/decision_making_rationale.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/research/paf24/planning/decision_making_rationale.md b/doc/research/paf24/planning/decision_making_rationale.md index b051b1d0..a5f3976c 100644 --- a/doc/research/paf24/planning/decision_making_rationale.md +++ b/doc/research/paf24/planning/decision_making_rationale.md @@ -1,4 +1,4 @@ -# Decision Makaing Approaches +# Decision Making Approaches Responsible for making decisions based on information provided by other system components. It should account for all possible traffic scenarios to ensure comprehensive coverage and safe driving behavior