diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index b30c0359..9a495da2 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -45,9 +45,10 @@ jobs: uses: docker/build-push-action@v3 with: context: . - file: ./build/docker/build/Dockerfile + file: ./build/docker/agent/Dockerfile push: true # tag 'latest' and version on push to main, otherwise use the commit hash tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest cache-from: type=gha cache-to: type=gha,mode=max + build-args: USERNAME=paf,USER_UID=1000,USER_GID=1000 diff --git a/build/agent_service.yaml b/build/agent_service.yaml index 13d8bab1..c1f1aa44 100644 --- a/build/agent_service.yaml +++ b/build/agent_service.yaml @@ -34,7 +34,7 @@ services: volumes: - /tmp/.X11-unix:/tmp/.X11-unix # if you change the volume here also change the copy command - # in ``build/docker/build/Dockerfile + # in ``build/docker/agent/Dockerfile - ../:/workspace # mount git config for dvc - ../.gitconfig:/home/carla/.gitconfig diff --git a/build/docker/build/Dockerfile b/build/docker/build/Dockerfile deleted file mode 100644 index cd8fcae8..00000000 --- a/build/docker/build/Dockerfile +++ /dev/null @@ -1,7 +0,0 @@ -# syntax = edrevo/dockerfile-plus - -INCLUDE+ ./build/docker/agent/Dockerfile - -# if you change the paths here also change the volume -# in docker-compose.yml for service agent -COPY ./code /workspace/code/ diff --git a/doc/assets/development/attach.png b/doc/assets/development/attach.png new file mode 100644 index 00000000..710c2c9a Binary files /dev/null and b/doc/assets/development/attach.png differ diff --git a/doc/assets/development/devcontainer.png b/doc/assets/development/devcontainer.png new file mode 100644 index 00000000..2368aae8 Binary files /dev/null and b/doc/assets/development/devcontainer.png differ diff --git a/doc/assets/development/docstring.png b/doc/assets/development/docstring.png new file mode 100644 index 00000000..aebf5091 Binary files /dev/null and b/doc/assets/development/docstring.png differ diff --git a/doc/assets/development/docstring_popup.png b/doc/assets/development/docstring_popup.png new file mode 100644 index 00000000..afada0de Binary files /dev/null and b/doc/assets/development/docstring_popup.png differ diff --git a/doc/development/README.md b/doc/development/README.md index 75a99b4c..9727da08 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -1,22 +1,28 @@ -# Development guidelines +# Development in PAF + +## First steps + +If this is your first time working with the project you can follow the first steps in [/doc/development/first_steps.md](/doc/development/first_steps.md). + +## Development guidelines If you contribute to this project please read the following guidelines first: 1. [Documentation Requirements](./documentation_requirements.md) 2. [Linting](./linting.md) -3. [Coding style](./coding_style.md) -4. [Git Style](./git_workflow.md) -5. [Reviewing](./review_guideline.md) -6. [Project management](./project_management.md) -7. Github actions +3. [Git Style](./git_workflow.md) +4. [Reviewing](./review_guideline.md) +5. [Project management](./project_management.md) +6. Github actions 1. [linting action](./linter_action.md) 2. [build action](./build_action.md) -8. [Install python packages](./installing_python_packages.md) -9. [Discord Webhook Documentation](./discord_webhook.md) + 3. [drive action](./drive_action.md) +7. [Install python packages](./installing_python_packages.md) +8. [Discord Webhook Documentation](./discord_webhook.md) ## Templates -Templates for documentation and code are provided in [`doc/development/templates`](./templates). +Templates for documentation and code are provided in [`doc/development/templates`](./templates). These templates are to be used for new files that are added to the project. ### [`template_class.py`](./templates/template_class.py) diff --git a/doc/development/build_action.md b/doc/development/build_action.md index 77f71011..6c2c9473 100644 --- a/doc/development/build_action.md +++ b/doc/development/build_action.md @@ -2,13 +2,9 @@ (Kept from previous group [paf22]) -**Summary:** This page explains the GitHub build action we use to first: - -- create an executable image of our work -- evaluate our Agent with the leaderboard +**Summary:** This page explains the GitHub build action we use to create an executable image of our work. - [GitHub actions](#github-actions) - - [Table of contents](#table-of-contents) - [General](#general) - [The Dockerfile (`build/docker/build/Dockerfile`)](#the-dockerfile-builddockerbuilddockerfile) - [The `build-and-push-image` job](#the-build-and-push-image-job) @@ -18,13 +14,6 @@ - [4. Bump version and push tag (`mathieudutour/github-tag-action`)](#4-bump-version-and-push-tag-mathieudutourgithub-tag-action) - [5. Get commit hash](#5-get-commit-hash) - [6. Build and push Docker image](#6-build-and-push-docker-image) - - [The drive job](#the-drive-job) - - [1. Checkout repository (`actions/checkout@v3`)](#1-checkout-repository-actionscheckoutv3-1) - - [2. Run agent with docker-compose](#2-run-agent-with-docker-compose) - - [3. Copy simulation results file out of container](#3-copy-simulation-results-file-out-of-container) - - [4. Stop docker-compose stack](#4-stop-docker-compose-stack) - - [5. Comment result in pull request `actions/github-script@v6`](#5-comment-result-in-pull-request-actionsgithub-scriptv6) - - [Simulation results](#simulation-results) ## General @@ -80,60 +69,3 @@ the [GitHub Actions cache](https://docs.docker.com/build/building/cache/backends is used to cache the image after build. If the action is run on a branch other than `main`, the image is tagged with the commit hash from Step 5. Otherwise, the image is tagged with both the tag created in Step 4 and `latest`. - -## The drive job - -The `drive` job is executed conditionally on `pull_request`, after the build successfully ran through. - -### 1. Checkout repository ([`actions/checkout@v3`](https://github.com/actions/checkout)) - -Same step as in the [build job](#1-checkout-repository--actionscheckoutv3-) - -### 2. Run agent with docker-compose - -Runs the agent with the [`build/docker-compose.cicd.yaml`](../../build/docker-compose.cicd.yaml) that only contains the -bare minimum components for test execution: - -- Carla Simulator -- roscore -- Agent container, run through the - Carla [`leaderboard_evaluator`](https://github.com/carla-simulator/leaderboard/blob/leaderboard-2.0/leaderboard/leaderboard_evaluator.py). - -### 3. Copy simulation results file out of container - -Copies the created `simulation_results.json` file out of the agent container into the current container - -### 4. Stop docker-compose stack - -Stops the remaining containers (Carla, roscore) and removes the volumes with: -`$ docker-compose down -v`. - -This step is important to clean up the remaining containers to have a clean run everytime. This is also the reason for -the `if: always()`, that ensures step execution. - -### 5. Comment result in pull request [`actions/github-script@v6`](https://github.com/marketplace/actions/github-script) - -This steps uses a JS script to parse the simulation results and add a comment with a results table to the corresponding -pull request. - -An example comment for this would be: - -## Simulation results - -| Metric | Value | -|--------------------------------------|---------| -| Avg. driving score | 0.06006 | -| Avg. route completion | 0.22 | -| Avg. infraction penalty | 0.273 | -| Collisions with pedestrians | 0.0 | -| Collisions with vehicles | 62.046 | -| Collisions with layout | 62.046 | -| Red lights infractions | 0.0 | -| Stop sign infractions | 0.0 | -| Off-road infractions | 0 | -| Route deviations | 0.0 | -| Route timeouts | 62.046 | -| Agent blocked | 0.0 | -| Yield emergency vehicles infractions | 0.0 | -| Scenario timeouts | 62.046 | -| Min speed infractions | 0.0 | diff --git a/doc/development/coding_style.md b/doc/development/coding_style.md deleted file mode 100644 index 544c397f..00000000 --- a/doc/development/coding_style.md +++ /dev/null @@ -1,1062 +0,0 @@ -# Coding style guidelines - -(Kept from previous group [paf22]) - -**Summary:** This page contains the coding rules we want to follow as a team to improve readability and reviewing of our code.This document is for reference only and should be consolidated in case of uncertainty of following the style guidelines. Based on PEP 8 () - -- [Coding style guidelines](#coding-style-guidelines) - - [Code lay-out](#code-lay-out) - - [Indentation](#indentation) - - [Tabs or Spaces?](#tabs-or-spaces) - - [Should a line break before or after a binary operator?](#should-a-line-break-before-or-after-a-binary-operator) - - [Blank Lines](#blank-lines) - - [Source File Encoding](#source-file-encoding) - - [Imports](#imports) - - [Module level dunder names](#module-level-dunder-names) - - [String Quotes](#string-quotes) - - [Whitespace in Expressions and Statements](#whitespace-in-expressions-and-statements) - - [Pet Peeves](#pet-peeves) - - [Other Recommendations](#other-recommendations) - - [When to use trailing commas](#when-to-use-trailing-commas) - - [Comments](#comments) - - [Block Comments](#block-comments) - - [Inline Comments](#inline-comments) - - [Documentation Strings](#documentation-strings) - - [Naming Conventions](#naming-conventions) - - [Overriding Principle](#overriding-principle) - - [Descriptive Naming Styles](#descriptive-naming-styles) - - [Prescriptive: Naming Conventions](#prescriptive-naming-conventions) - - [Names to Avoid](#names-to-avoid) - - [ASCII Compatibility](#ascii-compatibility) - - [Package and Module Names](#package-and-module-names) - - [Class Names](#class-names) - - [Type variable names](#type-variable-names) - - [Exception Names](#exception-names) - - [Global Variable Names](#global-variable-names) - - [Function Names](#function-names) - - [Function and method arguments](#function-and-method-arguments) - - [Method Names and Instance Variables](#method-names-and-instance-variables) - - [Constants](#constants) - - [Designing for inheritance](#designing-for-inheritance) - - [Public and internal interfaces](#public-and-internal-interfaces) - - [Programming Recommendations](#programming-recommendations) - - [Function Annotations](#function-annotations) - - [Footnotes](#footnotes) - - [Copyright](#copyright) - - [Source](#source) - -## Code lay-out - -### Indentation - -Use 4 spaces per indentation level. - -Continuation lines should align wrapped elements either vertically using Python’s implicit line joining inside parentheses, brackets and braces, or using a hanging indent 3. -When using a hanging indent the following should be considered; there should be no arguments on the first line and further indentation should be used to clearly distinguish itself as a continuation line. - -Yes: - -```python -# Aligned with opening delimiter. -foo = long_function_name(var_one, var_two, - var_three, var_four) - -# More indentation included to distinguish this from the rest. -def long_function_name( - var_one, var_two, var_three, - var_four): - print(var_one) - -# Hanging indents should add a level. -foo = long_function_name( - var_one, var_two, - var_three, var_four) -``` - -No: - -```python -# Arguments on first line forbidden when not using vertical alignment. -foo = long_function_name(var_one, var_two, - var_three, var_four) - -# Further indentation required as indentation is not distinguishable. -def long_function_name( - var_one, var_two, var_three, - var_four): - print(var_one) - -# The 4-space rule is optional for continuation lines. - -# Optional: - -# Hanging indents *may* be indented to other than 4 spaces. -foo = long_function_name( - var_one, var_two, - var_three, var_four) -``` - -The closing brace/bracket/parenthesis on multi-line constructs may either line up under the first non-whitespace character of the last line of list, as in: - -```python -my_list = [ - 1, 2, 3, - 4, 5, 6, - ] -result = some_function_that_takes_arguments( - 'a', 'b', 'c', - 'd', 'e', 'f', - ) -``` - -or it may be lined up under the first character of the line that starts the multi-line construct, as in: - -```python -my_list = [ - 1, 2, 3, - 4, 5, 6, -] -result = some_function_that_takes_arguments( - 'a', 'b', 'c', - 'd', 'e', 'f', -) -``` - -### Tabs or Spaces? - -Spaces are the preferred indentation method. - -Tabs should be used solely to remain consistent with code that is already indented with tabs. - -Python 3 disallows mixing the use of tabs and spaces for indentation. - -Limit all lines to a maximum of 79 characters. - -For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters. - -Limiting the required editor window width makes it possible to have several files open side-by-side, and works well when using code review tools that present the two versions in adjacent columns. - -The preferred way of wrapping long lines is by using Python’s implied line continuation inside parentheses, brackets and braces. Long lines can be broken over multiple lines by wrapping expressions in parentheses. These should be used in preference to using a backslash for line continuation. - -Backslashes may still be appropriate at times. For example, long, multiple with-statements cannot use implicit continuation, so backslashes are acceptable: - -```python -with open('/path/to/some/file/you/want/to/read') as file_1, \ - open('/path/to/some/file/being/written', 'w') as file_2: - file_2.write(file_1.read()) -``` - -### Should a line break before or after a binary operator? - -For decades the recommended style was to break after binary operators. But this can hurt readability in two ways: the operators tend to get scattered across different columns on the screen, and each operator is moved away from its operand and onto the previous line. -Here, the eye has to do extra work to tell which items are added and which are subtracted: - -```python -# No: operators sit far away from their operands -income = (gross_wages + - taxable_interest + - (dividends - qualified_dividends) - - ira_deduction - - student_loan_interest) -``` - -To solve this readability problem, mathematicians and their publishers follow the opposite convention. Donald Knuth explains the traditional rule in his Computers and Typesetting series: - -`` -“Although formulas within a paragraph always break after binary operations and relations, displayed formulas always break before binary operations” -`` - -Following the tradition from mathematics usually results in more readable code: - -```python -# Yes: easy to match operators with operands -income = (gross_wages - + taxable_interest - + (dividends - qualified_dividends) - - ira_deduction - - student_loan_interest) -``` - -In Python code, it is permissible to break before or after a binary operator, as long as the convention is consistent locally. For new code Knuth's style is suggested. - -### Blank Lines - -Surround top-level function and class definitions with two blank lines. - -Method definitions inside a class are surrounded by a single blank line. - -Use blank lines in functions, sparingly, to indicate logical sections. - -### Source File Encoding - -Code in the core Python distribution should always use UTF-8 (or ASCII in Python 2). - -Files using ASCII (in Python 2) or UTF-8 (in Python 3) should not have an encoding declaration. - -For Python 3.0 and beyond, the following policy is prescribed for the standard library (see PEP 3131): -All identifiers in the Python standard library MUST use ASCII-only identifiers, and SHOULD use English words wherever feasible (in many cases, abbreviations and technical terms are used which aren’t English). -In addition, string literals and comments must also be in ASCII. The only exceptions are (a) test cases testing the non-ASCII features, and (b) names of authors. -Authors whose names are not based on the latin alphabet MUST provide a latin transliteration of their names. - -Open source projects with a global audience are encouraged to adopt a similar policy. - -### Imports - -Imports should usually be on separate lines, e.g.: - -```python -# Yes: - -import os -import sys - -# No: - -import os, sys -``` - -It’s okay to say this though: - -```python -from subprocess import Popen, PIPE -``` - -Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants. - -Imports should be grouped in the following order: - -1. standard library imports -2. related third party imports -3. local application/library specific imports - -You should put a blank line between each group of imports. - -Absolute imports are recommended, as they are usually more readable and tend to be better behaved (or at least give better error messages) if the import system is incorrectly configured (such as when a directory inside a package ends up on sys.path): - -```python -import mypkg.sibling -from mypkg import sibling -from mypkg.sibling import example -``` - -However, explicit relative imports are an acceptable alternative to absolute imports, especially when dealing with complex package layouts where using absolute imports would be unnecessarily verbose: - -```python -from . import sibling -from .sibling import example -``` - -Standard library code should avoid complex package layouts and always use absolute imports. - -Implicit relative imports should never be used and have been removed in Python 3. - -When importing a class from a class-containing module, it’s usually okay to spell this: - -```python -from myclass import MyClass -from foo.bar.yourclass import YourClass -``` - -If this spelling causes local name clashes, then spell them : - -```python -import myclass -import foo.bar.yourclass -``` - -and use ``myclass.MyClass`` and ``foo.bar.yourclass.YourClass``. - -Wildcard imports (from `` import *``) should be avoided, as they make it unclear which names are present in the namespace, confusing both readers and many automated tools. -There is one defensible use case for a wildcard import, which is to republish an internal interface as part of a public API -(for example, overwriting a pure Python implementation of an interface with the definitions from an optional accelerator module and exactly which definitions will be overwritten isn’t known in advance). - -When republishing names this way, the guidelines below regarding public and internal interfaces still apply. - -### Module level dunder names - -Module level "dunders" (i.e. names with two leading and two trailing underscores) such as ```__all__```, ```__author__```, ```__version__```, etc. should be placed after the module docstring but before any import statements except from ```__future__``` imports. -Python mandates that future-imports must appear in the module before any other code except docstrings. - -For example: - -```python -"""This is the example module. - -This module does stuff. -""" - -from __future__ import barry_as_FLUFL - -__all__ = ['a', 'b', 'c'] -__version__ = '0.1' -__author__ = 'Cardinal Biggles' - -import os -import sys -``` - -## String Quotes - -In Python, single-quoted strings and double-quoted strings are the same. This PEP does not make a recommendation for this. Pick a rule and stick to it. When a string contains single or double quote characters, however, use the other one to avoid backslashes in the string. It improves readability. - -For triple-quoted strings, always use double quote characters to be consistent with the docstring convention in PEP 257. - -## Whitespace in Expressions and Statements - -### Pet Peeves - -Avoid extraneous whitespace in the following situations: - -Immediately inside parentheses, brackets or braces: - -```python -# Yes: - -spam(ham[1], {eggs: 2}) - -# No: - -spam( ham[ 1 ], { eggs: 2 } ) -``` - -Between a trailing comma and a following close parenthesis: - -```python -# Yes: - -foo = (0,) - -# No: - -bar = (0, ) -``` - -Immediately before a comma, semicolon, or colon: - -```python -# Yes: - -if x == 4: print x, y; x, y = y, x - -# No: - -if x == 4 : print x , y ; x , y = y , x -``` - -However, in a slice the colon acts like a binary operator, and should have equal amounts on either side (treating it as the operator with the lowest priority). In an extended slice, both colons must have the same amount of spacing applied. Exception: when a slice parameter is omitted, the space is omitted. - -```python -# Yes: - -ham[1:9], ham[1:9:3], ham[:9:3], ham[1::3], ham[1:9:] -ham[lower:upper], ham[lower:upper:], ham[lower::step] -ham[lower+offset : upper+offset] -ham[: upper_fn(x) : step_fn(x)], ham[:: step_fn(x)] -ham[lower + offset : upper + offset] - -# No: - -ham[lower + offset:upper + offset] -ham[1: 9], ham[1 :9], ham[1:9 :3] -ham[lower : : upper] -ham[ : upper] -``` - -Immediately before the open parenthesis that starts the argument list of a function call: - -```python -# Yes: - -spam(1) - -# No: - -spam (1) -``` - -Immediately before the open parenthesis that starts an indexing or slicing: - -```python -# Yes: - -dct['key'] = lst[index] - -# No: - -dct ['key'] = lst [index] -``` - -More than one space around an assignment (or other) operator to align it with another. - -```python -# Yes: - -x = 1 -y = 2 -long_variable = 3 - -# No: - -x = 1 -y = 2 -long_variable = 3 -``` - -### Other Recommendations - -Avoid trailing whitespace anywhere. Because it’s usually invisible, it can be confusing: e.g. a backslash followed by a space and a newline does not count as a line continuation marker. Some editors don’t preserve it and many projects (like CPython itself) have pre-commit hooks that reject it. - -Always surround these binary operators with a single space on either side: assignment (=), augmented assignment (+=, -= etc.), comparisons (==, <, >, !=, <>, <=, >=, in, not in, is, is not), Booleans (and, or, not). - -If operators with different priorities are used, consider adding whitespace around the operators with the lowest priority(ies). Use your own judgment; however, never use more than one space, and always have the same amount of whitespace on both sides of a binary operator. - -```python -# Yes: - -i = i + 1 -submitted += 1 -x = x*2 - 1 -hypot2 = x*x + y*y -c = (a+b) * (a-b) - -# No: - -i=i+1 -submitted +=1 -x = x * 2 - 1 -hypot2 = x * x + y * y -c = (a + b) * (a - b) -``` - -Don’t use spaces around the = sign when used to indicate a keyword argument or a default parameter value. - -```python -# Yes: - -def complex(real, imag=0.0): - return magic(r=real, i=imag) - -# No: - -def complex(real, imag = 0.0): - return magic(r = real, i = imag) -``` - -Function annotations should use the normal rules for colons and always have spaces around the -> arrow if present. (See Function Annotations below for more about function annotations.) - -```python -# Yes: - -def munge(input: AnyStr): ... -def munge() -> AnyStr: ... - -# No: - -def munge(input:AnyStr): ... -def munge()->PosInt: ... -``` - -When combining an argument annotation with a default value, use spaces around the = sign (but only for those arguments that have both an annotation and a default). - -```python -# Yes: - -def munge(sep: AnyStr = None): ... -def munge(input: AnyStr, sep: AnyStr = None, limit=1000): ... - -# No: - -def munge(input: AnyStr=None): ... -def munge(input: AnyStr, limit = 1000): ... -``` - -Compound statements (multiple statements on the same line) are generally discouraged. - -```python -# Yes: - -if foo == 'blah': - do_blah_thing() -do_one() -do_two() -do_three() - -# Rather not: - -if foo == 'blah': do_blah_thing() -do_one(); do_two(); do_three() -``` - -While sometimes it’s okay to put an if/for/while with a small body on the same line, never do this for multi-clause statements. Also avoid folding such long lines! - -```python -# Rather not: - -if foo == 'blah': do_blah_thing() -for x in lst: total += x -while t < 10: t = delay() - -# Definitely not: - -if foo == 'blah': do_blah_thing() -else: do_non_blah_thing() - -try: something() -finally: cleanup() - -do_one(); do_two(); do_three(long, argument, - list, like, this) - -if foo == 'blah': one(); two(); three() -``` - -## When to use trailing commas - -Trailing commas are usually optional, except they are mandatory when making a tuple of one element (and in Python 2 they have semantics for the print statement). For clarity, it is recommended to surround the latter in (technically redundant) parentheses. - -```python -# Yes: - -FILES = ('setup.cfg',) - -# OK, but confusing: - -FILES = 'setup.cfg', -``` - -When trailing commas are redundant, they are often helpful when a version control system is used, when a list of values, arguments or imported items is expected to be extended over time. -The pattern is to put each value (etc.) on a line by itself, always adding a trailing comma, and add the close parenthesis/bracket/brace on the next line. -However, it does not make sense to have a trailing comma on the same line as the closing delimiter (except in the above case of singleton tuples). - -```python -# Yes: - -FILES = [ - 'setup.cfg', - 'tox.ini', - ] -initialize(FILES, - error=True, - ) - -# No: - -FILES = ['setup.cfg', 'tox.ini',] -initialize(FILES, error=True,) -``` - -## Comments - -Comments that contradict the code are worse than no comments. Always make a priority of keeping the comments up-to-date when the code changes! - -Comments should be complete sentences. If a comment is a phrase or sentence, its first word should be capitalized, unless it is an identifier that begins with a lower case letter (never alter the case of identifiers!). - -If a comment is short, the period at the end can be omitted. Block comments generally consist of one or more paragraphs built out of complete sentences, and each sentence should end in a period. - -You should use two spaces after a sentence-ending period. - -When writing English, follow Strunk and White. - -Python coders from non-English speaking countries: please write your comments in English, unless you are 120% sure that the code will never be read by people who don’t speak your language. - -### Block Comments - -Block comments generally apply to some (or all) code that follows them, and are indented to the same level as that code. Each line of a block comment starts with a # and a single space (unless it is indented text inside the comment). - -Paragraphs inside a block comment are separated by a line containing a single #. - -### Inline Comments - -Use inline comments sparingly. - -An inline comment is a comment on the same line as a statement. Inline comments should be separated by at least two spaces from the statement. They should start with a # and a single space. - -Inline comments are unnecessary and in fact distracting if they state the obvious. - -```python -# Don’t do this: - -x = x + 1 # Increment x - -# But sometimes, this is useful: - -x = x + 1 # Compensate for border -``` - -### Documentation Strings - -Conventions for writing good documentation strings (a.k.a. “docstrings”) are immortalized in PEP 257. - -Write docstrings for all public modules, functions, classes, and methods. Docstrings are not necessary for non-public methods, but you should have a comment that describes what the method does. This comment should appear after the def line. - -PEP 257 describes good docstring conventions. Note that most importantly, the """ that ends a multiline docstring should be on a line by itself, e.g.: - -```python - """Return a foobang - - Optional plotz says to frobnicate the bizbaz first. - """ - -# For one liner docstrings, please keep the closing """ on the same line. -``` - -## Naming Conventions - -The naming conventions of Python’s library are a bit of a mess, so we’ll never get this completely consistent – nevertheless, here are the currently recommended naming standards. -New modules and packages (including third party frameworks) should be written to these standards, but where an existing library has a different style, internal consistency is preferred. - -### Overriding Principle - -Names that are visible to the user as public parts of the API should follow conventions that reflect usage rather than implementation. - -### Descriptive Naming Styles - -There are a lot of different naming styles. It helps to be able to recognize what naming style is being used, independently from what they are used for. - -The following naming styles are commonly distinguished: - -> b (single lowercase letter) -> -> B (single uppercase letter) -> -> lowercase -> -> lower_case_with_underscores -> -> UPPERCASE -> -> UPPER_CASE_WITH_UNDERSCORES -> -> CapitalizedWords (or CapWords, CamelCase5, StudlyCaps) -> -> mixedCase (differs from CapitalizedWords by initial lowercase character!) -> -> Capitalized_Words_With_Underscores (ugly!) - -Note: - -When using abbreviations in CapWords, capitalize all the letters of the abbreviation. Thus HTTPServerError is better than HttpServerError. - -There’s also the style of using a short unique prefix to group related names together. This is not used much in Python, but it is mentioned for completeness. For example, the os.stat() function returns a tuple whose items traditionally have names like st_mode, st_size, st_mtime and so on. -(This is done to emphasize the correspondence with the fields of the POSIX system call struct, which helps programmers familiar with that.) - -The X11 library uses a leading X for all its public functions. In Python, this style is generally deemed unnecessary because attribute and method names are prefixed with an object, and function names are prefixed with a module name. - -In addition, the following special forms using leading or trailing underscores are recognized (these can generally be combined with any case convention): - -> ``_single_leading_underscore``: weak “internal use” indicator. E.g. from M import * does not import objects whose name starts with an underscore. -> -> ``single_trailing_underscore_``: used by convention to avoid conflicts with Python keyword, e.g.: Tkinter.Toplevel(master, class_='ClassName') -> -> ``__double_leading_underscore``: when naming a class attribute, invokes name mangling (inside class FooBar, ``__boo`` becomes ``_FooBar__boo`; see below). -> -> ``__double_leading_and_trailing_underscore__``: “magic” objects or attributes that live in user-controlled namespaces. E.g. ``__init__``, ``__import__`` or ``__file__``. Never invent such names; only use them as documented. - -### Prescriptive: Naming Conventions - -#### Names to Avoid - -Never use the characters ‘l’ (lowercase letter el), ‘O’ (uppercase letter oh), or ‘I’ (uppercase letter eye) as single character variable names. - -In some fonts, these characters are indistinguishable from the numerals one and zero. When tempted to use ‘l’, use ‘L’ instead. - -#### ASCII Compatibility - -Identifiers used in the standard library must be ASCII compatible as described in the policy section of PEP 3131. - -#### Package and Module Names - -Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves readability. Python packages should also have short, all-lowercase names, although the use of underscores is discouraged. - -When an extension module written in C or C++ has an accompanying Python module that provides a higher level (e.g. more object oriented) interface, the C/C++ module has a leading underscore (e.g. _socket). - -#### Class Names - -Class names should normally use the CapWords convention. - -The naming convention for functions may be used instead in cases where the interface is documented and used primarily as a callable. - -Note that there is a separate convention for builtin names: most builtin names are single words (or two words run together), with the CapWords convention used only for exception names and builtin constants. - -#### Type variable names - -Names of type variables introduced in PEP 484 should normally use CapWords preferring short names: T, AnyStr, Num. It is recommended to add suffixes _co or_contra to the variables used to declare covariant or contravariant behavior correspondingly. Examples - -```python -from typing import TypeVar - - VT_co = TypeVar('VT_co', covariant=True) - KT_contra = TypeVar('KT_contra', contravariant=True) -``` - -#### Exception Names - -Because exceptions should be classes, the class naming convention applies here. However, you should use the suffix “Error” on your exception names (if the exception actually is an error). - -#### Global Variable Names - -(Let’s hope that these variables are meant for use inside one module only.) The conventions are about the same as those for functions. - -Modules that are designed for use via from M import * should use the **all** mechanism to prevent exporting globals, or use the older convention of prefixing such globals with an underscore (which you might want to do to indicate these globals are “module non-public”). - -#### Function Names - -Function names should be lowercase, with words separated by underscores as necessary to improve readability. - -mixedCase is allowed only in contexts where that’s already the prevailing style (e.g. threading.py), to retain backwards compatibility. - -#### Function and method arguments - -Always use self for the first argument to instance methods. - -Always use cls for the first argument to class methods. - -If a function argument’s name clashes with a reserved keyword, it is generally better to append a single trailing underscore rather than use an abbreviation or spelling corruption. Thus class_ is better than clss. (Perhaps better is to avoid such clashes by using a synonym.) - -#### Method Names and Instance Variables - -Use the function naming rules: lowercase with words separated by underscores as necessary to improve readability. - -Use one leading underscore only for non-public methods and instance variables. - -To avoid name clashes with subclasses, use two leading underscores to invoke Python’s name mangling rules. - -Python mangles these names with the class name: if class Foo has an attribute named __a, it cannot be accessed by Foo.__a. (An insistent user could still gain access by calling Foo._Foo__a.) -Generally, double leading underscores should be used only to avoid name conflicts with attributes in classes designed to be subclassed. - -Note: there is some controversy about the use of __names (see below). - -#### Constants - -Constants are usually defined on a module level and written in all capital letters with underscores separating words. Examples include MAX_OVERFLOW and TOTAL. - -#### Designing for inheritance - -Always decide whether a class’s methods and instance variables (collectively: “attributes”) should be public or non-public. If in doubt, choose non-public; it’s easier to make it public later than to make a public attribute non-public. - -Public attributes are those that you expect unrelated clients of your class to use, with your commitment to avoid backward incompatible changes. -Non-public attributes are those that are not intended to be used by third parties; you make no guarantees that non-public attributes won’t change or even be removed. - -We don’t use the term “private” here, since no attribute is really private in Python (without a generally unnecessary amount of work). - -Another category of attributes are those that are part of the “subclass API” (often called “protected” in other languages). Some classes are designed to be inherited from, either to extend or modify aspects of the class’s behavior. -When designing such a class, take care to make explicit decisions about which attributes are public, which are part of the subclass API, and which are truly only to be used by your base class. - -With this in mind, here are the Pythonic guidelines: - -Public attributes should have no leading underscores. - -If your public attribute name collides with a reserved keyword, append a single trailing underscore to your attribute name. This is preferable to an abbreviation or corrupted spelling. -(However, notwithstanding this rule, ‘cls’ is the preferred spelling for any variable or argument which is known to be a class, especially the first argument to a class method.) - -Note 1: See the argument name recommendation above for class methods. - -For simple public data attributes, it is best to expose just the attribute name, without complicated accessor/mutator methods. Keep in mind that Python provides an easy path to future enhancement, should you find that a simple data attribute needs to grow functional behavior. -In that case, use properties to hide functional implementation behind simple data attribute access syntax. - -Note 1: Properties only work on new-style classes. - -Note 2: Try to keep the functional behavior side-effect free, although side-effects such as caching are generally fine. - -Note 3: Avoid using properties for computationally expensive operations; the attribute notation makes the caller believe that access is (relatively) cheap. - -If your class is intended to be subclassed, and you have attributes that you do not want subclasses to use, consider naming them with double leading underscores and no trailing underscores. -This invokes Python’s name mangling algorithm, where the name of the class is mangled into the attribute name. This helps avoid attribute name collisions should subclasses inadvertently contain attributes with the same name. - -Note 1: Note that only the simple class name is used in the mangled name, so if a subclass chooses both the same class name and attribute name, you can still get name collisions. - -Note 2: Name mangling can make certain uses, such as debugging and ```__getattr__()```, less convenient. However the name mangling algorithm is well documented and easy to perform manually. - -Note 3: Not everyone likes name mangling. Try to balance the need to avoid accidental name clashes with potential use by advanced callers. - -### Public and internal interfaces - -Any backwards compatibility guarantees apply only to public interfaces. Accordingly, it is important that users be able to clearly distinguish between public and internal interfaces. - -Documented interfaces are considered public, unless the documentation explicitly declares them to be provisional or internal interfaces exempt from the usual backwards compatibility guarantees. All undocumented interfaces should be assumed to be internal. - -To better support introspection, modules should explicitly declare the names in their public API using the ```__all__``` attribute. Setting ```__all__``` to an empty list indicates that the module has no public API. - -Even with ```__all__``` set appropriately, internal interfaces (packages, modules, classes, functions, attributes or other names) should still be prefixed with a single leading underscore. - -An interface is also considered internal if any containing namespace (package, module or class) is considered internal. - -Imported names should always be considered an implementation detail. -Other modules must not rely on indirect access to such imported names unless they are an explicitly documented part of the containing module’s API, such as os.path or a package’s ```__init__``` module that exposes functionality from submodules. - -## Programming Recommendations - -Code should be written in a way that does not disadvantage other implementations of Python (PyPy, Jython, IronPython, Cython, Psyco, and such). - -For example, do not rely on CPython’s efficient implementation of in-place string concatenation for statements in the form ```a += b``` or ```a = a + b```. This optimization is fragile even in CPython (it only works for some types) and isn’t present at all in implementations that don’t use refcounting. -In performance sensitive parts of the library, the ```''.join()``` form should be used instead. This will ensure that concatenation occurs in linear time across various implementations. - -Comparisons to singletons like ```None``` should always be done with ```is``` or ```is not```, never the equality operators. - -Also, beware of writing ```if x``` when you really mean ```if x is not None``` – e.g. when testing whether a variable or argument that defaults to None was set to some other value. The other value might have a type (such as a container) that could be false in a boolean context! - -Use ```is not``` operator rather than ```not ... is```. While both expressions are functionally identical, the former is more readable and preferred. - -```python -# Yes: - -if foo is not None: - -# No: - -if not foo is None: -``` - -When implementing ordering operations with rich comparisons, it is best to implement all six operations (```__eq__```, ```__ne__```, ```__lt__```, ```__le__```, ```__gt__```, ```__ge__```) rather than relying on other code to only exercise a particular comparison. - -To minimize the effort involved, the ```functools.total_ordering()``` decorator provides a tool to generate missing comparison methods. - -PEP 207 indicates that reflexivity rules are assumed by Python. Thus, the interpreter may swap ```y > x``` with ```x < y```, ```y >= x``` with ```x <= y```, and may swap the arguments of ```x == y``` and ```x != y.``` -The ```sort()``` and ```min()``` operations are guaranteed to use the ```<``` operator and the ```max()``` function uses the ```>``` operator. However, it is best to implement all six operations so that confusion doesn’t arise in other contexts. - -Always use a def statement instead of an assignment statement that binds a lambda expression directly to an identifier. - -```python -# Yes: - -def f(x): return 2*x - -# No: - -f = lambda x: 2*x -``` - -The first form means that the name of the resulting function object is specifically ```‘f’``` instead of the generic ```‘’```. This is more useful for tracebacks and string representations in general. -The use of the assignment statement eliminates the sole benefit a lambda expression can offer over an explicit def statement (i.e. that it can be embedded inside a larger expression) - -Derive exceptions from Exception rather than BaseException. Direct inheritance from BaseException is reserved for exceptions where catching them is almost always the wrong thing to do. - -Design exception hierarchies based on the distinctions that code catching the exceptions is likely to need, rather than the locations where the exceptions are raised. -Aim to answer the question “What went wrong?” programmatically, rather than only stating that “A problem occurred” (see PEP 3151 for an example of this lesson being learned for the builtin exception hierarchy) - -Class naming conventions apply here, although you should add the suffix “Error” to your exception classes if the exception is an error. Non-error exceptions that are used for non-local flow control or other forms of signaling need no special suffix. - -Use exception chaining appropriately. In Python 3, “raise X from Y” should be used to indicate explicit replacement without losing the original traceback. - -When deliberately replacing an inner exception (using “raise X” in Python 2 or “raise X from None” in Python 3.3+), ensure that relevant details are transferred to the new exception -(such as preserving the attribute name when converting KeyError to AttributeError, or embedding the text of the original exception in the new exception message). - -The paren-using form also means that when the exception arguments are long or include string formatting, you don’t need to use line continuation characters thanks to the containing parentheses. - -When catching exceptions, mention specific exceptions whenever possible instead of using a bare except: clause. - -For example, use: - -```python -try: - import platform_specific_module -except ImportError: - platform_specific_module = None -``` - -A bare except: clause will catch SystemExit and KeyboardInterrupt exceptions, making it harder to interrupt a program with Control-C, and can disguise other problems. If you want to catch all exceptions that signal program errors, use except Exception: (bare except is equivalent to except BaseException:). - -A good rule of thumb is to limit use of bare ‘except’ clauses to two cases: -If the exception handler will be printing out or logging the traceback; at least the user will be aware that an error has occurred. -If the code needs to do some cleanup work, but then lets the exception propagate upwards with raise. try...finally can be a better way to handle this case. - -When binding caught exceptions to a name, prefer the explicit name binding syntax added in Python 2.6: - -```python -try: - process_data() -except Exception as exc: - raise DataProcessingFailedError(str(exc)) -``` - -This is the only syntax supported in Python 3, and avoids the ambiguity problems associated with the older comma-based syntax. -When catching operating system errors, prefer the explicit exception hierarchy introduced in Python 3.3 over introspection of errno values. - -Additionally, for all try/except clauses, limit the try clause to the absolute minimum amount of code necessary. Again, this avoids masking bugs. - -```python -# Yes: - -try: - value = collection[key] -except KeyError: - return key_not_found(key) -else: - return handle_value(value) - -# No: - -try: - # Too broad! - return handle_value(collection[key]) -except KeyError: - # Will also catch KeyError raised by handle_value() - return key_not_found(key) -``` - -When a resource is local to a particular section of code, use a with statement to ensure it is cleaned up promptly and reliably after use. A try/finally statement is also acceptable. - -Context managers should be invoked through separate functions or methods whenever they do something other than acquire and release resources. For example: - -```python -# Yes: - -with conn.begin_transaction(): - do_stuff_in_transaction(conn) - -# No: - -with conn: - do_stuff_in_transaction(conn) -``` - -The latter example doesn’t provide any information to indicate that the ```__enter__``` and ```__exit__``` methods are doing something other than closing the connection after a transaction. Being explicit is important in this case. - -Be consistent in return statements. Either all return statements in a function should return an expression, or none of them should. -If any return statement returns an expression, any return statements where no value is returned should explicitly state this as return None, and an explicit return statement should be present at the end of the function (if reachable). - -```python -# Yes: - -def foo(x): - if x >= 0: - return math.sqrt(x) - else: - return None - -def bar(x): - if x < 0: - return None - return math.sqrt(x) - -# No: - -def foo(x): - if x >= 0: - return math.sqrt(x) - -def bar(x): - if x < 0: - return - return math.sqrt(x) -``` - -Use string methods instead of the string module. - -String methods are always much faster and share the same API with unicode strings. Override this rule if backward compatibility with Pythons older than 2.0 is required. - -Use ```''.startswith()``` and ```''.endswith()``` instead of string slicing to check for prefixes or suffixes. - -```startswith()``` and ```endswith()``` are cleaner and less error prone. For example: - -```python -# Yes: - -if foo.startswith('bar'): - -# No: - -if foo[:3] == 'bar': -``` - -Object type comparisons should always use isinstance() instead of comparing types directly: - -```python -# Yes: - -if isinstance(obj, int): - -# No: - -if type(obj) is type(1): -``` - -When checking if an object is a string, keep in mind that it might be a unicode string too! In Python 2, str and unicode have a common base class, basestring, so you can do: - -```python -if isinstance(obj, basestring): -``` - -Note that in Python 3, unicode and basestring no longer exist (there is only str) and a bytes object is no longer a kind of string (it is a sequence of integers instead) - -For sequences, (strings, lists, tuples), use the fact that empty sequences are false: - -```python -# Yes: - -if not seq: -if seq: - -# No: - -if len(seq): -if not len(seq): -``` - -Don’t write string literals that rely on significant trailing whitespace. Such trailing whitespace is visually indistinguishable and some editors (or more recently, reindent.py) will trim them. - -Don’t compare boolean values to True or False using ==: - -```python -# Yes: - -if greeting: - -# No: - -if greeting == True: - -# Worse: - -if greeting is True: -``` - -### Function Annotations - -With the acceptance of PEP 484, the style rules for function annotations are changing. - -In order to be forward compatible, function annotations in Python 3 code should preferably use PEP 484 syntax. (There are some formatting recommendations for annotations in the previous section.) -The experimentation with annotation styles that was recommended previously in this PEP is no longer encouraged. - -However, outside the stdlib, experiments within the rules of PEP 484 are now encouraged. For example, marking up a large third party library or application with PEP 484 style type annotations, reviewing how easy it was to add those annotations, and observing whether their presence increases code understandability. - -The Python standard library should be conservative in adopting such annotations, but their use is allowed for new code and for big refactorings. - -For code that wants to make a different use of function annotations it is recommended to put a comment of the form: - -```python -# type: ignore -``` - -near the top of the file; this tells type checker to ignore all annotations. (More fine-grained ways of disabling complaints from type checkers can be found in PEP 484.) -Like linters, type checkers are optional, separate tools. Python interpreters by default should not issue any messages due to type checking and should not alter their behavior based on annotations. - -Users who don’t want to use type checkers are free to ignore them. However, it is expected that users of third party library packages may want to run type checkers over those packages. -For this purpose PEP 484 recommends the use of stub files: .pyi files that are read by the type checker in preference of the corresponding .py files. Stub files can be distributed with a library, or separately (with the library author’s permission) through the typeshed repo 6. - -For code that needs to be backwards compatible, type annotations can be added in the form of comments. See the relevant section of PEP 484 7. - -## Footnotes - -PEP 7, Style Guide for C Code, van Rossum↩ - -Barry’s GNU Mailman style guide ↩ - -Hanging indentation is a type-setting style where all the lines in a paragraph are indented except the first line. -In the context of Python, the term is used to describe a style where the opening parenthesis of a parenthesized statement is the last non-whitespace character of the line, with subsequent lines being indented until the closing parenthesis.↩ - -Donald Knuth's The TeXBook, pages 195 and 196.↩ - -↩ - -Typeshed repo ↩ - -Suggested syntax for Python 2.7 and straddling code ↩ - -## Copyright - -This document has been placed in the public domain. - -### Source - -based on PEP8: diff --git a/doc/development/documentation_requirements.md b/doc/development/documentation_requirements.md index 163db979..e75fa783 100644 --- a/doc/development/documentation_requirements.md +++ b/doc/development/documentation_requirements.md @@ -3,6 +3,7 @@ **Summary:** This document provides the guidelines for the documentation. - [Documentation Requirements](#documentation-requirements) + - [Python Docstrings](#python-docstrings) - [Readability and Maintainability](#readability-and-maintainability) - [Code Structure](#code-structure) - [Efficiency and Performance](#efficiency-and-performance) @@ -14,11 +15,22 @@ - [Scalability](#scalability) - [Consistency with Coding Standards](#consistency-with-coding-standards) +## Python Docstrings + +The project comes with an extension to automatically generate docstrings. It is mandatory to use it if you comment your code. + +To generate a docstring you have to be in the development container and then a popup will apear after writing three quotation marks: + +![docstring_popup.png](/doc/assets/development/docstring_popup.png) + +Press `Enter` or select the option and it should produce a docstring that looks like this: + +![docstring.png](/doc/assets/development/docstring.png) + ## Readability and Maintainability - **Consistent Formatting:** Code should follow a consistent and readable formatting style. Tools like linters or formatters can help enforce a consistent code style. - [linting](./linting.md) - - [coding_style](./coding_style.md) - **Meaningful Names:** Variable and function names should be descriptive and convey the purpose of the code. - **Comments:** Clear and concise comments should be used where necessary to explain complex logic or provide context. @@ -55,7 +67,6 @@ ## Version Control - **Commit Messages:** Use descriptive and meaningful commit messages to track changes effectively. - - [commit](./commit.md) - **Branching Strategy:** Follow a consistent and well-defined branching strategy to manage code changes. ## Scalability diff --git a/doc/development/drive_action.md b/doc/development/drive_action.md new file mode 100644 index 00000000..08c93037 --- /dev/null +++ b/doc/development/drive_action.md @@ -0,0 +1,69 @@ +# GitHub actions + +**Summary:** This page explains the GitHub build action we use to evaluate our agent. + +- [GitHub actions](#github-actions) + - [The drive job](#the-drive-job) + - [1. Checkout repository (`actions/checkout@v3`)](#1-checkout-repository-actionscheckoutv3) + - [2. Run agent with docker-compose](#2-run-agent-with-docker-compose) + - [3. Copy simulation results file out of container](#3-copy-simulation-results-file-out-of-container) + - [4. Stop docker-compose stack](#4-stop-docker-compose-stack) + - [5. Comment result in pull request `actions/github-script@v6`](#5-comment-result-in-pull-request-actionsgithub-scriptv6) + - [Simulation results](#simulation-results) + +## The drive job + +The `drive` job is executed conditionally on `pull_request`, after the build successfully ran through. + +### 1. Checkout repository ([`actions/checkout@v3`](https://github.com/actions/checkout)) + +Same step as in the [build job](#1-checkout-repository--actionscheckoutv3-) + +### 2. Run agent with docker-compose + +Runs the agent with the [`build/docker-compose.cicd.yaml`](../../build/docker-compose.cicd.yaml) that only contains the +bare minimum components for test execution: + +- Carla Simulator +- roscore +- Agent container, run through the + Carla [`leaderboard_evaluator`](https://github.com/carla-simulator/leaderboard/blob/leaderboard-2.0/leaderboard/leaderboard_evaluator.py). + +### 3. Copy simulation results file out of container + +Copies the created `simulation_results.json` file out of the agent container into the current container + +### 4. Stop docker-compose stack + +Stops the remaining containers (Carla, roscore) and removes the volumes with: +`$ docker-compose down -v`. + +This step is important to clean up the remaining containers to have a clean run everytime. This is also the reason for +the `if: always()`, that ensures step execution. + +### 5. Comment result in pull request [`actions/github-script@v6`](https://github.com/marketplace/actions/github-script) + +This steps uses a JS script to parse the simulation results and add a comment with a results table to the corresponding +pull request. + +An example comment for this would be: + +## Simulation results + +| Metric | Value | +|--------------------------------------|---------| +| Avg. driving score | 0.06006 | +| Avg. route completion | 0.22 | +| Avg. infraction penalty | 0.273 | +| Collisions with pedestrians | 0.0 | +| Collisions with vehicles | 62.046 | +| Collisions with layout | 62.046 | +| Red lights infractions | 0.0 | +| Stop sign infractions | 0.0 | +| Off-road infractions | 0 | +| Route deviations | 0.0 | +| Route timeouts | 62.046 | +| Agent blocked | 0.0 | +| Yield emergency vehicles infractions | 0.0 | +| Scenario timeouts | 62.046 | +| Min speed infractions | 0.0 | diff --git a/doc/development/first_steps.md b/doc/development/first_steps.md new file mode 100644 index 00000000..309679dd --- /dev/null +++ b/doc/development/first_steps.md @@ -0,0 +1,61 @@ +# First steps + +**Summary**: These first steps should help you get familiar with the project and how to work with it. + +- [First steps](#first-steps) + - [Prepare your environment](#prepare-your-environment) + - [Start the development container](#start-the-development-container) + - [Linting and formatting](#linting-and-formatting) + - [Python and Markdown linters](#python-and-markdown-linters) + - [Python formatter](#python-formatter) + - [Trigger linting and formatting manually](#trigger-linting-and-formatting-manually) + - [Further information](#further-information) + +## Prepare your environment + +After you cloned the repository open the corresponding folder with VS Code. The repository comes with a suite of recommended extensions that simplify the development process. Navigate to the `Extensions` tab in VS Code (Shortcut `Ctrl+Shift+X`) and install the extensions. + +## Start the development container + +In order to make changes to the code or add new functionality you can edit the files directly in VS Code now. However, for the default development environment head to the `/build` folder and execute the `docker-compose.dev.yaml` file via right-click and selecting `Compose Up` in the menu: + +![devcontainer.png](/doc/assets/development/devcontainer.png) + +> This is the default way to execute every docker-compose file. If you try to start a container without it things may break! + +Then navigate to the `Docker` tab in VS Code and attach a VS Code window to the `build-agent-dev` container: + +![attach.png](/doc/assets/development/attach.png) + +A new VS Code window will open, and you now have all the project files with the correct python paths and code completion. + +## Linting and formatting + +The linters and formatter are automatically running in the background as VS Code extensions. + +### Python and Markdown linters + +For Python and Markdown files the suggested changes are shown as warnings and are displayed in the file directly. These have to be resolved in order for the GitHub actions to complete on your pull request. + +### Python formatter + +The Python formatter automatically executes each time you save your file. + +### Trigger linting and formatting manually + +If you want to manually check the linting and formatting you can execute the `docker-compose.linter.yaml` file the same way you execute the dev file mentioned in [Start the development container](#start-the-development-container). + +If you want to trigger the code formatting manually you can type the following command in a shell of the container: + +```sh +black +``` + +To get a shell of the container you can either use the Terminal on the bottom of VS Code in the attached VS Code window (see [Start the development container](#start-the-development-container), Shortcut `` Ctrl+Shift+` ``) or attach a shell in the same right-click menu as attaching a VS Code window. + +## Further information + +These are additional resources that are useful for handling the project: + +- [Executing different scenarios](/doc/general/execution.md) +- [Documentation requirements](/doc/development/documentation_requirements.md) (especially docstrings) diff --git a/doc/development/linter_action.md b/doc/development/linter_action.md index 3cc5da67..e52a00ef 100644 --- a/doc/development/linter_action.md +++ b/doc/development/linter_action.md @@ -1,10 +1,10 @@ -# Github actions +# GitHub actions (Kept from previous group [paf22]) **Summary:** This page explains the GitHub lint action we use to unsure Code quality. -- [Github actions](#github-actions) +- [GitHub actions](#github-actions) - [General](#general) - [Pull requests](#pull-requests) - [🚨 Common Problems](#-common-problems) @@ -13,8 +13,8 @@ ## General -We use a github action to verify code quality. -These actions are defined in `.github/workflows/linter.yml`. +We use a GitHub action to verify code quality. +These actions are defined in `.github/workflows/linter.yml` and `.github/workflows/format.yml`. The actions are executed only on pull requests in order not to exceed the [minutes per month included in the Github](https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions) free plan. This is done by limiting the execution of the action by the following line: @@ -23,7 +23,7 @@ This is done by limiting the execution of the action by the following line: on: pull_request ``` -The actions uses the same linters described in the section [Linting](./linting.md). +The actions use the same linters described in the section [Linting](./linting.md). Event though the linters are already active during development, the execution on pull request ensures that nobody skips the linter during commit. @@ -31,7 +31,7 @@ the execution on pull request ensures that nobody skips the linter during commit ## Pull requests The action must pass successful to merge a pull request. -To enforce this behaviour, we set the action as requirement as described in the following: +To enforce this behavior, we set the action as requirement as described in the following: > You can add rules to your branches and how your merge request works. > diff --git a/doc/development/linting.md b/doc/development/linting.md index 16d887e8..37c6d5ce 100644 --- a/doc/development/linting.md +++ b/doc/development/linting.md @@ -17,7 +17,9 @@ To enforce unified standards in all python files, we use [Flake8](https://pypi.o - pycodestyle - Ned Batchelder’s McCabe script -. More details on it can be found in the documentation of Flake8. +More details on it can be found in the documentation of Flake8. + +We also use [black](https://github.com/psf/black) as a code formatter to unify the coding style. ## 💬 Markdown Linter diff --git a/doc/development/templates/template_class.py b/doc/development/templates/template_class.py index 76edcb37..617bf7b6 100644 --- a/doc/development/templates/template_class.py +++ b/doc/development/templates/template_class.py @@ -50,37 +50,17 @@ def __init__(self): # with reserved keywords self.if_ = False - # function names should be lower case with underscores to improve readability - # always use self as first argument for istance methods - def test_function1(self): - pass - - # single blank line between method definitions - # always use cls as first argument for class functions - def test_function2(cls): - pass - - ############################# - # 5. Comments # - ############################# - - def test_function3(self): # inline comment - # This is a block comment - # It goes over multiple lines - # All comments start with a blank space - pass - - ############################# - # 6. Docstrings # - ############################# - def test_function4(self, param1, param2): - # This docstring style is the default google style of the autoDocstring - # extension and helps with automated API documentation creation - """This is the description of the function. + # This docstring style is the default google style of the autoDocstring + # extension for automated API documentation creation + def test_function(self, param1: int, param2: float) -> str: + """_summary_ Args: - param1 (_type_): _description_ - param2 (_type_): _description_ + param1 (int): _description_ + param2 (float): _description_ + + Returns: + str: _description_ """ pass diff --git a/doc/development/templates/template_class_no_comments.py b/doc/development/templates/template_class_no_comments.py index 9d57c511..3d33c05f 100644 --- a/doc/development/templates/template_class_no_comments.py +++ b/doc/development/templates/template_class_no_comments.py @@ -15,41 +15,18 @@ def __init__(self): self._name = "Max" self.if_ = False - def test_function1(self, param1): - pass - - def test_function2(cls): - """ - - :return: - """ - pass - - def test_function3(self): # inline comment - # This is a block comment - # It goes over multiple lines - # All comments start with a blank space - pass + def test_function(self, param1: int, param2: float) -> str: + """_summary_ - def test_function4(self, param1, param2): - """ - This is the description of the function. + Args: + param1 (int): _description_ + param2 (float): _description_ - :param param1: first parameter - :param param2: second parameter - :return: return value(s) + Returns: + str: _description_ """ pass - def test_function5(self, param1, param2): - """_summary_ - - :param param1: _description_ - :param param2: _description_ - :return: _description_ - """ - return param1 - def main(self): """_summary_""" print("Hello World") diff --git a/doc/general/installation.md b/doc/general/installation.md index f5bc39a3..ba6b91d1 100644 --- a/doc/general/installation.md +++ b/doc/general/installation.md @@ -1,8 +1,9 @@ # 🛠️ Installation -**Summary:** This page explains the installation process for the +**Summary:** This page explains the installation process for the project. - [🛠️ Installation](#️-installation) + - [General](#general) - [Docker with NVIDIA GPU support](#docker-with-nvidia-gpu-support) - [Docker](#docker) - [Allow non-root user to execute Docker commands](#allow-non-root-user-to-execute-docker-commands) @@ -11,6 +12,10 @@ - [🚨 Common Problems](#-common-problems) - [Vulkan device not available](#vulkan-device-not-available) +## General + +Installation scripts are provided in the base folder of the repository ([pc_setup_admin.sh](/pc_setup_admin.sh) and [pc_setup_user.sh](/pc_setup_user.sh)). How to use these files is briefly described in [pc_setup.sh](/pc_setup.sh). The installation process in described in further detail here. + To run the project you have to install [docker](https://docs.docker.com/engine/install/) with NVIDIA GPU support, [nvidia-docker](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker). For development, we recommend Visual Studio Code with the plugins that are recommended inside the `.vscode` folder.