From 9cf2f6ada32a1c151d8d181a6cce22add1a08827 Mon Sep 17 00:00:00 2001 From: Jan Richter Date: Tue, 12 Sep 2023 16:55:22 +0200 Subject: [PATCH] =?UTF-8?q?Doc=20fixes=20before=20LTS=20-=20Avocado=20User?= =?UTF-8?q?=E2=80=99s=20Guide?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Updates to documentation in `Avocado User’s Guide` section to respects the latest changes. Reference: #5757 Signed-off-by: Jan Richter --- .../guides/contributor/chapters/plugins.rst | 2 + .../guides/contributor/chapters/runners.rst | 2 + docs/source/guides/user/chapters/advanced.rst | 39 ++++++++--------- .../guides/user/chapters/configuring.rst | 6 +-- docs/source/guides/user/chapters/datadirs.rst | 3 -- .../guides/user/chapters/dependencies.rst | 2 +- .../guides/user/chapters/installing.rst | 6 +-- .../guides/user/chapters/introduction.rst | 43 ++++++++++--------- .../guides/user/chapters/operations.rst | 15 ++++--- docs/source/guides/user/chapters/plugins.rst | 4 +- docs/source/guides/user/chapters/results.rst | 34 +++++++++------ docs/source/guides/user/chapters/tags.rst | 2 +- .../source/guides/writer/chapters/writing.rst | 2 + 13 files changed, 83 insertions(+), 77 deletions(-) diff --git a/docs/source/guides/contributor/chapters/plugins.rst b/docs/source/guides/contributor/chapters/plugins.rst index 235b1f762f..acf3b3ba1b 100644 --- a/docs/source/guides/contributor/chapters/plugins.rst +++ b/docs/source/guides/contributor/chapters/plugins.rst @@ -1,3 +1,5 @@ +.. _writing_plugin: + ************************* Writing an Avocado plugin ************************* diff --git a/docs/source/guides/contributor/chapters/runners.rst b/docs/source/guides/contributor/chapters/runners.rst index 1b9553672a..234af92e7a 100644 --- a/docs/source/guides/contributor/chapters/runners.rst +++ b/docs/source/guides/contributor/chapters/runners.rst @@ -81,6 +81,8 @@ doing:: To be able to complete such a command, Avocado needs to find the tests, and then to execute them. Those two major steps are described next. +.. _finding_tests: + Finding tests ~~~~~~~~~~~~~ diff --git a/docs/source/guides/user/chapters/advanced.rst b/docs/source/guides/user/chapters/advanced.rst index 85a4c36b72..9a3338355a 100644 --- a/docs/source/guides/user/chapters/advanced.rst +++ b/docs/source/guides/user/chapters/advanced.rst @@ -8,18 +8,15 @@ In some cases, you might have a wrapper as an entry point for the tests, so Avocado will use only the wrapper as test id. For instance, imagine a Makefile with some targets ('foo', 'bar') and each target is one test. Having a single test suite with a test calling `foo`, it will make Avocado print something like -this: +this:: - -``` -JOB ID : b6e5bdf2c891382bbde7f24e906a168af351154a -JOB LOG : ~/avocado/job-results/job-2021-09-24T17.39-b6e5bdf/job.log - (1/1) make: STARTED - (1/1) make: PASS (2.72 s) -RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 -JOB HTML : ~/avocado/job-results/job-2021-09-24T17.39-b6e5bdf/results.html -JOB TIME : 5.49 s -``` + JOB ID : b6e5bdf2c891382bbde7f24e906a168af351154a + JOB LOG : ~/avocado/job-results/job-2021-09-24T17.39-b6e5bdf/job.log + (1/1) make: STARTED + (1/1) make: PASS (2.72 s) + RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 + JOB HTML : ~/avocado/job-results/job-2021-09-24T17.39-b6e5bdf/results.html + JOB TIME : 5.49 s This is happening because Avocado is using the 'uri' as identifier with in the current Runnables. @@ -33,17 +30,15 @@ identifier_format = "{uri}-{args[0]}" ``` With the above adjustment, running the same suite it will produce something -like this: - -``` -JOB ID : 577b70b079e9a6f325ff3e73fd9b93f80ee7f221 -JOB LOG : /home/local/avocado/job-results/job-2021-11-23T13.12-577b70b/job.log - (1/1) "/usr/bin/make-foo": STARTED - (1/1) "/usr/bin/make-foo": PASS (0.01 s) -RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 -JOB HTML : ~/avocado/job-results/job-2021-11-23T13.12-577b70b/results.html -JOB TIME : 0.97 s -``` +like this:: + + JOB ID : 577b70b079e9a6f325ff3e73fd9b93f80ee7f221 + JOB LOG : /home/local/avocado/job-results/job-2021-11-23T13.12-577b70b/job.log + (1/1) "/usr/bin/make-foo": STARTED + (1/1) "/usr/bin/make-foo": PASS (0.01 s) + RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 + JOB HTML : ~/avocado/job-results/job-2021-11-23T13.12-577b70b/results.html + JOB TIME : 0.97 s For the `identifier_format` you can use any f-string that it will use `{uri}`, `{args}` or `{kwargs}`. By default it will use `{uri}`. diff --git a/docs/source/guides/user/chapters/configuring.rst b/docs/source/guides/user/chapters/configuring.rst index 464827b5ef..0229764c3f 100644 --- a/docs/source/guides/user/chapters/configuring.rst +++ b/docs/source/guides/user/chapters/configuring.rst @@ -81,7 +81,7 @@ So the file parsing order is: * ``avocado.plugins.settings`` plugins (but they can insert to any location) - For more information about this, visit the "Contributor's Guide" - section named "Writing an Avocado plugin" + section named :ref:`writing_plugin` * ``~/.config/avocado/avocado.conf`` @@ -229,10 +229,6 @@ Example using the command-line: $ avocado run --verbose /bin/true -.. note:: Currently we still have some "old style boolean" options where you - should pass "on" or "off" on the command-line. i.e: ``--json-job-result=off``. - Those options are going to be replaced soon. - Lists ~~~~~ diff --git a/docs/source/guides/user/chapters/datadirs.rst b/docs/source/guides/user/chapters/datadirs.rst index be3c8fd218..fc5fc94ef8 100644 --- a/docs/source/guides/user/chapters/datadirs.rst +++ b/docs/source/guides/user/chapters/datadirs.rst @@ -49,6 +49,3 @@ place temporary files used by tests. That directory will be in normal circumstan ``/var/tmp/avocado_XXXXX``, (where ``XXXXX`` is in actuality a random string) securely created on ``/var/tmp/``, unless the user has the ``$TMPDIR`` environment variable set, since that is customary among unix programs. - -The next section of the documentation explains how you can see and set config -values that modify the behavior for the Avocado utilities and plugins. diff --git a/docs/source/guides/user/chapters/dependencies.rst b/docs/source/guides/user/chapters/dependencies.rst index ed454efec2..1c84267d70 100644 --- a/docs/source/guides/user/chapters/dependencies.rst +++ b/docs/source/guides/user/chapters/dependencies.rst @@ -3,7 +3,7 @@ Managing Dependencies ===================== -.. note:: Test dependencies are supported only on the nrunner runner. +.. note:: Test dependencies are supported only on the nrunner runner which is the default one. A test's dependency can be fulfilled by the Dependencies Resolver feature. diff --git a/docs/source/guides/user/chapters/installing.rst b/docs/source/guides/user/chapters/installing.rst index d1fa709cbc..99f4935f0a 100644 --- a/docs/source/guides/user/chapters/installing.rst +++ b/docs/source/guides/user/chapters/installing.rst @@ -19,7 +19,7 @@ Installing from PyPI -------------------- The simplest installation method is through ``pip``. On most POSIX systems -with Python 3.4 (or later) and ``pip`` available, installation can be performed +with Python 3.7 (or later) and ``pip`` available, installation can be performed with a single command:: $ pip3 install --user avocado-framework @@ -56,7 +56,7 @@ version 29. To subscribe to the latest version stream, run:: Or, to use the LTS (Long Term Stability) version stream, run:: - $ dnf module enable avocado:82lts + $ dnf module enable avocado:103lts Then proceed to install a module profile or individual packages. If you're unsure about what to do, simply run:: @@ -127,7 +127,7 @@ Then to install Avocado from the git repository run:: $ cd avocado $ python3 setup.py install --user -Optionally, to install the plugins run: +Optionally, to install the plugins run:: $ python3 setup.py plugin --install=golang --user $ python3 setup.py plugin --install=html --user diff --git a/docs/source/guides/user/chapters/introduction.rst b/docs/source/guides/user/chapters/introduction.rst index f43ae2aba8..c3817d5720 100644 --- a/docs/source/guides/user/chapters/introduction.rst +++ b/docs/source/guides/user/chapters/introduction.rst @@ -13,11 +13,13 @@ test reference, which could be either a path to the file, or a recognizable name:: $ avocado run /bin/true - JOB ID : 3a5c4c51ceb5369f23702efb10b4209b111141b2 - JOB LOG : $HOME/avocado/job-results/job-2019-10-31T10.34-3a5c4c5/job.log - (1/1) /bin/true: PASS (0.04 s) + JOB ID : 89b5d609d4832c784f04cf14f4ec2d17917d419a + JOB LOG : $HOME/avocado/job-results/job-2023-09-06T15.52-89b5d60/job.log + (1/1) /bin/true: STARTED + (1/1) /bin/true: PASS (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 - JOB TIME : 0.15 s + JOB HTML : $HOME/avocado/job-results/job-2023-09-06T15.52-89b5d60/results.html + JOB TIME : 1.49 s You probably noticed that we used ``/bin/true`` as a test, and in accordance with our expectations, it passed! These are known as @@ -57,19 +59,19 @@ instrumented and executable tests:: Using a different runner ------------------------ -Currently Avocado has two test runners: ``nrunner`` (the new runner) and -``runner`` (legacy). You can find a list of current runners installed with the +Currently Avocado has only one runner: ``nrunner`` (the new runner) +But some plugins may use their own runners. +You can find a list of current runners installed with the ``avocado plugins`` command:: $ avocado plugins Plugins that run test suites on a job (runners): nrunner nrunner based implementation of job compliant runner - runner The conventional test runner During the test execution, you can select the runner using the option -``--test-runner``, where the default is the nrunner one:: +``--suite-runner``, where the default is the nrunner one:: - $ avocado run --test-runner='runner' /bin/true + $ avocado run --suite-runner='runner' /bin/true Interrupting tests @@ -170,7 +172,8 @@ them to tests. If one or more test references can not be resolved to tests, the Job will not be created. Example:: $ avocado run examples/tests/passtest.py badtest.py - Unable to resolve reference(s) 'badtest.py' with plugins(s) 'file', 'robot', try running 'avocado -V list badtest.py' to see the details. + No tests found for given test references: badtest.py + Try 'avocado -V list badtest.py' for details But if you want to execute the Job anyway, with the tests that could be resolved, you can use ``--ignore-missing-references``, a boolean command-line @@ -366,7 +369,7 @@ Avocado features. A reasonable effort will be made to not break backwards compatibility with applications that parse the current form of its JSON result. - **3. TAP:** +**3. TAP:** Provides the basic `TAP `__ (Test Anything Protocol) results, currently in v12. Unlike most existing Avocado machine readable @@ -377,7 +380,7 @@ outputs this one is streamlined (per test results):: ok 1 examples/tests/sleeptest.py:SleepTest.test - **4. Beaker:** +**4. Beaker:** When avocaodo finds itself running inside a beaker task the beaker_report plugin will send the test results and log files to the @@ -396,16 +399,14 @@ In order to do that, you can use ``avocado --show=job run ...``:: $ avocado --show=job run examples/tests/sleeptest.py ... - Job ID: f9ea1742134e5352dec82335af584d1f151d4b85 - - START 1-sleeptest.py:SleepTest.test - - PARAMS (key=timeout, path=*, default=None) => None - PARAMS (key=sleep_length, path=*, default=1) => 1 - Sleeping for 1.00 seconds - PASS 1-sleeptest.py:SleepTest.test + + avocado.job: Job ID: c9ca6a96d34fd0355f5f121af7fa87eef734a196 + avocado.job: + avocado.job: examples/tests/sleeptest.py:SleepTest.test: STARTED + avocado.job: examples/tests/sleeptest.py:SleepTest.test: PASS + avocado.job: More information in $HOME/avocado/job-results/job-2023-09-08T15.27-c9ca6a9/test-results/1-examples_tests_sleeptest.py_SleepTest.test + avocado.job: Test results available in $HOME/avocado/job-results/job-2023-09-08T15.27-c9ca6a9 - Test results available in $HOME/avocado/job-results/job-2015-06-02T10.45-f9ea174 As you can see, the UI output is suppressed and only the job log is shown, making this a useful feature for test development and debugging. diff --git a/docs/source/guides/user/chapters/operations.rst b/docs/source/guides/user/chapters/operations.rst index 36fa8c3502..e29db80c44 100644 --- a/docs/source/guides/user/chapters/operations.rst +++ b/docs/source/guides/user/chapters/operations.rst @@ -15,8 +15,10 @@ from the command line, such as:: $ avocado run /bin/true /bin/false JOB ID : 42c60bea72e6d55756bfc784eb2b354f788541cf JOB LOG : $HOME/avocado/job-results/job-2020-08-13T11.23-42c60be/job.log + (1/2) /bin/true: STARTED + (2/2) /bin/false: STARTED (1/2) /bin/true: PASS (0.01 s) - (2/2) /bin/false: FAIL: Exited with status: '1', stdout: '' stderr: '' (0.08 s) + (2/2) /bin/false: FAIL (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : $HOME/avocado/job-results/job-2020-08-13T11.23-42c60be/results.html JOB TIME : 0.41 s @@ -31,8 +33,10 @@ Resulting in:: JOB ID : f3139826f1b169a0b456e0e880ffb83ed26d9858 SRC JOB ID : latest JOB LOG : $HOME/avocado/job-results/job-2020-08-13T11.24-f313982/job.log - (1/2) /bin/true: PASS (0.01 s) - (2/2) /bin/false: FAIL: Exited with status: '1', stdout: '' stderr: '' (0.07 s) + (1-2/2) /bin/false: STARTED + (1-1/2) /bin/true: STARTED + (1-2/2) /bin/false: FAIL (0.01 s) + (1-1/2) /bin/true: PASS (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : $HOME/avocado/job-results/job-2020-08-13T11.24-f313982/results.html JOB TIME : 0.39 s @@ -159,7 +163,8 @@ using the ``--dry-run`` argument:: $ avocado run /bin/true --dry-run JOB ID : 0000000000000000000000000000000000000000 JOB LOG : /var/tmp/avocado-dry-run-k2i_uiqx/job-2020-09-02T09.09-0000000/job.log - (1/1) /bin/true: CANCEL: Test cancelled due to --dry-run (0.01 s) + (1/1) /bin/true: STARTED + (1/1) /bin/true: CANCEL: Test cancelled due to --dry-run (0.00 s) RESULTS : PASS 0 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 1 JOB HTML : /var/tmp/avocado-dry-run-k2i_uiqx/job-2020-09-02T09.09-0000000/results.html JOB TIME : 0.29 s @@ -205,4 +210,4 @@ tests:: Notice that the verbose flag also adds summary information. .. seealso:: To read more about test discovery, visit the section - "Understanding the test discovery (Avocado Loaders)". + :ref:`finding_tests`. diff --git a/docs/source/guides/user/chapters/plugins.rst b/docs/source/guides/user/chapters/plugins.rst index b95344963f..81d083337e 100644 --- a/docs/source/guides/user/chapters/plugins.rst +++ b/docs/source/guides/user/chapters/plugins.rst @@ -92,7 +92,7 @@ developers to make execution more effective. To list the plugins in execution order, you can use ``avocado plugins --ordered``. .. note:: For more information about how the execution order is set, please - visit visit the Plugin section on Contributor's Guide. + visit the :ref:`writing_plugin` section on Contributor's Guide. Changing the plugin execution order ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -109,7 +109,7 @@ That configuration sets the ``job.prepost.myplugin`` plugin to execute before the standard Avocado ``job.prepost.jobscripts`` does. .. note:: If you are interested on how plugins works and how to create your own - plugin, visit the Plugin section on Contributor's Guide. + plugin, visit the :ref:`writing_plugin` section on Contributor's Guide. Pre and post plugins -------------------- diff --git a/docs/source/guides/user/chapters/results.rst b/docs/source/guides/user/chapters/results.rst index e1c895b36d..842da012c6 100644 --- a/docs/source/guides/user/chapters/results.rst +++ b/docs/source/guides/user/chapters/results.rst @@ -8,7 +8,7 @@ name includes a timestamp, such as ``job-2021-09-28T14.21-e0775d9``. A typical results directory structure can be seen below :: $HOME/avocado/job-results/job-2021-09-28T14.21-e0775d9/ - ├── avocado.core.DEBUG + ├── full.log ├── id ├── jobdata │   ├── args.json @@ -16,7 +16,7 @@ results directory structure can be seen below :: │   ├── config │   ├── pwd │   ├── test_references - │   └── variants-1.json + │   └── variants-1-1.json ├── job.log ├── results.html ├── results.json @@ -84,27 +84,33 @@ results directory structure can be seen below :: │   │   └── version │   └── profile └── test-results - ├── 1-examples_tests_sleeptest.py_SleepTest.test + ├── 1-1-_bin_true │   ├── debug.log - │   └── whiteboard - ├── 2-examples_tests_sleeptest.py_SleepTest.test + │   ├── stderr + │   └── stdout + ├── 1-2-_bin_false │   ├── debug.log - │   └── whiteboard - └── 3-examples_tests_sleeptest.py_SleepTest.test - ├── debug.log - └── whiteboard + │   ├── stderr + │   └── stdout + └── by-status + ├── FAIL + │   └── 1-2-_bin_false -> ../../1-2-_bin_false + └── PASS + └── 1-1-_bin_true -> ../../1-1-_bin_true From what you can see, the results directory has: 1) A human readable ``id`` in the top level, with the job SHA1. 2) A human readable ``job.log`` in the top level, with human readable logs of the task -3) Subdirectory ``jobdata``, that contains machine readable data about the job. -4) A machine readable ``results.xml`` and ``results.json`` in the top level, +3) A human readable ``full.log`` in the top level, with human readable logs of + anything genereated inside the job. +4) Subdirectory ``jobdata``, that contains machine readable data about the job. +5) A machine readable ``results.xml`` and ``results.json`` in the top level, with a summary of the job information in xUnit/json format. -5) A top level ``sysinfo`` dir, with sub directories ``pre``, ``post`` and +6) A top level ``sysinfo`` dir, with sub directories ``pre``, ``post`` and ``profile``, that store sysinfo files pre/post/during job, respectively. -6) Subdirectory ``test-results``, that contains a number of subdirectories +7) Subdirectory ``test-results``, that contains a number of subdirectories (filesystem-friendly test ids). Those test ids represent instances of test execution results. @@ -113,7 +119,7 @@ Test execution instances specification The instances should have: -1) A top level human readable ``job.log``, with job debug information +1) A top level human readable ``job.log`` and ``full.log``, with job debug information 2) A ``sysinfo`` subdirectory, with sub directories ``pre``, ``post`` and ``profile`` that store sysinfo files pre test, post test and profiling info while the test was running, respectively. diff --git a/docs/source/guides/user/chapters/tags.rst b/docs/source/guides/user/chapters/tags.rst index 9d7578791b..3c6611e510 100644 --- a/docs/source/guides/user/chapters/tags.rst +++ b/docs/source/guides/user/chapters/tags.rst @@ -158,5 +158,5 @@ could use:: .. seealso:: If you would like to understand how write plugins and how describe - tags inside a plugin, please visit the section: `Writing Tests` on Avocado Test + tags inside a plugin, please visit the :ref:`writing_tests` on Avocado Test Writer's Guide. diff --git a/docs/source/guides/writer/chapters/writing.rst b/docs/source/guides/writer/chapters/writing.rst index 40a731a226..e43a6cce26 100644 --- a/docs/source/guides/writer/chapters/writing.rst +++ b/docs/source/guides/writer/chapters/writing.rst @@ -1,3 +1,5 @@ +.. _writing_tests: + Writing Avocado Tests with Python =================================