Skip to content

Commit

Permalink
Doc fixes before LTS - Avocado User’s Guide
Browse files Browse the repository at this point in the history
Updates to documentation in `Avocado User’s Guide` section to respects
the latest changes.

Reference: #5757
Signed-off-by: Jan Richter <[email protected]>
  • Loading branch information
richtja committed Sep 13, 2023
1 parent 6c339ca commit ec7952c
Show file tree
Hide file tree
Showing 13 changed files with 82 additions and 78 deletions.
2 changes: 2 additions & 0 deletions docs/source/guides/contributor/chapters/plugins.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _writing_plugin:

*************************
Writing an Avocado plugin
*************************
Expand Down
2 changes: 2 additions & 0 deletions docs/source/guides/contributor/chapters/runners.rst
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,8 @@ doing::
To be able to complete such a command, Avocado needs to find the tests, and
then to execute them. Those two major steps are described next.

.. _finding_tests:

Finding tests
~~~~~~~~~~~~~

Expand Down
39 changes: 17 additions & 22 deletions docs/source/guides/user/chapters/advanced.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,15 @@ In some cases, you might have a wrapper as an entry point for the tests, so
Avocado will use only the wrapper as test id. For instance, imagine a Makefile
with some targets ('foo', 'bar') and each target is one test. Having a single
test suite with a test calling `foo`, it will make Avocado print something like
this:
this::


```
JOB ID : b6e5bdf2c891382bbde7f24e906a168af351154a
JOB LOG : ~/avocado/job-results/job-2021-09-24T17.39-b6e5bdf/job.log
(1/1) make: STARTED
(1/1) make: PASS (2.72 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML : ~/avocado/job-results/job-2021-09-24T17.39-b6e5bdf/results.html
JOB TIME : 5.49 s
```
JOB ID : b6e5bdf2c891382bbde7f24e906a168af351154a
JOB LOG : ~/avocado/job-results/job-2021-09-24T17.39-b6e5bdf/job.log
(1/1) make: STARTED
(1/1) make: PASS (2.72 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML : ~/avocado/job-results/job-2021-09-24T17.39-b6e5bdf/results.html
JOB TIME : 5.49 s

This is happening because Avocado is using the 'uri' as identifier with in the
current Runnables.
Expand All @@ -33,17 +30,15 @@ identifier_format = "{uri}-{args[0]}"
```

With the above adjustment, running the same suite it will produce something
like this:

```
JOB ID : 577b70b079e9a6f325ff3e73fd9b93f80ee7f221
JOB LOG : /home/local/avocado/job-results/job-2021-11-23T13.12-577b70b/job.log
(1/1) "/usr/bin/make-foo": STARTED
(1/1) "/usr/bin/make-foo": PASS (0.01 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML : ~/avocado/job-results/job-2021-11-23T13.12-577b70b/results.html
JOB TIME : 0.97 s
```
like this::

JOB ID : 577b70b079e9a6f325ff3e73fd9b93f80ee7f221
JOB LOG : /home/local/avocado/job-results/job-2021-11-23T13.12-577b70b/job.log
(1/1) "/usr/bin/make-foo": STARTED
(1/1) "/usr/bin/make-foo": PASS (0.01 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML : ~/avocado/job-results/job-2021-11-23T13.12-577b70b/results.html
JOB TIME : 0.97 s

For the `identifier_format` you can use any f-string that it will use `{uri}`,
`{args}` or `{kwargs}`. By default it will use `{uri}`.
Expand Down
6 changes: 1 addition & 5 deletions docs/source/guides/user/chapters/configuring.rst
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ So the file parsing order is:
* ``avocado.plugins.settings`` plugins (but they can insert to any location)

- For more information about this, visit the "Contributor's Guide"
section named "Writing an Avocado plugin"
section named :ref:`writing_plugin`

* ``~/.config/avocado/avocado.conf``

Expand Down Expand Up @@ -229,10 +229,6 @@ Example using the command-line:
$ avocado run --verbose /bin/true
.. note:: Currently we still have some "old style boolean" options where you
should pass "on" or "off" on the command-line. i.e: ``--json-job-result=off``.
Those options are going to be replaced soon.

Lists
~~~~~

Expand Down
3 changes: 0 additions & 3 deletions docs/source/guides/user/chapters/datadirs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,3 @@ place temporary files used by tests. That directory will be in normal circumstan
``/var/tmp/avocado_XXXXX``, (where ``XXXXX`` is in actuality a random string) securely
created on ``/var/tmp/``, unless the user has the ``$TMPDIR`` environment variable set,
since that is customary among unix programs.

The next section of the documentation explains how you can see and set config
values that modify the behavior for the Avocado utilities and plugins.
2 changes: 0 additions & 2 deletions docs/source/guides/user/chapters/dependencies.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@
Managing Dependencies
=====================

.. note:: Test dependencies are supported only on the nrunner runner.

A test's dependency can be fulfilled by the Dependencies Resolver feature.

Test's dependencies are specified in the test definition and are fulfilled
Expand Down
6 changes: 3 additions & 3 deletions docs/source/guides/user/chapters/installing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Installing from PyPI
--------------------

The simplest installation method is through ``pip``. On most POSIX systems
with Python 3.4 (or later) and ``pip`` available, installation can be performed
with Python 3.7 (or later) and ``pip`` available, installation can be performed
with a single command::

$ pip3 install --user avocado-framework
Expand Down Expand Up @@ -56,7 +56,7 @@ version 29. To subscribe to the latest version stream, run::

Or, to use the LTS (Long Term Stability) version stream, run::

$ dnf module enable avocado:82lts
$ dnf module enable avocado:103lts

Then proceed to install a module profile or individual packages. If you're
unsure about what to do, simply run::
Expand Down Expand Up @@ -127,7 +127,7 @@ Then to install Avocado from the git repository run::
$ cd avocado
$ python3 setup.py install --user

Optionally, to install the plugins run:
Optionally, to install the plugins run::

$ python3 setup.py plugin --install=golang --user
$ python3 setup.py plugin --install=html --user
Expand Down
43 changes: 22 additions & 21 deletions docs/source/guides/user/chapters/introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,13 @@ test reference, which could be either a path to the file, or a recognizable
name::

$ avocado run /bin/true
JOB ID : 3a5c4c51ceb5369f23702efb10b4209b111141b2
JOB LOG : $HOME/avocado/job-results/job-2019-10-31T10.34-3a5c4c5/job.log
(1/1) /bin/true: PASS (0.04 s)
JOB ID : 89b5d609d4832c784f04cf14f4ec2d17917d419a
JOB LOG : $HOME/avocado/job-results/job-2023-09-06T15.52-89b5d60/job.log
(1/1) /bin/true: STARTED
(1/1) /bin/true: PASS (0.01 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 0.15 s
JOB HTML : $HOME/avocado/job-results/job-2023-09-06T15.52-89b5d60/results.html
JOB TIME : 1.49 s

You probably noticed that we used ``/bin/true`` as a test, and in
accordance with our expectations, it passed! These are known as
Expand Down Expand Up @@ -57,19 +59,19 @@ instrumented and executable tests::
Using a different runner
------------------------

Currently Avocado has two test runners: ``nrunner`` (the new runner) and
``runner`` (legacy). You can find a list of current runners installed with the
Currently Avocado has only one runner: ``nrunner`` (the new runner)
But some plugins may use their own runners.
You can find a list of current runners installed with the
``avocado plugins`` command::

$ avocado plugins
Plugins that run test suites on a job (runners):
nrunner nrunner based implementation of job compliant runner
runner The conventional test runner

During the test execution, you can select the runner using the option
``--test-runner``, where the default is the nrunner one::
``--suite-runner``, where the default is the nrunner one::

$ avocado run --test-runner='runner' /bin/true
$ avocado run --suite-runner='runner' /bin/true


Interrupting tests
Expand Down Expand Up @@ -170,7 +172,8 @@ them to tests. If one or more test references can not be resolved to tests, the
Job will not be created. Example::

$ avocado run examples/tests/passtest.py badtest.py
Unable to resolve reference(s) 'badtest.py' with plugins(s) 'file', 'robot', try running 'avocado -V list badtest.py' to see the details.
No tests found for given test references: badtest.py
Try 'avocado -V list badtest.py' for details

But if you want to execute the Job anyway, with the tests that could be
resolved, you can use ``--ignore-missing-references``, a boolean command-line
Expand Down Expand Up @@ -366,7 +369,7 @@ Avocado features. A reasonable effort will be made to not break backwards
compatibility with applications that parse the current form of its JSON result.


**3. TAP:**
**3. TAP:**

Provides the basic `TAP <https://testanything.org/>`__ (Test Anything Protocol)
results, currently in v12. Unlike most existing Avocado machine readable
Expand All @@ -377,7 +380,7 @@ outputs this one is streamlined (per test results)::
ok 1 examples/tests/sleeptest.py:SleepTest.test


**4. Beaker:**
**4. Beaker:**

When avocaodo finds itself running inside a beaker task the
beaker_report plugin will send the test results and log files to the
Expand All @@ -396,16 +399,14 @@ In order to do that, you can use ``avocado --show=job run ...``::

$ avocado --show=job run examples/tests/sleeptest.py
...
Job ID: f9ea1742134e5352dec82335af584d1f151d4b85

START 1-sleeptest.py:SleepTest.test

PARAMS (key=timeout, path=*, default=None) => None
PARAMS (key=sleep_length, path=*, default=1) => 1
Sleeping for 1.00 seconds
PASS 1-sleeptest.py:SleepTest.test
avocado.job: Job ID: c9ca6a96d34fd0355f5f121af7fa87eef734a196
avocado.job:
avocado.job: examples/tests/sleeptest.py:SleepTest.test: STARTED
avocado.job: examples/tests/sleeptest.py:SleepTest.test: PASS
avocado.job: More information in $HOME/avocado/job-results/job-2023-09-08T15.27-c9ca6a9/test-results/1-examples_tests_sleeptest.py_SleepTest.test
avocado.job: Test results available in $HOME/avocado/job-results/job-2023-09-08T15.27-c9ca6a9

Test results available in $HOME/avocado/job-results/job-2015-06-02T10.45-f9ea174

As you can see, the UI output is suppressed and only the job log is shown,
making this a useful feature for test development and debugging.
Expand Down
15 changes: 10 additions & 5 deletions docs/source/guides/user/chapters/operations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,10 @@ from the command line, such as::
$ avocado run /bin/true /bin/false
JOB ID : 42c60bea72e6d55756bfc784eb2b354f788541cf
JOB LOG : $HOME/avocado/job-results/job-2020-08-13T11.23-42c60be/job.log
(1/2) /bin/true: STARTED
(2/2) /bin/false: STARTED
(1/2) /bin/true: PASS (0.01 s)
(2/2) /bin/false: FAIL: Exited with status: '1', stdout: '' stderr: '' (0.08 s)
(2/2) /bin/false: FAIL (0.01 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML : $HOME/avocado/job-results/job-2020-08-13T11.23-42c60be/results.html
JOB TIME : 0.41 s
Expand All @@ -31,8 +33,10 @@ Resulting in::
JOB ID : f3139826f1b169a0b456e0e880ffb83ed26d9858
SRC JOB ID : latest
JOB LOG : $HOME/avocado/job-results/job-2020-08-13T11.24-f313982/job.log
(1/2) /bin/true: PASS (0.01 s)
(2/2) /bin/false: FAIL: Exited with status: '1', stdout: '' stderr: '' (0.07 s)
(1-2/2) /bin/false: STARTED
(1-1/2) /bin/true: STARTED
(1-2/2) /bin/false: FAIL (0.01 s)
(1-1/2) /bin/true: PASS (0.01 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML : $HOME/avocado/job-results/job-2020-08-13T11.24-f313982/results.html
JOB TIME : 0.39 s
Expand Down Expand Up @@ -159,7 +163,8 @@ using the ``--dry-run`` argument::
$ avocado run /bin/true --dry-run
JOB ID : 0000000000000000000000000000000000000000
JOB LOG : /var/tmp/avocado-dry-run-k2i_uiqx/job-2020-09-02T09.09-0000000/job.log
(1/1) /bin/true: CANCEL: Test cancelled due to --dry-run (0.01 s)
(1/1) /bin/true: STARTED
(1/1) /bin/true: CANCEL: Test cancelled due to --dry-run (0.00 s)
RESULTS : PASS 0 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 1
JOB HTML : /var/tmp/avocado-dry-run-k2i_uiqx/job-2020-09-02T09.09-0000000/results.html
JOB TIME : 0.29 s
Expand Down Expand Up @@ -205,4 +210,4 @@ tests::
Notice that the verbose flag also adds summary information.

.. seealso:: To read more about test discovery, visit the section
"Understanding the test discovery (Avocado Loaders)".
:ref:`finding_tests`.
4 changes: 2 additions & 2 deletions docs/source/guides/user/chapters/plugins.rst
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ developers to make execution more effective. To list the plugins in execution
order, you can use ``avocado plugins --ordered``.

.. note:: For more information about how the execution order is set, please
visit visit the Plugin section on Contributor's Guide.
visit the :ref:`writing_plugin` section on Contributor's Guide.

Changing the plugin execution order
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand All @@ -109,7 +109,7 @@ That configuration sets the ``job.prepost.myplugin`` plugin to execute before
the standard Avocado ``job.prepost.jobscripts`` does.

.. note:: If you are interested on how plugins works and how to create your own
plugin, visit the Plugin section on Contributor's Guide.
plugin, visit the :ref:`writing_plugin` section on Contributor's Guide.

Pre and post plugins
--------------------
Expand Down
34 changes: 20 additions & 14 deletions docs/source/guides/user/chapters/results.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,15 @@ name includes a timestamp, such as ``job-2021-09-28T14.21-e0775d9``. A typical
results directory structure can be seen below ::

$HOME/avocado/job-results/job-2021-09-28T14.21-e0775d9/
├── avocado.core.DEBUG
├── full.log
├── id
├── jobdata
│   ├── args.json
│   ├── cmdline
│   ├── config
│   ├── pwd
│   ├── test_references
│   └── variants-1.json
│   └── variants-1-1.json
├── job.log
├── results.html
├── results.json
Expand Down Expand Up @@ -84,27 +84,33 @@ results directory structure can be seen below ::
│   │   └── version
│   └── profile
└── test-results
├── 1-examples_tests_sleeptest.py_SleepTest.test
├── 1-1-_bin_true
│   ├── debug.log
│   └── whiteboard
├── 2-examples_tests_sleeptest.py_SleepTest.test
│   ├── stderr
│   └── stdout
├── 1-2-_bin_false
│   ├── debug.log
│   └── whiteboard
└── 3-examples_tests_sleeptest.py_SleepTest.test
├── debug.log
└── whiteboard
│   ├── stderr
│   └── stdout
└── by-status
├── FAIL
│   └── 1-2-_bin_false -> ../../1-2-_bin_false
└── PASS
└── 1-1-_bin_true -> ../../1-1-_bin_true

From what you can see, the results directory has:

1) A human readable ``id`` in the top level, with the job SHA1.
2) A human readable ``job.log`` in the top level, with human readable logs of
the task
3) Subdirectory ``jobdata``, that contains machine readable data about the job.
4) A machine readable ``results.xml`` and ``results.json`` in the top level,
3) A human readable ``full.log`` in the top level, with human readable logs of
anything genereated inside the job.
4) Subdirectory ``jobdata``, that contains machine readable data about the job.
5) A machine readable ``results.xml`` and ``results.json`` in the top level,
with a summary of the job information in xUnit/json format.
5) A top level ``sysinfo`` dir, with sub directories ``pre``, ``post`` and
6) A top level ``sysinfo`` dir, with sub directories ``pre``, ``post`` and
``profile``, that store sysinfo files pre/post/during job, respectively.
6) Subdirectory ``test-results``, that contains a number of subdirectories
7) Subdirectory ``test-results``, that contains a number of subdirectories
(filesystem-friendly test ids). Those test ids represent instances of test
execution results.

Expand All @@ -113,7 +119,7 @@ Test execution instances specification

The instances should have:

1) A top level human readable ``job.log``, with job debug information
1) A top level human readable ``job.log`` and ``full.log``, with job debug information
2) A ``sysinfo`` subdirectory, with sub directories ``pre``, ``post`` and
``profile`` that store sysinfo files pre test, post test and
profiling info while the test was running, respectively.
Expand Down
2 changes: 1 addition & 1 deletion docs/source/guides/user/chapters/tags.rst
Original file line number Diff line number Diff line change
Expand Up @@ -158,5 +158,5 @@ could use::


.. seealso:: If you would like to understand how write plugins and how describe
tags inside a plugin, please visit the section: `Writing Tests` on Avocado Test
tags inside a plugin, please visit the :ref:`writing_tests` on Avocado Test
Writer's Guide.
2 changes: 2 additions & 0 deletions docs/source/guides/writer/chapters/writing.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _writing_tests:

Writing Avocado Tests with Python
=================================

Expand Down

0 comments on commit ec7952c

Please sign in to comment.