Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New extended tutorial for test jobs (infra) #1405

Merged
merged 12 commits into from
Sep 11, 2024
Merged

Conversation

Hook25
Copy link
Collaborator

@Hook25 Hook25 commented Aug 8, 2024

Description

We are currently lacking a tutorial to get started with test authoring.

This is an attempt to give a comprehensive introduction to everything one may want to know about automated tests via a tutorial that guides the user through creating a new provider with:

  • automated tests
  • templates
  • resources
  • inter-test dependencies
  • binaries to build via mange build

This tutorial is only about tests, so the absence of test plans is intentional. This is meant to be the first in a chain of "advanced/extended" tutorials (this is why it also contains the instructions to create the provider and provision Checkbox from source)

Resolved issues

Fixes: https://warthogs.atlassian.net/browse/CHECKBOX-761

Documentation

N/A

Tests

N/A

@Hook25 Hook25 force-pushed the new_ext_tutorial_test_case branch from 87b024f to 0adcbfc Compare August 8, 2024 15:05
Copy link
Collaborator

@pieqq pieqq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool stuff indeed!

I only reviewed until the Template Jobs section.

Writing tutorials is very hard. It's tempting to try to explain many things at once, but this might become too hard to read for a newcomer. If you feel that you have to explain a concept while explaining another one, this might be better to split this into another section, later.

docs/.sphinx/wordslist.txt Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
@Hook25 Hook25 marked this pull request as ready for review August 20, 2024 09:49
@Hook25 Hook25 force-pushed the new_ext_tutorial_test_case branch from 5d1d68b to 5f05d54 Compare August 20, 2024 09:54
@Hook25 Hook25 assigned Hook25 and unassigned Hook25 Aug 21, 2024
Copy link
Collaborator

@pieqq pieqq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Brilliant work, really! This is a long tutorial, and you introduce a lot of concepts that are unique to Checkbox in a easy-to-follow way. Congratulations!

You even include obscure stuff like the ability to compile source code... impressive!

I've made a bunch of suggestions and typo fixes as well as other general comments. Once these land, we can merge this PR.

Thanks again, great work!

docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
tang-mm
tang-mm previously requested changes Sep 6, 2024
Copy link
Collaborator

@tang-mm tang-mm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kudos for devising such a thorough tutorial!!
It's easy to follow, clearly explains what/why we are doing so in each step, and all the tips/suggestions are very useful. I left a bunch of comments but most of them are nitpicks. Feel free to adapt as needed

And a thought for future consideration (for me):
This tutorial is a good candidate for documentation testing - all the code samples are executable and already in the repository. Instead of duplicating the code in the doc, maybe we can include the snippet from source files into the document, so the tutorial can be validated as well.

docs/tutorial/writing-tests/test-case.rst Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/writing-tests/test-case.rst Outdated Show resolved Hide resolved
docs/tutorial/index.rst Show resolved Hide resolved
Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Update docs/tutorial/writing-tests/test-case.rst

Co-authored-by: Pierre Equoy <[email protected]>

Apply suggestions from code review

Co-authored-by: tang-mm <[email protected]>
Co-authored-by: Pierre Equoy <[email protected]>
@Hook25 Hook25 force-pushed the new_ext_tutorial_test_case branch from 53b32d0 to 26651a5 Compare September 6, 2024 14:35
Copy link
Collaborator

@pieqq pieqq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Go go go! 🎉

@Hook25 Hook25 dismissed tang-mm’s stale review September 11, 2024 08:07

Implemented the changes

@Hook25 Hook25 merged commit 5cb117c into main Sep 11, 2024
11 checks passed
@Hook25 Hook25 deleted the new_ext_tutorial_test_case branch September 11, 2024 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants