-
Notifications
You must be signed in to change notification settings - Fork 92
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TRG Suggestion - Application Testing - Test Automation with Cucumber #1148
Open
ds-hzimmer
wants to merge
2
commits into
eclipse-tractusx:main
Choose a base branch
from
ds-hzimmer:application-testing-test-automation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+131
−0
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
{ | ||
"label": "TRG 10 - Application Testing" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,128 @@ | ||
--- | ||
title: TRG 10.02 - Test Automation with Cucumber | ||
sidebar_position: 2 | ||
--- | ||
|
||
| Status | Created | Post-History | | ||
|--------|-------------|-------------------------| | ||
| Draft | 07-Feb-2025 | Initial version created | | ||
|
||
## Why | ||
|
||
Test automation is essential for maintaining high-quality software in complex environments. It improves reliability by | ||
detecting issues early, ensures consistency across projects, and reduces the effort of | ||
repetitive manual testing. Automated testing provides faster feedback, enabling developers to address problems promptly. | ||
|
||
This guideline focuses on API tests at the integration test level (end-to-end). The described frameworks can also be used to cover tests of graphical user interfaces at the | ||
integration test level. The test automation at the unit test level should be implemented with other methods such as JUnit. | ||
|
||
## Description | ||
|
||
### 1. Test Case Creation | ||
|
||
#### 1.1. **Gherkin Syntax** | ||
|
||
- Test cases should be written using **Gherkin syntax** in `.feature` files. | ||
- Features should be expressed in a **domain-specific language (DSL)** for clarity and consistency. | ||
|
||
#### 1.2. **Feature File Structure** | ||
|
||
- Features should include: | ||
- A clear and concise title. | ||
- Background sections for shared setup steps. | ||
- Scenarios written in the Given-When-Then format. | ||
|
||
### 2. Test Implementation | ||
|
||
#### 2.1. **Cucumber with Maven** | ||
|
||
- Use **Cucumber** in combination with **Maven** to generate test classes and implement them in a structured manner. | ||
- Ensure modular and reusable test step implementations. | ||
|
||
#### 2.2. **API Test Focus** | ||
|
||
- Prioritize API tests at the integration test level to validate end-to-end functionality. | ||
- Optionally include UI tests at the integration level if applicable for your software component. | ||
|
||
### 3. Automation and Integration | ||
|
||
#### 3.1. **GitHub Actions** | ||
|
||
- Integrate test execution into **GitHub Actions** workflows for automated testing. | ||
- Trigger tests on key events such as pull requests and merges to main branches. | ||
|
||
#### 3.2. **Test Reporting** | ||
|
||
- Generate and store detailed test reports as part of the CI/CD pipeline outputs. | ||
- Provide clear pass/fail results for stakeholders. | ||
|
||
### 4. Examples and Resources | ||
|
||
#### 4.1. **Code Examples** | ||
|
||
An example of such a GitHub workflow integration can be found below. | ||
|
||
```yaml | ||
name: Run Cucumber Tests | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
pull_request: | ||
|
||
jobs: | ||
run-tests: | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
# Step 1: Checkout code | ||
- name: Checkout code | ||
uses: actions/checkout@v3 | ||
|
||
# Step 2: Set up Java | ||
- name: Set up Java | ||
uses: actions/setup-java@v3 | ||
with: | ||
distribution: 'temurin' | ||
java-version: '17' | ||
|
||
# Step 3: Cache Maven dependencies | ||
- name: Cache Maven dependencies | ||
uses: actions/cache@v3 | ||
with: | ||
path: ~/.m2/repository | ||
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} | ||
restore-keys: | | ||
${{ runner.os }}-maven- | ||
# Step 4: Run Maven tests | ||
- name: Run Cucumber tests | ||
run: mvn clean test | ||
|
||
# Step 5: Upload test results | ||
- name: Archive test results | ||
if: always() | ||
uses: actions/upload-artifact@v3 | ||
with: | ||
name: cucumber-test-results | ||
path: target/surefire-reports/ | ||
|
||
``` | ||
|
||
#### 4.2. **Optional Scope** | ||
|
||
- The Cucumber framework allows for importing test run results from the CI/CD pipeline into various Test Management systems. Reference implementations for this are available and in use in some Eclipse Tractus-X projects. However, this is not considered part of the | ||
current TRG version due Eclipse Tractus-X requirements on open source software. | ||
|
||
--- | ||
|
||
## Documentation | ||
|
||
All test automation configurations should be stored in the project repository. Test results should be uploaded | ||
to the execution workflow as an attachment. See [code-examples](#41-code-examples). | ||
|
||
--- | ||
|
||
## Compliance | ||
|
||
This TRG serves as a recommendation. While product teams may adopt other frameworks or tools to implement their test automation, the use of Cucumber with test cases written in Gherkin syntax as a basis is encouraged as a suggested best practice at the integration test level. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this should not be the case.
the way tests are expressed should be a choice made by the product team based on what's the desired outcome.
Certain products could benefit from the gherkin syntax because non-technical people could read it and understand how the product is working (please note that this is not always the case anyway :) ), but it would also be fair to point out that cucumber tests are harder to maintain because the additional ".feature to code" layer needed, and not all the teams want to deal with this burden.