Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TRG Suggestion - Application Testing - Test Automation with Cucumber #1148

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/release/trg-10/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"label": "TRG 10 - Application Testing"
}
128 changes: 128 additions & 0 deletions docs/release/trg-10/trg-10-02.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
---
title: TRG 10.02 - Test Automation with Cucumber
sidebar_position: 2
---

| Status | Created | Post-History |
|--------|-------------|-------------------------|
| Draft | 07-Feb-2025 | Initial version created |

## Why

Test automation is essential for maintaining high-quality software in complex environments. It improves reliability by
detecting issues early, ensures consistency across projects, and reduces the effort of
repetitive manual testing. Automated testing provides faster feedback, enabling developers to address problems promptly.

This guideline focuses on API tests at the integration test level (end-to-end). The described frameworks can also be used to cover tests of graphical user interfaces at the
integration test level. The test automation at the unit test level should be implemented with other methods such as JUnit.

## Description

### 1. Test Case Creation

#### 1.1. **Gherkin Syntax**

- Test cases should be written using **Gherkin syntax** in `.feature` files.
- Features should be expressed in a **domain-specific language (DSL)** for clarity and consistency.

#### 1.2. **Feature File Structure**

- Features should include:
- A clear and concise title.
- Background sections for shared setup steps.
- Scenarios written in the Given-When-Then format.

### 2. Test Implementation

#### 2.1. **Cucumber with Maven**

- Use **Cucumber** in combination with **Maven** to generate test classes and implement them in a structured manner.
- Ensure modular and reusable test step implementations.

#### 2.2. **API Test Focus**

- Prioritize API tests at the integration test level to validate end-to-end functionality.
- Optionally include UI tests at the integration level if applicable for your software component.

### 3. Automation and Integration

#### 3.1. **GitHub Actions**

- Integrate test execution into **GitHub Actions** workflows for automated testing.
- Trigger tests on key events such as pull requests and merges to main branches.

#### 3.2. **Test Reporting**

- Generate and store detailed test reports as part of the CI/CD pipeline outputs.
- Provide clear pass/fail results for stakeholders.

### 4. Examples and Resources

#### 4.1. **Code Examples**

An example of such a GitHub workflow integration can be found below.

```yaml
name: Run Cucumber Tests

on:
push:
branches:
- main
pull_request:

jobs:
run-tests:
runs-on: ubuntu-latest

steps:
# Step 1: Checkout code
- name: Checkout code
uses: actions/checkout@v3

# Step 2: Set up Java
- name: Set up Java
uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: '17'

# Step 3: Cache Maven dependencies
- name: Cache Maven dependencies
uses: actions/cache@v3
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-
# Step 4: Run Maven tests
- name: Run Cucumber tests
run: mvn clean test

# Step 5: Upload test results
- name: Archive test results
if: always()
uses: actions/upload-artifact@v3
with:
name: cucumber-test-results
path: target/surefire-reports/

```

#### 4.2. **Optional Scope**

- The Cucumber framework allows for importing test run results from the CI/CD pipeline into various Test Management systems. Reference implementations for this are available and in use in some Eclipse Tractus-X projects. However, this is not considered part of the
current TRG version due Eclipse Tractus-X requirements on open source software.

---

## Documentation

All test automation configurations should be stored in the project repository. Test results should be uploaded
to the execution workflow as an attachment. See [code-examples](#41-code-examples).

---

## Compliance

This TRG serves as a recommendation. While product teams may adopt other frameworks or tools to implement their test automation, the use of Cucumber with test cases written in Gherkin syntax as a basis is encouraged as a suggested best practice at the integration test level.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This TRG serves as a recommendation. While product teams may adopt other frameworks or tools to implement their test automation, the use of Cucumber with test cases written in Gherkin syntax as a basis is encouraged as a suggested best practice at the integration test level.
the use of Cucumber with test cases written in Gherkin syntax as a basis is encouraged as a suggested best practice at the integration test level.

this should not be the case.
the way tests are expressed should be a choice made by the product team based on what's the desired outcome.
Certain products could benefit from the gherkin syntax because non-technical people could read it and understand how the product is working (please note that this is not always the case anyway :) ), but it would also be fair to point out that cucumber tests are harder to maintain because the additional ".feature to code" layer needed, and not all the teams want to deal with this burden.