selenium system test into CI for html_report

.github/workflows/ci.yml

@@ -35,9 +35,21 @@ jobs:
       - name: Run TRLC
         run: |
           make trlc
+  selenium-tests:
+    name: Run Selenium Tests
+    runs-on: ubuntu-24.04
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install dependencies
+        run: |
+          python3 -m pip install --upgrade pip
+          python3 -m pip install selenium webdriver-manager
+      - name: Run selenium Tests
+        run: |
+          make selenium-tests
   test:
     name: TestSuite
-    needs: [lint, trlc]
+    needs: [lint, trlc, selenium-tests]
     strategy:
       matrix:
         os: [ubuntu-24.04, windows-2022, macos-13, macos-14]

Makefile

@@ -54,6 +54,10 @@ clang-tidy:
 	cmake -S llvm -B build -G Ninja -DLLVM_ENABLE_PROJECTS='clang;clang-tools-extra' -DCMAKE_BUILD_TYPE=Release && \
 	cmake --build build --target clang-tidy
 
+selenium-tests:
+	@echo "Running Selenium Tests..."
+	python3 ./tests-system/lobster-core/html_report/selenium-test/test_html_report.py
+
 integration-tests: packages
 	(cd tests-integration/projects/basic; make)
 	(cd tests-integration/projects/filter; make)

documentation/selenium_into_CI.md

@@ -0,0 +1,138 @@
+# **Document: Investigation on Integrating Selenium or XRAY into CI for GUI Tests**
+
+## **Purpose**
+As a software developer, the goal is to integrate GUI testing tools (Selenium or XRAY) into the CI pipeline for the `lobster-html-report` system. This would help automate system tests, ensure reliability, and simplify maintenance.
+
+---
+
+## **Objective**
+1. Research how Selenium and XRAY can be integrated into CI tools.
+2. Provide a solution for running GUI-based system tests in a Continuous Integration (CI) environment.
+3. Identify the pros, cons, and suitability of each tool.
+4. Recommend the most effective approach for the `lobster-html-report` system.
+
+---
+
+## **Tools Overview**
+### **1. Selenium**
+Selenium is a widely used open-source framework for automating web applications. It supports multiple browsers and languages (Python, Java, C#, etc.).
+- **Features:**
+  - Automates web GUI tests.
+  - Integrates well with CI tools (GitHub Actions)
+  - Supports headless browsers for CI environments.
+
+### **2. XRAY**
+XRAY is a test management tool that integrates with Jira to manage test cases, execution, and reporting.
+- **Features:**
+  - Integrates with CI/CD tools.
+  - Allows for test execution reporting (manual and automated tests).
+  - Works alongside automation frameworks (e.g., Selenium).
+
+---
+
+## **Integration Strategies**
+### **Option 1: Selenium Integration in CI**
+To integrate Selenium tests into the CI pipeline:
+1. **Set Up Selenium Environment:**
+   - Install required dependencies (Selenium libraries, drivers like ChromeDriver/GeckoDriver).
+   - Use a headless browser (Chrome/Firefox) for GUI tests in CI.
+
+2. **Add GUI Test Script:**
+   - Write Selenium test scripts for `lobster-html-report` (e.g., validate UI components, links, buttons).
+   - Save test scripts in the project repository (e.g., under a `tests/gui` folder).
+
+3. **Configure CI Pipeline:**
+   - **GitHub Actions Example**:
+     ```yaml
+     name: GUI Tests
+     on: [push]
+     jobs:
+       gui-tests:
+         runs-on: ubuntu-latest
+         steps:
+           - name: Checkout Code
+             uses: actions/checkout@v3
+           - name: Set Up Python
+             uses: actions/setup-python@v3
+             with:
+               python-version: '3.x'
+           - name: Install Dependencies
+             run: |
+               pip install selenium
+               wget https://chromedriver.storage.googleapis.com/114.0.5735.90/chromedriver_linux64.zip
+               unzip chromedriver_linux64.zip
+           - name: Run Selenium Tests
+             run: python tests/gui/test_lobster_html_report.py
+     ```
+
+4. **Test Execution:**
+   - Execute Selenium scripts automatically during pipeline runs.
+   - Capture test results in CI logs.
+
+---
+
+### **Option 2: XRAY Integration in CI**
+To integrate XRAY tests into CI for test management and reporting:
+1. **Set Up XRAY with CI/CD Tool:**
+   - Integrate XRAY with Jira and CI tools.
+   - Use REST APIs or XRAY CLI for uploading and executing tests.
+
+2. **Write Test Scripts (with Selenium):**
+   - Use Selenium to create GUI test scripts for `lobster-html-report`.
+   - Link these automated tests to XRAY test cases using XRAY's framework.
+
+3. **Upload and Execute Tests:**
+   - Example for uploading results:
+     ```yaml
+     - name: Run Selenium Tests
+       run: |
+         python tests/gui/test_lobster_html_report.py > results.xml
+         curl -H "Content-Type: application/json" \
+         -u $JIRA_USER:$JIRA_API_TOKEN \
+         -X POST "https://your-jira-instance/rest/raven/1.0/import/execution" \
+         --data @results.xml
+     ```
+
+4. **Test Reporting:**
+   - XRAY collects and manages test results.
+   - Reports can be viewed in Jira or exported for further analysis.
+
+---
+
+## **Comparison: Selenium vs XRAY**
+| Feature                   | Selenium                              | XRAY                                      |
+|---------------------------|---------------------------------------|-------------------------------------------|
+| Test Automation           | Full web UI automation               | Works with Selenium for automation        |
+| CI Integration            | Direct integration into pipelines    | Integration via APIs or CLI               |
+| Test Management           | Not included                         | Comprehensive test management             |
+| Reporting                 | Logs in CI/CD tools                  | Detailed reporting in Jira                |
+| Suitability for Project   | Highly suitable for GUI automation   | Suitable for tracking and reporting tests |
+
+---
+
+## **Recommendations**
+1. **Primary Approach**: Use **Selenium** for GUI test automation within the CI pipeline as it:
+   - Offers robust web automation capabilities.
+   - Easily integrates with GitHub Actions, GitLab CI/CD, or Jenkins.
+   - Supports headless browsers for faster CI execution.
+
+2. **Secondary Approach**: If test management and reporting are essential:
+   - Use XRAY alongside Selenium to manage, execute, and report tests in Jira.
+   - Integrate XRAY into the CI pipeline for test result uploads.
+
+---
+
+## **Conclusion**
+Integrating Selenium into the CI pipeline is the most effective solution for automating GUI system tests for `lobster-html-report`. If additional test management and reporting are required, XRAY can be integrated alongside Selenium. The proposed solution will simplify maintenance, improve reliability, and ensure automated testing becomes part of the development workflow.
+
+---
+
+## **Next Steps**
+1. Set up Selenium test scripts for `lobster-html-report`.
+2. Configure Selenium integration into the existing CI pipeline.
+3. If needed, explore XRAY integration for enhanced test management and reporting.
+
+---
+
+**End of Document**
+