Skip to content

Commit 4d0a414

Browse files
javiertuyajaviertuya
committed
Configure evaluation module in local environment
1 parent 7e28a73 commit 4d0a414

File tree

9 files changed

+156
-125
lines changed

9 files changed

+156
-125
lines changed

‎README.md

+23-16
Original file line numberDiff line numberDiff line change
@@ -2,37 +2,42 @@
22

33
This project contains the system and integration tests of *Test Data Generation for APIs* (TDG).
44
Includes the required test classes, result files and the SUT projects,
5-
along with an additional project (to convert into module) to evaluate the mutation
5+
along with an additional module to evaluate the mutation
66
score of the tests for one of the SUTs.
77

88
## Systems Under Test (SUTs)
99

10-
- Swagger Petstore (`swagger-petstore-main-fork`).
10+
- Swagger Petstore (`sut-petstore` folder).
1111
This is a detached fork of https://github.com/swagger-api/swagger-petstore at v1.0.17 (2022-09-30) with some additions
12-
- Market (`swagger-market-fork`).
12+
- Market (`sut-market` folder).
1313
This is a detached fork of https://github.com/aleksey-lukyanets/market (2022-02-07) with some additions
14-
- Gestao Hospital (`swagger-gestaoHospital-fork`).
14+
- Gestao Hospital (`sut-gestaoHospital` folder).
1515
This is a detached fork of https://github.com/ValchanOficial/GestaoHospital (2023-07-05) with some additions
1616

1717
## Structure of this project
1818

1919
- Module `st-tdg-test`:
2020
- Java tests (`src/test/java`) for each SUT.
2121
- Files for test result comparison (`src/test/resources`)
22-
- A folder for each of the SUTs that are evaluated in this replication package.
23-
- A folder `setup` with scripts to execute each SUT (backend and databases).
22+
- Clasess `*Petstore0*` contain some commented examples to illustrate
23+
some details of the TDG approach.
24+
- There is a common base class for all tests (`BaseAll`) and
25+
Each namespace has an additional base class
26+
with the particular configuration of each SUT.
27+
- A folder for each of the SUTs that are tested in this projects.
28+
- A folder `setup` with scripts to run each SUT (backend and databases).
2429
- Module `st-tdg-eval`: To evaluate the effectiveness of the test data my measuring
25-
the mutation score of the tests using PIT (pitest.org).
30+
the mutation score of the tests using PIT (https://pitest.org/).
2631

2732
## How to execute the System Tests
2833

29-
All experiments are enclosed in the Java tests.
34+
The tests are in the `st-tdg-test` module.
3035

3136
To run the tests of a SUT in your local development environment:
32-
- Ensure that the port that uses the SUT is not used by other SUT
37+
- Ensure that the port bindings of each SUT are not used by other application
3338
- Run the server from the `setup` folder and wait until it is up
34-
- Execute the tests in the namespace that corresponds with the project
35-
- To execute from maven (e.g. for the petstore): `mvn -pl st-tdg-test`
39+
- Execute the tests in the namespace that corresponds with the SUT
40+
- To execute this module from maven: `mvn -pl st-tdg-test test`
3641

3742
Port reservation and endpoints: To avoid conflicts when running in local,
3843
different host port are reserved for each SUT.
@@ -41,7 +46,7 @@ Below are the ports and the main endpoint urls to check that SUTs are working:
4146
[api example](http://localhost:8081/api/v3/pet/findByStatus?status=available) |
4247
[swagger-ui](http://localhost:8081/) |
4348
[api-docs](http://localhost:8081/api/v3/openapi.json)
44-
- Market: 8082 (web), 8083 (api), 8084 (db)
49+
- Market: 8082 (web), 8083 (api), 8087 (db)
4550
[api example](http://localhost:8083/products) |
4651
[swagger-ui](http://localhost:8083/swagger-ui/index.html) |
4752
[api-docs](http://localhost:8083/v2/api-docs)
@@ -51,14 +56,16 @@ Below are the ports and the main endpoint urls to check that SUTs are working:
5156

5257
The whole sequence of tests for all SUTs is run when executed in CI
5358

54-
## How to evaluate the mutation score
59+
## How to evaluate the Mutation Score
5560

56-
At `st-tdg-eval` run this maven command:
61+
The tests are located in the `st-tdg-eval` module.
62+
Run this maven command:
5763
```
58-
mvn test-compile org.pitest:pitest-maven:mutationCoverage
64+
mvn -pl st-tdg-eval test-compile org.pitest:pitest-maven:mutationCoverage
5965
```
6066

61-
Test results are in the `target` folder, and mutation report in `target/pit-reports`
67+
Test results of the module are in the `target` folder,
68+
and the mutation report in `target/pit-reports`
6269

6370
## Graphical TDM models
6471

‎pom.xml

+1-14
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525

2626
<modules>
2727
<module>st-tdg-test</module>
28+
<module>st-tdg-eval</module>
2829
</modules>
2930

3031
<dependencyManagement>
@@ -89,20 +90,6 @@
8990
<target>1.8</target>
9091
</configuration>
9192
</plugin>
92-
<plugin>
93-
<groupId>org.apache.maven.plugins</groupId>
94-
<artifactId>maven-surefire-plugin</artifactId>
95-
<version>${surefire.version}</version>
96-
<configuration>
97-
<testFailureIgnore>true</testFailureIgnore>
98-
<!-- Sets the VM argument line used when unit tests are run under JaCoCo -->
99-
<argLine>${surefireArgLine}</argLine>
100-
<redirectTestOutputToFile>true</redirectTestOutputToFile>
101-
<!-- evita fallo con jenkins slave linux y openjdk: https://stackoverflow.com/questions/23260057/the-forked-vm-terminated-without-saying-properly-goodbye-vm-crash-or-system-exi/53070605 -->
102-
<useSystemClassLoader>false</useSystemClassLoader>
103-
<skipTests>${skipTests}</skipTests>
104-
</configuration>
105-
</plugin>
10693
<plugin>
10794
<groupId>org.apache.maven.plugins</groupId>
10895
<artifactId>maven-antrun-plugin</artifactId>

‎st-tdg-eval/.classpath

+48
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<classpath>
3+
<classpathentry excluding="**/io/swagger/petstore/controller/**|**/io/swagger/petstore/exception/**|**/io/swagger/petstore/utils/**" kind="src" output="target/classes" path="src/main/java">
4+
<attributes>
5+
<attribute name="optional" value="true"/>
6+
<attribute name="maven.pomderived" value="true"/>
7+
</attributes>
8+
</classpathentry>
9+
<classpathentry kind="src" path="petstoreMainResources"/>
10+
<classpathentry excluding="io/swagger/petstore/controller/|io/swagger/petstore/exception/|io/swagger/petstore/utils/" kind="src" path="petstoreMainJava"/>
11+
<classpathentry kind="src" output="target/test-classes" path="src/test/java">
12+
<attributes>
13+
<attribute name="test" value="true"/>
14+
<attribute name="optional" value="true"/>
15+
<attribute name="maven.pomderived" value="true"/>
16+
</attributes>
17+
</classpathentry>
18+
<classpathentry excluding="**" kind="src" output="target/test-classes" path="src/test/resources">
19+
<attributes>
20+
<attribute name="test" value="true"/>
21+
<attribute name="maven.pomderived" value="true"/>
22+
<attribute name="optional" value="true"/>
23+
</attributes>
24+
</classpathentry>
25+
<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.8">
26+
<attributes>
27+
<attribute name="maven.pomderived" value="true"/>
28+
</attributes>
29+
</classpathentry>
30+
<classpathentry kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER">
31+
<attributes>
32+
<attribute name="maven.pomderived" value="true"/>
33+
</attributes>
34+
</classpathentry>
35+
<classpathentry excluding="**/io/swagger/petstore/controller/**|**/io/swagger/petstore/exception/**|**/io/swagger/petstore/utils/**" kind="src" output="target/classes" path="TUYA/IN2TEST/github-giis/tdrules-st-tdg/sut-petstore/src/main/java">
36+
<attributes>
37+
<attribute name="optional" value="true"/>
38+
<attribute name="maven.pomderived" value="true"/>
39+
</attributes>
40+
</classpathentry>
41+
<classpathentry excluding="**" kind="src" output="target/classes" path="src/main/resources">
42+
<attributes>
43+
<attribute name="maven.pomderived" value="true"/>
44+
<attribute name="optional" value="true"/>
45+
</attributes>
46+
</classpathentry>
47+
<classpathentry kind="output" path="target/classes"/>
48+
</classpath>

‎st-tdg-eval/.project

+35
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<projectDescription>
3+
<name>st-tdg-eval</name>
4+
<comment></comment>
5+
<projects>
6+
</projects>
7+
<buildSpec>
8+
<buildCommand>
9+
<name>org.eclipse.jdt.core.javabuilder</name>
10+
<arguments>
11+
</arguments>
12+
</buildCommand>
13+
<buildCommand>
14+
<name>org.eclipse.m2e.core.maven2Builder</name>
15+
<arguments>
16+
</arguments>
17+
</buildCommand>
18+
</buildSpec>
19+
<natures>
20+
<nature>org.eclipse.jdt.core.javanature</nature>
21+
<nature>org.eclipse.m2e.core.maven2Nature</nature>
22+
</natures>
23+
<linkedResources>
24+
<link>
25+
<name>petstoreMainJava</name>
26+
<type>2</type>
27+
<locationURI>PARENT-1-PROJECT_LOC/sut-petstore/src/main/java</locationURI>
28+
</link>
29+
<link>
30+
<name>petstoreMainResources</name>
31+
<type>2</type>
32+
<locationURI>PARENT-1-PROJECT_LOC/sut-petstore/src/main/resources</locationURI>
33+
</link>
34+
</linkedResources>
35+
</projectDescription>

‎st-tdg-eval/.settings/org.eclipse.core.resources.prefs

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
eclipse.preferences.version=1
2+
encoding//TUYA/IN2TEST/github-giis/tdrules-st-tdg/sut-petstore/src/main/java=UTF-8
3+
encoding//src/main/java=UTF-8
4+
encoding//src/test/java=UTF-8
5+
encoding//src/test/resources=UTF-8
6+
encoding/<project>=UTF-8

‎st-tdg-eval/.settings/org.eclipse.jdt.core.prefs

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
eclipse.preferences.version=1
2+
org.eclipse.jdt.core.compiler.codegen.targetPlatform=1.8
3+
org.eclipse.jdt.core.compiler.compliance=1.8
4+
org.eclipse.jdt.core.compiler.problem.enablePreviewFeatures=disabled
5+
org.eclipse.jdt.core.compiler.problem.forbiddenReference=warning
6+
org.eclipse.jdt.core.compiler.problem.reportPreviewFeatures=ignore
7+
org.eclipse.jdt.core.compiler.release=disabled
8+
org.eclipse.jdt.core.compiler.source=1.8

‎st-tdg-eval/pom.xml

+31-84
Original file line numberDiff line numberDiff line change
@@ -1,77 +1,55 @@
1-
<?xml version="1.0" encoding="UTF-8"?>
2-
3-
<project xmlns="http://maven.apache.org/POM/4.0.0"
4-
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
5-
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
1+
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
62
<modelVersion>4.0.0</modelVersion>
7-
8-
<groupId>giis</groupId>
9-
<artifactId>swagger-petstore-eval</artifactId>
10-
<version>1.0.0-SNAPSHOT</version>
11-
12-
<properties>
13-
<maven.compiler.source>1.8</maven.compiler.source>
14-
<maven.compiler.target>1.8</maven.compiler.target>
15-
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
16-
17-
<tdrules.version>4.3.0</tdrules.version>
18-
<qagrow.version>1.4.266</qagrow.version>
19-
20-
<!--
21-
<swagger-parser-version>2.0.27</swagger-parser-version>
22-
<swagger-core-version>2.1.10</swagger-core-version>
23-
<swagger-inflector-version>2.0.5</swagger-inflector-version>
24-
-->
25-
</properties>
26-
3+
<parent>
4+
<groupId>io.github.giis-uniovi</groupId>
5+
<artifactId>tdrules-st-tdg</artifactId>
6+
<version>1.5.0-SNAPSHOT</version>
7+
</parent>
8+
<artifactId>st-tdg-eval</artifactId>
9+
<packaging>jar</packaging>
10+
11+
<name>st-tdg-eval</name>
12+
<description>TDG - Test Data Generation for APIs - Petstore evaluation</description>
13+
<url>http://github.com/giis-uniovi/tdrules-st-tdg</url>
14+
<organization>
15+
<name>Software Engineering Research Group (GIIS) - Universidad de Oviedo, ES</name>
16+
<url>http://giis.uniovi.es/</url>
17+
</organization>
18+
2719
<dependencies>
2820
<dependency>
2921
<groupId>io.github.giis-uniovi</groupId>
3022
<artifactId>tdrules-client</artifactId>
31-
<version>${tdrules.version}</version>
32-
<scope>test</scope>
3323
</dependency>
3424
<dependency>
3525
<groupId>io.github.giis-uniovi</groupId>
3626
<artifactId>tdrules-client-oa</artifactId>
37-
<version>${tdrules.version}</version>
38-
<scope>test</scope>
3927
</dependency>
4028
<dependency>
4129
<groupId>io.github.giis-uniovi</groupId>
4230
<artifactId>tdrules-store-loader</artifactId>
43-
<version>${tdrules.version}</version>
44-
<scope>test</scope>
4531
</dependency>
4632
<dependency>
4733
<groupId>giis</groupId>
4834
<artifactId>qagrow</artifactId>
49-
<version>${qagrow.version}</version>
50-
<scope>test</scope>
5135
</dependency>
5236

5337
<dependency>
5438
<groupId>org.slf4j</groupId>
5539
<artifactId>slf4j-api</artifactId>
56-
<version>1.7.36</version>
5740
</dependency>
5841
<dependency>
59-
<groupId>ch.qos.logback</groupId>
60-
<artifactId>logback-classic</artifactId>
61-
<version>1.2.13</version>
62-
<scope>test</scope>
42+
<groupId>org.apache.logging.log4j</groupId>
43+
<artifactId>log4j-slf4j2-impl</artifactId>
6344
</dependency>
45+
6446
<dependency>
6547
<groupId>junit</groupId>
6648
<artifactId>junit</artifactId>
67-
<version>4.13.2</version>
68-
<scope>test</scope>
6949
</dependency>
7050
<dependency>
7151
<groupId>io.github.javiertuya</groupId>
7252
<artifactId>visual-assert</artifactId>
73-
<version>2.4.1</version>
74-
<scope>test</scope>
7553
</dependency>
7654
<dependency>
7755
<groupId>org.projectlombok</groupId>
@@ -81,10 +59,11 @@
8159
</dependency>
8260

8361
<!--
84-
Not latest versions, but we would needto compile the petstore files.
85-
To avoid conficts with dependencies we drop these dependencies and:
86-
only use the model and data folders from the petstore sources
87-
and create some dummy annotation classes to allow compile the model.
62+
Dependencies required by swagger inflector are not latest versions,
63+
but we would need to compile the petstore files.
64+
To prevent potential conficts with dependencies we drop these dependencies and:
65+
- only use the model and data folders from the petstore sources.
66+
- create some dummy annotation classes to allow compile the model.
8867
8968
<dependency>
9069
<groupId>io.swagger.core.v3</groupId>
@@ -122,7 +101,7 @@
122101
<configuration>
123102
<sources>
124103
<source>
125-
../swagger-petstore-main-fork/src/main/java
104+
../sut-petstore/src/main/java
126105
</source>
127106
</sources>
128107
</configuration>
@@ -132,7 +111,6 @@
132111
<plugin>
133112
<groupId>org.apache.maven.plugins</groupId>
134113
<artifactId>maven-compiler-plugin</artifactId>
135-
<version>3.13.0</version>
136114
<configuration>
137115
<excludes>
138116
<exclude>**/io/swagger/petstore/controller/**</exclude>
@@ -161,41 +139,10 @@
161139
-->
162140
</configuration>
163141
</plugin>
142+
<plugin>
143+
<groupId>org.apache.maven.plugins</groupId>
144+
<artifactId>maven-antrun-plugin</artifactId>
145+
</plugin>
164146
</plugins>
165147
</build>
166-
167-
<repositories>
168-
<repository>
169-
<id>giis</id>
170-
<url>https://in2test.lsi.uniovi.es/xlib/maven/</url>
171-
<snapshots>
172-
<enabled>false</enabled>
173-
</snapshots>
174-
<releases>
175-
<enabled>true</enabled>
176-
</releases>
177-
</repository>
178-
<repository>
179-
<id>giis-snapshots</id>
180-
<url>https://in2test.lsi.uniovi.es/xlib/maven-snapshots/</url>
181-
<snapshots>
182-
<enabled>true</enabled>
183-
</snapshots>
184-
<releases>
185-
<enabled>false</enabled>
186-
</releases>
187-
</repository>
188-
<!-- Repositorios de snapshots en GitHub -->
189-
<repository>
190-
<id>github-giis-uniovi</id>
191-
<url>https://maven.pkg.github.com/giis-uniovi/*</url>
192-
<snapshots>
193-
<enabled>true</enabled>
194-
</snapshots>
195-
<releases>
196-
<enabled>false</enabled>
197-
</releases>
198-
</repository>
199-
</repositories>
200-
201-
</project>
148+
</project>

0 commit comments

Comments
 (0)