Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Customization of the template #10

Merged
merged 13 commits into from
Apr 3, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
- uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '8'
java-version: '17'
cache: 'maven'

- name: Test and aggregate surefire report
Expand Down Expand Up @@ -76,7 +76,7 @@ jobs:
- uses: javiertuya/[email protected]
with:
token: ${{ secrets.GITHUB_TOKEN }}
java-version: '8'
java-version: '17'
mvn-deploy-args: '-P publish-github -DskipTests=true -Dmaven.test.failure.ignore=false -U --no-transfer-progress'
delete-old-snapshots: true
min-snapshots-to-keep: 2
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ pom.xml.versionsBackup
TESTS-TestSuites.xml
*.db
*/dat/out

/.settings/
suarezgpablo marked this conversation as resolved.
Show resolved Hide resolved
.idea
# to allow each developer customize sonarlint
org.sonarlint.eclipse.core.prefs
2 changes: 1 addition & 1 deletion .project
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>samples-giis-template</name>
<name>modevo</name>
<comment></comment>
<projects>
</projects>
Expand Down
86 changes: 75 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,78 @@
[![Status](https://github.com/giis-uniovi/modevo/actions/workflows/test.yml/badge.svg)](https://github.com/giis-uniovi/modevo/actions)
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=my%3Amodevo&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=my%3Amodevo)
# MoDEvo - Model-driven approach to maintain data integrity for evolution of column family DBMSs

suarezgpablo marked this conversation as resolved.
Show resolved Hide resolved
# samples-giis-template
This repository contains the modules of the Model-driven engineering approach MoDEvo, which provides the data migrations required to maintain data integrity in a column family DBSM when the database evolves:

Template project, includes:
- Configuration of a maven multimodule java project
- Aggregated test reports (surefire and junit style)
- SonarQube static test (includes coverage results)
- Publish a snapshot for each branch to GitHub Packages
- Publish releases to Maven Central (deactivated)
- Provide a data migration model given an evolution of the schema.
- Transformation of the data migration model in a script.
- Execution of the script against a Cassandra database.
- Available for Java 17 and higher.

See the general contribution policies and guidelines for *giis-uniovi* at
[CONTRIBUTING.md](https://github.com/giis-uniovi/.github/blob/main/profile/CONTRIBUTING.md).
## Quick Start

Currently, MoDEvo can be used by importing it as a Maven project in Eclipse. MoDEvo transforms the the input models of MoDEvo (Schema, Schema Evolution and Conceptual Model) requires into a Data Migration model by executing ATL transformations. There are two possible ways to execute these transformations:

- Using the ATL module: Run from Eclipse the MoDEvo.atl as an ATL transformation. In the configuration menu, you must insert the paths for the inputs models and the path where the output model will be generated. It is required to install the last version of the ATL module in Eclipse (https://download.eclipse.org/mmt/atl/updates/releases/)

- Using Java: Create a Main Java class or a test case using JUnit that imports the class `MainTransformations` and then call the method `createDataMigrationModelAndScript` with the corresponding input models paths and the output model path.

<details open><summary><strong>Java</strong></summary>

```Java Main class
import giis.modevo.transformations.MainTransformations;
public static void main (String[] args){
new MainTransformations().createDataMigrationModelAndScript("pathSchema.xml", "pathConceptuaModel.xml", "pathSchemaEvolution.xml", "outputPath.xml");
}
```
```Java Test Case
@Test
public void testExample () {
new MainTransformations().createDataMigrationModelAndScript("pathSchema.xml", "pathConceptuaModel.xml", "pathSchemaEvolution.xml", "outputPath.xml");
}
```



</details>


## Modules

MoDEvo is composed of the following modules:

- `modevo-transform`: Transforms the input models (Schema, Schema Evolution and Conceptual Model) in a Data Migration model that specifies the data migrations to perform.
- `modevo-script`: Transforms a Data Migration model in scripts stored as Java objects that can, optionally be executed against a Cassandra database using this module. It also provides a textual representation of the script in a Cassandra Query Language-like language.

## modevo-transform module

Determines the data migrations required to maintain the data integrity in a column family DBMS . It determines these migration through the execution of ATL transformations that are located in the package `giis.modevo.transformations.atl`. In order to execute these transformations, MoDEvo requires of the following input models in xml format:

- Conceptual Model: Contains the structure of the conceptual model, specifying its entities, their relationships and the attributes that are part of an entity or relationship.
- Schema: Contains the structure of the schema, specifying its tables and the columns of these tables. It also contains the association of each column with an attribute of the schema.
- Schema Evolution: Contains the evolution performed in the schema.

Given these three inputs models, MoDEvo transforms them into a Data Migration model:

- Data Migration: Contains all the migrations that are required to maintain the data integrity.

In addition, Java objects of each of the input and output models are created to be used in the MoDEvo-script module.

## modevo-script module

Transforms the data migration model generated in the MoDEvo-transformation module into a script that can be executed against a column family DBSM. Currently it supports the most used column family DBSM, Apache Cassandra. In order to execute this transformation, MoDEvo requires the Schema, Schema Evolution and Data Migration models as Java objects.

Given these objects, MoDEvo creates one script for each table that requires data migrations in two ways:

- Textual representation in a Cassandra Query Language-like format. It is composed of the three basic opeators FOR, SELECT and INSERT.
- Optionally, the script can be executed in a Cassandra database, requiring to configure the "MoDEvo.properties" file located in the root with the appropiate values.

## Citing this work

TODO

## Related work

Suárez-Otero, P., Mior, M. J., Suárez-Cabal, M. J., & Tuya, J. (2023). CoDEvo: Column family database evolution using model transformations. Journal of Systems and Software, 111743.

Suárez-Otero González, P., Mior, M. J., Suárez Cabal, M. J., & Tuya González, P. J. (2023, March). Data migration in column family database evolution using MDE. In Workshop Proceedings of the EDBT/ICDT 2023 Joint Conference (March 28-March 31, 2023, Ioannina, Greece). George Fletcher and Verena Kantere.

Suárez-Cabal, M. J., Suárez-Otero, P., de la Riva, C., & Tuya, J. (2023). MDICA: Mainte-nance of data integrity in column-oriented database applications. Computer Standards & In-terfaces, 83, 103642.
2 changes: 1 addition & 1 deletion samples-giis-template-bom/.project → modevo-bom/.project
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>samples-giis-template-bom</name>
<name>modevo-bom</name>
<comment></comment>
<projects>
</projects>
Expand Down
12 changes: 6 additions & 6 deletions samples-giis-template-bom/pom.xml → modevo-bom/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>io.github.giis-uniovi</groupId>
<artifactId>samples-giis-template</artifactId>
<artifactId>modevo</artifactId>
<version>1.0.1-SNAPSHOT</version>
</parent>
<artifactId>samples-giis-template-bom</artifactId>
<artifactId>modevo-bom</artifactId>
<packaging>pom</packaging>
<name>samples-giis-template-bom</name>
<description>Multimodule maven project template, bill of materials</description>
<name>modevo-bom</name>
<description>MoDEvo bill of materials</description>
<url>http://github.com/giis-uniovi/samples-giis-template</url>
suarezgpablo marked this conversation as resolved.
Show resolved Hide resolved
<organization>
<name>Software Engineering Research Group (GIIS) - Universidad de Oviedo, ES</name>
Expand All @@ -18,12 +18,12 @@
<dependencies>
<dependency>
<groupId>io.github.giis-uniovi</groupId>
<artifactId>samples-giis-template-m1</artifactId>
<artifactId>modevo-transform</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.github.giis-uniovi</groupId>
<artifactId>samples-giis-template-m2</artifactId>
<artifactId>modevo-script</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>samples-giis-template-m2</name>
<name>modevo-script</name>
<comment></comment>
<projects>
</projects>
Expand Down
10 changes: 5 additions & 5 deletions samples-giis-template-m2/pom.xml → modevo-script/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>io.github.giis-uniovi</groupId>
<artifactId>samples-giis-template</artifactId>
<artifactId>modevo</artifactId>
<version>1.0.1-SNAPSHOT</version>
</parent>
<artifactId>samples-giis-template-m2</artifactId>
<artifactId>modevo-script</artifactId>
<packaging>jar</packaging>
<name>samples-giis-template-m2</name>
<description>Multimodule maven project template, module with dependencies</description>
<name>modevo-script</name>
<description>MoDEvo script module</description>
<url>http://github.com/giis-uniovi/samples-giis-template</url>
<organization>
<name>Software Engineering Research Group (GIIS) - Universidad de Oviedo, ES</name>
Expand All @@ -17,7 +17,7 @@
<dependencies>
<dependency>
<groupId>io.github.giis-uniovi</groupId>
<artifactId>samples-giis-template-m1</artifactId>
<artifactId>modevo-transform</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>samples-giis-template-m1</name>
<name>modevo-transform</name>
<comment></comment>
<projects>
</projects>
Expand Down
8 changes: 4 additions & 4 deletions samples-giis-template-m1/pom.xml → modevo-transform/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>io.github.giis-uniovi</groupId>
<artifactId>samples-giis-template</artifactId>
<artifactId>modevo</artifactId>
<version>1.0.1-SNAPSHOT</version>
</parent>
<artifactId>samples-giis-template-m1</artifactId>
<artifactId>modevo-transform</artifactId>
<packaging>jar</packaging>
<name>samples-giis-template-m1</name>
<description>Multimodule maven project template, module without dependencies</description>
<name>modevo-transform</name>
<description>MoDEvo transform module</description>
<url>http://github.com/giis-uniovi/samples-giis-template</url>
<organization>
<name>Software Engineering Research Group (GIIS) - Universidad de Oviedo, ES</name>
Expand Down
16 changes: 8 additions & 8 deletions pom.xml
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>io.github.giis-uniovi</groupId>
<artifactId>samples-giis-template</artifactId>
<artifactId>modevo</artifactId>
<version>1.0.1-SNAPSHOT</version>
<packaging>pom</packaging>

<name>samples-giis-template</name>
<description>Multimodule maven project template</description>
<name>modevo</name>
<description>Model-driven approach to maintain data integrity for evolution of column family DBMSs</description>
<url>http://github.com/giis-uniovi/samples-giis-template</url>
<organization>
<name>Software Engineering Research Group (GIIS) - Universidad de Oviedo, ES</name>
<url>http://giis.uniovi.es/</url>
</organization>

<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<maven.compiler.target>17</maven.compiler.target>
<maven.compiler.source>17</maven.compiler.source>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

<surefire.version>3.2.5</surefire.version>
</properties>

<modules>
<module>samples-giis-template-m1</module>
<module>samples-giis-template-m2</module>
<module>samples-giis-template-bom</module>
<module>modevo-transform</module>
<module>modevo-script</module>
<module>modevo-bom</module>
</modules>

<dependencyManagement>
suarezgpablo marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
3 changes: 2 additions & 1 deletion sonar-project.properties
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ sonar.projectKey=my:modevo
sonar.projectName=modevo
sonar.organization=giis
sonar.projectVersion=1.0
sonar.java.source=1.8
sonar.java.source=17
sonar.modules=samples-giis-template-m1,samples-giis-template-m2
sonar.sources=src/main/java
sonar.sourceEncoding=UTF-8
Expand All @@ -12,3 +12,4 @@ sonar.java.test.binaries=target/test-classes
sonar.language=java
sonar.junit.reportsPath=**/target/surefire-reports
sonar.coverage.jacoco.xmlReportPaths=**/target/site/jacoco/jacoco.xml
sonar.java.libraries=${env.HOME}/.m2/repository/org/projectlombok/lombok/**/*.jar
Loading