Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create DB function to get all checkpoint data of a partitioning #189

Closed
Tracked by #172
benedeki opened this issue Apr 15, 2024 · 2 comments · Fixed by #191
Closed
Tracked by #172

Create DB function to get all checkpoint data of a partitioning #189

benedeki opened this issue Apr 15, 2024 · 2 comments · Fixed by #191
Assignees
Labels
DB Issues touching the Database part of the project enhancement New feature or request work in progress Work on this item is not yet finished (mainly intended for PRs)

Comments

@benedeki
Copy link
Contributor

benedeki commented Apr 15, 2024

Background

We need to be able to read saved checkpoint data of a given partitioning.

Feature

Create a DB function that will return all checkpoint and its measurements for the given partitioning.

Proposed Solution

Function signature

CREATE OR REPLACE FUNCTION runs.get_partitioning_checkpoints(
    IN  i_partitioning     JSONB,
    OUT status             INTEGER,
    OUT status_text        TEXT,
    OUT id_checkpoint      UUID,
    OUT checkpoint_name    TEXT,
    OUT measure_name       TEXT,
    OUT measure_columns    TEXT[],
    OUT measurement_value  JSONB,
    OUT checkpoint_time    TIMESTAMP WITH TIME ZONE
) RETURNS SETOF record AS

for output order by checkpoint_time and id_checkpoint

Depends on #178

@benedeki benedeki moved this from 🆕 To groom to 🔖 Sprint in CPS small repos project Apr 15, 2024
@benedeki benedeki changed the title Create DB function to get all checkpoint data Create DB function to get all checkpoint data of a partitioning Apr 15, 2024
@benedeki benedeki added enhancement New feature or request DB Issues touching the Database part of the project dependent The item depends on some other open item (Issue or PR) labels Apr 15, 2024
@benedeki benedeki removed the dependent The item depends on some other open item (Issue or PR) label Apr 15, 2024
@benedeki
Copy link
Contributor Author

1 similar comment
@benedeki
Copy link
Contributor Author

@TebaleloS TebaleloS added the work in progress Work on this item is not yet finished (mainly intended for PRs) label Apr 18, 2024
TebaleloS added a commit that referenced this issue Apr 19, 2024
TebaleloS added a commit that referenced this issue Apr 19, 2024
TebaleloS added a commit that referenced this issue Apr 19, 2024
TebaleloS added a commit that referenced this issue Apr 25, 2024
TebaleloS added a commit that referenced this issue Apr 29, 2024
TebaleloS added a commit that referenced this issue Apr 30, 2024
TebaleloS added a commit that referenced this issue May 6, 2024
TebaleloS added a commit that referenced this issue May 6, 2024
TebaleloS added a commit that referenced this issue May 6, 2024
TebaleloS added a commit that referenced this issue May 6, 2024
TebaleloS added a commit that referenced this issue May 7, 2024
TebaleloS added a commit that referenced this issue May 7, 2024
TebaleloS added a commit that referenced this issue May 7, 2024
…#191)

* Fixes #189 - Define and implemented a db function to return the checkpoint

* Fixes #189 - Modifying a db function

* Fixes #189 - Modifying a db function

* Fixes #189 - Adding test cases

* Fixes #189 - Added two input fields to the db function

* Fixes #189 - Fixing type mismatch

* Testing #120 db integration

* Fixes #189 - Adding test cases

* Fixes #189 - Changing back the details of flows.flows

* Fixes #189 - Implementing GitHub comments

* Update database/src/main/postgres/runs/V1.8.3__get_partitioning_checkpoint.sql

Co-authored-by: David Benedeki <[email protected]>

* Update database/src/main/postgres/runs/V1.8.3__get_partitioning_checkpoint.sql

Co-authored-by: David Benedeki <[email protected]>

* Update database/src/main/postgres/runs/V1.8.3__get_partitioning_checkpoint.sql

Co-authored-by: David Benedeki <[email protected]>

* Update database/src/main/postgres/runs/V1.8.3__get_partitioning_checkpoint.sql

Co-authored-by: David Benedeki <[email protected]>

* Fixes #189 - Addressing GitHub comments

* Closes #189

* Closes #189 - Addressing GitHub comments

* Closes #189 - Addressing GitHub comments

---------

Co-authored-by: David Benedeki <[email protected]>
@github-project-automation github-project-automation bot moved this from 🔖 Sprint to ✅ Done in CPS small repos project May 7, 2024
TebaleloS added a commit that referenced this issue Jun 13, 2024
…190 (#194)

* Fixes #189 - Define and implemented a db function to return the checkpoint

* Fixes #189 - Modifying a db function

* Fixes #189 - Modifying a db function

* Fixes #189 - Adding test cases

* Fixes #189 - Added two input fields to the db function

* Fixes #189 - Fixing type mismatch

* Testing #120 db integration

* Fixes #189 - Adding test cases

* Fixes #189 - Changing back the details of flows.flows

* Fixes #189 - Implementing GitHub comments

* Fixes #190 - defined the dto to submit

* Update V1.7.1__flows.alter.ddl

* Delete database/src/main/postgres/runs/V1.8.3__get_partitioning_checkpoint.sql

* Update GetPartitioningMeasuresTest.scala

* Delete database/src/test/scala/za/co/absa/atum/database/runs/GetPartitioningCheckpoints.scala

* Fixees #190 - Added checkpoint query resultDTO, GetPartitioningCheckpoints class and its object

* Fixes #190 - implementing get checkpoints functionality in the repository

* Fixes #190 - renamed CheckpointDTO to CheckpointSubmitDTO

* Fixes #190 - Implementing partitioningCheckpoint service

* Fixes #190 - Implementing partitioningCheckpoint in the partitioning related classes.

* Fixes #190 - Implementing partitioningCheckpoint in the partitioning related classes.

* Fixes #190 - fixing controller

* Fixes #190 - Adding layers

* Fixes #190 - Adding doobie implicit

* Fixes #190 - Fixing the build

* Fixes #190 - removing unwanted layer from checkpoint repository

* Fixes #190 - adding mock layer from PartitioningRepositorySpec

* Fixes #190 - adding the implicits for the added dto's and defining the endpoint.

* Fixes #190 - defining current version

* Fixes #190 - Fixing type mismatch

* Fixes #190 - Implementing DTO instances

* Fixes #190 - Implementing DTO instances

* Fixes #190 - adding CheckpointMeasurements model

* Fixes #190 - adding CheckpointMeasurements test

* Read[CheckpointMeasurements]

* Saving changes

* Fixes #190 - Read[CheckpointMeasurements] test

* Fixes #190 - transforming CheckpointMeasurements to CheckpointDTO

* removing default values on output

* partitioning checkpoint test

* partitioning checkpoint test repository

* partitioning checkpoint test for service

* partitioning checkpoint test controller

* partitioning checkpoint test controller

* removed the mapError from getPartitioningCheckpoints

* Fixing getPartitioningCheckpoints test and implements GitHub suggestions

* Fixing merging conflicts

* Removing unused import

* implementing GitHub comments

* implementing GitHub comments in service

* implementing GitHub comments

* Fixing the build

* Fixing V1.8.3__get_partitioning_checkpoints.sql fields

* Fixing service test cases

* Fixing measure_columns to measured_columns

* Changing from object CheckpointFromDBObject to CheckpointFromDB

* Changing from object CheckpointFromDBObject to CheckpointFromDB

* Defining partitioningJson in a common way

* Implementing GitHub comments

---------

Co-authored-by: Pavel Salamon <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DB Issues touching the Database part of the project enhancement New feature or request work in progress Work on this item is not yet finished (mainly intended for PRs)
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants