Get Atum Context, by getting existing partitioning from DB - retrieve Measures and AdditionalData for a given partitioning #120 #167

TebaleloS · 2024-03-11T09:00:01Z

Partitioning DB-Functions Calls

Added class to call partitioning measures function
Added class to call partitioning additional data DB function
Defined two methods getPartitioningMeasures and getPartitioningAdditionalData and implemented them.
Defined and implemented a method in the service returnAtumContext to package the above two methods including createPartitioningIfNotExists method in order to return the desired atum context.
Added few test-cases for the newly added code.
Included some test cases for createOrUpdateAdditionalData method as well.

closes #120

…nalData-for-a-given-partitioning

… with the partitioning

…nd-AdditionalData-for-a-given-partitioning' into feature/#120-retrieve-Measures-and-AdditionalData-for-a-given-partitioning

…dditional_data function

…al_data function

…ervice

…nalData-for-a-given-partitioning # Conflicts: # server/src/main/scala/za/co/absa/atum/server/Main.scala # server/src/main/scala/za/co/absa/atum/server/api/controller/PartitioningController.scala # server/src/main/scala/za/co/absa/atum/server/api/repository/PartitioningRepository.scala # server/src/main/scala/za/co/absa/atum/server/api/service/PartitioningService.scala

...scala/za/co/absa/atum/server/api/database/runs/functions/GetPartitioningAdditionalData.scala

…e service

TebaleloS · 2024-04-19T20:24:21Z

Addressed all the comment.
No integration testing was done in this PR, only the mocks were performed.

…#120-retrieve-Measures-and-AdditionalData-for-a-given-partitioning # Conflicts: # database/src/main/postgres/flows/V1.7.4__flows.alter.ddl # database/src/test/scala/za/co/absa/atum/database/runs/GetPartitioningAdditionalDataTest.scala # database/src/test/scala/za/co/absa/atum/database/runs/GetPartitioningMeasuresTest.scala

lsulak

LGTM, can be merged - the only thing I have is that we already have these 2 functions in the repository and I don't know what that would do to integration tests and/or Flyway, as their definitions are the same.

benedeki

The latest little comments don't warrant postponing the merge. It's good already. 😸

benedeki · 2024-04-26T14:52:01Z

server/src/main/scala/za/co/absa/atum/server/api/controller/PartitioningControllerImpl.scala

@@ -26,7 +26,9 @@ import zio._
 class PartitioningControllerImpl(partitioningService: PartitioningService)
  extends PartitioningController with BaseController {

-  override def createPartitioningIfNotExists(partitioningSubmitDTO: PartitioningSubmitDTO): IO[ErrorResponse, AtumContextDTO] = {
+  override def createPartitioningIfNotExists(


Tiny:
Weird allignment

benedeki · 2024-04-26T15:21:17Z

server/src/test/scala/za/co/absa/atum/server/api/TestData.scala

-  protected val partitioningDTO2: PartitioningDTO = Seq(PartitionDTO("Invali", "invalid"), PartitionDTO("invalid", "invalid"))
+  protected val partitioningDTO1: PartitioningDTO = Seq(
+    PartitionDTO("key1", "val1"),
+    PartitionDTO("key2", "val2")


Nah, let it be. What I meant was, that a bigger variability in test cases (record count), would cover more of the testable space. But this is already good enough 😉

benedeki · 2024-04-26T15:29:05Z

server/src/main/scala/za/co/absa/atum/server/api/controller/PartitioningController.scala

@@ -23,6 +23,6 @@ import zio.macros.accessible

 @accessible
 trait PartitioningController {
-  def createPartitioningIfNotExists(partitioning: PartitioningSubmitDTO): IO[ErrorResponse, AtumContextDTO]
+  def createPartitioningIfNotExists(partitioningSubmitDTO: PartitioningSubmitDTO): IO[ErrorResponse, AtumContextDTO]


OK, good reason. It's just it's not the best idea to put the type name into the parameter/variable name (only with rare exceptions). Next time I would for example use submittedPartitioning: PartitioningSubmitDTO.

But don't change it anymore! It's OK, good enough 😉

github-actions · 2024-04-29T06:43:31Z

JaCoCo server module code coverage report - scala 2.13.11

Overall Project	76.61% `-9.59%`	🍏
Files changed	85.1%	🍏

File	Coverage
PartitioningRepositoryImpl.scala	100%	🍏
PartitioningControllerImpl.scala	100%	🍏
PartitioningServiceImpl.scala	100%	🍏
GetPartitioningMeasures.scala	71.18% `-35.37%`	🍏
GetPartitioningAdditionalData.scala	70.56% `-35.93%`	🍏

Fixes #120 - Added class to call partitioning measures function

74af153

TebaleloS self-assigned this Mar 11, 2024

Merge branch 'master' into feature/#120-retrieve-Measures-and-Additio…

74993c0

…nalData-for-a-given-partitioning

TebaleloS added the work in progress Work on this item is not yet finished (mainly intended for PRs) label Mar 11, 2024

TebaleloS added 10 commits March 12, 2024 10:41

Fixes #120 - Removed status 16 option from the DB function associated…

9b8dc52

… with the partitioning

Merge remote-tracking branch 'origin/feature/#120-retrieve-Measures-a…

b0949ba

…nd-AdditionalData-for-a-given-partitioning' into feature/#120-retrieve-Measures-and-AdditionalData-for-a-given-partitioning

Fixes #120 - Changed measures function definitions

daa1a16

Fixes #120 - defined the class to communicate with get_partitioning_a…

9680a1b

…dditional_data function

Fixes #120 - implementing the repository of get_partitioning_addition…

ff6edb5

…al_data function

Fixes #120 - Calling getPartioningAdditionalData of the repo in the s…

086525e

…ervice

Fixes #120 - Calling getPartioningAdditionalData of the repo in the s…

52449d3

…ervice

Fixes #120 - added value parameters

57a1fd7

Fixes #120 - Merged master

1896eba

lsulak reviewed Mar 20, 2024

View reviewed changes

...scala/za/co/absa/atum/server/api/database/runs/functions/GetPartitioningAdditionalData.scala Outdated Show resolved Hide resolved

lsulak reviewed Mar 20, 2024

View reviewed changes

...scala/za/co/absa/atum/server/api/database/runs/functions/GetPartitioningAdditionalData.scala Outdated Show resolved Hide resolved

TebaleloS and others added 14 commits March 25, 2024 16:21

Fixes #120 - refactored the controller added computation method in th…

ac6aa3a

…e service

Fixes #120 - refactored the controller added computation method in th…

94703b8

…e service

Fixes #120 - changed type from error response to service error

3b05824

Fixes #120 - Changed inheritance from service trat to implementation

c063480

Fixes #120 - Fixing bugs

c5c5a28

Fixes #120 - Fixing type mismatch

e6e48c9

Fixes #120 - removed unused imports

f43f67f

Fixes #120 - Adding implicit conversion

baf89a2

Fixes #120 - Added package object

c86914b

implicits needed for Read instance derivation

5b3ceb5

Fixes #120 - Fixing bugs in PartitioningRepositorySpec

9100092

Fixes #120 - removed dead code in PartitioningRepositorySpec

1212cd6

Fixes #120 - fixing type mismatch in PartitioningRepositorySpec

a1c6cca

Fixes #120 - Adding test cases in PartitioningRepositorySpec

1301c24

Close #120 - Addressing GitHub comments

44c3664

Testing #120 db integration

7e30a62

benedeki mentioned this pull request Apr 24, 2024

#187: get all checkpoints for a flow #192

Merged

Closes #120 - Cleaning unwanted code lines

d1b6f7c

TebaleloS force-pushed the feature/#120-retrieve-Measures-and-AdditionalData-for-a-given-partitioning branch from 450a45f to d1b6f7c Compare April 26, 2024 07:31

TebaleloS and others added 6 commits April 26, 2024 09:36

Closes #120 - Fixing the build

42e9604

Closes #120 - Fixing the build

66af419

Closes #120 - Fixing the tests

31dea31

Closes #120 - Fixing the tests

ef537a5

Closes #120 - Fixing the tests cases

30c5f3f

lsulak previously approved these changes Apr 26, 2024

View reviewed changes

fixing the balta tests

14d8244

lsulak dismissed their stale review via 14d8244 April 26, 2024 14:58

lsulak previously approved these changes Apr 26, 2024

View reviewed changes

benedeki previously approved these changes Apr 26, 2024

View reviewed changes

AbsaOSS deleted a comment from github-actions bot Apr 29, 2024

benedeki added 2 commits April 29, 2024 09:16

* Fixing jacoco_check.yml setup (hopefully)

d62f574

* Decreasing overall coverage req

57ffa95

benedeki dismissed stale reviews from lsulak and themself via 57ffa95 April 29, 2024 07:47

salamonpavel added 2 commits April 30, 2024 14:49

jacoco excludes

7ab2215

added tests, decreased coverage level, added jacoco exclusions

f6ef5df

benedeki approved these changes Apr 30, 2024

View reviewed changes

TebaleloS merged commit 4d83d79 into master Apr 30, 2024
9 checks passed

TebaleloS deleted the feature/#120-retrieve-Measures-and-AdditionalData-for-a-given-partitioning branch April 30, 2024 14:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Get Atum Context, by getting existing partitioning from DB - retrieve Measures and AdditionalData for a given partitioning #120 #167

Get Atum Context, by getting existing partitioning from DB - retrieve Measures and AdditionalData for a given partitioning #120 #167

TebaleloS commented Mar 11, 2024 •

edited

Loading

TebaleloS commented Apr 19, 2024 •

edited

Loading

lsulak left a comment

benedeki left a comment

benedeki Apr 26, 2024

benedeki Apr 26, 2024

benedeki Apr 26, 2024

github-actions bot commented Apr 29, 2024 •

edited

Loading

Get Atum Context, by getting existing partitioning from DB - retrieve Measures and AdditionalData for a given partitioning #120 #167

Get Atum Context, by getting existing partitioning from DB - retrieve Measures and AdditionalData for a given partitioning #120 #167

Conversation

TebaleloS commented Mar 11, 2024 • edited Loading

Partitioning DB-Functions Calls

TebaleloS commented Apr 19, 2024 • edited Loading

lsulak left a comment

Choose a reason for hiding this comment

benedeki left a comment

Choose a reason for hiding this comment

benedeki Apr 26, 2024

Choose a reason for hiding this comment

benedeki Apr 26, 2024

Choose a reason for hiding this comment

benedeki Apr 26, 2024

Choose a reason for hiding this comment

github-actions bot commented Apr 29, 2024 • edited Loading

JaCoCo server module code coverage report - scala 2.13.11

TebaleloS commented Mar 11, 2024 •

edited

Loading

TebaleloS commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 29, 2024 •

edited

Loading