-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
streaming functions for doobie and slick with tests #108
base: master
Are you sure you want to change the base?
Conversation
…ts and mixed into db functions with status
slick/src/it/scala/za/co/absa/fadb/slick/SlickStreamingResultFunctionTest.scala
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Found a few tiny things and Jacoco coverage action is still failing (not on insufficient coverage though), so please take a look. Otherwise LGTM.
I pulled it, reviewed, compiled and ran tests.
# Conflicts: # README.md # build.sbt # core/src/main/scala/za/co/absa/fadb/DBFunction.scala # core/src/main/scala/za/co/absa/fadb/DBSchema.scala # doobie/src/main/scala/za/co/absa/fadb/doobie/DoobieFunction.scala # project/Dependencies.scala # slick/src/main/scala/za/co/absa/fadb/slick/SlickFunction.scala
JaCoCo code coverage report - scala 2.12.17
Files
|
JaCoCo code coverage report - scala 2.13.12
Files
|
Fixed this. The problem was in node version of Github's updated runner. Solved by an update of the jacoco github action version. |
@lsulak @jakipatryk Thanks for the initial review here and on the calls we had yesterday. Based on your comments I have made the streaming part for Slick as a separate package since additional dependencies are needed and we don't want to be importing transitive dependencies into our projects when streaming is not used. At the moment there is core module for non-streaming, streaming module for streaming, slick module for slick non-streaming, slick-streaming module for slick streaming and doobie module. Doobie module provides both non-streaming and streaming functions since Doobie already bring all the dependencis needed (works with fs2 streams under the hood) so there would be no benefit in splitting this into separate modules. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- pulled
- built
- test run
- jacoco parts - code review
${{ github.workspace }}/slick/target/scala-${{ matrix.scalaShort }}/jacoco/report/jacoco.xml | ||
${{ github.workspace }}/slick-streaming/target/scala-${{ matrix.scalaShort }}/jacoco/report/jacoco.xml | ||
${{ github.workspace }}/doobie/target/scala-${{ matrix.scalaShort }}/jacoco/report/jacoco.xml |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do not agree with removing the code coverage fail-check logic - see the comment on the end of the file.
As I know there is no hard merge block enabled.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@miroslavpojer I am not satisfied with this situation either. Unfortunately I couldn't perform merge of previous pull request with this check in place. Let's work together to find a good solution so we can perform tests against a database both locally and within a Github pipeline; for instance with a dockerized Postgres instance.
@@ -93,7 +97,9 @@ jobs: | |||
with: | |||
paths: > | |||
${{ github.workspace }}/core/target/scala-${{ matrix.scalaShort }}/jacoco/report/jacoco.xml, | |||
${{ github.workspace }}/streaming/target/scala-${{ matrix.scalaShort }}/jacoco/report/jacoco.xml, | |||
${{ github.workspace }}/slick/target/scala-${{ matrix.scalaShort }}/jacoco/report/jacoco.xml |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
${{ github.workspace }}/slick/target/scala-${{ matrix.scalaShort }}/jacoco/report/jacoco.xml | |
${{ github.workspace }}/slick/target/scala-${{ matrix.scalaShort }}/jacoco/report/jacoco.xml, |
${{ github.workspace }}/slick/target/scala-${{ matrix.scalaShort }}/jacoco/report/jacoco.xml | ||
${{ github.workspace }}/slick-streaming/target/scala-${{ matrix.scalaShort }}/jacoco/report/jacoco.xml |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
${{ github.workspace }}/slick-streaming/target/scala-${{ matrix.scalaShort }}/jacoco/report/jacoco.xml | |
${{ github.workspace }}/slick-streaming/target/scala-${{ matrix.scalaShort }}/jacoco/report/jacoco.xml, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just read the code
Added support for streaming for both Slick and Doobie. Since Slick works with Futures but for streaming it relies on IOs or other effect types there was a need to abstract the streaming operation in a new DBStreamingEngine.
There are following modules:
Closes #107