Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patch distributions in import batch #4880

Merged
merged 3 commits into from
Apr 23, 2024

Conversation

imsdu
Copy link
Contributor

@imsdu imsdu commented Apr 18, 2024

Fixes #4876

@@ -0,0 +1,7 @@
CREATE TABLE IF NOT EXISTS public.ship_original_project_context(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To save the original project context to expand selfs when needed

* Allows to keep track of the original project context which will change because of project renaming / patching
* configuration so as to be able to use them to expand iris in resource payload
*/
class OriginalProjectContext(xas: Transactors) extends FetchContext {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to implement FetchContext for this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only place it's needed is in FileSelf, so you could just split the interface. I think it's likely that a lot of places it would make sense to just pass onRead rather than all of these methods

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the methods come from FetchContext and the import batch is a special use case, if we are to refactor it, it is outside the scope of this PR

}
}(_)

def single: Json => IO[Json] = (json: Json) => fileContentUrl(json).map(toS3Location)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is never used in real code, the tests can easily call the other one

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would still be more complicated as an extra level would be required

import io.circe.syntax.EncoderOps
import io.circe.{Codec, Json}

/**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is going in postgres so we can import in multiple sessions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The batch will run in an incremental way so we need to keep track of those contexts in the different runs.
Is that what you are asking ?

@imsdu imsdu merged commit 090a78e into BlueBrain:master Apr 23, 2024
9 checks passed
@imsdu imsdu deleted the 4876-patch-distributions branch April 23, 2024 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Patching distributions content urls in resources original payload
2 participants