-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Patch distributions in import batch #4880
Conversation
@@ -0,0 +1,7 @@ | |||
CREATE TABLE IF NOT EXISTS public.ship_original_project_context( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To save the original project context to expand selfs when needed
* Allows to keep track of the original project context which will change because of project renaming / patching | ||
* configuration so as to be able to use them to expand iris in resource payload | ||
*/ | ||
class OriginalProjectContext(xas: Transactors) extends FetchContext { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You don't need to implement FetchContext for this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only place it's needed is in FileSelf, so you could just split the interface. I think it's likely that a lot of places it would make sense to just pass onRead
rather than all of these methods
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Most of the methods come from FetchContext
and the import batch is a special use case, if we are to refactor it, it is outside the scope of this PR
} | ||
}(_) | ||
|
||
def single: Json => IO[Json] = (json: Json) => fileContentUrl(json).map(toS3Location) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This method is never used in real code, the tests can easily call the other one
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would still be more complicated as an extra level would be required
import io.circe.syntax.EncoderOps | ||
import io.circe.{Codec, Json} | ||
|
||
/** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess this is going in postgres so we can import in multiple sessions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The batch will run in an incremental way so we need to keep track of those contexts in the different runs.
Is that what you are asking ?
Fixes #4876