Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add endpoint to update previously registered S3 files #4893

Closed
wants to merge 2 commits into from

Conversation

dantb
Copy link
Contributor

@dantb dantb commented Apr 23, 2024

@olivergrabinski is this enough to unblock you? Still need to verify all the file attribute fields in the E2E test response and the API design could be better.

Some points of note

  1. We change the FileAttributesOrigin from External to Client.
  2. The S3 path isn't provided by the client - it comes from the file state.
  3. Generate another UUID that's stored in the event but isn't used.
  4. Content type and filename now come from the request, other attributes are populated in the S3 client (same as for an initial save).

pathPrefix("register-update") {
(idSegment & indexingMode) { (id, mode) =>
pathEndOrSingleSlash {
operationName(s"$prefixSegment/files/{org}/{project}/register-update/{id}") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not important but we don't need this operationName anymore as the Kamon feature is disabled because it made Prometheus crash by sending data with a lot of distinct values (which Prometheus is not fond about)

id: FileId,
storageId: Option[IdSegment],
rev: Int,
entity: HttpEntity,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have a http entity here ?
It should be a path like the registerFile no ?

@@ -290,6 +290,32 @@ final class FilesRoutes(
}
}
}
},
pathPrefix("register-update") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be fixed in a later PR but we have a problem, it clashes with this endpoint:
https://bluebrainnexus.io/docs/delta/api/files-api.html#create-using-put

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not in the update operation but for the create one, we will need to provide a way to "register" a file where the user leaves Delta generating the id

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah this was intentional - can we do without the version where we generate the Id? Fewer endpoints => smaller API surface => less maintenance (tests, docs, code, etc)

@@ -138,6 +137,9 @@ object StorageFileRejection {
s"File cannot be saved because it already exists on path '$path'."
)

final case class FileNotFound(path: String)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if we can rename it in a way that makes it different from the FileNotFound from FileRejection ?


def overwriteFile(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The physical files should not be overwritte, updating a file means a new revision in the primary store and a new physical file.
When updating a file through register, we should just provide a path to another file which should already exist in S3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants