Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: File API module #2418

Merged
merged 6 commits into from
Nov 14, 2023
Merged

feat: File API module #2418

merged 6 commits into from
Nov 14, 2023

Conversation

DafyddLlyr
Copy link
Contributor

@DafyddLlyr DafyddLlyr commented Nov 14, 2023

What does this PR do?

  • Group file related API endpoints into a module
  • Document endpoints in Swagger
  • Rename some routes for a more logical structure
  • Add validation and types
  • Adds auth to private file upload endpoint
  • file_typefileType
image

@DafyddLlyr DafyddLlyr changed the title feat: Files API module feat: File API module Nov 14, 2023
.field("filename", "")
.attach("file", Buffer.from("some data"), "some_file.txt")
.expect(422)
.expect(400)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One issue with the current simplified Zod error handling is that the validation checker in api.planx.uk/shared/middleware/validate.ts just returns a 400 so we lose some nuance.

I'll open a GH issue describing this small improvement as well as nicer error formatting using something like https://github.com/causaly/zod-validation-error maybe.

Comment on lines +68 to +69
expect(res.body).toHaveProperty("issues");
expect(res.body).toHaveProperty("name", "ZodError");
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above - I'd like there to be more meaningful errors returned here. It does show that the error is the missing filename field but it's not in a super friendly format.

Comment on lines +39 to +40
public: `${process.env.REACT_APP_API_URL}/file/public/upload`,
private: `${process.env.REACT_APP_API_URL}/file/private/upload`,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These routes are only used internally and referenced here - it made sense to pick this up as a small tidy up to standardise our routes a little.

@DafyddLlyr DafyddLlyr requested a review from a team November 14, 2023 09:06
@DafyddLlyr DafyddLlyr marked this pull request as ready for review November 14, 2023 09:07
Copy link

github-actions bot commented Nov 14, 2023

Removed vultr server and associated DNS entries

@@ -11,7 +11,7 @@ export const getFileFromS3 = async (fileId: string) => {
const file = await s3.getObject(params).promise();

return {
body: file.Body,
body: file.Body as Buffer,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be S3.Body which is just Buffer | Uint8Array | Blob | string | Readable but this was causing Vultr to run out of heap memory when compiling with tsc. It was going from 350mb to over 1gb just for this change - presumably it was importing and compiling a ton of additional AWS types?

Copy link
Member

@jessicamcinchak jessicamcinchak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes make this really clean & consistent, thanks!

@DafyddLlyr DafyddLlyr merged commit 7cce5f9 into main Nov 14, 2023
12 checks passed
@DafyddLlyr DafyddLlyr deleted the dp/files-module branch November 14, 2023 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants