feat: add send events and API endpoint for uploading submissions to S3 #3003
Conversation
|
Removed vultr server and associated DNS entries |
| ContentDisposition: `inline;filename="${filename}"`, | ||
| ContentType: file.mimetype, | ||
| ContentType: file?.mimetype || "application/json", |
There was a problem hiding this comment.
A bit surprisingly, exportData (our DigitalPlanningApplication typed object) satisfies type Express.Multer.File expected by these functions. It won't have a buffer & mimetype properties, but the S3.PutObjectRequest.Body accepts any string so I've just defined this simple JSON.stringify() fallback here.
Would be a good one to more intentionally revisit in a follow-up PR if this grows beyond a proof-of-concept!
| }); | ||
| }); | ||
|
|
||
| it.todo("succeeds"); // mock uploadPrivateFile ?? |
There was a problem hiding this comment.
I'm pretty sure there are examples of this (mocking aws-sdk/clients/s3 basically) in file.test.ts 👍
| // `/upload-submission/:localAuthority` is only called via Hasura's scheduled event webhook, so body is wrapped in a "payload" key | ||
| const { payload } = req.body; | ||
| const localAuthority = req.params.localAuthority; | ||
|
|
||
| if (!payload?.sessionId) { | ||
| return next({ | ||
| status: 400, | ||
| message: `Missing application payload data to send to email`, | ||
| }); | ||
| } | ||
|
|
||
| try { | ||
| const { sessionId } = payload; | ||
|
|
||
| // Only prototyping with Barnet to begin | ||
| // In future, confirm this local authority has an S3 bucket/folder configured in team_integrations or similar | ||
| if (localAuthority !== "barnet") { | ||
| return next({ | ||
| status: 400, | ||
| message: `Send to S3 is not enabled for this local authority (${localAuthority})`, | ||
| }); | ||
| } |
There was a problem hiding this comment.
nit: /send is one of the modules that still doesn't have the validation layer finished.
It could be nice to call validate() as a middleware in the route before getting here which would save us some of this manual checking.
Totally fair if this is a follow up, post-prototype tidy up tbh (I think there's shared validation logic between various /send routes so this might be nice to tackle in one go).
| if ( | ||
| value === Destination.S3 && | ||
| newCheckedValues.includes(value) && | ||
| teamSlug !== "barnet" | ||
| ) { | ||
| alert( | ||
| "AWS S3 uploads are currently being prototyped with Barnet only. Please do not select this option for other councils yet.", | ||
| ); | ||
| } | ||
| }; |
There was a problem hiding this comment.
(nit / no action needed): As this grows it looks like a good case for a formik validation schema, similar to how we handle logic for other forms.
There was a problem hiding this comment.
Thanks for these validation reminders - will address all in followup PRs - going to merge this initial work so I can get test links over to Kev sooner than later 👍
There was a problem hiding this comment.
- The success message will include a link the uploaded submission on S3 like below
- Confirm that JSON file is private and you can download successfully if authenticated
- The lowcal_session database record should also be marked as submitted
Tested and working as expected! ✅
These are the backend changes following on from #2999:
/upload-submisison/:localAuthoritywithin Send module to generate digital planning payload JSON & upload as a private file to a specified/static S3 folder within our main user data bucketTesting:
/publishedlink and fill out application (note that Pay is not enabled for Barnet and will show error, but you can "Continue" through without paying for this test and still generate a valid payload)lowcal_sessiondatabase record should also be marked as submitted