Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: create submission_services_log view #3144

Merged
merged 1 commit into from
May 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions hasura.planx.uk/metadata/tables.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1360,6 +1360,9 @@
- locked_at:
_is_null: true
check: null
- table:
name: submission_services_log
schema: public
- table:
name: submission_services_summary
schema: public
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
drop view if exists public.submission_services_log cascade;
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
create or replace view public.submission_services_log as
with payments as (
select
session_id,
payment_id::text as event_id,
'Pay' as event_type,
Copy link
Member Author

@jessicamcinchak jessicamcinchak May 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future we may want to expand this to more clearly distinguish 'Pay' & 'Invite to Pay' - but we know the latter is rarely used currently and all payment attempts will be captured here regardless of method.

initcap(status) as status,
'{}'::jsonb as response,
created_at
from payment_status
where status != 'created'
and created_at >= '2024-01-01'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason to query & display submissions prior to January 2024? This filter gives us a bit more consistency in event formatting & we could always look up submissions outside this timeframe on request.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally on board with keeping this filter in place - makes a whole lot of sense.

), submissions as (
select
(seil.request -> 'payload' -> 'payload' ->> 'sessionId')::uuid as session_id,
se.id as event_id,
case
when se.webhook_conf::text like '%bops%' then 'Submit to BOPS'
when se.webhook_conf::text like '%uniform%' then 'Submit to Uniform'
when se.webhook_conf::text like '%email-submission%' then 'Send to email'
when se.webhook_conf::text like '%upload-submission%' then 'Upload to AWS S3'
else se.webhook_conf::text
end as event_type,
case
when seil.status = 200 then 'Success'
else format('Failed (%s)', seil.status)
end as status,
seil.response::jsonb,
seil.created_at
from hdb_catalog.hdb_scheduled_events se
left join hdb_catalog.hdb_scheduled_event_invocation_logs seil on seil.event_id = se.id
where se.webhook_conf::text not like '%email/%'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm filtering out email events (reminders, confirmations, saves etc) - they'd be easy to add in the future if we want, but let's keep straightforward for now 👍

and seil.created_at >= '2024-01-01'
), all_events as (
select * from payments
union all
select * from submissions
)
SELECT
ls.flow_id,
ae.*
FROM all_events ae
left join public.lowcal_sessions ls on ls.id = ae.session_id
WHERE ls.flow_id is not null
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flow_id is null in older cases where we used to create send events on /draft or /preview routes - this filters those out & there's been a fix in place to not create events on those routes at all anymore.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, could we remove those records which should not have been created?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Event logs will simply be cleaned up by scheduled data sanitation, right? Personally would like to avoid a manual removal of records - they were arguably created "correctly" at the time, so I don't see an issue with keeping the audit history of them!

order by ae.created_at desc;
Loading