Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add missing postgres indexes #3910

Merged
merged 2 commits into from
Nov 5, 2024
Merged

fix: add missing postgres indexes #3910

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1a6ed0f
add missing indexes
jessicamcinchak Nov 4, 2024
e40a3d1
add DESC order to published flows date index
jessicamcinchak Nov 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 38 additions & 38 deletions hasura.planx.uk/metadata/tables.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@
definition:
enable_manual: false
insert:
columns: "*"
columns: '*'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is only formatting changes 🌀

retry_conf:
interval_sec: 30
num_retries: 1
Expand All @@ -171,7 +171,7 @@
query_params:
type: bops-submission
template_engine: Kriti
url: "{{$base_url}}/webhooks/hasura/send-slack-notification"
url: '{{$base_url}}/webhooks/hasura/send-slack-notification'
version: 2
- table:
name: document_template
Expand Down Expand Up @@ -225,7 +225,7 @@
definition:
enable_manual: false
insert:
columns: "*"
columns: '*'
retry_conf:
interval_sec: 30
num_retries: 1
Expand All @@ -239,7 +239,7 @@
query_params:
type: email-submission
template_engine: Kriti
url: "{{$base_url}}/webhooks/hasura/send-slack-notification"
url: '{{$base_url}}/webhooks/hasura/send-slack-notification'
version: 2
- table:
name: feedback
Expand Down Expand Up @@ -483,9 +483,9 @@
forward_client_headers: false
headers:
- name: authorization
value: "{{HASURA_PLANX_API_KEY}}"
value: '{{HASURA_PLANX_API_KEY}}'
timeout: 10
url: "{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html"
url: '{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html'
type: http
- role: demoUser
permission:
Expand Down Expand Up @@ -529,9 +529,9 @@
forward_client_headers: false
headers:
- name: authorization
value: "{{HASURA_PLANX_API_KEY}}"
value: '{{HASURA_PLANX_API_KEY}}'
timeout: 10
url: "{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html"
url: '{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html'
type: http
- role: teamEditor
permission:
Expand Down Expand Up @@ -562,9 +562,9 @@
forward_client_headers: false
headers:
- name: authorization
value: "{{HASURA_PLANX_API_KEY}}"
value: '{{HASURA_PLANX_API_KEY}}'
timeout: 10
url: "{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html"
url: '{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html'
type: http
select_permissions:
- role: api
Expand Down Expand Up @@ -617,7 +617,7 @@
- 1
- 29
- 30
comment: "For the demo user, we want to ensure they can only see their own flows, and flows from the Open Digital Planning [id = 30], Open Systems Lab [id = 1], and Templates [id = 29] team "
comment: 'For the demo user, we want to ensure they can only see their own flows, and flows from the Open Digital Planning [id = 30], Open Systems Lab [id = 1], and Templates [id = 29] team '
- role: platformAdmin
permission:
columns:
Expand Down Expand Up @@ -698,9 +698,9 @@
forward_client_headers: false
headers:
- name: authorization
value: "{{HASURA_PLANX_API_KEY}}"
value: '{{HASURA_PLANX_API_KEY}}'
timeout: 10
url: "{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html"
url: '{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html'
type: http
- role: demoUser
permission:
Expand Down Expand Up @@ -740,9 +740,9 @@
forward_client_headers: false
headers:
- name: authorization
value: "{{HASURA_PLANX_API_KEY}}"
value: '{{HASURA_PLANX_API_KEY}}'
timeout: 10
url: "{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html"
url: '{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html'
type: http
- role: teamEditor
permission:
Expand All @@ -767,9 +767,9 @@
forward_client_headers: false
headers:
- name: authorization
value: "{{HASURA_PLANX_API_KEY}}"
value: '{{HASURA_PLANX_API_KEY}}'
timeout: 10
url: "{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html"
url: '{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html'
type: http
delete_permissions:
- role: demoUser
Expand Down Expand Up @@ -809,9 +809,9 @@
forward_client_headers: false
headers:
- name: authorization
value: "{{HASURA_PLANX_API_KEY}}"
value: '{{HASURA_PLANX_API_KEY}}'
timeout: 10
url: "{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html"
url: '{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html'
type: http
select_permissions:
- role: demoUser
Expand Down Expand Up @@ -849,9 +849,9 @@
forward_client_headers: false
headers:
- name: authorization
value: "{{HASURA_PLANX_API_KEY}}"
value: '{{HASURA_PLANX_API_KEY}}'
timeout: 10
url: "{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html"
url: '{{HASURA_PLANX_API_URL}}/webhooks/hasura/validate-input/jsonb/clean-html'
type: http
- table:
name: lowcal_sessions
Expand Down Expand Up @@ -998,7 +998,7 @@
method: POST
query_params: {}
template_engine: Kriti
url: "{{$base_url}}/send-email/confirmation"
url: '{{$base_url}}/send-email/confirmation'
version: 2
- name: setup_lowcal_expiry_events
definition:
Expand Down Expand Up @@ -1028,7 +1028,7 @@
method: POST
query_params: {}
template_engine: Kriti
url: "{{$base_url}}/webhooks/hasura/create-expiry-event"
url: '{{$base_url}}/webhooks/hasura/create-expiry-event'
version: 2
- name: setup_lowcal_reminder_events
definition:
Expand Down Expand Up @@ -1058,7 +1058,7 @@
method: POST
query_params: {}
template_engine: Kriti
url: "{{$base_url}}/webhooks/hasura/create-reminder-event"
url: '{{$base_url}}/webhooks/hasura/create-reminder-event'
version: 2
- table:
name: operations
Expand Down Expand Up @@ -1278,7 +1278,7 @@
definition:
enable_manual: false
insert:
columns: "*"
columns: '*'
retry_conf:
interval_sec: 10
num_retries: 3
Expand All @@ -1300,13 +1300,13 @@
method: POST
query_params: {}
template_engine: Kriti
url: "{{$base_url}}/webhooks/hasura/create-payment-expiry-events"
url: '{{$base_url}}/webhooks/hasura/create-payment-expiry-events'
version: 2
- name: setup_payment_invitation_events
definition:
enable_manual: false
insert:
columns: "*"
columns: '*'
retry_conf:
interval_sec: 10
num_retries: 3
Expand All @@ -1328,13 +1328,13 @@
method: POST
query_params: {}
template_engine: Kriti
url: "{{$base_url}}/webhooks/hasura/create-payment-invitation-events"
url: '{{$base_url}}/webhooks/hasura/create-payment-invitation-events'
version: 2
- name: setup_payment_reminder_events
definition:
enable_manual: false
insert:
columns: "*"
columns: '*'
retry_conf:
interval_sec: 10
num_retries: 3
Expand All @@ -1356,7 +1356,7 @@
method: POST
query_params: {}
template_engine: Kriti
url: "{{$base_url}}/webhooks/hasura/create-payment-reminder-events"
url: '{{$base_url}}/webhooks/hasura/create-payment-reminder-events'
version: 2
- name: setup_payment_send_events
definition:
Expand Down Expand Up @@ -1385,7 +1385,7 @@
method: POST
query_params: {}
template_engine: Kriti
url: "{{$base_url}}/webhooks/hasura/create-payment-send-events"
url: '{{$base_url}}/webhooks/hasura/create-payment-send-events'
version: 2
- table:
name: payment_status
Expand Down Expand Up @@ -1647,12 +1647,12 @@
definition:
enable_manual: false
insert:
columns: "*"
columns: '*'
retry_conf:
interval_sec: 30
num_retries: 1
timeout_sec: 60
webhook: "{{HASURA_PLANX_API_URL}}"
webhook: '{{HASURA_PLANX_API_URL}}'
headers:
- name: authorization
value_from_env: HASURA_PLANX_API_KEY
Expand All @@ -1661,7 +1661,7 @@
query_params:
type: s3-submission
template_engine: Kriti
url: "{{$base_url}}/webhooks/hasura/send-slack-notification"
url: '{{$base_url}}/webhooks/hasura/send-slack-notification'
version: 2
- table:
name: sessions
Expand Down Expand Up @@ -1804,7 +1804,7 @@
- created_at
- submitted_at
filter: {}
comment: "For future, if this moves outside of the Flow to somewhere like Team, we should update 'demoUser' to only see submission data related to only their flows. "
comment: 'For future, if this moves outside of the Flow to somewhere like Team, we should update ''demoUser'' to only see submission data related to only their flows. '
- role: platformAdmin
permission:
columns:
Expand Down Expand Up @@ -2365,7 +2365,7 @@
- 29
- 30
- 32
comment: "For the demo user, we want to ensure they can only see their own team [id = 32], and the teams: Open Digital Planning [id = 30], Open Systems Lab [id = 1], and Templates [id = 29] team "
comment: 'For the demo user, we want to ensure they can only see their own team [id = 32], and the teams: Open Digital Planning [id = 30], Open Systems Lab [id = 1], and Templates [id = 29] team '
- role: platformAdmin
permission:
columns:
Expand Down Expand Up @@ -2482,7 +2482,7 @@
definition:
enable_manual: false
insert:
columns: "*"
columns: '*'
retry_conf:
interval_sec: 30
num_retries: 1
Expand All @@ -2496,7 +2496,7 @@
query_params:
type: uniform-submission
template_engine: Kriti
url: "{{$base_url}}/webhooks/hasura/send-slack-notification"
url: '{{$base_url}}/webhooks/hasura/send-slack-notification'
version: 2
- table:
name: user_roles
Expand Down
7 changes: 7 additions & 0 deletions hasura.planx.uk/migrations/1730742914548_create_index_bops_applications_session_id/down.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
DROP INDEX IF EXISTS "public"."bops_applications_session_id";
DROP INDEX IF EXISTS "public"."uniform_applications_session_id_idx";
DROP INDEX IF EXISTS "public"."email_applications_session_id_idx";
DROP INDEX IF EXISTS "public"."s3_applications_session_id_idx";
DROP INDEX IF EXISTS "public"."feedback_flow_id_team_id_idx";
DROP INDEX IF EXISTS "public"."reconciliation_requests_session_id_idx";
DROP INDEX IF EXISTS "public"."published_flows_created_at_idx";
14 changes: 14 additions & 0 deletions hasura.planx.uk/migrations/1730742914548_create_index_bops_applications_session_id/up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
CREATE INDEX "bops_applications_session_id_idx" on
"public"."bops_applications" using hash ("session_id");
CREATE INDEX "uniform_applications_session_id_idx" on
"public"."uniform_applications" using hash ("submission_reference");
CREATE INDEX "email_applications_session_id_idx" on
"public"."email_applications" using hash ("session_id");
CREATE INDEX "s3_applications_session_id_idx" on
"public"."s3_applications" using hash ("session_id");
CREATE INDEX "feedback_flow_id_team_id_idx" on
"public"."feedback" using btree ("team_id", "flow_id");
CREATE INDEX "reconciliation_requests_session_id_idx" on
"public"."reconciliation_requests" using hash ("session_id");
CREATE INDEX "published_flows_created_at_idx" on
"public"."published_flows" using btree ("created_at");
Copy link
Member Author

@jessicamcinchak jessicamcinchak Nov 4, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple notes about this migration:

  • Why a mix of hash versus btree index types?
    • I'm using hash indices when it's a column that is exclusively queried via an equality check (eg session id) and btree indices when a) it's a column that is queried via equality or range checks or b) multiple columns are combined into the same index
    • Postgres docs on this are helpful https://www.postgresql.org/docs/current/indexes-types.html
  • I'm adding an index to published_flows.created_at because that is how all of our current queries are ordering, but I have not actually checked/compared yet if there's a difference to sort by published flow id (or jogged my memory about why we don't do this already). Even if we swap how we order in the future, an index on created_at should still be useful/hit here when we do date comparisons for reconciliation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may also be worth adding a DESC in here as we're always searching in this order (docs).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool good shout - that's now updated 👍 Worth noting order isn't possible to specify in Hasura GUI for adding indexes, but worked no problem when added via "SQL" panel directly
Screenshot from 2024-11-05 09-13-40

Loading