Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Capture node_data directly alongside feeback #3156

Merged
merged 3 commits into from
May 16, 2024

Conversation

DafyddLlyr
Copy link
Contributor

@DafyddLlyr DafyddLlyr commented May 16, 2024

What does this PR do?

  • Adds new node_data column to feedback table
  • Writes to the node_data column when feedback is logged
  • Rebuilds feedback_summary view to read from this column

Context

The current feedback_summary view in pretty slow running, likely due to the filtered join per-row to the published_flows table. This solution stores the node data directly alongside the feedback, thus mitigating this issue (hopefully!).

Local testing is good - significantly faster to query the view now.

Next steps...

Once the following changes are live on production, I'm proposing to run the following SQL to populate the new column. I'm opting to do this manually instead of as a Hasura migration as it will be slightly easier to manage and monitor, and it's pretty low-risk (we're just populating a new column).

UPDATE feedback
SET node_data = (
    SELECT 
        (published_flows.data -> feedback.node_id) -> 'data'
    FROM 
        published_flows
    WHERE 
        published_flows.flow_id = feedback.flow_id 
        AND published_flows.created_at < feedback.created_at
    ORDER BY 
        published_flows.created_at DESC
    LIMIT 1
);

I've tested this locally with a copy of feedback data, however the sync script just copies a single published flow so it only populates a few records.

This matches the current join on the flow, as seen here -

LEFT JOIN LATERAL
(
SELECT
(published_flows.data -> fb.node_id) -> 'data' AS data
FROM
published_flows
WHERE
published_flows.flow_id = fb.flow_id
AND published_flows.created_at < fb.created_at
ORDER BY
published_flows.created_at DESC
LIMIT 1
) AS published_flow_node ON true;

Copy link

github-actions bot commented May 16, 2024

🤖 Hasura Change Summary compared a subset of table metadata including permissions:

Updated Tables (1)

  • public.feedback permissions:

    insert select update delete
    public /
    1 added column permissions
    insert select update
    public ➕ node_data

Copy link

github-actions bot commented May 16, 2024

Removed vultr server and associated DNS entries

@DafyddLlyr DafyddLlyr requested a review from a team May 16, 2024 14:59
@DafyddLlyr DafyddLlyr marked this pull request as ready for review May 16, 2024 14:59
Copy link
Member

@jessicamcinchak jessicamcinchak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good & plan sounds solid! Agree since it's a new column, it's an easy to manage manual migration for existing records (with comfort we can remove/retry if any surprises) 👍

LEFT JOIN
flows f ON f.id = fb.flow_id
LEFT JOIN
teams t ON t.id = fb.team_id;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

@DafyddLlyr DafyddLlyr merged commit 8033d92 into main May 16, 2024
12 checks passed
@DafyddLlyr DafyddLlyr deleted the dp/node-data-feedback branch May 16, 2024 15:19
@DafyddLlyr
Copy link
Contributor Author

Update: SQL executed on production, getting expected results and feedback_summary view now significantly faster and loading in Metabase without issue 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants