Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace temporary table with CTE in stats calculation #370

Merged
merged 9 commits into from
Jun 12, 2024

Conversation

candleindark
Copy link
Collaborator

@candleindark candleindark commented May 9, 2024

This PR replaces the use of temporary tables with CTEs (Common table expressions) in stats calculation of search results. The use of CTEs requires all subqueries to be executed in a combined query order to avoid repeated execution of a CTE. Consequently, some round trips to the DB server have been eliminated as well due to the use.

This PR closes #361.

Note: Changes in this PR have been applied to the Typhon and Falkor instances.

References:

The result of `CTE` is reused in a query, so performance is increased
…ups`

The result of `CTE` is reused in a query, so performance is increased
The result of `CTE` is reused in a query, so performance is increased
The result of `CTE` is reused in a query, so performance is increased
The result of `CTE` is reused in a query, so performance is increased
Creation of temporary tables are not allow in read-only db instances.
We support the use of read-only db instances
Copy link

codecov bot commented May 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.76%. Comparing base (7ede3ae) to head (897cb34).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #370      +/-   ##
==========================================
- Coverage   98.77%   98.76%   -0.01%     
==========================================
  Files          55       55              
  Lines        2605     2599       -6     
==========================================
- Hits         2573     2567       -6     
  Misses         32       32              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Instead of executing a selectable in this function. Returning a
scalar selectable call allow the result to be obtained
in an enclosing query along with other subqueries
Instead of executing a selectable in a function. Returning a
scalar selectable allow the stats to be obtained through execution
of a single query containing many subqueries
It is no longer needed since temporary tables are not used anymore
@candleindark candleindark marked this pull request as ready for review May 9, 2024 21:02
@yarikoptic
Copy link
Member

Did you have a chance to compare performance - is there a notable difference?

@candleindark
Copy link
Collaborator Author

candleindark commented May 10, 2024

Did you have a chance to compare performance - is there a notable difference?

Yes, I compared the the performance of the two different setup. No, I couldn't perceive any difference in performance.

@yarikoptic yarikoptic merged commit f4aed5d into datalad:master Jun 12, 2024
5 checks passed
@candleindark candleindark deleted the cte branch June 12, 2024 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants