Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: modify queryset from prefetch to select related #190

Merged
merged 2 commits into from
Jan 9, 2024

Conversation

uzairr
Copy link
Contributor

@uzairr uzairr commented Jan 5, 2024

PROD-3860

This PR modifies prefetch queryset to select_related query to reduce db round-trip for a skill fetching.

Testing:

  • Clone taxonomy in edx/src directory.
  • Install taxonomy via discovery shell in editable mode
pip install -e /edx/src/taxonomy-connector
  • Comment cache-related code to verify the impact
  • Add Course Skills to courses via discovery admin.
  • Check query count on master and ensure the response has skills information.
  • Checkout modify-queryset and hit the endpoint again. Note the decrease in query count.

  • Tested search/all endpoint with 2 scenarios:
    Course with multiple skills -> (with no skill_category and skill_subcategory)
    • with select_related: 94-96 queries
    • with prefetch_related: 93 queries
      Courses with multiple skill -> (having skill_category and skill_subcategory)
    • with select_related: 106 queries.
    • with prefetch_related: 112 queries

Merge checklist:

  • Any new requirements are in the right place (do not manually modify the requirements/*.txt files)
    • make upgrade && make requirements have been run to regenerate requirements
  • ./manage.py makemigrations has been run
    • Checkout the Database Migration Confluence page for helpful tips on creating migrations.
    • Note: This must be run if you modified any models.
      • It may or may not make a migration depending on exactly what you modified, but it should still be run.
  • Version bumped
  • Changelog record added

Post merge:

  • Tag pushed and a new version released
    • Note: Assets will be added automatically. You just need to provide a tag (should match your version number) and title and description.
  • After versioned build finishes in GitHub Actions, verify version has been pushed to PyPI
    • Each step in the release build has a condition flag that checks if the rest of the steps are done and if so will deploy to PyPi.
      (so basically once your build finishes, after maybe a minute you should see the new version in PyPi automatically (on refresh))
  • PR created in course-discovery to upgrade dependencies (including taxonomy-connector)
    • This must be done after the version is visible in PyPi as make upgrade in course-discovery will look for the latest version in PyPi.

CHANGELOG.rst Outdated
@@ -13,6 +13,10 @@ Change Log

Unreleased

[1.46.1] - 2024-01-05
---------------------
* feat: Modify prefetch queryset to select related of whitelisted product skills.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: prefetch related to select related for whitelisted product skills.

@AfaqShuaib09 AfaqShuaib09 merged commit df8c5cd into master Jan 9, 2024
10 checks passed
@AfaqShuaib09 AfaqShuaib09 deleted the modify-queryset branch January 9, 2024 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants