Pull more product data from the Learn API #193
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…able ids, fall back to initial data - The metadata retrieval is now in an api function so it's easier to test. - Added a parser for readable ID - skus are generally readable IDs, but if we have a sku that's for a specific run, the learning_resources API won't know what to do with it. - If the data returned from the API doesn't contain data, or doesn't contain _some_ data, fall back to the data that was in the product already. - Added tests for API function - Fixed some names (we're not just pulling image data now)
…spec; remove a test endpoint - Pre-load API: a separate API that can be used to load and build a product for a given system and SKU - Created a function to load the data from Learn for a given system and SKU (but just load it) - Fixed some test issues - Removed the traefik_test_request endpoint, since we're not doing Traefik at all now - Added some settings for throttling
- Adds import_product management command to build out a product out of Learn data - Renames update_product_image_data to update_product_data since it's not _just_ image data now - Refactored all the Learn pulling code into api.get_product_metadata - Updated the preload API to use the new stuff
no-verify, maybe that'll help with the formatting thing
jkachel
added
Needs Review
cp-at-mit
approved these changes
Jan 11, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What are the relevant tickets?
Closes mitodl/hq#6309
Description (What does it do?)
Expands the data retrieval from Learn to include:
This is used in the existing management command
update_product_data, which will now update all of those fields in addition to the images. (It's also now namedupdate_product_data.)Since this is effectively all the relevant data for the product, this also adds an endpoint and a management command to do that specifically:
/api/v0/meta/products/preload/<system_slug>/<sku>/will trigger a query to Learn to retrieve the data for the specified resource, and will create the product in UE if it's found in Learn but not in UEimport_productdoes this same thing, but as a management command.The API endpoint is rate limited for unauthenticated users, and it calls the Learn API in real time. The use case is that the integrated system will ping this endpoint when a learner opens a resource page to make sure UE has a product for the resource. The API won't update a product that exists, and will return errors if the item isn't found via Learn's
learning_resourcesAPI (so you can't just load whatever you want). The integrated system doesn't necessarily need to wait for the call to finish.This also updates the data retrieval code to handle resource runs better. Prior to this, the code simply checked for the entire SKU in Learn (SKUs are generally the resource's readable ID). This is fine for a course or a program but not for a run of that program - the Learn API won't return data if you have a run in the readable ID. So, this now splits off the run and resource IDs so they can be matched separately.
Finally, this culls the
traefik_test_requestendpoint; we haven't supported Traefik in a while so this isn't particularly useful to have.How can this be tested?
Create an Integrated System that matches a platform in Learn, and a product for that system that matches a resource. For example, create
mitxonlineand a product forcourse-v1:MITxT+14.100x+2T2025. Then, runupdate_product_datafor this product. The product should be updated with the image data, as well as the pricing, title, and description from Learn.Import a product using the command line:
import_product mitxonline course-v1:MITxT+11.S952x+2T2022(or similar). This should build out the product with the data from Learn.Run the preload API on a SKU that does not yet exist:
/api/v0/meta/product/preload/mitxonline/course-v1%3AMITxT%2B14.73x%2B1T2025/This should build out a product with the data from Learn.Run the last two for SKUs that don't match an integrated system or a resource that exists. You should see an appropriate error message.
course-v1:MITxT+24.118x+3T4096). The API will have data from Learn to work with in this case but it should still fail because the run shouldn't exist.Additional Context
Maybe we want to restrict the preload API to require an API key or something since this could be used to generate products for things in Learn that shouldn't have them (or shouldn't have them quite yet).
This will take the highest price out of the prices that are captured by Learn. It will fall back to the resource price if the run doesn't have a price attached. Most (all?) things have two or fewer prices so this is fine, but if there's something with more than that, then that product should be managed manually. (We may want a flag for this in the product record - ideally these should update via scheduled task.)