Skip to content
This repository has been archived by the owner on Oct 18, 2024. It is now read-only.

feat: ✨ implement degreeworks scraper and requirements endpoint #140

Merged
merged 57 commits into from
May 19, 2024

Conversation

ecxyzzy
Copy link
Member

@ecxyzzy ecxyzzy commented Mar 9, 2024

Summary

Add the DegreeWorks scraper and relevant REST endpoints/GraphQL queries for fetching the data.

Endpoints

  • /v1/rest/degrees/all (allDegrees): Get all degree types as a top-level object, including its ID, dotted abbreviation, and all programs offered under that degree.
  • /v1/rest/degree/majors (major/majors): Get one degree by its ID, or all degrees that fulfill some criteria. Currently supported filters are degree ID (e.g. BS) and/or case-insensitive name substring.
  • /v1/rest/degree/minors (minor/minors): Get one minor by its ID, or all minors by case-insensitive substring.
  • /v1/rest/degree/specializations (specialization/specializations/specializationsByMajorId): Get one specialization by its ID, or all specializations by case-insensitive substring, or all specializations for a major's ID.

Issues

Closes #137.

ecxyzzy added 30 commits August 14, 2023 11:52
- Use static async factories for the DegreeWorks and PPAPI clients. This
  prevents logging weirdness and also ensures that all required fields
  are actually initialized before instance methods are called.
- Encapsulate all audit-related functions into an AuditParser class.
  Like the other clients this also uses a static async factory, since it
  requires the PPAPI client to normalize course numbers.
@ecxyzzy ecxyzzy temporarily deployed to staging-140-docs March 9, 2024 22:09 — with GitHub Actions Inactive
@github-actions github-actions bot changed the title feat: implement degreeworks scraper and requirements endpoint feat: ✨ implement degreeworks scraper and requirements endpoint Mar 9, 2024
Copy link
Contributor

@ap0nia ap0nia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a lot of stuff, and most of it looks reasonable. Some adjustments/clean up would be good 👍

tools/degreeworks-scraper/src/components/Scraper.ts Outdated Show resolved Hide resolved
tools/degreeworks-scraper/src/components/Scraper.ts Outdated Show resolved Hide resolved
@ecxyzzy ecxyzzy temporarily deployed to staging-140-docs May 19, 2024 19:57 — with GitHub Actions Inactive
@ecxyzzy ecxyzzy temporarily deployed to staging-140-docs May 19, 2024 20:16 — with GitHub Actions Inactive
@ecxyzzy ecxyzzy requested a review from ap0nia May 19, 2024 20:28
Copy link
Contributor

@ap0nia ap0nia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@ecxyzzy ecxyzzy merged commit 0b16a9c into main May 19, 2024
4 checks passed
@ecxyzzy ecxyzzy deleted the degreeworks-scraper branch May 19, 2024 23:37
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Degree requirements endpoint
2 participants