Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to Parcel #57

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
REACT_APP_GOOGLE_CLIENT_ID=706375540729-sqnnig7o0d0uqmvav0h8nh8aft6l55u3.apps.googleusercontent.com
API_BASE=https://hydrant.xvm.mit.edu
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably need to use a separate xvm instance to do the server properly, unless we want to implement it all in CGI. @psvenk thoughts?

2 changes: 2 additions & 0 deletions .env.development
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
API_PORT=6873
API_BASE=http://localhost:$API_PORT
1 change: 1 addition & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
index.js
3 changes: 3 additions & 0 deletions .eslintrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"extends": ["react-app"]
}
7 changes: 6 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,14 +1,19 @@
# artifacts
scrapers/catalog.json
scrapers/fireroad.json
public/latest.json

src-server/data/latest.json

# python
__pycache__

# node
/node_modules
/build
/server

# build tooling
/.parcel-cache

deploy.sh

Expand Down
5 changes: 5 additions & 0 deletions .parcelrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"extends": "@parcel/config-default",
"resolvers": ["@parcel/resolver-glob", "..."],
"reporters": ["...", "./scripts/parcel-dev-reporter.js"]
}
22 changes: 12 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,42 +5,44 @@
Install:

- Python 3, at least Python 3.6.
- Node.js 16, at least Node 16.16.
- Careful, the latest version is 18!
- Node.js 18.17 or above.
- One way manage Node versions is using [nvm](https://github.com/nvm-sh/nvm).

In the root directory, run:

- `pip install -r requirements.txt` to install dependencies.
- `npm install` to install dependencies.

## Updating

### Local development

There's the frontend, which is the website and the interface. Then there's the backend, or the schedules, which are the files that have information about class schedules.
There's the frontend, which is the website and the interface. Then there's the backend, which serves information about class schedules and links data stored in Hydrant to the FireRoad API.

To update the frontend, you can run `npm start`, which will open a browser tab that updates live as you edit code.
To update the frontend (source code in the `src/` directory), you can run `npm start` and open the link that shows up in your terminal window. As you change the source code, the site should update in real-time, but you can reload if you want to make sure you're using the latest version.

To update the backend, `cd scrapers` then run `python update.py`.
To work on the backend (located in `src-server/`), it's a similar process. If you run `npm start`, you can edit any of the files in `src-server/` to reload the Node.js server automatically with your changes. If you just want to tinker with the API without touching the frontend code, you can alternatively run `npm run start:frontend`.

Make sure to update the backend before you run the frontend for the first time.
*NB: Hydrant is currently in the process of moving to a Node.js-only backend! For now, you'll need to do the following steps as well.*

To update the backend's data files, `cd scrapers` then run `python update.py`.

Make sure to update the backend data before you run the frontend for the first time.

### Changing semesters

Let's say you're updating from e.g. Spring 2023 to Fall 2023.

First, archive the old semester. Make sure you have updated schedule files. Then run `mv public/latest.json public/s23.json`.
First, archive the old semester. Make sure you have updated schedule files. Then run `mv src-server/data/latest.json src-server/data/s23.json`.

Then, update the new semester. Open `public/latestTerm.json`, change `urlName` to `f23`, and update the dates per [Registrar](https://registrar.mit.edu/calendar).
Then, update the new semester. Open `src-server/data/latestTerm.json`, change `urlName` to `f23`, and update the dates per [Registrar](https://registrar.mit.edu/calendar).

Finally, run the normal update process and commit the results to the repo.

### Updating the server

The server's frontend updates based on the `deploy` branch on GitHub, so any changes pushed there will become live.

The server's backend is updated through a cron script that runs `update.py` every hour.
The server's backend is updated by an automatic deploy to an XVM instance via the same `deploy` branch.

See `deploy/README.md` for more info.

Expand Down
6 changes: 3 additions & 3 deletions deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ Note that the GitHub token in `ci_secrets` must be regenerated yearly.

## Backend

The entire backend is the `latest.json` file in `~/web_scripts/hydrant/latest.json`.
The backend runs on an XVM instance at `hydrant.xvm.mit.edu`. It is deployed separately from the frontend code.

We have a [cron job](https://en.wikipedia.org/wiki/Cron) that runs every hour that updates this file. Cron is configured with a crontab file, in `~/cron_scripts/crontab`. There's a line (the one starting with `0 * * * *`) that calls the `~/cron_scripts/update_latest.sh` every hour.
After the backend is build with `npm run build`, we start the server on the XVM instance with [PM2](https://pm2.keymetrics.io/) using the `pm2.config.js` file located in this folder.

The `~/cron_scripts/update_latest.sh` pulls the latest version of the `deploy` branch on GitHub to the folder `~/hydrant`. Inside that folder, it then runs the `scrapers/update.py` script. Then it copies the `latest.json` from that folder to `~/web_scripts/hydrant`, where it is served to the internet.
*NB: This documentation is incomplete! It will be updated after the migration to a Node.js backend is finished and we finalize the deployment process.*
47 changes: 0 additions & 47 deletions deploy/cron_scripts/cronic

This file was deleted.

19 changes: 0 additions & 19 deletions deploy/cron_scripts/crontab

This file was deleted.

29 changes: 0 additions & 29 deletions deploy/cron_scripts/update_latest.sh

This file was deleted.

7 changes: 7 additions & 0 deletions deploy/pm2.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
// TODO
module.exports = {
app: [{
name: "hydrant-backend",
script : "../server/index.js"
}]
}
Loading