New kerckhoff external sites JSON endpoint

[DumboOctopus]

Background

The Daily Bruin often creates interactive pages for special content. For example, every year we make a special interactive page for our registration issue. You can see 2020 registration issue here.

One major issue with the development of these interactive is filling/updating content. Before kerckhoff existed, the developers would hardcode the content in. This includes the title of all articles, the article image urls, the article urls and any long paragraphs. This is problematic because if the content people wish to update this info, the developers would have to change the code and redeploy the website

The original kerckhoff was designed as a middle man between content people and the code.

On the content people side, they create a kerckhoff package for the interactive. Then they create AML files (basically human friendly json) that has all of their text and image urls inside it.

For the developers, kerckhoff exposes an API which returns the AML as json. For example, for registration issue used this url https://kerckhoff.dailybruin.com/api/packages/flatpages/interactive.2020.registration-issue/. When the interactive page performs a GET request to this endpoint, it will receive all the content data as JSON. Then the interactive page will use that data to render the page.

The nice thing about this system is that if the content people wish to update something, they just need to update the google doc and refresh kerckhoff's cache. No change is required on the developer side. No redeployment is required. The interactive's frontends always fetch from kerckhoff so any change to keckhoff will be reflected the next time the page is requested.

Task

We wish to have a similar endpoint for new kerckhoff. This endpoint should have a similar format and we should try to make it backwards compatible with old kerckhoff. That means it should generally have the same json structure.

When you perform a GET request to api/v1/package-sets/<package_set_slug>/packages/<package_slug>/data, kerckhoff should return a json object which has all the data from the latest version of the package <package_slug> which is in the package set <package_set_slug>.

I recommend using this json structure

{
slug: ...
version: 
package_set: 
publish_date:
images:{
   s3: {} 
}
data: {
  //all of data.aml goes here
  // usually in the interactives, we only use this data
  "article.aml":  {  // this key would exist if there was an "article.aml" file inside the google drive folder
      // all the data inside the AML file as json
   },
   "some_other_file.aml": { 
     // all the data inside some_other_file.aml as JSON
   },
}
}

Since backwards compatibility is important, playing around with old kerckhoff is a must. You can create a test package on kerckhoff to play around with it. First go to kerckhoff and sign in. Then click on Packages and create a new package. You can call it anything but just make sure it is clear that this package is a test package. Once the package is created, click on it in the list. Then there should be a link to the google drive folder. Go to that folder and create an article.aml. Copy paste some aml from another package. Then go back to kerckhoff and press update from gdrive. Lastly, go to this url https://kerckhoff.dailybruin.com/api/packages/<the package set of the package you made>/<slug of package you made>/. That will should show all the data from the package. Feel free to add images to the google drive folder and then update from gdrive. Then you should see stuff in the images.s3 part of the json response. Tbh, in the interactives, we usually don't read the images.s3 keys. We usually expect the content editors to copy paste those links into the AML files.

Kerckhoff Packages, Package Sets and Versions

packages: Packages are a collection of AML and images for single interactive. There is two places that their data is stored. First the package is linked with a google drive folder. This google drive folder contains images and AML files. Whenever someone refreshes kerckhoff's cache, kerckhoff will fetch the latest data from this google drive folder. Then the person can create a version of this package

version: Packages change over time. Maybe in the first version it only had articles from News, Arts, Sports and Quad but maybe in the last week, some articles from Sports get cancelled. Then the content people will update the AML and tell kerckhoff to create a new version. Kerckhoff keeps track of all versions but for this task, we only want to serve the latest version (this may change down the road but for now we will do this since it is the simplest).

package set: A package set is a collection of packages. In old kerckhoff, we had 3 package sets: prime, flatpages and alumni-site. This is just to keep things organized because we have A LOT of packages.

Testing

To preface this, these test cases do NOT cover all cases. The developer working on this is expected to come up with some more test cases as they implement the feature. By the time the developer finishes the task, they will probably know the details of the task better than I do. During the code review, we can go over these developer designed test cases.

1. Test that the endpoint returns the AML file output.
2. Test it returns the S3 links for the images (assets.dailybruin.com/something)
3. Test that if you make a new version, it returns the correct AML and S3 links
4. Test removing an image and creating a new version causes the endpoint to no longer list that image
5. Test that if there are no version, it has acceptable behavior (this can be pretty much anything as long as its clear there is no version for this package yet)

Recommended Resources

Django Views
Django URL dispatcher