Skip to content

Search Algolia

egauzens edited this page Oct 17, 2023 · 8 revisions

Table of Contents

Technical Plan

Overview

One source, powered by Algolia, to search all content for the SPARC Portal. Data will come from the Pennsieve Discover API, select Contentful entry types, and the scicrunch API.

This project was a SPARC Year 4 milestone.

Objectives and Milestones

The objectives of the project and the metrics that will be used to measure success and serve as milestones to provide the leadership team with rough estimations.

Objectives and Milestones

Usage

Algolia index SPARC_pr is used for searching among datasets on the portal at sparc.science, while Algolia index SPARC_k-core-dev is used for searching among datasets on the portal at staging.sparc.science

The following api is utilized by the SPARC portal for search and faceting https://www.algolia.com/doc/api-reference/api-methods/

Solution Design

Pennsieve Discover

JavaScript code using the Algolia JavaScript client to get data from the Discover API, and write it to Algolia. This will utilize the replace all objects function, and will run every hour via GitHub  Actions

Contentful

Use webhooks in Contentful that will push data to Algolia when publishing and deleting entries.

Webhooks are setup in Contentful

Entry types

  • News
  • Event
  • About page
  • Projects
  • Tools and Resources

Algolia

Create indices for each type above

Pennsieve Discover → Algolia script

About

This script will populate the Algolia index with datasets from the Pennsieve API. More info can be found in the repo's readme.

Repository

GitHub

Usage

  • GitHub Actions will run a script to get all published datasets from Pennsieve Discover every hour
  • The GitHub action is configured in the repo at [/.github/workflows/algolia.yml](https://github.com/nih-sparc/algolia-search-discover/blob/main/.github/workflows/algolia.yml)

Other Resources