Jai Pise, Piranava Abeyakaran, Rishi Soni, Shyam Sai Bethina
The main scripts that drive the backend logic and run in Google Cloud Functions are in backend/main.py and the UI code is in the frontend folder. Inspiration
With the rising academic pressures on college students, there's an increasing demand for efficient study tools. We noticed that while students take extensive lecture notes, they often struggle to review them in a manner that truly tests their understanding. We also noticed that ChatGPT and other major LLMS have a character limit that can limit how many pages of lecture notes students can input. The idea of converting these notes into a quiz format and key bullet points emerged as an effective way to reinforce learning, while giving students the freedom to input as many notes as they want to.
What it does
Our product, named "Fountain," allows college students to upload their lecture notes or any study material. The platform then processes the content and generates a comprehensive quiz and review notes based on the uploaded notes. This not only helps students test their knowledge but also aids in active recall, a proven method to strengthen memory retention. Fountain allows students to upload multiple notes or lecture slides to create sample quiz questions that students can use to understand the content. By allowing multiple notes to be uploaded, students can easily switch between notes and different classes to review different topics faster. By also providing key points, Fountain makes it much easier to digest large amounts of notes and lectures.
How we built it
Frontend: We have a friendly looking UI, built using React.js and Tailwind.css, where users can upload their notes as a PDF. The interface also allows users to view the notes or lecture material side by side with the key points from it or a generated quiz from it. Middleware: Once a document has been uploaded into the Firebase Cloud Storage, it triggers a Python cloud function that parses the PDF file. This parsed PDF file is then tokenized using OpenAI’s tiktoken, and separated into smaller chunks. These smaller chunks are then passed to OpenAI’s embedding API and the embeddings for each chunk are stored as vectors in Pinecone for fast similarity retrieval. We used Langchain as a connecting service for similarity search, text chunking, and to get structured prompts and outputs. Backend: Once a user uploads a PDF document, it is uploaded to Firebase’s Cloud Storage automatically. Additionally, when a file has been parsed and the questions, key points, and sources have been added to the Cloud Storage bucket, this information is retrieved by the application and parsed to be displayed to the user.
Challenges we ran into
One of the primary challenges we ran into was that although we specified in our API requests through Langchain that the information generated by the LLM should be returned as JSON, OpenAI was finicky about this aspect and was not as consistent as we would have liked. As a result we initially had to conduct a lot of data preprocessing and parsing to ensure we could convert it to JSON and display it effectively in the User Interface. Another challenge we ran into was managing our Google Cloud resources as the processing of data through cloud functions required more cloud memory than we initially assumed we would require.
Accomplishments that we're proud of
None of us had any previous experience in integrating LLMs into web apps and working with the full stack technologies to effectively implement these. We gained a lot of technical skills as we learned a lot about vector databases and how they tie into LLMs and generative AI in general. Working with the full stack was a great learning experience as we understood how to automate AI processes with cloud functions using Google Cloud. It was also fantastic to work in a group that was motivated and supportive.
What we learned
This was a transformative experience as we all gained first-hand experience with the technology empowering AI models. It was also fascinating seeing how all the parts of the project fit together at the end as different team members collaborated on uniting the entire tech stack into a cohesive application. Throughout the development process, we gained insights into the intricacies of NLP and the importance of context in language. We also learned the value of user feedback in refining the product, ensuring it truly addresses students' needs.
What's next for Fountain
We plan to incorporate adaptive learning algorithms that adjust the quiz difficulty based on user performance. Additionally, we're exploring partnerships with educational institutions to integrate Fountain directly into their learning management systems. Collaboration with professors to input lecture-specific nuances could further enhance the accuracy and relevance of the quizzes.