Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Forecaster framework #217

Merged
merged 14 commits into from
Jul 31, 2024
Merged

New Forecaster framework #217

merged 14 commits into from
Jul 31, 2024

Conversation

jpwchang
Copy link
Collaborator

Description

A rewrite of the Forecaster and ForecasterModel framework, offering greatly improved ease of use for both end users and developers, and consistency with the latest formal definitions of conversational forecasting.

Motivation and Context

Forecaster was initially released as a supplement to the "Trouble on the Horizon" paper (Chang and Danescu-Niculescu-Mizil, 2019), when conversational forecasting models as a concept were in their infancy. It came with several issues. First, it was not user-friendly, having a large number of confusing parameters related to minutiae of the CRAFT model specifically. Relatedly, despite claiming to be general, the interface was too specifically centered around CRAFT. Finally, it is somewhat obsolete as more recent work has put forward a more general formalization of conversational forecasting (see https://www.cs.cornell.edu/~cristian/papers/chang_thesis.pdf) which the existing Forecaster does not fully adhere to.

The new implementation is built from the ground up starting from formal definitions as a starting point, has far fewer parameters for the end user to worry about, and is more thoroughly documented. It aims to serve as a starting point not only for end users interested in using forecasting models, but also developers working on new conversational forecasting models, who can use Forecaster as a general framework to help bootstrap their work.

How has this been tested?

The newly written example notebook, included as part of this pull request, has validated that the full CRAFT fine-tuning pipeline can be reproduced using only the new Forecaster API, and gets the expected results.

@seanzhangkx8 seanzhangkx8 merged commit 1b1a385 into master Jul 31, 2024
9 checks passed
@seanzhangkx8 seanzhangkx8 deleted the forecaster-rewrite branch July 31, 2024 03:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants