Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
A rewrite of the
Forecaster
andForecasterModel
framework, offering greatly improved ease of use for both end users and developers, and consistency with the latest formal definitions of conversational forecasting.Motivation and Context
Forecaster
was initially released as a supplement to the "Trouble on the Horizon" paper (Chang and Danescu-Niculescu-Mizil, 2019), when conversational forecasting models as a concept were in their infancy. It came with several issues. First, it was not user-friendly, having a large number of confusing parameters related to minutiae of the CRAFT model specifically. Relatedly, despite claiming to be general, the interface was too specifically centered around CRAFT. Finally, it is somewhat obsolete as more recent work has put forward a more general formalization of conversational forecasting (see https://www.cs.cornell.edu/~cristian/papers/chang_thesis.pdf) which the existingForecaster
does not fully adhere to.The new implementation is built from the ground up starting from formal definitions as a starting point, has far fewer parameters for the end user to worry about, and is more thoroughly documented. It aims to serve as a starting point not only for end users interested in using forecasting models, but also developers working on new conversational forecasting models, who can use
Forecaster
as a general framework to help bootstrap their work.How has this been tested?
The newly written example notebook, included as part of this pull request, has validated that the full CRAFT fine-tuning pipeline can be reproduced using only the new
Forecaster
API, and gets the expected results.