Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing Implementation for Calculating Likelihood Values as per Equation (10) #2

Open
Monohydroxides opened this issue Mar 22, 2024 · 1 comment

Comments

@Monohydroxides
Copy link

Hello, thank you for your outstanding work! In the code, I don't seem to find the implementation for calculating likelihood values as given by equation (10) in the paper. Could you please inform me about when you plan to upload it to the GitHub repository? Or perhaps I missed a part of the code that covers this calculation. Could you please assist me in identifying where the code for calculating likelihood values is located?

@alexanderswerdlow
Copy link

Following up on this, it seems the only perplexity metric in the repo is using the SEDD predictions as GT and measuring the perplexity of a pre-trained GPT-2 model, presumably to measure alignment with that model.

Would it be at all possible to provide the perplexity metric used in the paper?

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants