Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Influence for sentiment analysis notebook #483

Merged
merged 22 commits into from
Jan 16, 2024

Conversation

Xuzzo
Copy link
Collaborator

@Xuzzo Xuzzo commented Dec 29, 2023

Description

This PR closes #452

Changes

  • Adds new notebook, influence_sentiment_analysis.ipynb, which showcases ekfac for LLMs
  • Fixes small issue in Ekfac implementation
  • Adds methods to EkfacInfluence that return influence values for each layer
  • Adds tests for new by_layer methods

Checklist

  • Wrote Unit tests (if necessary)
  • Updated Documentation (if necessary)
  • Updated Changelog
  • If notebooks were added/changed, added boilerplate cells are tagged with "tags": ["hide"] or "tags": ["hide-input"]

@Xuzzo Xuzzo self-assigned this Dec 29, 2023
@Xuzzo Xuzzo marked this pull request as draft December 29, 2023 00:36
@Xuzzo Xuzzo changed the base branch from develop to feature/ekfac_new_framework December 29, 2023 00:37
@Xuzzo Xuzzo marked this pull request as ready for review January 3, 2024 23:30
@Xuzzo Xuzzo requested a review from schroedk January 3, 2024 23:30
@schroedk
Copy link
Collaborator

schroedk commented Jan 4, 2024

You have to add the new notebook to the mkdocs.yml

      - For sentiment analysis: examples/influence_sentiment_analysis.ipynb

@Xuzzo Xuzzo force-pushed the feature/ekfac_new_framework branch from 9c5c8e7 to 5307c1a Compare January 4, 2024 15:08
@Xuzzo Xuzzo changed the base branch from feature/ekfac_new_framework to develop January 8, 2024 13:14
AnesBenmerzoug
AnesBenmerzoug previously approved these changes Jan 14, 2024
Copy link
Collaborator

@AnesBenmerzoug AnesBenmerzoug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late review. Overall it looks good to me.
Feel free to merge it once the comments I left are addressed.

@AnesBenmerzoug
Copy link
Collaborator

I just had a look at the rendered notebook in the documentation and some things are off:

  • The table of content in the beginning doesn't render properly

image

  • Plotting code should be hidden using the 'hide-input' tag

image

@Xuzzo Xuzzo merged commit 3e7b4c8 into develop Jan 16, 2024
18 checks passed
@mdbenito mdbenito deleted the feature/ekfac_notebook_new_framework branch February 13, 2024 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Write EKFAC notebook
3 participants