Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google CoLab tweaks #296

Closed
PhilipQuirke opened this issue Nov 25, 2024 · 1 comment · Fixed by #299
Closed

Google CoLab tweaks #296

PhilipQuirke opened this issue Nov 25, 2024 · 1 comment · Fixed by #299

Comments

@PhilipQuirke
Copy link

When reproducing the nnsight https://nnsight.net/notebooks/tutorials/activation_patching/ tutorial on Google Colab, I ran into two issues:

  1. The result visualizations did not show anything. The cause turned out to be:
    pio.renderers.default = "plotly_mimetype+notebook_connected+colab+notebook"
    Replacing this line with the following resolved the issue:
    pio.renderers.default = "colab"

  2. Using a Google Colab A100 instance with 40Gb of GPU RAM I got an out of memory error. The cause turned out to be this line of code which saves large structures:
    _ioi_patching_results.append(patched_result.save())
    Replacing this line with the following reduced the memory usage from 40+GB to 3GB:
    _ioi_patching_results.append(patched_result.item().save())
    One line of code later in the tutorial needs to be deleted to cope with the above change:
    ioi_patching_results = util.apply(ioi_patching_results, lambda x: x.value.item(), Proxy)

Please consider incorporating these changes to your tutorial.

@AdamBelfki3 AdamBelfki3 linked a pull request Dec 2, 2024 that will close this issue
@AdamBelfki3
Copy link
Collaborator

Hi Philip! Thanks for bringing up these changes, they will be incorporated.

Please see the linked PR. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants