On Karpathy's Pytorch course, they found out Numerical instability in Google Colab, but no issues on local device.

[JonathanSum]

We tried to hand calculate the derivative and do a comparing to the Pytorch derivative and our hand calculated.
It didn't show issues on the local device, but it showed issues on the colab.

karpathy/nn-zero-to-hero#13
I got no answer from the Pytorch forum. I am thinking maybe the issue is colab itself?

[cperry-goog]

I'm guessing this is version skew between dependencies and will be very hard to debug. Is it possible to ensure your dependencies are all the same version as our base image? If so, you can share a minimal repro notebook.

[JonathanSum]

I'm guessing this is version skew between dependencies and will be very hard to debug. Is it possible to ensure your dependencies are all the same version as our base image? If so, you can share a minimal repro notebook.

Colab notebook: https://colab.research.google.com/drive/1HmZ8bgtAfvyMaZyu3Sr1Bgxsj35jitTs?usp=sharing

[mco-gh]

How about setting a fixed number of digits of precision in your maxdiff calculation?