A question more than an issue...

[sjscotti]

Hi Blake
I just discovered your videos/repos and you do a great job walking viewers through the "how to do" steps of working with AI.
My question is about the hardware/software you are using for the GPT efforts in your repos.
It sounds like you are working on a RYZEN powered PC with a couple of Nvidia graphics cards. Are you running Windows and doing your AI work in WSL (Windows Subsystem for Linux) or do you have a standalone version of linux installed?
I have been using Windows for my exploration of facebookresearch/parlai repos, and it would be easiest to stay in Windows to try out some of your GPT codes if possible. But using WSL would not be too difficult as a fallback. Going to a standalone version of linux would be a lot more effort I think.
Thanks in advance!

P.S. One nice thing about Windows that I am not sure is available with the different flavors of linux on a PC is that when I try to load data into the GPU that is larger than the GPU physical memory, it automatically uses CPU memory as virtual GPU memory (Windows calls it "Shared GPU memory"). So even though I only have an 8GB Nvidia card, I can run models that put up to 40GB on the card memory because the overflow goes to this virtual GPU memory. From my experience using parlai repos, the penalty of using virtual GPU memory isn't too bad as long as what is in the virtual GPU memory is only the optimizer states and not the model weights and gradients.

[mallorbc]

Thanks for watching my videos!
My PC has changed over time. My CPU in all my videos has either been a Ryzen 3600 or Ryzen 5950x. I have both a 3090 and 3060 but typically use the 3090 for larger models out of speed and or need. I now have 128 GB of RAM, which is needed for finetuning GPTJ.
I dual boot my system with Windows and Ubuntu 20.04 . Most of my videos are on Ubuntu 20.04 as things seem easier on that system and I am more familar with Linux at this point than Windows. I have had issues with WSL so I avoid it for anything compute-heavy(but its nice for SSH and the like).

I can not guarantee that my videos work on Windows, especially the newer the tech is.

I was not aware of that. If true then limitations on loading models would be greatly reduced. I will have to look into that, especially with even larger models coming out.