Need advice on new hardware build

[sbob84]

Hi all,

I'm new to LLM, recently I had an idea to build a PC to run DeepSeek at home, I read a bunch of threads on Reddit and was about to get a 64 core EPYC Milan + 1024GB DDR-3200 to run the widely used llama.cpp on it, but I came across ktransofrmers.

Now I'm thinking about going with a 56-core Xeon 9480 + ~512GB DDR-4800 + RTX 3090 instead of the quiet old EPYC setup. It's a bit more expensive, but with ktransformers it seems like a better option since the CPU has AMX units, 64GB on-die HBM memory, and 24GB of VRAM on the GPU.

Unfortunately, I couldn't find any information about the 9480 CPU in the context of using it with ktransformers. There are some reports of using it on llama.cpp, but the numbers look disappointing.

I would appreciate it if you could tell me if this would really be the best option for a home DeepSeek inference setup.