If you want to inference big LLMs on CPU, you really want those 2x 12 memory cha... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		Tepix on Nov 30, 2024 \| parent \| context \| favorite \| on: Myths and Urban Legends About Dual-Socket Servers If you want to inference big LLMs on CPU, you really want those 2x 12 memory channels a Dual EPYC system offers you for decent speeds.

chipdart on Nov 30, 2024 [–]

> (..) you really want those 2x 12 memory channels a Dual EPYC system offers (...)

I had to check and I was amazed that there are companies selling workstations with dual EPYC processors, providing a whopping 256 CPU cores and over 2TB of DDR5. All in a desktop form factor. Amazing.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact