Pretty much. I do my work on a headless workstation that I SSH into, so it's not... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		jart on March 31, 2023 \| parent \| context \| favorite \| on: Llama.cpp 30B runs with only 6GB of RAM now Pretty much. I do my work on a headless workstation that I SSH into, so it's not like competing with Chrome tabs or anything like that. But I do it mostly because that's what I've always done. The point of my change is you won't have to be like me anymore. Many of the devs who contacted after using my change have been saying stuff like, "yes! I can actually run LLaMA without having to close all my apps!" and they're so happy.

dekhn on April 1, 2023 [–]

Linux has a command to drop caches at runtime (https://www.tecmint.com/clear-ram-memory-cache-buffer-and-sw...) which is VERY useful during debugging.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact