Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
startupsfail
on March 31, 2023
|
parent
|
context
|
favorite
| on:
Llama.cpp 30B runs with only 6GB of RAM now
Hmm, can you try running iotop, to see how much is being red from disk? Is it 20Gb or 6Gb? Maybe the prefetch is able to fill in before page faults are happening? Or maybe you are hitting the disk cache?
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search: