Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm fairly certain compressing random data is nonsensical. In fact there's a cash prize challenge for doing it that has been unmet for 20 years. https://marknelson.us/posts/2012/10/09/the-random-compressio...

Not sure I can explain it properly myself but it think it's impossible by definition. Intuitively, if you could compress any random data then you should be able to compress 2 bits into 1, which is clearly impossible.



It's possible to compress (some) random data, based on just luck. It's not generally possible to compress any random data.

If I generate a thousand random 1024 byte sequences, I'm sure I could pick a few of them that I can compress, so long as I don't have to compress all of them.

It's also possible that the total of my compressed subset and the uncompressed rest, is smaller than the sum of the sequences! This still doesn't mean I could compress any random sequence, just specific ones. Ones that had lower entropy, by pure luck.

To translate to your analogy: I can make a compressor that compresses 2 bits into 1 BUT it works only works for 50% of all two bit sequences...


You're right... it's impossible.

This quote from the competition sums it up:

"Again, I will emphasize that Challenge 2 is at its heart provably impossible to beat. No program can compress all files of a given size, and the chances of any program being able to compress 1,000 different files of this length is so vanishingly small that it can safely be ruled out, even if every computer on earth was working on it from now until we are engulfed in flames during Sol’s red giant phase."




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: