BitTorrent will solve the distribution problem but not magically provide more storage. Someone still has to foot the bill for storing gigabytes (or terabytes) worth of docker images.
Have a free client that seeds any images that you download, and a paid for one that doesn't. Now you have all those who don't want to pay providing your storage and bandwidth.
This is an excellent use of p2p incentives. Share to pay.
Tricky bit is, for some users, you’ll either abandon them with no way to share or you will still be paying their ingress/egress fees when their client falls back to your TURN server if NAT hole punching fails.
You’ll also have to solve image expiration gracefully. Hosting a “publish as much as you want” append-only ledger isn’t going to scale infinitely. There needs to be garbage collection, rate limiting, fair-use policies, moderation, etc. Otherwise you’re still going to outstrip your storage pool.
This doesn't make sense as an argument at all. If there isn't anyone using the image, no one will have it on their computer... Sure - but that isn't as much of an issue if you have a build file that constructs the image up from more basic parts. Secondly, the popular files get way way faster with the more their used/downloaded.
Torrent is a phenomenal *wonderful* system to distribute Machine Learning weights, docker images, and databases.
It's a developers dream for a basic utility of distributing data.
Potentially ipfs could be useful too, but idk much about it specifically.
One of the most revolutionary and fundamental tools to be made is a basic way / template / paradigm which constructs databases in a replicable way, such that the hash of the code is mapped to the hash of the data. Then the user could either just download the data or reproduce it locally, depending on their system's capabilities, and automatically become a host for that data in the network.
It should probably work like ipfs, with pinning services. You can pin (provide a server that stores and shares the contents) yourself, or pay for a commercial pinning service (or get one from an OSS-friendly org, etc).
I think something like IPFS would be perfect for that, you have some layers pulled in your storage anyways.
big projects could self host easily, as their popularity would quickly give them enough seeds to not need to provide much traffic themselves.
also I think adapting docker way of storing layers as tars is fundamentally broken, maybe with combination with something like ostree as a storage to decrease duplicates we could really cut a lot of storage.
imagine how much unique content does your average docker image have? 1 binary and maybe few text files? rest is probably os and deps anyways.
