@lxpz It would be great to do a follow up to this blog post with the latest Peergos. All the issues with baseline bandwidth and requests have gone away, even with federation on. The baseline is now 0, and even many locally initiated requests will be served directly from a Peergos cache without touching S3.
You might be interested in Peergos [0][1] (creator here) which has official Linux apps, is E2EE, fully open source (including the server), and self-hostable. It's also recommended by privacy guides: https://www.privacyguides.org/en/cloud/#peergos
You might be interested in Peergos [0][1] which is E2EE, fully open source (including the server), and self hostable. We've been audited by Cure53 and Radically Open Security.
Yes your files stay on your server, unless you share them with a friend on another server. It basically just uses libp2p as an internode communication protocol, and you need auth to retrieve ciphertext blocks.
Cool thanks! So does that mean if I want more durability/copies of the data I can just link up additional servers and they’ll sync all the data via IPFS? So like if I started with just my home NAS, then later added a VPS, and later a computer sitting at my brother’s house, I would end up with a copy of all data in all 3 locations and could access it if at least one of them was online at any given time?
At the moment if your client accesses a server with a mirror, and your primary is offline, then you can read, but not write. Writes are proxied to your primary, and thus need your primary to be accessible.
I couldn't find an email in your bio. You can reach me via the email at the bottom of my website (in my HN bio).
Looking through the docs on Peergos, it looks like it's built on top of IPFS.
I've been meaning to write some documentation for Blobcache comparing it to IPFS. I can give a quick gist here.
Blobcache Volumes are similar to an IPNS name, and the set of IPFS blocks that can be transitively reached from it.
A significant difference is that Blobcache Volumes expose a transaction API with serializable isolation semantics.
IPFS provides distributed, available-but-inconsistent, cryptographically signed cells.
IPFS chooses availability, and Blobcache chooses consistency.
A Blobcache Volume corresponds to a specific entity maintained and controlled by a specific Node.
An IPFS name exists as a distributed entity on the network.
Most applications need some sort of consistent transactional cell (even if they don't realize it), but in order to be useful, inconsistent-but-available cells have to be used carefully in an application specific way.
I blame this required application-specific care for the lack of adoption of CRDTs.
There's a long tail of other differences too.
IPFS was pretty badly behaved the last time I used it, trying to configure my router, and creating lots of connections to other nodes.
Blobcache is more like a web browser; it creates transient connections in immediate response to user actions.
That whole ecosystem is filled with complicated abstractions. Just as an example, the Multihash format is pervasive.
It amounts to a tag for the algorithm used to create a hash, and then the hash output.
I'd rather not have that indirection.
All the hashes in Blobcache are 256 bits, and you set the algorithm per Volume.
In Go that means the hashes can just be `[32]byte` instead of a slice and a tag and a table of algorithms.
I haven't used IPFS in a while, but I became pretty familiar with it awhile ago. Had I been able to build any of the stuff I was interested in on top of it, I probably wouldn't have written Blobcache.
The good news is Peergos also has serializable transactional modifications. This comes from us storing signed roots in a db on your home server (not ipns). We also have our own minimal ipfs implementation that uses 1000x fewer resources than kubo, aka go-ipfs.
The same API part isn't surprising, content addressed stores are the most natural way to accept encrypted data.
The public storage networks are targeting a different use case than Blobcache though, which I think of as a private or web-of-trust storage network. To use a cryptocurrency backed storage solution, one must manage accounts, or a wallet of transaction outputs, connect to unknown parties on the internet, and pay for the increased redundancy.
There's also legal risk, depending on the jurisdiction, when allowing untrusted parties to store arbitrary information on one's devices.
I don't want to consult the global economy in order to make use of my extra hard drives, which would otherwise be idle.
re legal risks: no one knows what their machines are storing in swarm without also holding a key and a hash. the pieces are distributed based on the hash of the encrypted value.
That looks really promising. It checks a lot of the boxes I already had in mind for such a system, like being able to continue a thread without exposing the whole thing to untrusted parties
https://garagehq.deuxfleurs.fr/blog/2022-ipfs/
Let's talk!