1. You can lazy download and checkout data as you need it. Assuming your build system is sufficiently realized to hide the latency this will save a lot of time as long as accessed data is significantly less then used data (say <80%).
2. You don't need to scan the filesystem to see what has changed. So things like commits, status checks and even builds checking for changes can target just what has actually changed.
Neither of these are particularly complex. The hardest part is integrating with the VCS and build system to take advantage of the change tracking. Git has some support for this (see fsmonitor) build I'm not aware of any build systems that do. (But you still get a lot of benefits without that.)
I have thought about this, but also wondered if it would be as magic without the highly paid team of fantastic SRE and maintainers, and the ridiculous amount of disk and compute available to them.
I imagine it would be as magic as blaze vs bazel out in the wild. That is, you need still someone(s) to do a ton of hard work to make it work right but when it does you do get the magic.
You’re absolutely right - SrcFS and EdenFS were inspirations for SourceFS.
The challenge with those systems is that they’re tightly coupled with the tools, infrastructure, and even developer distros used internally at Google and Meta, which makes them hard to generalize. SourceFS aims to bring that “Piper-like” experience to teams outside Google - but in a way that works with plain Git, Repo, and standard Linux environments.
Also, if I’m not mistaken, neither SrcFS nor EdenFS directly accelerate builds - most of that speed comes from the build systems themselves (Blaze/Buck). SourceFS goes a step further by neatly and simply integrating with the build system and caching/replay pretty much any build step.
The Android example we’ve shown is just one application - it’s a domain we know well and one where the pain is obvious - but we built SourceFS in a way where we can easily integrate with a new build system and speed up other big codebases.
Also you’re spot on that this problem mostly affects big organizations with complex codebases. Here without the infrastructure and SRE support the magic does not work (e.g. think the Redis CVE 10.0 of last week or the AWS downtime of this week) - and hence the “talk to us”.
We plan to gradually share more interesting details about how SourceFS works. If there’s something specific you’d like us to cover - let us know - and help us crowd source our blogpost pipeline :-).
It's a shame that AI is ruining certain phrases, the "You’re absolutely right" was appropriate but I've been trained reading so many AI responses to roll my eyes at that.
WDYM this seems very familiar. At commit deadbeef I don't need to materialize the full tree to build some subcomponent of the monorepo. Did I miss something?
And as for pricing... are there really that many people working on O(billion) lines of code that can't afford $TalkToUs? I'd reckon that Linux is the biggest source of hobbyist commits and that checks out on my laptop OK (though I'll admit I don't really do much beyond ./configure && make there...)
Oh yea, this is "srcfs the idea" but not "srcfs the project".
I.e. this isn't something battel tested for hundreds of thousands of developers 24/7 over the last years.
But a simple commercial product sold by people that liked what they used.
Well, since android is their flagship example, anyone that wants to build custom android releases for some reason.
With the way things are, you don't need billions of code of your own code to maybe benefit from tools that handle billions of lines of code.
Just knowing when stuff happens outside your house when you're not home. Like, someone in the household came back from work, a package arrived, or a cool animal showed up. Or you're home and appreciate a little more notice before a visitor rings the doorbell.
Personally not a compelling enough reason to buy the camera in the first place, but those non crime notifications end up being the most common once it's up.
Because you might want VLAN. Plus you cant just start blasting reply without a preamble, so its still 14 bytes after receiving just the dest MAC. Then you get potential IFG, FEC, scrambling, it all adds up, no way any switch can do 4ns without heavy lawyer talk in the small print.
Why? For example to not have diacritics in month names? Take them as examples as you can easily add them to a shell script to make in work the way you want.
reply