> If the unwrap hadn't caused an exit, the process would've run out of memory
It was trying to push an element into a full ArrayVec. The options are:
- Blindly write off the end of the array. Obviously no one wants this, despite the decades of tradition...
- Panic and unwind, as the program actually did in this case.
- Return an error.
Some folks assume that returning an error instead of unwinding would've been better. But my assumption is that the outcome would've been the same. I think the issue came up when loading configs, which isn't usually a recoverable situation. If you have an "invalid config error", you're probably just going to return that all the way up, which is effectively the same outcome as unwinding: your process exits with an error code. There are cases where the difference matters a lot, but I don't think this was one of them.
The real gap seems to be why it took hours for folks to notice that this service was crash looping. That should normally be really prominent in alerts and dashboards. (Probably part of the story is that alerts were firing all over the place. Tough day at the office.)
> similar to what Bao encoding is used for in BLAKE3
In my mind Sakura and Bao are doing very different things. Sakura is a general framework for defining sound hash functions, while Bao is a BLAKE3-specific interleaving of hash function input and output that let's you do partial/streaming verification.
There was an interesting issue where it turned out that the Sakura security properties (which I studied while I was working on Bao, very helpful) weren't sufficient for what Bao wanted to do. I didn't realize that until a couple colleagues pointed it out years later. Luckily it didn't turn out to be a break in practice: https://github.com/oconnor663/bao/issues/41
BLAKE3 does reduce the round count relative to BLAKE2, and the underlying compression functions are similar enough that it is an apples-to-apples comparison. Our rationale for doing that was described in https://eprint.iacr.org/2019/1492.pdf, which also argued that Keccak could reduce their round count even farther than they did.
This is a difference between BLAKE3 and most other hash functions. In the usual arrangement ("Merkle–Damgård"), each block depends on the previous one, so the only way to verify some "slice" of the input is to re-hash the whole thing. But when you arrange the input into a tree shape (a "Merkle tree") instead, suddenly the right half of the tree does not depend on the left half until the very last step at the very top. If you give me the input to that last step, I can verify that it matches the root hash that I know, now I have the hashes ("chaining values") I'd need to verify either the left half or the right half without the other. Then I do our favorite trick in computer science, which is to recursively apply that same procedure all the way down, until I have an efficient "path" to whatever part of the tree I actually care about.
> to rewrite some feature for a tiny security benefit
For what it's worth, the zero->one introduction of a new language into a big codebase always comes with a lot of build changes, downstream impact, debate, etc. It's good for that first feature to be some relatively trivial thing, so that it doesn't make the changes any bigger than they have to be, and so that it can be delayed or reverted as needed without causing extra trouble. Once everything lands, then you can add whatever bigger features you like without disrupting things.
> They did this by not breaking the universe in these major updates
I don't think the amount of breakage per se was the problem with Python 3. I think the problem was that for a long time (until u"" strings in 3.3? four years after 3.0?) there was ~no way for a library to support both 2 and 3 in a single codebase. That meant that each project had to either maintain a fork, or do a "flag day" migration, which in practice meant you couldn't migrate until all your dependencies were migrated. Big mistake in retrospect.
When I was doing undergrad CS in 2006, we had a choice between Perl and Python for our scripting assignments. As far as I know, no one chose Perl. It's comparatively a huge pain to get ramped up on, and the promised payoff is that you get all these implicitly stateful sigils that let you write contest-winning one-liners. But by the early 2000's, the culture was getting up to speed on obvious-in-retrospect principles like "avoid global variables" and "you should give your variables names that someone else can understand", and even as students we could all see that Perl wasn't going to be the future.
I chose Perl first. At that time around early 2000s it was the more popular. I liked how my scripts were short and compact but then I I had noticed that I couldn't understand what I did week or so later. The "write-only language" joke was kind of true for me. Then remember finding Python and rewriting all my script in Python and have been using it ever since.
The boring answer is that standard atomics didn't exist until C++11, so any compiler older than that didn't support them. I think most platforms (certainly the popular desktop/server platforms) had ways to accomplish the same thing, but that was up to the vendor, and it might not've been well documented or stable. Infamously, `volatile` used to be (ab)used for this a lot before we had proper standards. (I think it still has some atomic-ish properties in MSVC?)
I think it's pretty rare to do a straight up atomic load of a refcount. (That would be the `use_count` method in C++ or the `strong_count` method in Rust.) More of the time you're doing either a fetch-add to copy the pointer or a fetch-sub to destroy your copy, both of which involve stores. Last I heard the fetch-add can use the "relaxed" atomic ordering, which should make it very cheap, but the fetch-sub needs to use the "release" ordering, which is where the cost comes in.
I imagine it's kind of like "What's stopping someone from forging your signature on almost any document?" The point is less that it's hard to fake, and more that it's a line you're crossing where everyone agrees you can't say "oops I didn't know I wasn't supposed to do that."
It was trying to push an element into a full ArrayVec. The options are:
- Blindly write off the end of the array. Obviously no one wants this, despite the decades of tradition...
- Panic and unwind, as the program actually did in this case.
- Return an error.
Some folks assume that returning an error instead of unwinding would've been better. But my assumption is that the outcome would've been the same. I think the issue came up when loading configs, which isn't usually a recoverable situation. If you have an "invalid config error", you're probably just going to return that all the way up, which is effectively the same outcome as unwinding: your process exits with an error code. There are cases where the difference matters a lot, but I don't think this was one of them.
The real gap seems to be why it took hours for folks to notice that this service was crash looping. That should normally be really prominent in alerts and dashboards. (Probably part of the story is that alerts were firing all over the place. Tough day at the office.)
reply