>It has a great backwards-compatibility story -- a lot better than Python's, for example, which I use in my current dayjob.
I think this is a really important point. I worked intensively with Python for years, but it was a deep disappointment to see how little the core devs and the community care about backwards compatibility in the 2/3 transition, and things have not improved. That might be due to what Python is dominantly used for today (The Numeric/Numpy transition which happened around 2000 was totally different). Breaking things every few years might be OK for unicorn startups or companies that move fast and, well, break things, but there is a lot of valuable programming from people and in organisations which can simply not afford, or do not see it as a priority to re-write mature algorithms every few years.
One feeling I have is that while Clojure is incredibly tasteful, well-designed and elegant, it is less focused on Correctness as Racket. And this gives me the feeling that if I make some mistake (say, erroneously swapping arguments of a complex function that e.g. computes on list as sets), I do not know what is going to happen.
I think Guile is a good example that one can have an SBCL-like workflow of live coding (with debugging and jumping right into a stack trace) in Scheme.
I have heard that this is not possible because Racket needs to make a stricgter separation of compile time and run time... but I am not sure whether this is actually true.
Regarding the stricter separation, Racket and its module system have nicely clear concepts of phases. (I really like how the Racket module system has been used these to benefit the language.)
But, though I don't know current Racket internals, I'm not sure those need to be a barrier to monkeypatching that approaches CL.
Other than optimizations that make some debugging info and dynamic changes difficult-to-impossible, if the system had a mode to disable these optimizations (or to maintain some information in parallel that permits a cascading unraveling of optimizations affected by a change), I'm not aware of a fundamental reason the system couldn't support more loose poking at things.
Though, one implementation complication that Schemes do have is first-class continuations.
I guess one can think of Clojure as a very Scheme-affine Lisp. Which is not surprising since Scheme was a kind of distilled iteration of Common Lisp as a common denominator.
Are there any Schemes which are as "Batteries included" as Clojure with broad and deep library support? (I think Racket has quite good support!)
I am using Racket once in a while, and I like it. Though I have also learned that some Lisps, and especially Common Lisps (and, I suspect, Guile), are a bit under-hyped.
But a question to people who know it better: What are features of Racket which at first might not seen practical, relevant, or useful, but are surprisingly useful for programming when one gives them a try?
The web of trust is a really nice idea, but it works badly against that kind of attacks. Just consider that in the real world, most living people (all eight billions) are linked by only six degrees of separation. It really works, for code and for trusted social relations (like "I lend you 100 bucks and you pay me them back when you get your salary") mostly when you know the code author in person.
This is also not a new insight. In the beginning of the naughties, there was a web site named kuro5hin.org, which experiemented with user ratings and trust networks. It turned out impossible to prevent take-overs.
IIRC, kuro5hin and others all left out a crucial step in the web-of-trust approach: There were absolutely no repercussions when you extended trust to somebody who later turned out to be a bad actor.
It considers trust to be an individual metric instead of leaning more into the graph.
(There are other issues, e.g. the fact that "trust" isn't a universal metric either, but context dependent. There are folks whom you'd absolutely trust to e.g. do great & reliable work in a security context, but you'd still not hand them the keys to your car)
At least kuro5hin modeled a degradation of trust over time, which most models still skip.
It'd be a useful thing, but we have a long way to go before there's a working version.
Once you add punishment for handing out trust to bad actors, even in good faith (which you can't prove/disprove anyway), then you also need to somehow provide siginificant rewardsf for handing out trust to good actors - otherwise everyone is going to play it safe and not vouch for anyone and your system becomes useless.
There were experiments back in the day. Slashdot had one system based on randomly assigned moderation duty which worked pretty great actually, except that for the longest time you couldn't sort by it.
Kuro5hin had a system which didn't work at all, as you mentioned.
But the best was probably Raph Levien's Advogato. That had a web of trust system which actually worked. But had a pretty limited scope (open source devs).
Now everyone just slaps an upvote/downvote button on and calls it a day.
clickhouse has pretty good github_events dataset on playground that folks can use to do some research - some info on the dataset https://ghe.clickhouse.tech/
Yeah. It would be interesting to see who adopted to the compromised versions and how quickly, compared to how quickly they normally adopt new versions (not bots pulling upgrades, but how quickly maintainers approve and merge them)
If there were a bunch of people who adopted it abnormally fast compared to usual, might point to there being more "bad actors" in this operation (said at the risk of sounding paranoid if this turns out to be a state run thing)
> If the code is complex, there could be many more exploits hiding.
Then the code should not be complex. Low-level hacks and tricks (like pointer juggling) should be not allowed and simplicity and readability should be preferred.
Yes, but my point was that at the level of performance tools like this are expected to operate at, it’s highly probable that you’ll need to get into incredibly esoteric code. Look at ffmpeg – tons of hand-written Assembly, because they need it.
To be clear, I have no idea how to solve this problem; I just don’t think saying that all code must be non-hacky is the right approach.
Performance can be bought with better hardware. It gets cheaper and cheaper every year. Trustworthiness cannot be purchased in the same way. I do not understand why performance would ever trumph clean code, especially for for code that processes user provided input.
This attitude is how we get streaming music players that consume in excess of 1 GiB of RAM.
Performant code needn’t be unclean; it’s just often using deeper parts of the language.
I have a small project that became absolute spaghetti. I rewrote it to be modular, using lots of classes, inheritance, etc. It was then slower, but eminently more maintainable and extensible. I’m addressing that by using more advanced features of the language (Python), like MemoryView for IPC between the C libraries it calls. I don’t consider this unclean, but it’s certainly not something you’re likely to find on a Medium article or Twitter take.
I value performant code above nearly everything else. I’m doing this for me, there are no other maintainers, and it’s what I enjoy. You’re welcome to prioritize something else in your projects, but it doesn’t make other viewpoints objectively worse.
Performant code does not need to be unclean, exactly! My original point was just to not put performance on a pedestal. Sure, prioritize it, but correct and clean should come first - at least for foundational libraries that others are supposed to build upon.
I maintain ML libraries that run on microcontrollers with kilobytes of memory, performance is a friend of mine ;)
I suggest you run your browsers Javascript engine in interpreter mode to understand how crippling the simple and sraight forward solution is to performance.
I guess because at server farm level, performance/efficiency translate to real million USD savings. In general, at scale ends (the cloud and the embedded) this matters a lot. In resource limited environments like raspberry pi, this design philosophy wins over many users between DIY and the private industry.
I hate this argument. If current hardware promises you a theoretical throughput of 100 MB/s for an operation, someone will try to hit that limit. Your program that has no hard to understand code but gives me 5 MB/s will loose in the face of a faster one, even if that means writing harder to understand code.
No, but often it is far worse than 95%. A good example is random.randint() vs math.ceil(random.random() * N) in Python. The former is approximately 5x slower than the latter, but they produce effectively the same result with large enough values of N. This isn’t immediately apparent from using them or reading docs, and it’s only really an issue in hot loops.
Another favorite of mine is bitshifting / bitwise operators. Clear and obvious? Depends on your background. Fast as hell? Yes, always. It isn’t always needed, but when it is, it will blow anything else out of the water.
Bitwise is highly context dependent. There are simple usages like shifts to divide/multiply by 2. Idiomatic patterns that are clean when wrapped in good reusable and restricted macros, like for common registers manipulation in microcontrollers. And other uses that are anything from involuntary obfuscation to competition grade obfuscation.
> There are simple usages like shifts to divide/multiply by 2.
Clean code should not do that as the compiler will do that.
Clean code should just say what it wants to do, not replace that with low-level performance optimizations. (Also wasn't performance to be obtained from newer hardware?)
Faster and more complex hardware can also have bugs or back doors, as can cheaper hardware. That said, I'm not happy with buggy and untrustworthy code either.
If this is a conspiracy or a state-sponsored attack, they might have gone specifically for embedded devices and the linux kernel. Here archived from tukaani.org:
> XZ Embedded is a relatively small decompressor for the XZ format. It was developed with the Linux kernel in mind, but is easily usable in other projects too.
> *Features*
> * Compiled code 8-20 KiB
> [...]
> * All the required memory is allocated at initialization time.
This is targeted at embedded and real-time stuff. Could even be part of boot loaders in things like buildroot or RTEMS. And this means potentially millions of devices, from smart toasters or toothbrushes to satellites and missiles which most can't be updated with security fixes.
One scenario for malicious code in embedded devices would be a kind of killswitch which listens to a specific byte sequence and crashes when encountering it. For a state actor, having such an exploit would be gold.
One of my complaints about so many SciFi stories is the use of seemingly conventional weapons. I always thought that with so much advanced technology that weapons would be much more sophisticated. However if the next "great war" is won not by the side with the most destructive weapons but by the side with the best kill switch, subsequent conflicts might be fought with weapons that did not rely on any kind of computer assistance.
This is eerily similar to Einstein's (purported) statement that if World War III was fought with nuclear weapons, World War IV would be fought with sticks and stones. Similar, but for entirely different reasons.
I'm trying to understand why the characters in Dune fought with swords, pikes and knives.
> I'm trying to understand why the characters in Dune fought with swords, pikes and knives.
At least part of the reason is that the interaction between a lasgun and a shield would cause a powerful explosion that would kill the shooter too. No one wants that and no one will give up their shield, so they had to go back to melee weapons.
No, there is a in-world reason at least for no drones. Wikipedia:
> However, a great reaction against computers has resulted in a ban on any "thinking machine", with the creation or possession of such punishable by immediate death.
tl;dr - Machine intelligences existed in Dune history, were discovered to be secretly controlling humanity (through abortion under false pretenses, forced sterilization, emotional/social control, and other ways), then were purged and replaced with a religious commandment: "Thou shalt not make a machine in the likeness of a human mind"
No, and there is a (piloted) drone attack in the first book -- Paul is attacked by a hunter-seeker.
The reason nobody tries to use the lasgun-shield interaction as a weapon is because the resulting explosion is indistinguishable from a nuclear weapon, and the Great Convention prohibits the use of nukes on human targets.
Just the perception of having used a nuclear device would result in the House which did so becoming public enemy #1 and being eradicated by the Landsraad and Sardaukar combined.
@Potro: If you liked the movie, read the books. I don't read a lot anymore, but during sick leave I started with the first book. Didn't stop until I finished the main story, including the sequels by Frank Herbert's son about a month later. That's like... uh... nine books?
In the book Paul is attacked by an insect drone while in his room. The drone was controlled by a Harkonnen agent placed weeks in anticipation inside a structure of the palace so it was also a suicide attack as the agent had no chance to escape and would die of hunger/thirsty if not found.
It's related to excessive coupling between modules and low coherence.
There is a way for programs to implement the systemd readiness notification protocol without using libsystemd, and thus without pulling in liblzma, which is coupled to libsystemd even though the readiness notification protocol does not require any form of compression. libsystemd provides a wide range of things which have only weak relationships to each other.
There are in fact two ways, as two people independently wrote their own client code for the systemd readiness notification protocol, which really does not require the whole of libsystemd and its dependencies to achieve. (It might be more than 2 people nowadays.)
This is only evidence that libsystemd is popular. If you want to 0wn a bunch of systems, or even one particular system but make it non-obvious, you choose a popular package to mess with.
BeOS isn't getting a lot of CVEs attached to it, these days. That doesn't mean its good or secure, though.
It's easy to have your existing biases validated if you already dislike systemd. The reality is that systemd is much more coherently designed than its predecessors from a 'end user interface' point of view, hence why its units are largely portable etc. which was not the case for sysvinit.
The reality is that it is not systemd specifically but our modern approach to software design where we tend to rely on too much third party code and delight in designing extremely flexible, yet ultimately extremely complex pieces of software.
I mean this is even true as far as the various CPU attack vectors have shown in recent years, that yes speculative execution is a neat and 'clever' optimization and that we rely on it for speed, but that maybe that was just too clever a path to go down and we should've stuck with simpler designs that would maybe led to slower speedups but a more solid foundation to build future CPU generations on.
Let's be real, sshd loading random libraries it doesn't actually need because distros patched in a kitchen sink library is inexcusable. That kitchen sink library is libsystemd and it follows the same kitchen sink design principle that systemd-opponents have been criticising all along. But its easier to accuse them of being biased rather consider that maybe they have a point.
People hate systemd from an ethical, philosophical, and ideological standpoint.
People love systemd for the efficiency, economics, etc.
It's like ideal vs production.
That is just technical disagreements and sour grapes by someone involved in a competing format (Lzip).
There’s no evidence Lasse did anything “wrong” beyond looking for / accepting co-maintainers, something package authors are taken to task for not doing every time they have life catching up or get fed up and can’t / won’t spend as much time on the thing.
Yes, nothing points to the inventor of the format and maintainer for decades has done anything with the format to make it suspect. If so, the recent backdoor wouldn't be needed.
It's good to be skeptic, but don't drag people through the mud without anything to back it up.
If a project targets a high-profile, very security sensitive project like the linux kernel from the start, as the archived tukaani web site linked above shows, it is justified to ask questions.
Also, the exploit shows a high effort, and a high level of competence, and a very obvious willingness to play a long game. These are not circumstances for applying Hanlon's razor.
Are you raising the same concerns and targeting individuals behind all other sensitive projects? No, because that would be insane.
It's weird to have one set of standards to a maintainer since 2009 or so, and different standards for others. This witch hunt is just post-hoc smartassery.
Yes, I think if a project has backdoors and its old maintainers are unable to review them, I am more critical than with normal projects. As said, compression is used everywhere and in embedded systems, it touches a lot of critical stuff. And the project went straight for that since the beginning.
And this is in part because I can not even tell for sure that he even exists. If I had met him a few times in a bar, I would be more inclined to believe he is not involved.
> As said, compression is used everywhere and in embedded systems, it touches a lot of critical stuff. And the project went straight for that since the beginning.
> You appeal to trust people and give them the benefit of doubt which is normally a good thing. But is this appropiate here?
Yes.
Without evidence to the contrary there is no reason to believe Lasse has been anything other than genuine so all you're doing is insulting and slandering them out of personal satisfaction.
And conspiratorial witch hunts are actively counter-productive, through that mode of thinking it doesn't take much imagination to figure out you are part of the conspiracy for instance.
1. An important project has an overburdened / burnt out maintainer, and that project is taken over by a persona who appears to help kindly, but is part of a campaign of a state actor.
2. A state actor is involved in setting up such a project from the start.
The first possibility is not only being an asshole to the original maintainer, but it is also more risky - that original maintainer surely feels responsible for his creation and could ring alarm bells. This is not unlikely because he knows the code. And alarm bells is something that state actors do not like.
The second possibility has the risk of the project not being successful, which would mean a serious investment in resources to fail. But that could be countered by having competent people working on that. And in that case, you don't have any real persons,just account names.
I don't think state actors would care one bit about being assholes. Organized crime black hats probably wouldn't either.
The original maintainer has said in the past, before Jia Tan's increased involvement and stepping up as a maintainer, that he couldn't put as much into the project due to mental health and other reasons [1]. Seems to fit possibility number one rather well.
If you suspect that Lasse Collin was somehow in it from the start, that'd mean the actor orchestrated the whole thing about mental health and not being able to keep up with sole maintainership. Why would they even do that if they had the project under their control already?
Of course we don't know what's really been happening with the project recently, or who's behind the backdoor and how. But IMO creating suspicions about the original maintainer's motives based entirely on speculation is also a bit assholey.
More layers of obfuscation. For example in order to be able to attribute the backdoor to a different party.
It is of course also possible that Lasse Collins is a nice real person who just has not been able to review this. Maybe he is too ill,or has to care for an ill spouse, or perhaps he is not even alive any more. Who knows him as a person (not just an account name) and knows how he is doing?
That is kinda crazy - state actors don't need to care about that level of obfuscation. From a state's perspective the situation here would be simple - hire a smart & patriotic programmer to spend ~1+ years maintaining an important package, then they slip a backdoor in. There isn't any point in making it more complicated than that.
They don't even need plausible deniability, groups like the NSA have been caught spying on everyone and it doesn't hurt them all that much. The publicity isn't ideal. But it only confirms what we already new - turns out the spies are spying on people! Who knew.
There are probably dozens if not hundreds of this sort of attempt going on right now. I'd assume most don't get caught. Or go undetected for a many years which is good enough enough. If you have government money on the budget, it makes sense to go with large-volume low-effort attempts rather than try some sort of complex good-cop-bad-cop routine.
You're correct about a great many things.
State actors do things in broad-daylight, get exposed, and it's no fuss to them at all.
But that depends on which "sphere of influence" you live in.
Russia and China have made major changes to key parts of their critical infrastructure based on revelations that might only result in a sub-committee in US Congress.
But to establish a significant contributor to a key piece of software, not unlike xz, is an ideal position for a state actor.
The developer doesn't even need to know who/why, but they could be financially/ideologically aligned.
This is what intelligence officers do. They manage real human assets who exist naturally.
But to have someone long-established as an author of a project is the exact type of asset they want. Even if they push the code, people immediately start considering how it could have been done by someone else.
Yes, it's conspiratorial/paranoid thinking but there's nothing more paranoid than state intelligence trade craft.
It makes me wonder. Is it possible to develop a robust Open Source ecosystem without destroying the mental health of the contributors? Reading his posting really made me feel for him. There are exceedingly few people who are willing do dedicate themselves to developing critical system in the first place. Now there is the burden of extensively vetting every volunteer contributor who helps out. This does not seem sustainable. Perhaps users of open source need to contribute more resources/money to the software that makes their products possible.
False dichotomy much? It doesn't have to be a motivated state actor pulling the strings from the begging. It could also just be some guy, who decided he didn't care anymore and either wanted to burn something or got paid by someone (possibly a state actor) to do this.
Recall that the original maintainer had mental health issues and other things that likely led to the perceived need to bring on someone to help maintain xz.
This brings up some integrity questions about you and other people bringing forth accusations in order to make the original maintainer feel pressure to bring on someone else to replace the one that inserted a backdoor after several years of ostensibly legitimate commits.
Hopefully this helps you see that these sorts of accusations are a slippery slope and unproductive. Heck, you could then turnaround and accuse me of doing something nefarious by accusing you.
I don’t stalk all of your social media posts, so from my perspective I don’t see any of the solutions you’ve posted elsewhere — which brings up a good point to keep in mind: none of us see the complete picture (or can read minds to know what someone else really thinks).
The possibility can be kept in mind and considered even if it isn’t being actively discussed. I think in this case, most people think he is not malicious — and feel that unless new compelling evidence to show otherwise appears, potentially starting a harmful rumor based on speculation is counterproductive.
You might not be trying to start a rumor, but other people could when they try to answer the questions from a place of ignorance — if you take a look at the comments on a gist summarizing the backdoor, there are quite a few comments by z-nonymous that seem to be insinuating that other specific GitHub users are complicit in things by looking at their commits in various non-xz repositories.
No one is running cover, just that most information so far points to the original maintainer not knowing that the person brought on to help out had ulterior motives, and likely wasn’t even who they purported to be. If you were running an open source project and facing burnout as the sole maintainer, I’d imagine you’d exercise perfect judgement and do a full background check on the person offering to help? I think many of us would like to believe we’d do better, but the reality is, most of us would have fallen for the same trick. So now imagine having to deal with the fallout not just on the technical side, but also the never-ending questions surrounding your professional reputation that people just keep bring up — sounds like a recipe for depression, possibly even suicidal thoughts.
I am running an open source project. Yes if someone was eager to help and was making changes to things that involved security, I would make them doxx themselves and submit to a background check
Well, good for you being one of the few exceptions who would make everyone submit themselves to a proper background check (presumably also covering the cost) before giving any write/commit access to the repo. That’s more than even most large open source projects do before giving access.
Thanks, but you assume too much. I outlined the circumstances under which i would require a background check, so you might want to reread. any other questions?
As I understand it Jia was contributing things like tests, not making changes that involve “security”. They just turned the commit, and eventual ability to make releases on the xz GitHub after “earning” more trust (+ access to GitHub pages hosted under tukaani domain), into something they could use to insert a backdoor.
No questions. Anyone can become a victim to social engineering — I believe the short answer to your question about all the downvotes is that a lot of people recognize how they could have fallen for something similar, and empathize that Lasse is likely now going through a rather difficult time.
I have no question about the downvotes, bud. You're very verbose. Still not sure why you revived an account you haven't commented with in 6 years just to run cover. I find you to be a highly suspicious individual and I really have nothing more to say to you.
I suppose I think verbose-ness will help people see the other side of things. I think I was also trying to convince myself that you aren’t just into conspiracy theories, but given that you’re now accusing me of being suspicious… :shrug: it did come full circle where in my first comment I said you would start accusing me. I guess neither of us have anything more to say to each other because we are both too locked into our own beliefs.
This person revived an account they haven't touched since 2018 in an attempt to convince ME SPECIFICALLY that there is nothing wrong with the original repo maintainer. They gloss over my arguments, use logical fallacies and are generally antagonistic in a way that is not immediately obvious. You be the judge, dear readers.
At any rate, this person has failed their cause and has actually made me double down on the conspiracy theory :)
It's possible that he was intentionally pressured and his mental health made bad or worse by the adversary to increase stress. The adversary would then propose to help them reduce the stress.
It argues the topic pretty well: xz is unsuitable for long-term archival. The arguments are in-depth and well worded. Do you have any argument to the contrary beyond "sour grapes"?
I can understand wanting your project to succeed, it's pretty natural and human, but it's flagrant Antonio had a lot of feels about the uptake of xz compared to lzip, as both are container formats around raw lzma data streams and lzip predates xz by 6 months. His complaint article about xz is literally one of the "Introductory links" of lzip.
Neither is lzip since it doesn't contain error correction codes. You can add those with an additional file (to any archive) e.g. via par2 but then most of the points in the linked rant become irrelevant.
Collateral damage yes, but it seems like he is currently away from the internet for an extended time. So it could be that Github needed to suspend his account in order to bypass things that he would otherwise have to do/approve? Or to preempt the possibility that his account was also compromised and we don't know yet.
No. I mean that the link you shared is a opinion piece about the xz file format, and those opinions are fully unrelated to today's news and only serve to further discredit Lasse Collin who for all we know have been duped and tricked by a nation state, banned by github and is having a generally shitty time.
There may be some suboptimal things about security of the XZ file format, I don't know.
I bet you there are less than optimal security choices in your most cherished piece of software as well.
This thread is about an exploit that does not rely on any potential security problems in the DESIGN of the xz FORMAT. Therefore your point, even if valid as a general one, is not really relevant to the exploit we're discussing.
Further, there's some proof needed that any potential suboptimal aspects of the security design of the xz FORMAT was designed such so that it could be exploited later or simply because no programmer is an expert on every aspect of security ever. I mean you could be the most security conscious programmer and your chain could still be compromised.
Security today is such a vast field and it takes so little to get you compromised that proclaiming anything 'secure design' these days is practically impossible.
I bet you an audit of lzip would find plenty of security issues, would those be intentional?
1) Are there no legit code reviews from contributors like this? How did this get accepted into main repos while flying under the radar? When I do a code review, I try to understand the actual code I'm reviewing. Call me crazy I guess!
2) Is there no legal recourse to this? We're talking about someone who managed to root any linux server that stays up-to-date.
> 2) Is there no legal recourse to this? We're talking about someone who managed to root any linux server that stays up-to-date.
Any government which uses GNU/Linux in their infrastructure can pitch this as an attempt to backdoor their servers.
The real question is: will we ever even know who was behind this? If it was some mercenary hacker intending to resell the backdoor, maybe. But if it was someone working with an intelligence agency in US/China/Israel/Russia/etc, I doubt they'll ever be exposed.
Reflecting on the idea of introducing a validation structure for software contributions, akin to what RPKI does for BGP routing, I see significant potential to enhance security and accountability in software development.
Such a system could theoretically bring greater transparency and responsibility, particularly in an ecosystem where contributions come from all corners.
Implementing verifiable identity proofs for contributors might be challenging, but it also presents an opportunity to bolster security without compromising privacy and the freedom to contribute under pseudonyms.
The accountability of those accepting pull requests would also become clearer, potentially reducing the risk of malicious code being incorporated.
Of course, establishing a robust validation chain for software would require the commitment of everyone in the development ecosystem, including platforms like GitHub. However, I view this not as a barrier but as an essential step towards evolving our approach to security and collaboration in software development.
The actual inclusion code was never in the repo. The blobs were hidden as lzma test files.
So you review would need to guess from 2 new test files that those are, decompressed, a backdoor and could be injected which was never in the git history.
Ok, go ahead and scrutinize those files without looking at the injection code that was never in the repo? Can you find anything malicious? Probably not - it looks like random garbage which is what it was claimed to be.
"Jia Tan" was not a contributor, but a maintainer of the project. The key point here is that this is a multi-year project of infiltrating the xz project and gaining commit access.
In a large tech company (including ones I have worked at), sometimes you have policy where every change has to be code reviewed by another person. This kind of stuff isn't possible when the whole project only has 1-2 maintainers. Who's going to review your change other than yourself? This is the whole problem of OSS right now that a lot of people are bringing up.
I maintain a widely used open source project myself. I would love it if I could get high quality code review for my commits similar to my last workplace lol, but very very few people are willing to roll up their sleeves like that and work for free. Most people would just go to the Releases page and download your software instead.
>How did this get accepted into main repos while flying under the radar? When I do a code review, I try to understand the actual code I'm reviewing. Call me crazy I guess!
And? You never do any mistakes? Google "underhanded C contest"
It's hardly surprising given that parsing is generally considered to be a tricky problem. Plus, it's a 15 years old project that's widely used. 750 commits is nothing to sneer about. No wonder the original maintainer got burned out.
For the duration of a major release, up until ~x.4 pretty much everything from upstream gets backported with a delay of 6-12 months, depending on how conservative to change the rhel engineer maintaining this part of the kernel is.
After ~x.4 things slow down and only "important" fixes get backported but no new features.
After ~x.7 or so different processes and approvals come into play and virtually nothing except high severity bugs or something that "important customer" needs will be backported.
Sadly, 8.6 and 9.2 kernel are the exception to this. Mainly as they are openshift container platform and fedramp requirements.
The goal is that 8.6, 9.2 and 9.4 will have releases at least every two weeks.
Maybe soon all Z streams will have a similar release cadence to keep up with the security expectations, but will keep a very similar expectations that you outlined above.
There is also a variety of new, parallelized implementations of compression algorithms which would be good to have a close look at. Bugs causing undefined behaviour in parallel code are notoriously hard to see, and the parallel versions (which are actually much faster) could be take the place of well-established programs which have earned a lot of trust.
The time stamp of a git commit depends on the system clock of the computer the commit was checked in. This cannot be checked by github & co (except that they could reject commits which have time stamps in the future).
I think this is a really important point. I worked intensively with Python for years, but it was a deep disappointment to see how little the core devs and the community care about backwards compatibility in the 2/3 transition, and things have not improved. That might be due to what Python is dominantly used for today (The Numeric/Numpy transition which happened around 2000 was totally different). Breaking things every few years might be OK for unicorn startups or companies that move fast and, well, break things, but there is a lot of valuable programming from people and in organisations which can simply not afford, or do not see it as a priority to re-write mature algorithms every few years.