> Microsoft didn't even bother with key rotations anymore after 2021 because one time they fucked it up and it caused an outage, so they decided to just not do that anymore.
Key rotation is almost like restoring from backups. It's an absolutely necessary capability and practice.
You'd be surprised at how little cloud vendors give a shit about security internally. Story time: I recently went ahead and implemented key rotation for one of our authz services, since it had none, and was reprimanded for "not implementing it like Google". Fun fact: Google's jwks.json endpoint claims to be "certs" from the path (https://www.googleapis.com/oauth2/v3/certs). They are not certs - there is no X.509 wrapper, no stated expiration, no trust hierarchy. Clients are effectively blind when performing token validation with this endpoint, and it's really shitty.
Other nonsense I've seen: leaking internally signed tokens for external use (front-channel), JWTs being validated without a kid claim in the header - so there's some sketchy coupling going on, skipping audience validation, etc...
Not much surprises me anymore when it comes to this kinda stuff - internally, I suspect most cloud providers operate like "feature factories" and security is treated as a CYA/least-concern thing. Try pushing for proper authz infrastructure inside your company and see what kinda support you'll get.
Are there any large companies that don't operate like feature factories? It seems to be such a common issue and the natural result of the incentive structure.
although this is a valid insight, it reduces the detail of the conversation into "yes or no" on a topic that is not a "yes or no" topic.. it is behavior and messaging among a dozen critical functions of business. Almost every business is different in their mix.. perhaps faced with similar rhetoric, law says "show me an example then we can discuss" instead of "classify all examples then apply to a situation"
Not necessarily. Scheduled key rotation has a lot of conceptual problems:
1. As MS found, revoking old keys is very risky because doing so creates outages. But if you don't do it then changing keys is useless. This isn't a problem specific to Microsoft. Lots of companies have learned this lesson the hard way.
2. It assumes that attackers don't just use stolen keys immediately (e.g. to issue more keys, change passwords, create new accounts etc). In practice they usually do.
3. It assumes that if you change the keys the attackers can't just immediately re-steal the new keys.
So it's only really a useful practice in one very specific scenario: you do something that boots undetected attackers out of your network without realising that's what it did, and the attackers need ongoing access that only that key can provide, and they can't use that key to elevate permissions in a more permanent way like by creating a new account on the system or stealing a user password. Pretty specific scenario.
Unfortunately, key rotation also comes with big downsides. Any software that works with keys has to be built to tolerate a change silently, because now it's a regular occurrence instead of a rare one (where maybe a bit of disruption can be absorbed). That creates complexity and therefore bugs. And because it's a repetitive piece of fiddly and complex work that can break your entire service if you get it wrong it inevitably gets automated, and that in turn means that you end up with a large collection of highly privileged subsystems that have the power to silently change keys in ways admins won't notice because they are expecting it: exactly the sort of thing attackers will immediately focus on.
Overall it's not an obviously winning move. Opportunity cost matters too. Whilst you're setting up all the infrastructure to do this, ironing out the bugs, cleaning up after the outages etc, your competitors might be investing in other kinds of security best practices that are more effective. It's especially useless here because MS don't know how the key was stolen to begin with, so there's no reason to think that if they changed it that would have had any effect. Most likely it could have just been immediately restolen and all the effort would have been theatre.
Key rotation is almost like restoring from backups. It's an absolutely necessary capability and practice.