This never should have been on by default. The end user (read: administrator) needs to know they want to use the TPM.
This is a huge foot gun for many devices.
The accompanying changelog note hints at why:
> Failure to load hardware attestation keys no longer prevents the client from starting. This could happen when the TPM device is reset or replaced.
This is unfortunate as for many, many deployments, you absolutely want this on. But because it's a time bomb for certain device/OS combinations that Tailscale can't control or predict, and folks install Tailscale on pretty much everything, then the incidence of borked installs can only rise.
As someone with a passing interest in using TPM for crypto things, everytime I think deeply about the implementation details like this, I come back to needing some kind of recovery password/backup key setup that entirely negates the point of the TPM in the first place. They seem really neat, but I struggle to see the benefit they have for doing crypto when a tiny slip up means your users' keys are poof, gone. And the tiny slip up may not even be with your software, but some edge case in the OS/TPM stack.
The TPM was never designed to be the only holder of a key that cannot be reset. The idea was that it prevents you from typing in a password or reseting an attestation signature in a database for 99% of boots, but if certain things in the boot process change (as determined by the firmware, the CPU, the OS, and the application using the TPM) it’s designed to lock you out so those things cannot change without anyone’s notice.
For that purpose they’re pretty good, though there are advantages to a more signature-oriented boot security option like Apple’s Secure Enclave. But that only works so well because Apple simply doesn’t permit altering parts of the macOS boot process. For Windows/Linux, you have a variety of hardware, firmware, and OS vendors all in the mix and agreeing on escrow of keys for all of them is hard.
The presumption is that the contents being secured are /so/ valuable that locking my device is preferable to any leak of them whatsoever.
This is military level security and just isn't appropriate for most consumers. Particularly around something so rarely exercised and utilized by users as the boot process. A simple warning with a long timeout would have sufficed.
Aside from that you have a hardware vendor, sourced into an integrated product from another vendor, sold to a user, with various third party software interacting with it. This was always going to result in questionable experiences for end users.
A warning doesn’t help at all. The main threat model for FDE is that someone steals your device and dumps the disk. If you don’t protect the boot process somehow, then you’re just storing the encryption key next to the data.
If you don’t care about that (which is not “military level security”, laptop thieves stealing creds is a thing), just don’t use FDE or use it with an on-boot password every time. No point in the theater.
The primary argument in favor of TPM's is the desire to assert against tampering to the boot system, and as a secondary effect it can be one of the solutions to reduce the need for users to type in passwords.
You can still use crypto without a TPM, including with full disk encryption, and for LUKS specifically you can use multiple passwords and mechanisms to unlock the system. Different solutions will give different benefits and drawbacks. Me and a friend wrote a remote password provider for Debian called Mandos which uses machines on the local network as a way for unattended boots. It does not address the issue of tampering with the bios/boot loader, but for the primary purpose of protecting against someone stealing the server/disks it serves the purpose of allowing us to use encrypted disk without drawbacks of typing in passwords, and the backup server, itself with encrypted disks, handles the risk of needing recovery passwords. At most one needs to have an additional backup key installed for the backup server.
TPM keys are great for things like SSH keys or Passkeys, which surprisingly works well even in Windows.
The private key is safe from any exfiltration, and usage only requires a short PIN instead of a long passphrase. The TPM ensures you're physically typing that PIN at the machine not a remote desktop window or other redirection that could be hacked.
Obviously, this is problematic/annoying for scripts and things that can't share the SSH session, because you need to PIN with every authentication. Also, for encryption, you want to use something where you can backup the private key before stashing it in the TPM. Windows allows you to do this with certificates that are exported for backup prior to encrypting the private key with an unexportable TPM key in Hello.
An easy solution to having to put your PIN in too often for SSH is to use the `ControlPersist` option in your SSH client config. This lets you only create a new SSH connection every 30s (or whatever you put), even if you’re doing separate operations. With a low timeout, there’s no realistic security risk (what’s the chance an attacker will only have control of your machine for 30s?).
I do this for GitHub in particular, because of tools that connect to the remote multiple times. Works with anything that uses the actual ssh executable under the hood.
Ideally this should have been hashed out before deploying passkeys everywhere, but I guess you can always register multiple passkeys for the sites that allow you to.
Iirc the original idea was that passkeys should be device specific. Of course that's impractical so now they're morphing to a long password that a human can't process.
In a few years someone will post "how about a long human retainable passphrase?" as a new and improved discovery.
They are still different to a password in that the service you are logging in to never gets the private key. So in the case the database gets compromised, if the service provider ensures no edits were made / restores a backup, there is no need to change your passkey since it was never exposed.
But e.g. Windows uses a TPM by default now ? If TPMs were such a major issue then there would be millions of Windows users with TPM problems, no ?
I have no inside info, but this strikes me more as a bit of a "sledgehammer to crack a nut". Tailscale turning off important functionality due to small-but-vocal number of TPM edge cases ?
It is also very unfortunate they did not manage to find any middle ground between the hard-binary all-on or all-off.
Windows uses TPM for Bitlocker. A very common scenario where TPMs get reset is BIOS updates (when a TPM is implemented in firmware).
AFAIK, Windows cheats here because it also manages BIOS updates. When an update happens, it takes extra steps to preserve the Bitlocker encryption key in plaintext, and re-seals it to the TPM after the update completes.
Apart from Windows, there are many setups that fail in fun ways: Kubernetes pods that migrate from one VM with a TPM to another one, hypervisors that mount a virtual TPM to VMs, containers or VM images that do Tailscale registration on one machine and then get replicated to others, etc.
Tailscale already did some attempts at cleverness when deciding whether to enable features using a TPM (e.g. probing for TPM health/version on startup, disabling node state encryption on Kubernetes pods), but there was still a long tail of edge cases.
Actually, this is not the case. BitLocker wraps the key, meaning even if the TPM were compromised, one would still have to brute-force the PIN for the actual key. It’s cryptsetup on Linux that stores the key on the TPM in plaintext. This vulnerability has been known for quite a while and nothing has been done about it so far.
Slightly off-topic: it also cheats in how TPM works for Bitlocker when you do TPM + PIN. One would assume PIN becomes part of the encryption key, but in reality, it's just used as the auth for TPM to release the key. So while it sounds like a two-factor solution, in reality it's just single factor.
So the Bitlocker without TPM is actually a better idea and Windows makes it very painful to do if TPM is on.
I don’t know much about the TPM but if it’s anything like Apple’s Secure Enclave, it should require exponentially longer time after each incorrect PIN past the first one, making it so you can’t reasonably brute force it without getting lucky.
I’m not sure how the typical “two factor” best practices would interpret one of the factors basically self destructing after 10 guesses, but IMO it’s a pretty decent system if done right.
That's not the issue. The TPM isn't blinded in the above description meaning that if someone cracks the TPM they can get your key. Ideally both factors are always required to access the secret.
If you're wondering, yes this is a security issue in practice. There have been TPM vulnerabilities in the past that enabled exfiltration of secrets.
Aren't PINs usually short, and might even be really be made out of just digits in the first place? So would there be real security benefits in adding that to the key?
You can make PINs as complex as you want, there's only a maximum length limitation of 20 characters. There's no difference between passwords and PINs in Windows except that Windows calls it a PIN if it's used with TPM. And yes, it does nudge you in the direction of making it simple because "TPM guarantees security", but you don't have to.
> Windows cheats here because it also manages BIOS updates
Is this (relatively) new?
I don't use TPM and I rarely update BIOS unless I really need to, but I thought there was an option on my BIOS/UEFI to use USB drive to update it. How would Windows know about it?
Window can get BIOS updates through windows update, if the OEM participates and packages them. I haven't seen BIOS updates through windows update on my systems where I built it from components, I've only seen it on integrated systems from major builders (HP, Lenovo, etc).
The BIOS update instructions for my retail packaged motherboard indicate to turn off BitLocker before doing upgrades to prevent loss of TPM turning into a loss of access, but it'd be easier if it were automated.
You can update with a USB drive, but if you have bitlocker enabled and don't temporarily disable it before the BIOS update, you'll need to reformat and reinstall Windows.
I believe you can also get it from your online Microsoft account if that's what you logged in with once. I ran into this a while ago and had to do it that way. I didn't even know I'd set up Bitlocker.
Windows seems to do two big things with a TPM. Bitlocker encryption and some microsoft account stuff.
If the bitlocker stuff goes wrong, big problem, hopefully you printed and kept your recovery key.
If the microsoft account stuff goes wrong, mostly the microsoft store and microsoft store apps break in subtle ways... but that's also how that ecosystem normally works, so how are you supposed to know it's the TPM problem?
Windows automatically reinitializes the TPM if it's reset boots normally, most end users will not notice any issues unless they have Bitlocker or biometrics configured.
Sure, but most users probably don't actually want this level of defense.
For the same reason that most folks don't use bank vault doors on their house.
Ex - even reasonably technical people hit this footgun in lots of edge cases... like updating their bios, changing the host of a vm running the tool, or having a k8s pod get scheduled on a different node.
No, it might figure out the solution but even after many days there's no assurance that it won't get stuck making the same mistakes over and over again, never getting closer to a solution. I've seen this many times.
Getting in a loop does still happen, yes. If you run codex in tmux and let another agent just occasionally check on progress, it can be prevented. That’s not even expensive - checking every 30 minutes suffices. The watchdog agent can then press Esc in tmux and send a message, maybe do some research to get it unstuck etc
Neither have I, personally, but I’ve seen reports this can happen on very hard problems, where the goal just cannot be reached from a local optimum. Getting unstuck by trying something new is something a watchdog agent could prompt it.
> There are ways to make go get use ssh but even with that approach the repository needs to be accessible over https.
No, that's false. You don't need anything to be accessible over HTTP.
But even if it did, and you had to use mTLS, there's a whole bunch of ways to solve this. How do you solve this for any other software that doesn't present client certs? You use a local proxy.
Yeah, I'd have thought so. The lad's a little behind on appreciating narrative, though. Loves books, but can't yet focus on anything that isn't ~= pictures::words. It's fine; he'll get there.
Love both Pooh, and Wind in the Willows, and will enjoy seeing how they take him.
> I’ve found reading aloud helpful for staying engaged — limiting myself to mouth-speed rather than eye-speed means I won’t rush, miss important details, and then lose interest, which has always been a problem for me.
This worked for me... for a time. And then what happened surprised me (but maybe shouldn't have): I started zoning out and thinking about other things, missing important details, while reading aloud. Wild that we can even do that.
I initially thought this was just a function of reading the same thing multiple times, but I’ve since had it happen many times when reading something completely new. Somehow my mind wanders and when I tune back in, not only am I reading clearly, I’m still doing the voicing for different characters. It’s so weird.
I haven't had it happen while reading aloud (since I almost never read aloud), but I've definitely had it happen while reading something new that I hated. I'd end up having to read the same page (or more!) three or four times because I kept zoning out.
I always find these takes curious because they could not be further from my experience. I'm still discovering tons of good music. Perhaps it's specific to genres, but I haven't encountered any generated junk tracks.
It depends on the algorithm which often preferences "similarity" (for whatever definition of similarity is).
This year I got into some pretty generic blues/rock when driving and really liked one of the songs in some playlist/radio [1]. Little did I know that the song was AI. So when I started a radio based on that song, the resulting radio was 99% AI though I didn't even realise that until after a second/third listen through.
There's also obvious care and human creation there as well.
They have several "artists". Bertha Mae Lightning gets the better lyrics and artangements. Virgil Dillard gets simpler tunes and the occasional weird grammar/lyrics. And so on.
Really? How about asking google to "play bloomberg news on spotify" next time. Then see if you can remove the resulting chaos from your history so it won't start feeding you slop.
I had a similar bizarre experience recently where when "Walmart" would be mentioned in an outgoing message, instead of sending the message it would change the nav destination.
> I believe that process was better than me just fully validating it myself
Why?
> and part of the process toward encouraging them to use LLM as a tool for their work.
Did you look at it from their perspective? You set the exact opposite example and serve as a perfect example for TFA: you did not deliver code you have proven to work. I imagine some would find this demoralizing.
I've worked with a lot of director-level software folk and many would just do the work. If they're not going to do the work, then they should probably assign someone to do it.
What if it didn't work? What if you just wasted a bunch of engineering time reviewing slop? I don't comprehend this mindset. If you're supposedly a leader, then lead.
If you can't tell the difference between active tracking and inspecting request headers, then it's worth committing a bit of time to ponder. Particularly the costs associated with IP tracking at scale.
This is a huge foot gun for many devices.
The accompanying changelog note hints at why:
> Failure to load hardware attestation keys no longer prevents the client from starting. This could happen when the TPM device is reset or replaced.
This is unfortunate as for many, many deployments, you absolutely want this on. But because it's a time bomb for certain device/OS combinations that Tailscale can't control or predict, and folks install Tailscale on pretty much everything, then the incidence of borked installs can only rise.
reply