Not the guy you’re asking, but I’d say risk of fire, yes. The laptop will be safer without a battery than it is with one, regardless of safeguards.

As said by others, mostly the fire risk. They can catch on fire, although rare, and a bad contact or flaky power source could make it go into many charge/discharge cycles in a short period of time. Batteries also degrade faster if it is too warm, cheap laptops often have terrible thermals and you could also shove it in a closet. A combination of those will increase the fire risk.

Also when using an old laptop, the battery could be pretty beaten up (too many cycles or prolonged exposure to heat) or it could have been replaced by a cheap non-compliant alternative, making it harder to trust wrt fire risk. And if you have to buy a brand-new one to reduce that risk, it immediately changes all the economic incentives to use an old laptop (if you are gonna spend money, might as well buy something more suitable).

> many won't have an easily removable battery

That’s true, although I’d guess majority can still have the battery disconnected once you get access to the motherboard.

I wish I took a picture of my MacBook pro mid-2015, which happens to be my home hosted stuff server, before I changed it's battery. As it was just sitting in a corner, almost forgotten, I noticed the problem when cleaning, one day, and it started wobbling when I moved the piece of furniture it was sitting on. Once I gave it to a guy who disposes of such things, he told me I was lucky it didn't explode.