Automatic light sleep maintains a WiFi connection with only 1 or 2 milliamps, I'm using the exact same ESP as server model but with Websockets, and it all just works.

There's even some PRs for automatic light sleep I think in ESPHome, but they're not merged yet.