It’s usually SSH that’s the bottleneck though. If you just expose your local X11 TCP server port and connect to it from the server (X11 client) by setting the DISPLAY env properly, it will be much more responsive. Often indistinguishable from local apps. Secure the port with Wireguard, if you want to be a bit more responsible and still have good performance. Checked with Firefox, which works like a charm this way, but is unusable over SSH.
> If you just expose your local X11 TCP server port and connect to it from the server (X11 client) by setting the DISPLAY env properly, it will be much more responsive.
That's fine on a LAN with low or sub ms latency. If you need to connect to a server on the other side of the country, you'll want a less chatty protocol, e.g. X2GO (which gives additional benefits, e.g. of restartable sessions).