HTTP 2 and above will use one connection to retrieve several files. Caddy [1] can act as a static file server that will default to HTTP 2 if all parties support it. No configuration required.

If you allow UDP connections in your firewall, it will upgrade to HTTP 3 automagically as well.

I highly recommend it

[1] https://caddyserver.com/