The main delay in VNC is the software stack encoding and decoding the data.
A video-over-ethernet solution would likely have the GPU directly packetize the data into IP packets and route them direct to the network hardware without the CPU touching anything. The network would have QoS to guarantee the necessary bandwidth and guarantee a fixed latency and no packet loss (ie. there is no chance of a packet being dropped due to a queue overflowing, because we have already arranged a fixed bandwidth allocation for the whole network route).
A video-over-ethernet solution would likely have the GPU directly packetize the data into IP packets and route them direct to the network hardware without the CPU touching anything. The network would have QoS to guarantee the necessary bandwidth and guarantee a fixed latency and no packet loss (ie. there is no chance of a packet being dropped due to a queue overflowing, because we have already arranged a fixed bandwidth allocation for the whole network route).