But you don't link against the X11 server, you communicate via sockets with it. So the client application has no direct dependency of the X server implementation. Like a web server, the X server could be written in any language.