Must Read: Ian Lance Taylor's 20-part series on Linkers (the articles are on his blog but linked to in one place here) - https://lwn.net/Articles/276782/

But Honjiu Lu's ELF: From The Programmer's Perspective (https://beefchunk.com/documentation/sys-programming/binary_f...) is what did the trick for me. It provides a very good and succinct overview.

Reference: See also Using ld the GNU Linker co-authored by Ian Lance Taylor.

My gripe with linkers is: why can't they add the minimal metadata recording the actual paths of the shared libraries an executable or shared library was linked to. You can provide search paths to the linker with -L and LIBRARY_PATH and what not, but actually recording _what_ has been linked would be great for reproducibility. Right now you only have a DT_NEEDED entry with the soname of the lib, and at runtime this library has to be located... again.

Dynamic libraries are not linked at all, so the linker can't do that. The linker just records what shared objects the loader needs to find when the linked artifact itself is loaded, the path to those objects is therefore the responsibility of the loader to resolve and report.

On Linux you can find the shared libraries of a running process in /proc/<PID>/maps, or what libraries will be loaded when the executable runs/shared object is run in the current shell with ldd.

The location of DSOs on the build machine is rarely the same as on the target machine the software is run on, except for the corner case of building software locally on the target. Embedding detailed information about the build environment is rarely useful to anyone but those with malicious intent.

Vendoring dependencies and static linking everything lets you avoid a whole world of pain including the issue you ran into.

For those of us on OS X (or anyone using llvm-nm I assume), pass `-f sysv` to `nm` to get the table format shown in the article.

On a slightly unrelated note, my current linker for llvm projects is mold[0]. It's delightfully fast and speeds up linking for some of my projects tremendously. I would recommend anyone using an llvm based compiler, like rustc or clang, to give it a try.

[0]: https://github.com/rui314/mold

I used to work with Rui-san before he went full time to work on mold. He's a real awesome guy and super fun to talk with, he's extremely knowledgeable as well. I was very sad when I heard he was going to leave the company but looking at how popular mold has gotten and how well the project seems to be doing I really couldn't be happier for him.

AMA

Are shared libraries a mistake? Wouldn't it be better just to statically build everything?

Also at what point should a well-used dynamic library simply be part of the OS? E.g. practically everything depends on a libc, why can't this simply be part of the OS?

Would you consider Docker assembling an image a linker? If not, what are the principal differences, something which is contrary in principle to the idea of linking, which is in Docker?

What is the think you wish compiler writers knew about linkers, but usually don't?

What have been the biggest advances in linker technology in the last decade or so?