"In Go object files and binaries, the full name of a symbol is the package path followed by a period and the symbol name: fmt.Printf or math/rand.Int. Because the assembler's parser treats period and slash as punctuation, those strings cannot be used directly as identifier names. Instead, the assembler allows the middle dot character U+00B7 and the division slash U+2215 in identifiers and rewrites them to plain period and slash."[0]
What slimsag wrote is correct. It makes cross-compiling code that needs to call C functions as easy a setting the GOOS and GOARCH and just building. This means no need to worry about building a C cross-compiler.
I do want to write an article about how purego works under the hood.
I don't have either. I was gonna figure out how to post it after I actually sat down and wrote it lol. I'll probably post it in the golang subreddit and maybe link to it in the README.md since it describes how purego works.
If memory serves, dev.to tends to be downranked or outright filtered by a bunch of places.
(I have no idea how or why that came about, I've merely observed people having all sorts of trouble getting posts on there visible in aggregators and etc.)
I’m one of the main contributors. I’ve looked into it bc I wanted to know if I could build iOS apps without Cgo. ATM, it is not possible. The reason is because when you run go build creating a shared object it runs the go cgo tool. That tool although written entirely in Go doesn’t know about purego and so will go ahead and import runtime/cgo which requires a C toolchain. Now it could be possible to circumvent that with using a custom Go build toolchain but the goal of purego was to be seemless to use in a project. Just use it and then go build like any other dependency.
It’s pretty simple to use if you are familiar with dlopen and friends.
Just call purego.Dlopen(“libname.so”, purego.RTLD_GLOBAL)
Take the returned library (make sure to check for errors with purego.Dlerror() first) and call purego.Dlsym(lib, “cfuncName”). If it exists than u can call it with either purego.SyscallN or purego.RegisterFunc
It would be good to have more documentation on usage, though; things like how to deal with struct padding (or packed structs), common OS API types (presumably manual munging of UCS2/UTF16 is needed for Windows), etc; at least to mention that it's unchanged from …/x/sys?
It's easier with dlopen because it's still C and therefore you have the normal headers…
Contributor here: Purego doesn’t do anything to improve the overhead of calling into C. It uses the same mechanisms that Cgo does to switch to the system stack and then call the C code. Purego just avoids having to need a C toolchain to cross compile code that calls into C from Go.
I’ve actually been quite interested in Zig. If that built-in was added than it would likely be possible to grow the goroutine stack to the proper size and than call the Zig code. Very interesting stuff!
Makes sense! I also wonder (if you know): last I looked I recall that each CGO call requires switching to the system stack, but I can't recall what happens after. Does it switch back to a regular goroutine stack once the syscall has completed?
I wonder if a more tailored CGO implementation could pin a goroutine to a thread which is guaranteed to have a system stack available, so that each CGO call need not worry about that switching at all. Maybe that'd require runtime changes though?
Stack switching isn't that much of the overhead. "ordinary" cgo overhead is <100ns now, has been for a few years, and is much closer to 30 than 80 on recent processors. Most of the overhead is a set of 4 CAS operations (incidentally this means that AMD has measurably lower cgo overhead because of something with its caching model I don't understand).
If cgo's only overhead was the "ordinary" overhead, most people wouldn't have an issue with it. It's downright zippy, in fact... as long as your syscall/C call takes less than 1us. If you stay under the 1us threshold, go will put the OS thread used for the syscall back where it found it and everything moves on.
The issue is that the OS thread was previously serving N goroutines that other parts of the program may be waiting on to move forward, and the OS thread is in a state where go can't pre-empt it and allow those other goroutines to move forward, and it has no idea how long it will be until it can move forward.
As a result, if a syscall/c call takes longer than 1us, go has no choice at this time but to resume a new thread, context switch all the old work onto that thread, and then suspend the syscall thread when it comes back. If you do this a lot, your performance will crater.
There's also separately a few issues around how go chooses to resume/suspend OS threads (for instance, if an os-locked goroutine does coooperative park for any reason to wait on another thread to do something, go will suspend the thread it was on, context switch to a different thread, then when the goroutine wakes up, it will realize its mistake, resume the thread it was on and context switch again).
This is all fixable stuff, but all the use cases that google cares about are working fine so it doesn't really get any attention.
Yeah the default behavior is to switch back. It’s possible to pin a goroutine to a thread with runtime.LockOSThread(). However I don’t believe it avoids the stack switching. It’s purpose it to make sure that Thread Local Storage works properly. The runtime is pretty smart though so it might already do the optimization you suggested in someway. I know it has a goroutine specifically for monitoring if a thread is stuck in a external call and therefore spawns a new thread to continue work (sysmon)
Maybe I should play with removing the stack switching from purego so that under condition of a locked OS thread you can avoid that overhead :) I might give that a shot sometime
[0] - https://go.dev/doc/asm