Nice writeup! You definitely chose a pretty hard way, but the project setup is always the most complex part. Bonus points for immediately running into security/header issues, but my bet would have been CORS.

At $WORK, we're also building with emscripten/C++. We'll add WebGPU/shaders and WebAudio for bonus pain.