Right, that was my point about the physical layout not really being conducive to NN-driven stabilization, on my cursory read.
Digging in, you could probably bitbang DShot on Orin GPIOs. It would be really sketchy since it's presumably running Linux on out-of-order cores with giant cache, so there's nothing close to realtime going on, but it's so damn fast that the theoretical problem probably doesn't matter and I bet it would work fine.
Orin also has "safety island" for ASIL applications which has lockstep Cortex-R52s. This would be ideal to run an integrated flight controller RTOS but it's supposedly fused off on consumer boards, lame. There's also a Cortex-R5 called "SPE" that could probably run a flight control RTOS as well.
Regardless of all of this theory craft, I was either totally wrong or accidentally right in two ways, depending on how you want to look at it: according to a sibling post, the STM32 board actually runs the NN-backed stabilization loop rather than a PID one! I've seen this done in research before but never in a meaningfully successful way, so I'm even more impressed now - as far as I can tell, it's _two_ novel solves, an actually working PID-or-better NN-based stabilization system on STM32 _and_ inside-out vision based flight planning for racing on the Nvidia board.
Its not just a stabilization/rate control loop but the NN does both guidance and control. It is an MLP that takes state + upcoming gate positions/waypoints and controls the motors directly. I.e. there is no explicit path planning going on but rather the NN outputs motor commands that hopefully help the drone traverse the given waypoints in minimum time. I think this is quite cool because running a similar Optimal Control MPC setup is really not at all feasible on a normal STM32H7 Flight controller, especially at 1kHz.
Oh, I see how this works now! That's _incredibly_ clever, not at _all_ what I would have guessed, and quite innovative. Thank you so much for taking the time to post these explanations.
Digging in, you could probably bitbang DShot on Orin GPIOs. It would be really sketchy since it's presumably running Linux on out-of-order cores with giant cache, so there's nothing close to realtime going on, but it's so damn fast that the theoretical problem probably doesn't matter and I bet it would work fine.
Orin also has "safety island" for ASIL applications which has lockstep Cortex-R52s. This would be ideal to run an integrated flight controller RTOS but it's supposedly fused off on consumer boards, lame. There's also a Cortex-R5 called "SPE" that could probably run a flight control RTOS as well.
Regardless of all of this theory craft, I was either totally wrong or accidentally right in two ways, depending on how you want to look at it: according to a sibling post, the STM32 board actually runs the NN-backed stabilization loop rather than a PID one! I've seen this done in research before but never in a meaningfully successful way, so I'm even more impressed now - as far as I can tell, it's _two_ novel solves, an actually working PID-or-better NN-based stabilization system on STM32 _and_ inside-out vision based flight planning for racing on the Nvidia board.