Neat! This may have not been well known when they kicked off the project and wrote their reasoning. Here is what they had to say about Scala at the time of the document linked up-thread[0]:
"Java / C# / Scala (and other OOP languages with pervasive dynamic dispatch): These languages share most of the static analysis problems as Python: their primary abstraction features (classes and interfaces) are built on highly dynamic constructs, which means that static analysis of Tensor operations depends on "best effort" techniques like alias analysis and class hierarchy analysis. Further, because they are pervasively reference-based, it is difficult to reliably disambiguate pointer aliases."
If they were wrong about that, or if the state of the art has progressed in the meantime, that's great! You may well be right that Scala would be a good / the best choice if they started the project today.
There is a paper published in 2019 from Purdue University where they show how AD can be combined with multi-stage programming in Scala. [1]
DeepLearning.scala that does compile-time reverse AD. [2]
Compute.scala for operator fusion at runtime (JITifying) [3]
Here [4] is a Pre-SIP to add reifiable control flow to Scala compiler so AD and operator fusion could happen at compile-time.
Moreover, Kotlin also has a library that does type-safe AD. [5]
nVidia and Oracle want better interopabiltiy between CUDA and GraalVM through grCuda. [6]
[1] Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator - https://arxiv.org/pdf/1803.10228.pdf
[2] https://github.com/ThoughtWorksInc/DeepLearning.scala
[3] https://github.com/ThoughtWorksInc/Compute.scala
[4] https://contributors.scala-lang.org/t/pre-sip-for-with-contr...
[5] https://github.com/breandan/kotlingrad
[6] https://developer.nvidia.com/blog/grcuda-a-polyglot-language...