It's interesting to compare Swift to Julia here – Julia had a base of people who absolutely loved it from the 0.2 days, and has managed to draw many users from Python/Matlab/R. There are many reasons for that, but overall Julia had something that really appealed to scientists: syntax they liked + competitive-with-C performance.
In contrast, S4TF's target audience seems to have been Swift developers, and they didn't really appeal to the data science crowd.
Precisely, S4TF felt like a startup product with no market research. It really felt like the only reason Swift was chosen was because the main dev created it.
We see this all the time right, the next big ML company is going to come from some scrappy startup with a precise focus on what to do (either because they're geniuses or because they're just the lucky idiot amongst 10,000 ML startups), not from some behemoth that decides "we'll eat that for breakfast". Swift4ML suffers from the fact that Swift exists. To make S4ML work you need to abandon atleast some of the things that makes Swift Swift.
In contrast, S4TF's target audience seems to have been Swift developers, and they didn't really appeal to the data science crowd.