That's a very interesting question. My guess is that the physics of grabbing things, especially non-rigid things, is very messy and difficult to simulate. It would be great if someone here were able to give a detailed answer to this question though.
1. The best / most recent attempt at this was for the DARPA robotics challenge and the Gazebo simulator.
This was still very buggy and prone to hilarious / depressing physics.
2. Almost all game physics engines start from rigid body and slap on particles, deformables, etc.
An exciting counter example to this is nVidia Flex which starts with unified particle simulation (much closer to molecular dynamics simulation used for, you know, real work).
3. From the perspective of AI, accurate simulation might not be required.
Intelligence requires complexity and a certain degree of predictability. So as long as you can build a rich and consistent / learnable world then whatever simulation you have could be super useful.
From the perspective of transferring that knowledge into a robot though you need accurate physics.
4. Natural touch sensors are hard to do in rigid body simulators but are super important to naturalistic learning.
There's a ton of information that your sense of touch and body position provide about how the world works, and getting the tens of thousands of soft-contact touch points simulated you need for this kind of sensing is pretty challenging today.
Lots of physics engines do all sorts of things to minimize contact points, or ignore them if there's no motion. You have to work against optimization a lot if you want mechanoreceptors and proprioception.
I agree with all the points you made but in addition I would add another - with external cameras for positioning and movement feedback, you don't need to have accurate geartrains or encoders nor have a rigid robot. Since the localization is all in software (and software is scalable/free from Google's standpoint) there are potentials for lots of weight and cost savings on the hardware side.
Kind of like my robot:
https://github.com/jonnycowboy/YARRM
I was hoping to use a robotic arm for a project I'm working on, and wondering if you guys could answer a question about motors. In my very limited research it looked like one of the factors that make the industrial (kuka, etc) robots so expensive was that they use backlash-free motors. What does that even mean?
I also saw a couple startups aimed at sub-$5k robots (like carbon.ai). Are they solving this problem in some novel way?
Backlash free motors are motors where the output shaft begins moving as soon as the motor starts moving. In particular, when the motor reverses direction there is no "slack" to pick up before the output shaft starts to move. The slack is called backlash when talking about gears and motors and what have you.
It's important for robots to not have backlash, because as movements are repeated, each bit of backlash adds up into a potentially big cumulative error. It could end up with the robot operating outside of the intended design envelope, which might be a safety problem.
We're getting outside of my home hobbyist experience here, but I think you could guarantee the robot would be in a particular position, but the backlash might make it hard to say when the robot will get to the particular position. Using encoders on all the motors would require having inputs for each encoder, which can get complex.
My guess is you can go pretty far with janky parts if you don't run for long periods of time and also measure where they are.
I think you mean backlash-free gearboxes (ie: cable driven gearboxes, harmonic drives or spring-loaded gearboxes that always apply a minimal but constant tension).
It is difficult, but doable assuming Coulomb friction.
The two main issues are that it is computationally expensive, but also that your mechanical modeling has to closely match that of the actual robot (especially the contact model), otherwise I suspect the training data will be useless in the end.
So if you can afford an actual robot, it makes sense to do the training using it.
How computationally expensive ? Are we talking supercomputer time to simulate the few seconds it takes to grab an object ? Advanced robots are expensive too and usually much harder to get access to then computational ressources (ie. AWS).
It depends: rigid/non-rigid objects, stiffness for non-rigid objects, approximate/exact Coulomb model, spatial/ temporal resolutions, and solution precision.
On a typical desktop computer, that would probably range from real time for the fast/imprecise simulation, to maybe one day for a full-blown simulation.
But again, most roboticists will tell you there is a world between the simulation (even an accurate one) and the actual robot.
