Toyota Teaching Robots to Properly Use Their Fingers Without Code Programming

By clumsy I mean most of them are not capable of performing delicate operations like pouring water into a glass or moving around objects that can easily be deformed. When they do become capable of doing so, that only happens at the end of a long and complicated programming process, and requires a lot of trial and error.

The Japanese from Toyota believe they are on the verge of a breakthrough that should allow humans to faster and better teach robots how to do such things. And that breakthrough requires not a single line of code to be fed to the robot.

The project is being run by the Toyota Research Institute (TRI), and the first details on it were announced this week. If anything, it paints a picture of an approach that may very well revolutionize robotics.

Traditionally, machines are not taught, but programmed to perform tasks through lines of code. Depending on the complexity and finesse of the task at hand, that code may require a lot of work and time, and a lot of test cycles after that.

Toyota does not use code for its teaching method but shows the robot how to use cameras, tactile sensors, and learned representations of the objects that need to be manipulated.

Here's how that works. A robot's behavior model is subjected to learning lessons at the hands of a "teacher," as Toyota calls its people. The teacher performs haptic demos, and feeds the machine with a language description of the task at hand.

The robotic system, loaded with an AI-based tech called Diffusion Policy, simply learns the skill it was shown, and it is able to then perform it consistently and repeatedly. It is, if you like, not that different from how kids learn, just by simply watching and replicating actions.

Diffusion Policy is something Toyota and researchers from Columbia University came up with, and it's described as "a new way of generating robot behavior by representing a robot's visuomotor policy as a conditional denoising diffusion process."

The idea was already put to the test on robots, testing four manipulation benchmarks across 12 different tasks. For the Toyota tests, the robots used had been custom-built for dexterous dual-arm manipulation and fully equipped with haptic feedback and tactile sensing.

It seems the project is in fairly advanced stages, despite it kicking off in the truest sense no more than a year ago. To date, Toyota says it has taught its robots no less than 60 dexterous skills, but plans a huge leap in that number by the end of the year.

Because this way of programming robots is so effective, Toyota is confident it can "quickly and confidently teach robots" new tricks, but also ways to function in new scenarios. The skills its robots are about to get are not limited to pick and place operations, but will allow the machines to interact with their surroundings in more creative and useful way.

The goal Toyota has set for itself is to teach robots hundreds of new skills by the end of the year (that's just three months from now), and to reach 1,000 by the end of the next one.

Some of you may of course have some questions as to where all of this is going, but the company has an answer for that too. The stated goal of the research is to "one day allow robots to support people in everyday situations and unpredictable, ever-changing environments."

These robots and the new tricks they are capable of performing are not meant to replace people, TRI says, but amplify their capabilities. Only time will tell if that'll be indeed so or not.

As far as safety is concerned, strong but undisclosed safeguards are in place, and are meant to protect both the robots themselves and the humans working around them.

Toyota does not say how far in time we are from a real-life use of a robot taught through Diffusion Policy, but at the rate things seem to be moving, we're probably closer than we think.

A paper on Diffusion Policy was published earlier this year. Its findings are meant to become a building block for something called Large Behavior Models (LBMs), which are basically generative AI teaching techniques for physical machines, not just their brains.

Until we get to a point when we can see robots build and pilot spaceships on their own, you can see them in the video below pouring liquids and solids into bowls, peeling potatoes, operating a manual mixer, and all sorts of other mostly kitchen-related things we human see as mundane.