Learn
The world of robotics today is a lot like the world of computing 50 years ago. There’s lots of talk and optimism about what robots could do to help people in their everyday lives, yet that future is still a ways off. Most robots are where mainframe computers were in the ’60s and ’70s: expensive specialist machines, operated by experts, performing specialized tasks in specially-designed environments.
A series of innovations, most notably the invention of the microprocessor, saw computers become cheaper, smaller, and much more powerful than anyone had thought possible. I believe we’re on the cusp of a similar transition in robotics. Recent advances in machine learning, combined with increasingly sophisticated sensor technology and low cost hardware, mean that we are much closer than ever to robots becoming mainstream.
For the last few years my team, The Everyday Robot Project, has been working to see if it’s possible to create robots that can do a range of useful tasks in the messy, unstructured spaces of our everyday lives. Our moonshot is to see if we can make robots as helpful to people in the physical world as computers now are in the virtual world. It’s still early days, but I’m excited to share some of our progress today.
Everyday environments like our homes or offices aren’t governed by a set of straightforward rules that robots can follow. Think about the people around you right now, wandering as they walk and text, or stopping suddenly to have a chat or make a detour to the refrigerator. Even everyday objects, from chairs to coffee cups, appear, move, and disappear in ways that we expect and anticipate, but that are very mysterious to a robot. Where humans naturally combine seeing, understanding, navigating, and acting to move around and achieve their goals, robots typically need careful instruction and coding to do each of these things. This is why it quickly gets very complicated for robots to perform tasks we find easy in highly changeable environments.
For robots to be useful in everyday environments we need to move away from painstakingly coding them to do specific and structured tasks in exactly the right way at exactly the right time. We have concluded that you have to teach machines to perform helpful tasks; you cannot program them.
In the early days of our project, we did this work in the lab, collaborating with teams at Google AI. Our tests showed that by giving robots simple tasks and then having them practice, it is indeed possible to teach them to develop new and better capabilities. We investigated how robots can learn from human demonstration, from shared experience, and how we can accelerate learning by simulating robots in the cloud. Once we saw what was possible, we began plotting our path out of the lab and into the real world to test their skills on useful tasks and to see if they could do them reliably and repeatedly. This is where we have some optimistic results to report.
Over the last few months we’ve been running an experiment at our offices that puts the robots to work on a task that has just the right amount of complexity: sufficiently hard that we honestly weren’t sure whether it could be done, but not so hard that it would take a year to get a clear “it’s working” or “it’s impossible” signal. We also wanted to do something clearly useful. So we decided to teach robots how to sort waste — dividing cups, bottles, snack wrappers, and more across landfill, recycling, and compost bins.
Everyone has put waste into the wrong bin at some point, whether as an “oops” or because our mind was somewhere else. As a result, contaminated items that could have been composted or recycled have to be sent to landfill. In a typical office, no one sifts through items to check for contamination, so this felt like a valuable problem to put our robots to work on.
For our robots to learn how to do these tasks, we’re using a variety of machine learning techniques. These include simulation, reinforcement learning, and collaborative learning. Each night, tens of thousands of virtual robots practice sorting the waste in a virtual office in our cloud simulator; we then move the training to real robots to refine their sorting ability. This real-world training is then integrated back into the simulated training data and shared back with the rest of the robots so that the experience and learning of each robot is shared with them all.
During the last few months, our robots have sorted thousands of pieces of trash and reduced our office’s waste contamination levels from 20% — which is what it is when people put objects in the trays — to less than 5%. These results are promising for a few reasons. First, they show that we’re able to create a robotic system that integrates all of the robot’s capabilities to do something genuinely useful: reducing the contamination levels to less than 5% successfully diverts waste from going to landfill. Second, they prove that it’s possible for robots to learn how to perform new tasks in the real world just through practice, rather than having engineers “hand code” every new task, exception, or improvement.
Our next challenge is to see if we can take what the robot learned in this task and apply that learning to another task without rebuilding the robot or writing a ton of code from scratch. This could prove to be impossible, but we’ll give it a shot.
As I’ve shared before, building cool robot technology is not an end in itself; instead, we see robots as tools that we can put to work to help us extend our own capabilities. It will be years before the helpful robots we imagine are here, but we’re looking forward to sharing more robot adventures along the way. You can learn more about what we’re up to on our project page.