Last week, I ventured a whopping 15 minutes from my house to see robots do some mind-boggling, jaw-dropping stuff.
I visited the Cambridge, Massachusetts, offices of a startup called Generalist AI, where I watched robot arms perform simple chores like stacking cups, putting blocks into bowls, and the like. I was astonished by how quickly they figured things out—it was reminiscent of a flesh-and-blood person.
The arms mastered a range of tasks after ingesting a short, instructional video and, most impressively, no specific training for a given task. One of the most striking examples involved a robot that was instructed to sweep a block into a bowl using a dustpan and brush. When the brush was removed from the scene, the robot improvised by using the dustpan like a brush and flicking the block into the bowl.
Courtesy of Generalist AI
In another case, a two-armed robot watched a videoclip of someone unzipping a purse before removing some banknotes. I watched—somewhat slack-jawed—as the robot unzipped a different kind of purse and carefully removed the notes. Most amazingly, when it couldn’t grab the money, it switched from using its right gripper to its left to get a better angle of attack. “Ha,” said one engineer standing nearby. “It never did that before.”
“This is exactly the kind of thing people were really excited about with GPT-3,” Generalist cofounder and CEO Pete Florence told me, in reference to OpenAI’s breakthrough large language model, released in 2020. “You could take that model and just prompt it to do a new task and it would have a real shot at doing it.”
Generalist appears to be focused on teaching its robots about the physics of the world, which seems inspired by the intuitive sense of physics humans exhibit from an early age. That may well contribute to the model’s ability to transfer what it has learned in one scenario to another. In fact, some of the company’s demos made me think of how children improvise and experiment when shown a task. The researchers have often been surprised by what the robot decides to do—one chose to sweep up items with a banana when it was placed in front of it, for example. This might seem trivial, but physical intelligence is something still largely lacking in machines, and the way babies learn so efficiently about their world may offer important insights for AI researchers.