Using a handheld device, Ajay Mandelkar, Jim Fan and Yuke Zhu use their software to control a robot arm. Source: L.A. CiceroUsing a handheld device, Ajay Mandelkar, Jim Fan and Yuke Zhu use their software to control a robot arm. Source: L.A. Cicero

Students at Stanford University have developed two frameworks that help robots learn from humans rather than from a large data set or by exploring their environment. The frameworks speed up the learning process.

Bender is one of the robots the students used to test the frameworks. Bender has a screen attached to a red robotic arm and two cartoon eyes pop up on the screen, giving Bender a face. Bender learned how to do tasks, like picking up objects and moving them, from humans.

The first framework, RoboTurk, allows people to direct robotic arms in real time. The team used an app and a browser on a smartphone to tell Bender how to pick up a block of wood shaped like a steak. RoboTurk shows the robot how to carry out such tasks.

The second framework is SURREAL. SURREAL speeds up the learning process by running multiple experiences through the robot’s algorithms at the same time. This allows robots to learn from multiple experiences at one time, significantly speeding up the process.

"With RoboTurk and SURREAL, we can push the boundary of what robots can do by combining lots of data collected by humans and coupling that with large-scale reinforcement learning," said Ajay Mandlekar, Ph.D. student in electrical engineering and member of the team that developed the frameworks.

But the researchers know that their process isn’t perfect yet, with the majority of mistakes made due to human error. In one of the testing sessions, Bender dropped the wooden steak in the wrong area.

"Humans are by no means optimal at this," Mandlekar said, "But this experience is still integral for the robots."

RoboTurk gives robots background knowledge to jumpstart learning. SURREAL runs thousands of simulated experiences through the robot so the overall process of learning a new task is significantly sped up.

"With SURREAL, we want to accelerate this process of interacting with the environment," said Linxi Fan, a Ph.D. student in computer science and a member of the team. These frameworks drastically increase the amount of data for the robots to learn from.

"The twin frameworks combined can provide a mechanism for AI-assisted human performance of tasks where we can bring humans away from dangerous environments while still retaining a similar level of task execution proficiency," said postdoctoral fellow Animesh Garg, a member of the team that developed the frameworks.

"You shouldn't have to tell the robot to twist its arm 20 degrees and inch forward 10 centimeters," said Yuke Zhu, a Ph.D. student in computer science and a member of the team. "You want to be able to tell the robot to go to the kitchen and get an apple."

The team is presenting their work at the Conference on Robot Learning 2018 in Zurich, Switzerland.