The 'pick-and-place' system consists of a standard industrial robotic arm that the researchers outfitted with a custom gripper and suction cup. Source: Melanie Gonick/MIT

Smart devices and robotics are being developed to help ease human life every day. The newest development in helpful robotics comes from MIT and Princeton University. Researchers have developed a robotic system that they hope will be able to help humans do tasks around the house like organizing and putting away groceries. They also believe that this robot could be helpful in warehouse environments, clearing debris and more.

The researchers called this system the “pick-and-place” system. It entails a standard industrial robotic arm that has an outfitted custom gripper and a suction cup that helps the system pick up items. The researchers developed an algorithm called “object-agnostic.” This algorithm allows the robot to evaluate a bin of objects and then figure out how to best grip and organize the item. The robot doesn’t even need to know anything about the object before it picks it up.

When the robot picks up the item and lifts it out of the bin, a series of cameras takes photos of the image from many angles. The robot then uses the algorithm to compare the item it is holding to previous images taken of other items so it can figure out the best way to store the item.

The robot follows a “grasp-first-then-recognize” method. Experiments have proven that this is an effective sequence when compared to other similar technologies.

"This can be applied to warehouse sorting, but also may be used to pick things from your kitchen cabinet or clear debris after an accident. There are many situations where picking technologies could have an impact," says Alberto Rodriguez, the Walter Henry Gale Career Development Professor in Mechanical Engineering at MIT.

Usually, pick-and-place technologies have many applications. There are already some of these systems in use, but they are made only for controlled environments.

Most of the current industrial picking robots have been developed to do one task. But the team wanted to redesign these types of robots to be more flexible, adaptable and intelligent for less rigid settings like a warehouse or home.

The new system has two general operations: picking up an object and recognizing the object once it has picked it up.

The robot arm has three main grasping behaviors: suctioning onto an object vertically or from the side, gripping the object vertically like a claw in an arcade game, or gripping an object that is against a wall using a flexible spatula to pull the object back from the wall.

The team showed the robot images of bins with objects from the robot’s eye and then the robot determined which objects it could grasp. They did this for hundreds of objects so the robot could build up many examples for it to compare new objects to later on. This information was loaded into a deep learning network.

"We developed a system where, just by looking at a tote filled with objects, the robot knew how to predict which ones were graspable or suctionable, and which configuration of these picking behaviors was likely to be successful," Rodriguez says. "Once it was in the gripper, the object was much easier to recognize, without all the clutter."

To further help the robot determine the size and shape of objects, the team developed a perception system. They created a library of images of products from online and then further developed a learning algorithm that relates pixels in an image to add a label to the object.

"We're comparing things that, for humans, may be very easy to identify as the same, but in reality, as pixels, they could look significantly different," Rodriguez says. "We make sure that this algorithm gets it right for these training examples. Then the hope is that we've given it enough training examples that, when we give it a new object, it will also predict the correct label."

The new robot had a 54 percent success rate in picking up objects with the suction and 75 percent success rate with the grasping technique. It was also able to recognize objects with 100 percent accuracy.

"Picking in unstructured environments is not reliable unless you add some level of reactiveness," Rodriguez says. "When humans pick, we sort of do small adjustments as we are picking. Figuring out how to do this more responsive picking, I think, is one of the key technologies we're interested in."

"The gripper now has tactile sensors, and we've enabled a system where the robot spends all day continuously picking things from one place to another. It's capturing information about when it succeeds and fails, and how it feels to pick up, or fails to pick up objects," Rodriguez says. "Hopefully it will use that information to start bringing that reactiveness to grasping."

To read more about this new robotics system, visit the MIT site here.