Advances in artificial intelligence continue to push the boundaries of what machines are capable of. Already, computers can drive cars, understand human speech and defeat the most-skilled human players in games like chess and Go.

Columbia Engineering’s robot starts with zero knowledge about what it is and learns about itself by moving around randomly – like a baby in a crib. Source: Columbia Engineering

Up until now, however, AI has lacked a key aspect of human intelligence: self-awareness.

A team of researchers at Columbia Engineering has taken an important step toward endowing machines with this crucial aspect of intelligence by developing a robot arm that can imagine itself.

The development could lead to AI that is more versatile and adaptable, moving beyond traditional AI that is designed for one task but cannot perform others. Self-driving cars and language recognition are impressive tasks in their own right, but represent narrow-focus applications in which AI is programmed or trained to complete a specific job. AI designed for self-driving cars is incapable of playing a game of chess or even understanding what the game is.

This limitation is a result of AI design. Thus far, AI has been unable to improve itself or learn on its own. AI algorithms have been built on either rule-based, pre-programmed routines or training-based, machine-learning techniques like neural networks. These approaches freeze the AI in a fixed state after the programming or training phases; following the preparation phase, the AI can no longer improve further, learn from errors or learn to deal with new situations.

Blank slate

The AI driving the Columbia Engineering robot is different. The device – a robotic arm with three bending joints, the ability to rotate around its anchor point and a gripping end effector – starts with zero knowledge about what it is and no understanding of the rules that govern its existence, such as physics and geometry. From this blank slate, the robot learns on its own about its form to enable it eventually to succeed in a task to pick up an object and place it in a jar.

Columbia’s robot generates its own self-model through deep learning, references its model in performing tasks, and can even detect damage to itself by recognizing discrepancies between its self-model and how its body is performing in reality. Source: Columbia Engineering. (Click image to enlarge.)

The robot begins its life much like a human baby in a crib, moving around randomly, bending its joints and rotating through various trajectories to collect data on its range of motion and capabilities. It then generates a self-model – essentially a simulation of itself that it can use to think about how its body will move – by processing the data with a machine learning technique known as deep learning.

Within hours, this internal self-model matched the robot’s physical reality accurately enough to perform a variety of tasks, including writing with a marker and performing a pick-and-place task consisting of gripping a series of objects on the ground, lifting them to a jar, and dropping them into it. The robot accomplished this task in a closed-loop, step-by-step process, adjusting its position at each step based only on its self-model.

The robot was even able to recover from damage. Researchers tested this ability by replacing a piece of the arm with a different 3D-printed shape. The robot was able to notice the change and update its self-model to successfully continue the pick-and-place job.

The new research is a pioneering foray into a world where self-aware robots create their own self-models. Previous robots have executed pre-programmed routines based on models explicitly created by humans.

“If we want robots to become independent, to adapt quickly to scenarios unforeseen by their creators, then it’s essential that they learn to simulate themselves,” said Hod Lipson, director of the Creative Machines Lab at Columbia Engineering, in a press release.

[Discover industrial robots on Engineering360.]

The work is supported by the Defense Research Projects Agency (DARPA) through its Lifelong Learning Machines (L2M) program, which is seeking to develop techniques that enable AIs to improve themselves over time and adapt to new situations. To achieve this goal, old rigid models for AI must be replaced with flexible architectures based on continuous adaptation mechanisms.

Dactyl’s shadow

Columbia’s robotic arm isn’t the only machine learning how to move in a purposeful manner. AI research lab OpenAI is teaching a robotic hand to manipulate objects. While Columbia’s arm learns about itself through random motion in the real world, OpenAI takes a different approach. It trains the robot hand in a simulated environment and transfers the knowledge to the real world.

Dactyl learns to manipulate objects with human-like fidelity by training neural networks with data from a massive number of simulations. Source: OpenAI

OpenAI’s system, called Dactyl, uses the Shadow Dexterous Hand – a robot hand with 24 joints and an electric “smart motor” actuation system built to imitate a human hand as closely as possible, including its size, shape, dexterity and range of motion. The Shadow Hand has been on the market since 2005 but hasn’t seen much use, which the researchers speculate is due to the difficulty of developing an effective control system for the complex device.

With machine learning, however, OpenAI has created an autonomous control system for the hand that can change the orientation of simple objects, such as a cube, with a level of dexterity on par with a human hand.

The researchers accomplish this by combining two machine learning approaches. The first uses a convolutional neural network to predict the pose, or orientation, of the object based on images from cameras pointed at the robot. The object-orientation estimates from this pose-prediction network, along with fingertip positions, are fed to a second network. This network in turn consists of a recurrent neural network and reinforced learning system that uses this information to inform a control policy that decides what motions the robot should perform to manipulate the object.

The researchers train these two machine learning networks with data collected from a huge number of simulations that run through randomized parameters like object size, surface friction coefficients and actuator force gains.

Through these simulations, Dactyl on its own learned a variety of the same manipulation techniques that humans use to move objects held in their hands, including finger pivoting, sliding and finger gaiting. Dactyl accomplished this with no prior knowledge or demonstration of the human techniques.

Dactyl autonomously learned several emergent behaviors that mirror the techniques used by humans to manipulate objects in their hands. Source: OpenAI

Illusion becomes reality

The field of robotics may be moving into new territory as it works toward self-aware machines able to learn on their own.

“Self-awareness will lead to more resilient and adaptive systems, but also implies some loss of control,” said Lipson and co-author Kwiatkowski. “It’s a powerful technology, but it should be handled with care.”

OpenAI’s robotic hand setup, with the Shadow Dexterous Hand in the center, surrounded by PhaseSpace motion tracking cameras that track fingertip locations and Basler RGB cameras that capture images of the object held in the hand. Source: OpenAI

For example, humans have time to think about how to design these machines and the consequences of their creation. The robotic AIs developed thus far have reached only primitive states of self-awareness, if it can be called that at all.

The actual level of self-awareness of Columbia’s robot is a matter of debate. Its creators are setting up a framework for its AI to learn and directing the robot to complete specific actions. OpenAI, meanwhile, makes no claim of self-awareness with Dactyl, but its AI is also, in effect, creating a self-model by learning through experience.

Some humans argue that machines will never be truly self-aware or conscious, but only capable of mimicking those attributes that have remained up until now uniquely human.

As AI continues to advance the philosophical nuances of consciousness may not matter if robots are able to simulate self-awareness so accurately that they appear to any human in any situation to be self-aware. In that case, illusion will have become reality.

Further Reading

Kwiatkowski, R., & Lipson, H. (2019). Task-agnostic self-modeling machines. Science Robotics, 4(26), eaau9354.

OpenAI, Andrychowicz, M. et al. (2018). Learning Dexterous In-Hand Manipulation. CoRR, abs/1808.00177.