We have created electronic devices that can engage the human senses of hearing and vision. Smell, taste and touch have been neglected due to a lack of technology to allow these devices to interact with these senses. Recently, however, the electronic world has been developing devices and methods to interface the user and connected device so they can produce tactile feedback and kinematic feedback to the user through the sense of touch by applying forces, vibrations and motion to the users. Force feedback is a term used to describe both tactile and kinematic feedback.

This technology is called “haptics” and its origins can be found in the Greek words haptikos (meaning able to touch) and haptesthai (meaning able to lay hold of). The haptics field is multidisciplinary. It involves engineering, robotics, developmental and experimental psychology, cognitive science, computer science and others.

Haptics incorporates tactile (touch) and kinesthetic (motion) components to produce interfaces that return force feedback about properties of virtual objects created by a computer. For example a haptic joystick produces real (force) resistance to the user while playing a video game, even if the game does not have any real physical object. Yet, if you feel the virtual objects of the game with any accuracy and fidelity, force feedback is precisely the information you receive through the haptic interface. When your smart phone vibrates in your pocket, force feedback is letting you know that a phone message is awaiting your response.

How the system works

The minimum components of a haptics (interface) system must have the following components:

• Sensors
• One or more actuators (motors) that either produces vibrations of force
• Actuator control system (microprocessor)
• Haptic software: real-time algorithms, a haptic effect library, and others
• A user interface, normally called application programming interface (API)

The simple diagram below shows how these components interact together.

Figure 1: Block diagram of a haptic system

The force feedback in this particular diagram is vibration. The haptic vibrations are created by the motor (actuator), and then are controlled by the microprocessor and its embedded software.

A specific example of the relationship between the user and the interface is depicted in Figure 2. Using regular mouse information is unidirectional (arrow directions in the diagram.) The regular mouse user receives no information from its movements, even if the mouse friction and inertial properties could be used for intelligent interaction with the computer. With a haptically enabled mouse the user can receive programmable feedback entirely based on the sense of touch. This allows for a faster and more accurate interaction with the computer. From this example we can conclude that bi-directionality is the most important attribute of haptic interfaces. These devices are designed to “read” from and to “write” to the human hand (or any other body parts). Because of this bi-directional exchange of information between user and computer, haptic technology is closely related to robotics. Like robots, haptic interfaces have special tasks to perform that normally occur mostly through the hands.

Figure 2: Block diagram of a regular and a haptic computer mouse. In haptic interface there is a simultaneous exchange of information between the user and the computer.

Types of haptics

Haptic technology is normally classified intro three categories:

• Human haptics: This is related to human touch and manipulation and provides force-feedback and tactile sensations to users as they interact with a virtual object. This is the way people perceive touch as neuro-physiological signals.
• Machine haptics: The implementation and control of human-computer-interface devices to send and receive force & torque. At present this is mostly dedicated to technology for robot arms and hands.
• Computer haptics: Related to computer-user interactions. Includes simulation (encoding and rendering) and communication of haptic signals in digital environments.

Why haptics?

Professor Srinivasan M.A, Director, Touch Lab, MIT, summarizes why haptic is studied as follows:

“Given the ever-increasing quantities and types of information that surround us, there is a critical need to explore new ways to interact with information. Our haptic system – with its tactile, kinesthetic and motor capabilities together with the associated cognitive processes – present a uniquely bidirectional information channel to our brain and offers such a fantastic medium!”

Haptic interfaces

There are many haptic interfaces on them market. Some specific examples - or those soon to be available - are discussed below.

Haptic paddle. This is a 1-degree of freedom (1-DO) motorized force-feedback joystick designed to allow users to feel forces generated by a virtual environment. It was developed by researchers at Stanford University, but many universities have contributed to the development of inexpensive, portable devices. The driver of current haptic paddle devices is an electromagnetic actuator such as a DC motor to create force feedback at the end of a rotating joystick-like handle.

Figure 3: Haptic paddles developed by Rice University used to teach Mechatronics. Credit: Rice University

Haptic knobs. Typical standard knobs are used in many environments such as in automotive gear shifts. A haptic knob adds intelligence by sending back force signals to the user to more accurately perform the knob's designed operation. A haptic feedback shift knob adds force (haptic) and visual feedback to help drivers shift appropriately. It is also possible to use these devices on touch-sensitive displays such as display screens in mobile telephones, PDAs, smart watches, portable and stationary computers, games consoles, tablet PCs, laptops, etc.

Sensible Phantom Omni. This is a six degree-of-freedom (6-DOF) linkage haptic force-feedback device. It is used in research, OEM designs, 3D modeling, medical training and other applications. In medical applications, for instance, a surgeon can plan and rehearse by operating on a virtual patient. Figure 4 shows the use of this device on cerebral surgery. This device is now called Geomagic^R Touch, after acquisition by the Geomagic Company. The phantom is basically a small robot with three joints each connected to a computer-controled DC motor. A stylus connected to the motors allows the user to excerpts up to 1.5 pounds of force at the tip of a stylus controlled by the user. This interface simulates touching at a single point of contact (the stylus). The three motors give feedback to the user by exerting pressure to the stylus. In this way a surgeon can feel the elasticity of an organ or a general user can feel the solidity of a building wall. The phantom can provide feelings of temperature, texture and weight, among others.

Figure 4: The Cerebral Surgery Rehearsal Platform (SRP) shows Dr. Warren Selman working with a Geometric Touch at the Case University Hospital of Cleveland. Credit: University of Cleveland.

Cyber Globe or Cyber Grasp. This is a haptic globe with sensors that can measure the position and movement of the fingers and the wrist. Cyber globes are used in medical application to simulate the feeling of human organs by working with virtual organs generated by a computer. A cyber grasp system is designed to allow users to feel the size and shape of virtual objects, features that are important in virtual surgical applications.

The Novint Falcon^R Controller

This is a low cost (around $250) interface that provides highly accurate three-dimensional force-feedback. The controller can move in all directions including movements up and down. The Falcon has a detachable grip that is used to move a cursor to interact with a virtual object. Once the cursor pointer is in the desirable location, actuators (motors) in the Falcon turn on letting the user to feel haptic effects such as texture, weight, shape, etc. The motor's operations are updated 1,000 times per second.

The Falcon. Credit: Novint Corp.

Applications

There many areas where haptics is a powerful tool with possible limitless applications in the future. Some of the application areas include:

Medicine

• Simulators for surgery training where medical students can learn delicate surgical techniques on the computer, feeling what it's like to suture blood vessels or inject Botox into the muscle tissue of a virtual face.
• Telemedicine and remote diagnosis.
• Interfaces for the blind and hearing aids for the deaf. Greece is introducing haptic technology into touchable maps for the blind. The first step is to create a video of a real world place such as the Parthenon, a city block, or a whole city. Computer scientists edit each video frame to determine the shape an location of each object in the frame, and add a force field to each structure. Using a haptic interface such as the Novint Falcon, a blind person can feel these forces and along with audio streams, get an almost an accurate feeling of the city or the monument.

Entertainment

• Haptic video games increase the reality of the game giving more satisfaction to the user.

Education

• Giving students a feel of physical phenomena in physics, chemistry and other subjects.
• Hands-on learning will greatly improve with haptic technology.

High risk training

• Aircraft mechanics can diagnose service and repair complex parts of an aircraft just by “touching” virtual images on a computer screen.
• Soldiers can prepare for battle in virtual environments. They can learn how to defuse a ­bomb or how to operate a tank or helicopter in virtual combat scenarios.

Art

• Virtual concert halls where the user can play musical instruments remotely.
• Museums that allow touching and feel of expensive and rare objects without the risk of corrupting them.

Consumer products: smart phones

• Smart phones companies are ready to introduce haptic technology. Samsung is introducing the Anycall Haptic Phone. Nokia is not far behind with their tactile touchscreens, and Apple will introduce “Apple’s Taptic Engine” and “Force Touch” technologies into their new iPhone. These technologies are already part of the latest MacBook and Apple Watch.