Technical
Article
Gesture Sensors Revolutionize User Interface Control
By Dan JacobsArticle from ams
Gesture Sensors Revolutionize User Interface Control
By Dan Jacobs
Senior Product Manager, ams AG
www.ams.com
Designers face a variety of challenges when selecting the most appropriate type of buttons or con-trols for basic user interfaces.
Mechanical switches are sensitive to reliability risk; they also require increased design effort to pro-tect them from the environment. Electrical controls, such as capacitive or resistive buttons or dis-plays, bypass the problems of mechanical switches, but they require the user’s physical touch to operate.
Optical sensors, in contrast, alleviate reliability risk and mechanical complexity while also enabling touchless interaction. Optical sensors such as proximity detection sensors are found in basic appli-cations such as soap or water dispensers, but the real potential for optical sensors lies in recogniz-ing user gestures that reduce system complexity while enhancrecogniz-ing user functionality. Today’s ges-ture sensors have achieved the ideal combination of functionality, performance, and ease of imple-mentation to revolutionize user interface control.
Highly Functional User Interface Attributes
Highly functional user interfaces are intuitive, dependable and versatile.
Intuitiveness and Dependability
To be intuitive a gesture interface must respond to predictable physical motion and operate only in a controlled field of view.
For the former, device functions should be controlled by user interface actions matching the physical motion that would be used for the same action in other circumstances. For exam-ple, it is intuitive to turn a page or change views on a display with the same sweeping mo-tion as turning a page in a book.
For the latter, users should have clear control of when their actions will affect the interface. This functionality is easier to control with certain types of gesture sensors. For instance, ac-tive gesture sensors synchronize emitted pulses and measure reflected signal, resulting in a controlled field of view and a limited working distance. The field of view and working dis-tance produce a fixed space in which the user interface controls will take effect; any user
action outside that space is safely ignored by the sensor. Once the user is aware of this working space, he or she will be able to interact effectively and intuitively with the interface. The working space is also critical for achieving high dependability. Gesture interface mo-tions can achieve 100% success rate without false positives or negatives thanks to this steady, limited working space.
Versatility
A highly functional gesture interface is versatile enough to meet all user control require-ments and add new functionality that enhances the user interface beyond the capabilities of previous user interface technologies. The precursor to gesture sensing is proximity sensors that provide the system with detect and release events. This information allows the system to start and stop events; for example, turning an automatic water faucet on and off.
Gesture sensors can add the next level of complexity by providing the system with infor-mation about the direction of the user’s motion.
Fig. 1: Possibility of two, four or eight fixed directions
As shown in Figure 1, two fixed directions (for example, left and right or up and down) can be used for various controls such as page turning and volume. Four fixed directions or eight fixed directions are possible with the same types of sensors, enabling even more complex options such as controlling volume or changing tracks on a music player or radio. And be-yond eight fixed directions, the angular direction of the motion could be used by a system, as shown in Figure 2.
Fig. 2: Possibility to move in any angular direction
For example, a user can scroll around a display of a 2D or 3D image in any direction with gestures that match the motions they would use to move the equivalent 2D or 3D object with his or her hand.
High Performance Standards
Today’s gesture sensors are more dependable and versatile thanks to high performance standards. The “sweet spot” for active gesture sensors is to recognize the motion of a finger or hand at 10 cen-timeters to 20 cencen-timeters above the sensor without consuming much power.
The trade-off between working distance and power consumption depends most on the emission effi-ciency and signal-to-noise (SNR) ratio of the sensor. Current gesture sensors have low enough noise that they are capable of working at the 10 centimeter to 20 centimeter sweet spot with an av-erage active current of 5mA or less.
This power consumption is cut by half in some sensors, such as ams’ TMG3992, which automati-cally combines proximity and gesture detection into two modes. While an object is not present, the TMG3992 goes into a monitor mode that uses only 50% of the power while the user’s hand is not present. Upon first detection, the sensor automatically increases its sensitivity to achieve high SNR during the gesture motion. For gesture applications, the user is motioning only a small portion of the time—typically less than 10%—which means that the TMG3992 feature reduces the overall power consumption by nearly 50%.
Ease of Implementation
New gesture sensors are promising tools for users, and are practical for product developers be-cause of their ease of implementation. Most electronic devices already use microcontrollers and an I2C interface, and many practical gesture sensors are ready to interface with these electronics
effi-ciently.
Gesture sensors such as the TMG3992 have fully functional, I2C-compatible digital interfaces and
do not require significant processor or memory bandwidth to operate. These sensors are interrupt-driven which means that the system only needs to interact with the sensors when a recognized event occurs. Polling data wastes power and processor bandwidth. In addition, reference code and driver tools are readily available for these sensors. Two and four direction gesture sensing applica-tions can be enabled with straightforward electrical and software designs. The mechanical design is similarly uncomplicated. The sensor can work behind plastic or glass that is transparent to infrared light. Many electronic devices use plastic housings that are already transparent to infrared or can easily incorporate these materials without adding complexity or reliability risk.
Novel Applications
Touchless user interfaces will improve a variety of applications in multiple ways.
New User Interface Options
Some trades and activities have restrictions that limit the types of controls and displays available. For example, gloves—particularly heavy ones—limit user interface options. Ca-pacitive touchscreens do not work with most types of gloves, so users need specialty gloves to operate them. Gesture sensors overcome this limitation by working with any type of glove. There are a variety of applications for this technology such as industrial applica-tions—construction, chemical industries, and clean room manufacturing—and recreational applications like cold-weather and aerial sports. For example, a skier could manipulate the functions on his or her self-mounted camera with ease or operate a smartphone while still keeping his or her hands warm.
Similarly, underwater applications also provide a variety of challenges. In this environment, touchscreens do not work. However, gesture sensors are fully functional. While water does attenuate infrared light, which restricts the working distance or demands more power con-sumption, this is a minor restriction when compared with the benefit. For example, the user interface can be greatly simplified with underwater cameras, by using a small gesture sen-sor such as TMG3992 from ams, which can replace several mechanical buttons for a smaller, more reliable interface. In some underwater-capable smartphones, adding the ges-ture feages-ture will allow the phone to take underwater photos without adding dedicated but-tons to the device.
Convenience
Smartphones already have many user interface options offering multiple solutions for a vari-ety of tasks. However there are many situations—such as cooking and exercising—when it is convenient to avoid touching the phone while performing tasks. With gesture controls, a user can interact in a variety of ways, such as checking notifications and scrolling through them. He or she could identify a caller and then select from a variety of options; for exam-ple, answer the call but with speaker already enabled, ignore the call with no response, or ignore the call but send a pre-defined text message or send the caller to voicemail. Some phones feature touchless interaction using proximity detection, but this is limited to only one action and is usually the first step before touching the screen. As shown in Figure 3, gesture sensors make the process completely touchless.
Fig. 3: Control panel that is completely touchless
Other examples of how gesture sensors make life more convenient and versatile include home and business use where gesture sensors can upgrade simple interfaces such as light switches and thermostats. A 4-direction single sensor can integrate on, off, and dimming functions together in a touchless switch. For thermostats, a similar application of the ges-ture sensor can adjust temperages-tures and switch modes and configurations without touch.
Improving Hygiene
Touchless user interfaces also help halt the spread of germs and open the door to improve-ments towards a more hygienic society. Everything that people touch has a chance of being contaminated. A good example is the door handle to enter or exit a public bathroom. In some designs, valuable building space is sacrificed to integrate a double-back style entry to remove the door but maintain privacy. If a gesture sensor is used, this extra building space can be recovered for other uses. The sensors inside and outside of the bathroom could
open the door with a simple downward motion similar to turning a door handle. In particular, the gesture sensor is most useful because it requires this specific feedback—a certain mo-tion within a certain distance—to open the door. Proximity detectors would generate false positives whenever a person walks in front of the sensor.
While public hygiene is important, cleanliness in a medical environment is critical for health and safety. There are many instruments in hospitals and at surgery, and each one requires some user interface and manipulation. With gesture detection integrated, medical staff can avoid touching more surfaces, particularly those surfaces that other people must also touch. For non-sterile medi-cal environments, this may be more important. There can be heavy traffic in a hospital between pa-tients, doctors, nurses, and visitors. Each instance where touch is removed within the hospital re-duces the risk of infections spreading or contamination.
The Right Balance
Today’s active gesture sensor components have application benefits that outweigh the burdens of implementation and adoption, benefiting both manufacturers and users alike. For product designers, today’s gesture sensors simplify system design and increase user control options. For users, these sensors provide a variety of benefits to a number of applications, with improvements ranging from evolutionary to revolutionary. There is a balance that is best achieved with today’s lower-power ac-tive gesture sensors, such as the TMG3992. The goal is to painlessly add new functions or open new possibilities in user interfaces in ways that users find intuitive and simple to adopt, and these active gesture sensors strike the right balance.
For more information about ams’ TMG3992 optical sensor, please go to