Haptic holography, also known as mid-air haptics, has the potential to bring virtual reality to life, but a recent study has shown an unexpected physical barrier that must be overcome.
Image Credit: metamorworks/shutterstock.com & Vector Tradition/shutterstock.com
A team of researchers at UC Santa Barbara has found a new phenomenon that underpins emerging holographic haptic displays and has the potential to develop more engaging virtual reality experiences. The team’s findings were published in the journal Science Advances.
Holographic haptic displays employ phased arrays of ultrasonic emitters to concentrate on ultrasound in the air. They enable users to touch, feel, and control three-dimensional virtual objects in mid-air with their bare hands, eliminating the need for a physical device or interface.
While these displays show tremendous potential for usage in different application areas, including augmented reality, virtual reality, and telepresence, their tactile sensations are diffuse and feeble, like a “breeze” or “puff of air”.
“Our new research explains why such holograms feel much more diffuse or indistinct than would be expected,” says Yon Visell, an associate professor in UCSB's College of Engineering specializing in interactive technologies, particularly haptics, robotics, and electronics.
The study, conducted by Yon Visell and Gregory Reardon, a doctoral student researcher, employed laser vibration measurement, simulations, and perceptual studies to thoroughly investigate ultrasound-excited waves that occur in the skin during haptic holography.
They observed that holographic displays cause extensive skin vibration patterns (known as shear shock waves).
According to Visell, shock waves are formed when ultrasonic waves are focused and scanned in mid-air, creating vibrations in the skin. These vibrations can interfere with one another, increasing their strength in some regions, a process called constructive interference.
The creation of shock waves causes a trailing wake pattern to extend beyond the intended focal point, diminishing tactile perceptions' spatial accuracy and clarity.
According to the researchers, if the concentrated sound beam represents a fast-moving boat on the water, the shock wave pattern is a wake trailing the boat. Currently used holographic haptic displays produce shock wave patterns so dispersed in the skin that the sensations feel diffuse.
The researchers employed a PSV Polytec Scanning Vibrometer to image the vibrations of human hand surfaces while providing holographic haptic feedback. The findings indicated a strong presence of ultrasound-evoked shear shock wave patterns in the skin.
“Our study reveals how holographic haptic displays – a promising new technology for virtual reality and telepresence – require new knowledge in acoustics innovations in design,” Visell explains.
“By understanding the underlying physics of ultrasound-generated shear shock waves in the skin, we hope to improve the design of haptic holographic displays and make them more realistic and immersive for users.”
“Such haptic displays could enable us to augment our physical surroundings with a limitless variety of virtual objects, interactive animated characters, or graspable tools that can be not only seen but also touched and felt with the hands."
After anticipating the shear shock wave using numerical models, Visell and his team employed a scanning laser vibrometer to visualize the actual wave patterns produced by focused ultrasound on the skin. Subsequent perception investigations correlated the wave patterns with user perception to determine their impact on user experience.
Fig. 1 depicts the experimental setup. The scanning vibrometer collected surface vibrations triggered by a tissue phantom with mechanical characteristics similar to human skin. This phantom was activated by focused ultrasound delivered from a phased array of ultrasound transducers depicted in green (Fig. 1A).
Finally, in vivo vibrometry measurements were obtained from human hands at more than 300 measurement points (Fig. 1B).
Figure 1. Time-resolved optical vibrometry characterizations were obtained during focused ultrasound stimulation of a tissue phantom and human hands. Image Credit: Courtesy of Authors
When the ultrasound source scanned across the surface at transonic to supersonic speeds, large-amplitude shear shock wave patterns with speed-dependent wake angles were produced, agreeing with wave mechanics and numerical simulation results (Fig. 2 right).
These shear shock patterns lagged behind the focal site and developed in milliseconds. The surface area of their wake zone was orders of magnitude greater than the effective acoustic focal area. As a result, shock wave formation dominated effective focusing resolution over ultrasound focusing.
Wave energy tracked the focal position in a wake 10 cm or more in length and extended several centimeters in directions perpendicular to the scanning path. Lower scanning speeds (v = 2 m/s, Fig. 2 left) resulted in wave patterns concentrated at the focal point, partly due to viscous damping.
These waves, however, were less in amplitude and frequency; thus, the skin would only feel them faintly. The insets in Fig. 2 show the frequency content of the excited wave patterns. As the Mach number increased, a wider spectrum of higher frequencies was excited.
Figure 2. Shear shock wave formation in a tissue phantom excited via focused ultrasound. Image Credit: Courtesy of Authors
Subsonic Ultrasound Source
Video Credit: Polytec
Supersonic Ultrasound Source
Video Credit: Polytec
The measurement findings agree quite well with the numerical simulation. Visell and his team then measured human hands in vivo (Fig. 3). In these trials, ultrasonic scanning paths extended along an axis of the volar hand surface, from the wrist to the tip of the index finger.
The linear pathways were laterally modulated transverse to the motion direction (resulting in zigzag path shapes), as this modulation has been shown to generate stronger sensations. The lowest and maximum scanning speeds (vl = 1 and 11 m/s) were within the subsonic and supersonic regions.
Low scanning speeds (vl = 1 m/s) produced shear wave patterns that expanded outward from the instantaneous focal site, consistent with theory and observations from the numerical simulations and phantom tissue studies.
At higher scanning speeds (vl > 4 m/s), the patterns created wakes that tracked the motion for 10 cm or more. The length of these wave patterns surpassed the approximate focal width (0.6 cm) by more than an order of magnitude.
Figure 3. Shear shock formation diminishes the perception of haptic feedback via focused ultrasound. Viscoelastic wave patterns excited via focused ultrasound stimulation of the volar hand surface. Image Credit: Courtesy of Authors
These phenomena are represented in human haptic perception, as demonstrated by comparing the findings of the in vivo vibrometry test data (Fig. 4A, blue) to behavioral data acquired from a tactile motion perception experiment (Fig. 4A, red).
During each trial, participants felt a focused ultrasound stimulus that moved from the wrist to the tip of the index finger or vice versa, and they reported the direction of the scanning motion.
Perceptual accuracy was higher at lower scanning speeds (P < 0.0001) and decreased to chance levels at the three highest scanning speeds (vl = 4, 7, and 11 m/s), which resulted in the longest wakes.
An integrated analysis of the perception and vibrometry test data demonstrated that perceptual accuracy decreased monotonically with increasing wake length (Fig. 4B).
Figure 4A. The wake length increases with increasing scanning speed (blue: median, interquartile range, and violin plot). Accuracy of tactile motion perception follows an opposite trend (red: median, interquartile range, and extrema). Image Credit: Courtesy of Authors
Figure 4B. As the wake length increases, the perceptual accuracy decreases monotonically (median and interquartile range are shown). Image Credit: Courtesy of Authors
The team’s discovery of the previously unknown shock wave phenomena that underpin haptic holography is a significant step forward in developing haptic holographic displays that will allow users to interact more realistically and immersively in the future metaverse.
This information has been sourced, reviewed and adapted from materials provided by Polytec.
For more information on this source, please visit Polytec