Imec Adds Machine Learning to its 140 GHz Radar Technology to Enable Intuitive Man-Machine Interactions

This week, at FutureSummits 2019, imec, a world-leading research and innovation hub in nanoelectronics and digital technologies, presents a compact highly-sensitive 140 GHz MIMO  (multiple-input multiple-output) radar system. 

The MIMO setup is demonstrated for gesture recognition, supporting intuitive man-machine interactions. In addition, the ultra-fine resolution of this radar allows the detection micro-skin movements related to vital signs serving applications like non-contact driver monitoring or patient monitoring.

Key differentiators of imec’s 140 GHz radar-on-chip prototype system are its small size and high radar performance – in terms of resolution and motion sensitivity. The radar operates up to 10m range, with 15 mm range resolution and 10 GHz of RF bandwidth. Multiple antenna paths are incorporated to enable a complete (virtual) 1x4 MIMO configuration to achieve angular target separation. The transceiver chip features on-chip antennas, and are integrated in 28 nm bulk CMOS technology, ensuring a low-cost solution at high volume production. These properties make the radar system particularly appealing for applications where high-precision, small-motion based detection is key.

By adding machine learning capabilities, imec has now demonstrated the feasibility of the radar to detect and classify small motions based on Doppler information. “This opens new opportunities, for example, enabling gesture recognition for intuitive man-machine interactions”, adds Barend van Liempd, R&D manager at imec. “Think about the AR/VR space, where the new radar can support intuitive interaction with virtual objects. Gesture recognition can potentially also enable intuitive device control – complementary to existing interfaces such as voice control or smart touch screens.” Being insensitive to lighting conditions and preserving privacy (a radar can so far not recognize humans), a radar solution has particular advantages over other types of motion sensors, for example time-of-flight-based infrared cameras. And, being extremely compact, imec’s 140 GHz radar system can be integrated invisibly in almost every device, such as laptops, smartphones or screen bezels.

Imec has developed a specific machine learning algorithm based on a multi-layer neural network including an LSTM layer and using supervised learning to train the inference model by using in-house labeled recordings of more than 25 people, including several captures for each of 7 different gestures. Against the experimental dataset, the model classifies the recorded 7 gestures and predicts the right gesture at least 94% of the time.

Aside from gestures, vital signs can also be measured with very high precision thanks to the high radio frequency. Therefore, the radar is an excellent candidate for in-car vital sign monitoring systems, to enable non-contact tracking of the driver’s state, e.g. to detect falling asleep, abnormal stress levels or possibly to prevent accidents due to acute health hazards, e.g. heart or epilepsy attacks. Another possible application is to monitor small children using motion and vital signs detection, even when the infant is covered by a blanket and asleep, e.g. to provide an alert in case a child is unintentionally left in a vehicle.

To enlarge data richness and spatial information, imec is currently building a 4x4 MIMO radar system, for which a new generation of radar chips is under development – incorporating the TX and RX as separate chips. This will allow a greater flexibility in distributing the MIMO array elements across the available area. It will also be explored if the functionality of the standalone radar chips can be increased, to enable MIMO systems with even larger arrays of chips.

Imec’s 140 GHz radar was developed in its open innovation R&D collaborative program on radar technology. Interested companies can partake in the program, or in a bilateral R&D project, or license the technology building blocks.