(left) A pair of Wi-Fi transceivers are inserted outside. The transmitter sends a wireless signal whose received power (or magnitude) is measured by the receiver. Then, given the video footage on the right — and by using only such received power measurements — XModal-ID can determine if the person behind the wall of the left figure is the same person in the video footage. Source: B. Korany et al.Researchers from the University of California, Santa Barbara, have developed technology that is capable of matching a subject captured in video footage to that same subject concealed behind a wall using just a pair of external Wi-Fi transceivers.

The researchers refer to the system for matching the subject, dubbed XModal-ID (which is pronounced Cross-Modal-ID) as a video-Wi-Fi cross-modal gait-based person identification system.

To demonstrate how the technology works, the team placed one Wi-Fi transmitter and one Wi-Fi receiver behind the walls outside of a room where a person is walking around. As the person walks, the transmitter communicates a wireless signal whose received power is gauged by the receiver. Meanwhile, video footage of the subject, captured from another location, is given to the system where the receiver can discover if the subject walking behind the wall is the same person captured in the video footage.

This development is largely due, according to researchers, to the uniqueness of a person’s gait. To determine if a subject’s movements as captured in the video footage match those of the movements of the subject concealed by the wall, the team worked with the video footage and used a human mesh recovery algorithm to glean the 3D mesh detailing the human body’s outer surface as a function of time. The team then applied Born’s electromagnetic wave approximation to mimic the RF signal that would have been produced if the subject had been moving through a Wi-Fi area.

The team then applied the time frequency processing approach to gather key gait features from the real Wi-Fi signal as measured from behind the wall and the video-based mimicked signal. Those two signals are eventually compared to reveal whether or not the person in the Wi-Fi area is the same person as the one who had appeared in the video.

The team’s processing pipeline includes a variety of mathematical functions, including the short-time Fourier transform and Hermite functions, to capture the spectrogram of the received signal. "A spectrogram carries the frequency-time content of the signal, which implicitly carries the gait information of the person," explained Belal Korany, a Ph.D. student involved in the research.

Several key gait features are then collected from both spectrograms and compared to determine if the subject in the video is behind the wall.

During testing of the technology, the team reportedly achieved an 84% rate of accuracy in correctly identifying the person behind the wall.

The system is designed for an assortment of applications that range from security and surveillance to the smart home. An example of a possible use case for the system, according to researchers, might be law enforcement agents obtaining footage of a robbery and believing that a suspect is hiding out in a particular house. Using the system, law enforcement might be able to potentially use the pair of Wi-Fi receivers placed outside of the house to determine whether the person within the structure is actually the suspect appearing in the video based on their gait.