Modified virtual reality tech can measure brain activity

Researchers have modified a commercial virtual reality headset, giving it the ability to measure brain activity and examine how we react to hints, stressors and other outside forces.

The research team at The University of Texas at Austin created a noninvasive electroencephalogram (EEG) sensor that they installed in a Meta VR headset that can be worn comfortably for long periods. The EEG measures the brain's electrical activity during the immersive VR interactions.

The device could be used in many ways, from helping people with anxiety, to measuring the attention or mental stress of aviators using a flight simulator, to giving a human the chance to see through the eyes of a robot.

"Virtual reality is so much more immersive than just doing something on a big screen," said Nanshu Lu, a professor in the Cockrell School of Engineering's Department of Aerospace Engineering and Engineering Mechanics who led the research. "It gives the user a more realistic experience, and our technology enables us to get better measurements of how the brain is reacting to that environment."

The research is published in Soft Science.

The pairing of VR and EEG sensors has made its way into the commercial sphere already. However, the devices that exist today are costly, and the researchers say their electrodes are more comfortable for the user, extending the potential wearing time and opening up additional applications.

The best EEG devices today consist of a cap covered in electrodes, but that does not work well with the VR headset. And individual electrodes struggle to get a strong reading because our hair blocks them from connecting with the scalp. The most popular electrodes are rigid and comb-shaped, inserting through the hairs to connect with the skin, an uncomfortable experience for the user.

"All of these mainstream options have significant flaws that we tried to overcome with our system," said Hongbian Li, a research associate in Lu's lab.

For this project, the researchers created a spongy electrode made of soft, conductive materials that overcome those issues, an effort led by Li. The modified headset features electrodes across the top strap and forehead pad, a flexible circuit with conductive traces similar to Lu's electronic tattoos, and an EEG recording device attached to the back of the headset.

This technology will play into another major research project at UT Austin: A new robot delivery network that will also serve as the largest study to date on human-robot interactions.

Lu is a part of that project, and the VR headsets will be used by people either traveling with robots or in a remote "observatory." They will be able to watch along from the robot's perspective, and the device will also measure the mental load of this observation for long periods.

"If you can see through the eyes of the robot, it paints a clearer picture of how people are reacting to it and lets operators monitor their safety in case of potential accidents," said Luis Sentis, a professor in the Department of Aerospace Engineering and Engineering Mechanics who is co-leading the robot delivery project and is a co-author on the VR EEG paper.

To test the viability of the VR EEG headset, the researchers created a game. They worked with José del R. Millán, a faculty member in the Chandra Family Department of Electrical and Computer Engineering and the Dell Medical School and an expert in brain-machine interfaces, to develop a driving simulation that has the user press a button to react to turn commands.

The EEG measures the brain activity of the users as they make driving decisions. In this case, it shows how closely the subjects are paying attention.

The researchers have filed preliminary patent paperwork for the EEG, and they’re open to partner with VR companies to create a built-in version of the technology.

Other members of the research team include Hyonyoung Shin, Minsu Zhang, Nicholas Riveira and Susmita Gangopadahyay of the Chandra Family Department of Electrical and Computer Engineering; Andrew Yu, Heeyong Huh, Zhengjie Li, and Yifan Rao from the Department of Aerospace Engineering and Engineering Mechanics; Sangjun Kim from the Walker Department of Mechanical Engineering, Jessie Peng of the Department of Biomedical Engineering; and Gubeum Kwon of Artue Associates Inc. in South Korea.