EEG and Eye Tracking

 

 

Fast Quantitative Brain Mapping in Real Time Combination with Eye Tracking

Introduction

Since the discovery of the EEG by Hans Berger visual analysis of the signal curves has only provided limited success with respect to understanding brain function or deviation from normality. However, Hans Berger published a paper together with Dietsch already in 1932 proposing quantitative assessment of the EEG by frequency analysis. Common use of this approach had to await help by computers for faster calculation. For this type of analysis, time periods of 2s and longer were used in order to be able to evaluate also the extent of slow waves. But the brain works faster than that. Processing time for a visual or auditory stimulus in the brain is about 300 to 400 ms. On the base of certain preconditions we now describe a new approach for fast dynamic quantitative analysis of the EEG including documentation of frequency changes by means of electric brain maps. Interpretation of single short-term maps of less than 400 ms duration is achieved by use of real time combination with conventional eye tracking as used in market research.

Methods

Recording of the EEG is performed as published earlier. In short, a 16 channel EEG is taken by using a conventional electrode cap and signals are amplified by commercially available devices like those from DeMeTec GmbH., Langgöns, Germany or g.tec medical engineering, Schiedlberg, Austria. Besides numerical calculation of 6 frequency ranges at 17 electrode positions (102 parameters) data are coded into colour maps by additive colour mixture according to the RGB mode as used in TV, but using spectral colours representing frequencies from 0.25 to 35.00 Hz. Map data are here presented in the so-called “global median mode” (one can also use absolute spectral power values). This mode represents an individual normalization, since spectral power values from all electrode positions are collected for each frequency range followed by calculation of the median value, which is set to 100%. Spectral power values are now calculated in % of this median and used to show the distribution of spectral power for each cortical region of the brain. Frequency content of the spectra after Fast Fourier Transformation (FFT) was transformed into spectral colours. Additive colour mixture according to RGB mode (like in TV) produced maps reflecting all changes of spectral power within one map. Slow frequencies are thus represented by red colour, medium ones by yellow and green colours and fast frequencies by blue colours. An overview on cortical “hot spots”, where main changes in electric activity have been found is depicted in Fig. 1.

 

Fig. 1 Overview on representative cortical areas of electric pattern changes (“Hot Spots” of spectral power) reflecting also changes in chemical neurotransmission as derived from preclinical work.

The newly developed software package neo-CATEEM® from Mewicon med.-wiss. Beratung GmbH, Schwarzenberg, Austria, was used for documentation of changes of spectral frequency content during performance of mental tasks or watching TV commercials besides watching single emotional images. Concomitant eye tracking was performed using the device of Interactive Minds, Dresden, Germany, with NYAN2® software. Changes of the EEG frequency maps were documented by screen capture using Adobe Captivate resulting in a video containing all changes over time. The eye tracking device provided a gaze overlay video in separate on a second independent computer. Both videos – the one reflecting the EEG changes and the one obtained from eye tracking – were now synchronized by using an audio signal presented on begin of the experimental session. This audio signal was transferred from the computer processing the eye tracking data to the computer processing the EEG data and was therefore visible in the corresponding audio time lines of the film cutting software “Final Cut” from Apple. The start of the audio signal in both videos was taken for synchronization of both videos. However, due to the processing time of the brain (300 – 400 ms) and the processing time of the computer (approximately 600 ms) the data from the eye tracking were shifted for 1 second in order to obtain more exact synchronization of both videos. A consecutive sequence of 3 TV commercials, 3 memory tasks and 6 emotional images was presented at the eye track computer. Up to now, eight subjects (four male and four female) took part in this first trial.

Results

From combining quantitative EEG mapping with conventional eye tracking it became obvious that one can monitor the electric activity of the brain with a time resolution of 364 ms. This time window corresponds very well to the processing time of the brain for one auditory or visual stimulus. To our surprise we observed large changes of spectral power from sweep to sweep. For example during performance of a memory task (7 numbers or spells were presented for 5 s for memorizing) very often representative pictures of the momentarily frequency content were observed as documented in Fig. 2, consisting of dominant beta activity (blue colour according to the coding of the maps). Interestingly, there appeared different consecutive maps during the 4 seconds during which one task was worked on. Since every presentation of the number-spell combination was continued by a 10 seconds lasting black screen, memorizing was followed by emergence of a sequence of different maps. Similar enkephaloglyphs were observed for 3 consecutive tasks.

Fig. 2 Short dynamic frequency map (spectral signature of electric brain activity also called an enkephaloglyph) during the performance of a memory task. The task is given in the upper part of the image. Information on the raw EEG signal is documented at the left part of the screenshot, focal individual distribution of the frequencies (in this case dominance of fast beta waves in the left temporal lobe) and quantitative documentation of frequency content at all 17 electrode positions is shown in the lower middle bar graph of the screenshot. Time course for one selected electrode position (in this case T3 according to the 10-20 system) is given on the lower right side of the screenshot.

A different short-term frequency pattern (spectral signature = enkephaloglyph) becomes visible during watching emotional images. One representative spectral signature is depicted in the upper part of Fig. 3 showing an image taken at “Helloween”. Here fronto-temporal increases in theta power dominate as can be seen in the bar graph (orange coloumns).

A different enkephaloglyph merged when the subject looked at a crying kid (lower part of Fig. 3. In this representative example left frontal slow frequency delta spectral power emerges in combination with temporal beta spectral power. In addition to the dominant frequencies as depicted within the map further prominent high spectral power is seen at different other electrode positions as depicted in the bar graph of Fig. 3 in the lower middle part for example in the parietal lobe.

 

Fig. 2 Enkephaloglyph showing the electric reaction to emotional pictures. Information on the raw EEG signal is depicted on the left part of the screenshot, focal individual distribution of the frequencies and quantitative documentation of frequency content at all 17 electrode positions in the lower middle bar graph of the screenshot. Time course for one selected electrode position is given on the lower right side of the screenshot. Please note that in left frontal lobe dominant delta spectral power (red according to colour coding) has emerged in combination to dominant beta spectral power (blue according to colour coding) within the temporal lobe watching at the crying kid but not in the picture taken at “Helloween”.

Finally, the new combination of both technologies also allowed fast dynamic analysis of TV commercials. Comparing now the enkephaloglyphs from four different subjects at a particular scenery of the TV commercial with each other surprisingly similar distributions of spectral power become visible.

Discussion

These preliminary results clearly indicate a very specific activation of electric circuits, much more complex than has been suggested by fNMRI experiments measuring blood flow as indirect representation of neuronal activity. Opposite to this, the current measurement of cortical electric activity probably reflects brain activity to a better degree and in more detail, since electric activity can be related much better to cognition and emotion than in NMRI measurements. However, very accurate synchronization of the gaze overlay film with the video containing the fast dynamic EEG changes is recommended when using such short time periods for analysis. In addition, interpretation of focal changes of electric brain activity has become feasible in terms of neurotransmitter action. In rats it has been shown, that for example slow delta waves are under the control of the cholinergic system, alpha2 waves under the control of the dopaminergic system. There is also evidence, that beta1 waves are controlled by glutamate and beta2 waves by GABA. If these data are confirmed in humans a new base for EEG evaluation may arise. The more data are available from future experiments, the better we will learn the electric language of the brain. The tools to enlighten specific brain activities are available. Data from this combination of eye tracking and EEG recording will help us to better understand our brain.