Enkephaloglyphs and Marketing Research

Enkephaloglyphs and Marketing Research

Enkephaloglyphs represent spectral signatures of electric brain activity. What does this mean? What is a spectral signature? Spectral signatures are better known from astronomy. Quite a bit of knowledge of astronomy and astrophysics is based on a mathematical transformation named frequency analysis. For example atom-absorption-spectrometry provides information on the composition of distant stars. It is also possible to decompose light in its single colours by use of a prism. We are dealing then with spectral colours, which consist of particular frequencies given in waves per second (named as hertz (Hz)). Within the field of capturing electric brain activity called electroencephalography (EEG), as Hans Berger, the discoverer of human brain electricity, named it, it stands for a transformation of the signals from time dependency into frequency dependency. The result consists in a power-density-spectrum. More details will be given later. This kind of approach using frequency analysis is the base for the mapping of brain electricity as realized in the software-hardware combination of neo-CATEEM^®. This idea of using an additive color mixture of spectral colors for depiction of all frequencies of an EEG within one map (an idea of Hans Carlos Hofmann) is not only still up to date but has put us into the position to create what we now call a spectral signature of electric brain activity or an Enkephaloglyph (Fig. 1 gives an example from a recent marketing research project dealing with web surfing of bank portals).

Fig. 1 Information content of an enkephaloglyph showing numeric content of the current frequency pattern of 10 subjects (bar chart on the left side of the figure) as well as the resulting averaged map. Please remark significant increases of electric power at frontal sites represented by electrode positions F7 and F8 (cognitive process of 364 milliseconds duration). Eye tracking (right) provided the documentation of a “hot spot” representing the location, where the majority of the subjects looked at in this moment.

With the extension of the CATEEM^® system into the direction of higher time resolution completely new applications have arisen for example within marketing research as control for success of commercials or pitch-perimeter advertising during sport events. Just imagine, your TV program is interrupted by a commercial. After some reaction of disappointment you decide to watch “this nonsense” since you don`t have a better idea for the moment. In addition, you never know the duration of the advertisements. But these commercials are sometimes very short (about 10 s) but nevertheless may be very efficient. The advertising industry always is very keen to receive hints on the individual success of their spots or pitch-perimeter advertisements. Since, as we shall see later, electric brain activity mirrors our interior with respect to cognitive as well as emotional features, its analysis can give us information with respect to individual and averaged resonances. In order to correlate electric activity to single scenes of a film or commercial, time resolution must be well beyond one second. Since the original CATEEM^® provided only a time resolution of 4 s, as practiced now for 20 years, this time resolution has not been good enough for this purpose.

In the past we had concentrated only on analyzing the averaged electric brain activity with regard to a certain time. That means, we averaged the electric signal over a time of several minutes and median activity was depicted as a map. Normally, this kind of activity was shown as difference to a baseline value recorded under the condition of open eyes. This has been used for a long time to characterize drug actions. But what about those ultra-quick processes, which we relate to single thoughts? Thanks to a genious idea of the physicist and mathematician Hans-Carlos Hofmann (with whom I work together for more than a quarter of a century) we are able to record the electric features of the brain now with a time resolution of 364 milliseconds. That means we receive about 3 pictures of brain activity per second. This enables us to relate particular scenes from films to the electric pattern in a very exact way. One could say that we capture the reaction of the TV watching people nearly as quick as thoughts come and go.

It has been known for a long time that one can record the electric reaction of the brain in the presence of acoustic or visual stimuli. However, administration of a large number of stimuli and averaging their response is needed. This kind of analysis of brain electric activity is known under the heading of “evoked potentials” in the neurophysiological literature. I have dedicated my effort for many years to this kind of analysis. Inspired by the work of Polich and colleagues my team succeeded to present this kind of brain responses as maps. For example we presented acoustic stimuli 1 second apart from each other during about 5 minutes. High frequency tones were alternated with low frequency tones (1:5) and subjects were asked to count the more seldom tones. Using this approach one can check the ability to concentrate quite well because the brain processes both kind of responses in a different way. From the difference one can calculate the degree of attention. However, this kind of analysis asks for several minutes of testing and the result consists in an average response. From this approach it became obvious that the brain needed about 300 to 360 milliseconds of processing time, which is age dependent. Since this kind of signal is depicted as a positive wave, it has been denominated as “P300” in the literature. This means, that quantitative analysis of single responses to acoustic or visual stimuli would ask for such short time of analysis. Unfortunately, this kind of signal escapes detection in the EEG trace because of the small signal to noise ratio, which means that you cannot see it during the course of conventional EEG recording (only as averaged signal as mentioned above).

However, we have seen that frequency analysis is a valuable tool for describing electric brain activity in an exact manner.  By coding single frequency ranges into a map by use of spectral colours the result can be depicted graphically in order to describe brain functions. After twenty years of practicing frequency analysis of this kind of signal a solution was detected, which allows spectral analysis of these ultra-short grapho-elements of the EEG and depict them graphically according to existing algorithms used already twenty years ago. Based on a sweep time of 364 milliseconds we are now well within the desired velocity of mental processes and thoughts. The result of spectral analysis of the EEG has been named an Enkephaloglyph by myself “(enkephalo” comes from ancient Greek language and means “what is in the head”; a glyph is a kind of pictogram, namely a graphic description in form of a signature). Enkephaloglyphs are electric correlates thought to mirror or reflect psychic processes like cognition and emotion among other mental processes. In any case they represent the electric response of our brain to acoustic or visual stimuli.

One of the problems arising with the availability of the new ultra-fast technology was the interpretation of the obtained Enkephaloglyphs. A picture can tell you more than a thousand words, as a German phrase puts it. That means also that during presentation of pictures or movies it is very important to know where the eyes look at. What part of the picture or video catches our attention? A solution was found by combining our EEG with another well-established technology. Within marketing research other scientists succeeded in developing a method, which now is known under the name of “eye tracking”. Basically, this method registers eye movements and projects its coordinates with high time resolution as a spot on the picture or movie under examination. The momentarily recorded eye-gaze is documented continuously as i.e. a red spot on the video presentation (s. Fig. 2).

In the case of concomitant recording of the EEG, both films - representing the EEG and the eye tracking – can be synchronized. Practically, the eye-tracking system transmits a trigger at begin of the presentation which is seen on the EEG recording by means of our newly developed Windows based neo-CATEEM^®. By it synchronization is achieved in a very accurate way. By use of a film cut program depiction of focused attention of a subject with a time resolution of 364 milliseconds per sweep is achieved. It allows to relate the brain`s electric response to a visual challenge to a very short eye-gaze.

The combination of these two physiological methods opens completely new perspectives for example in marketing research. If evaluation and control of success of advertising like commercials or presentation of logos as well as pitch-perimeter advertising during sport events was only accessible in a very rough manner, the combination of eye tracking and neo-CATEEM^® provides now the possibilities of individual quantitative analysis. First results show that the electric response of the brain differs quite a bit depending on the momentary eye-gaze. Prominent differences are seen between the sport events and gazes on pitch-perimeter advertising. Even the extent of the qualitative reaction as well as quantitative differences to presentations of single companies can be evaluated. On the other side similar enkephaloglyphs were recorded in different subjects with respect to identical advertising presentations. An example of a subject watching different advertisements is documented in Fig. 2 (see also videos on YouTube). It is very interesting that in both cases slow delta and theta activity increases at the lateral cortex (the frequencies of EEG waves have been named historically according to the Greek alphabet). Taking a look on the details of frequency changes one can see a central increase of alpha1 waves (depicted as yellow color) and a reaction on the electrode position F3 representing the frontal brain during watching the Mercedes presentation. One can conclude from this that a thought-association has occurred since similar enkephaloglyphs have been observed during recognition of subjects. Obviously, the goal of the advertisement had been reached. Further studies will lead to construction of a library of enkephaloglyphes allowing extended interpretations.

 

Fig. 2 Ultra short electric reactions on advertisements. Upper part: Pitch perimeter advertising during a socker game with significant increase of central beta waves (blue) and increase of delta, theta and alpha1 waves (red, orange and yellow, respectively) at the forebrain. Lower part: Reaction during a gaze on an advertisement of “Mercedes” (red spot represents momentary gaze), which is quite similar to the reaction on “Gazprom” but not identical.

Further information on the technology is provided at http://www.mewicon.at and its use in research at http://www.neurocode-ag.com    

EEG and Eye Tracking

 

 

Fast Quantitative Brain Mapping in Real Time Combination with Eye Tracking

Introduction

Since the discovery of the EEG by Hans Berger visual analysis of the signal curves has only provided limited success with respect to understanding brain function or deviation from normality. However, Hans Berger published a paper together with Dietsch already in 1932 proposing quantitative assessment of the EEG by frequency analysis. Common use of this approach had to await help by computers for faster calculation. For this type of analysis, time periods of 2s and longer were used in order to be able to evaluate also the extent of slow waves. But the brain works faster than that. Processing time for a visual or auditory stimulus in the brain is about 300 to 400 ms. On the base of certain preconditions we now describe a new approach for fast dynamic quantitative analysis of the EEG including documentation of frequency changes by means of electric brain maps. Interpretation of single short-term maps of less than 400 ms duration is achieved by use of real time combination with conventional eye tracking as used in market research.

Methods

Recording of the EEG is performed as published earlier. In short, a 16 channel EEG is taken by using a conventional electrode cap and signals are amplified by commercially available devices like those from DeMeTec GmbH., Langgöns, Germany or g.tec medical engineering, Schiedlberg, Austria. Besides numerical calculation of 6 frequency ranges at 17 electrode positions (102 parameters) data are coded into colour maps by additive colour mixture according to the RGB mode as used in TV, but using spectral colours representing frequencies from 0.25 to 35.00 Hz. Map data are here presented in the so-called “global median mode” (one can also use absolute spectral power values). This mode represents an individual normalization, since spectral power values from all electrode positions are collected for each frequency range followed by calculation of the median value, which is set to 100%. Spectral power values are now calculated in % of this median and used to show the distribution of spectral power for each cortical region of the brain. Frequency content of the spectra after Fast Fourier Transformation (FFT) was transformed into spectral colours. Additive colour mixture according to RGB mode (like in TV) produced maps reflecting all changes of spectral power within one map. Slow frequencies are thus represented by red colour, medium ones by yellow and green colours and fast frequencies by blue colours. An overview on cortical “hot spots”, where main changes in electric activity have been found is depicted in Fig. 1.

 

Fig. 1 Overview on representative cortical areas of electric pattern changes (“Hot Spots” of spectral power) reflecting also changes in chemical neurotransmission as derived from preclinical work.

The newly developed software package neo-CATEEM® from Mewicon med.-wiss. Beratung GmbH, Schwarzenberg, Austria, was used for documentation of changes of spectral frequency content during performance of mental tasks or watching TV commercials besides watching single emotional images. Concomitant eye tracking was performed using the device of Interactive Minds, Dresden, Germany, with NYAN2® software. Changes of the EEG frequency maps were documented by screen capture using Adobe Captivate resulting in a video containing all changes over time. The eye tracking device provided a gaze overlay video in separate on a second independent computer. Both videos – the one reflecting the EEG changes and the one obtained from eye tracking – were now synchronized by using an audio signal presented on begin of the experimental session. This audio signal was transferred from the computer processing the eye tracking data to the computer processing the EEG data and was therefore visible in the corresponding audio time lines of the film cutting software “Final Cut” from Apple. The start of the audio signal in both videos was taken for synchronization of both videos. However, due to the processing time of the brain (300 – 400 ms) and the processing time of the computer (approximately 600 ms) the data from the eye tracking were shifted for 1 second in order to obtain more exact synchronization of both videos. A consecutive sequence of 3 TV commercials, 3 memory tasks and 6 emotional images was presented at the eye track computer. Up to now, eight subjects (four male and four female) took part in this first trial.

Results

From combining quantitative EEG mapping with conventional eye tracking it became obvious that one can monitor the electric activity of the brain with a time resolution of 364 ms. This time window corresponds very well to the processing time of the brain for one auditory or visual stimulus. To our surprise we observed large changes of spectral power from sweep to sweep. For example during performance of a memory task (7 numbers or spells were presented for 5 s for memorizing) very often representative pictures of the momentarily frequency content were observed as documented in Fig. 2, consisting of dominant beta activity (blue colour according to the coding of the maps). Interestingly, there appeared different consecutive maps during the 4 seconds during which one task was worked on. Since every presentation of the number-spell combination was continued by a 10 seconds lasting black screen, memorizing was followed by emergence of a sequence of different maps. Similar enkephaloglyphs were observed for 3 consecutive tasks.

Fig. 2 Short dynamic frequency map (spectral signature of electric brain activity also called an enkephaloglyph) during the performance of a memory task. The task is given in the upper part of the image. Information on the raw EEG signal is documented at the left part of the screenshot, focal individual distribution of the frequencies (in this case dominance of fast beta waves in the left temporal lobe) and quantitative documentation of frequency content at all 17 electrode positions is shown in the lower middle bar graph of the screenshot. Time course for one selected electrode position (in this case T3 according to the 10-20 system) is given on the lower right side of the screenshot.

A different short-term frequency pattern (spectral signature = enkephaloglyph) becomes visible during watching emotional images. One representative spectral signature is depicted in the upper part of Fig. 3 showing an image taken at “Helloween”. Here fronto-temporal increases in theta power dominate as can be seen in the bar graph (orange coloumns).

A different enkephaloglyph merged when the subject looked at a crying kid (lower part of Fig. 3. In this representative example left frontal slow frequency delta spectral power emerges in combination with temporal beta spectral power. In addition to the dominant frequencies as depicted within the map further prominent high spectral power is seen at different other electrode positions as depicted in the bar graph of Fig. 3 in the lower middle part for example in the parietal lobe.

 

Fig. 2 Enkephaloglyph showing the electric reaction to emotional pictures. Information on the raw EEG signal is depicted on the left part of the screenshot, focal individual distribution of the frequencies and quantitative documentation of frequency content at all 17 electrode positions in the lower middle bar graph of the screenshot. Time course for one selected electrode position is given on the lower right side of the screenshot. Please note that in left frontal lobe dominant delta spectral power (red according to colour coding) has emerged in combination to dominant beta spectral power (blue according to colour coding) within the temporal lobe watching at the crying kid but not in the picture taken at “Helloween”.

Finally, the new combination of both technologies also allowed fast dynamic analysis of TV commercials. Comparing now the enkephaloglyphs from four different subjects at a particular scenery of the TV commercial with each other surprisingly similar distributions of spectral power become visible.

Discussion

These preliminary results clearly indicate a very specific activation of electric circuits, much more complex than has been suggested by fNMRI experiments measuring blood flow as indirect representation of neuronal activity. Opposite to this, the current measurement of cortical electric activity probably reflects brain activity to a better degree and in more detail, since electric activity can be related much better to cognition and emotion than in NMRI measurements. However, very accurate synchronization of the gaze overlay film with the video containing the fast dynamic EEG changes is recommended when using such short time periods for analysis. In addition, interpretation of focal changes of electric brain activity has become feasible in terms of neurotransmitter action. In rats it has been shown, that for example slow delta waves are under the control of the cholinergic system, alpha2 waves under the control of the dopaminergic system. There is also evidence, that beta1 waves are controlled by glutamate and beta2 waves by GABA. If these data are confirmed in humans a new base for EEG evaluation may arise. The more data are available from future experiments, the better we will learn the electric language of the brain. The tools to enlighten specific brain activities are available. Data from this combination of eye tracking and EEG recording will help us to better understand our brain.

Understanding Brain Function

See more details on http://www.mewicon.at

Rational Pharmacotherapy

Rational Drug Therapy Based on

Quantitative Real Time Electric Brain Mapping with CATEEM

Conventional recording of the Electroencephalogram (EEG) results in a very complicated depiction of potential differences whose interpretation takes a lot of skill and also is very time consuming. Fig. 1 documents an example of such a recording from a human scalp.

Fig. 1

Fig.2

But Hans Berger, the discoverer of human brain electric activity already in 1932 suggested together with Dietsch to perform a frequency analysis of the signal in order to receive quantitative parameters for better interpretation. About three weeks of calculation made it impossible for practical use at that time. However, today by aid of computers frequency analysis is performed in real time. Result of the analytical procedure named after French mathematician Fourier as “Fast Fourier Transformation” (FFT) consists in documentation of spectral power within certain specially defined frequency ranges historically known as delta, theta, alpha and beta waves. Fig. 3 gives an example of such a power spectrum.

Fig. 3

The power spectrum quantitatively depicts the electric power within delta waves (red), theta waves (orange), alpha1 waves (yellow), alpha2 waves (green), beta 1 waves (turquoise) and beta2 waves (blue). According to basic research the electric power within particular frequency ranges like delta or alpha2 reflect the activity of classic neurotransmitter activities. Thus, delta activity seems to be under the control of the cholinergic transmitter system, whereas alpha2 waves correspond to the activity of the dopaminergic system. In order to use this information derived from quantitative analysis of the EEG for diagnostic purposes, reference data are needed derived from healthy people. Therefore, EEG`s from more than 500 healthy volunteers have been collected using this methodology. They now serve for determination of the aberration from normality of individual patient data. Using this approach a so-called aberration index (AI) can be determined, which provides evidence for statistic deviation from normality with respect to each brain region and frequency content.

Results of the FFT are also used to construct a brain map of electric activity. Using the technique of additive colour mixture (like used for red, green and blue as “RGB” in TV pictures) 140 frequency ranges are coded into spectral colours and depicted as colour mixture. Nonlinear interpolation from 17 electrode positions according to LaGrange allows visualization of frequency changes throughout the whole scalp. An example of a young epileptic patient is given in Fig. 4.

Fig. 4

As one can see, electric activity within the temporal lobe (electrode position T3) deviates from normality with respect to delta and theta activity, whose content is considerably higher than in normal healthy volunteers (marked as solid line throughout all brain electrode positions). Rational drug therapy now consists in finding a medication, which more or less is able to change these two particular frequencies in the brain. This can be achieved by two different ways: Firstly, pharmaceutical industry very often provides information on which neurotransmitter systems a particular drug predominantly acts. Having the information on the relationship between special frequencies and neurotransmitter activities mentioned above a suitable drug can be chosen with a higher probability of success. Secondly, if there are data on changes of frequency content of the EEG by particular drugs available (clinically or from animal research) again a particular drug may be chosen which prevalently is able to target the frequencies recognized as deviating from normality. In the case of epilepsy depicted above the drug Valproic Acid was chosen, since the electropharmacogram of Valproic Acid as determined from EEG analysis in freely moving rats provided information on predominant effects on delta and theta activity.

Due to the ability of the drug valproic acid to change delta and theta activity the treatment of the epileptic patient was successful. No seizures were observed anymore and the electric brain map (lower part of Fig. 4) no longer showed a temporal deviation of delta and theta waves! EEG data obviously can allow non-invasive therapy control.

EnkephaloVision

 EnkephaloVision

(Synchronization of Neurocode Tracking and Eye Tracking)

How can advertisement research profit from brain research? Due to new progress in brain research with respect to measuring brain electric activity, also known as ElectroEncephaloGram (EEG), a combination has now been developed with Eye Tracking for analysis of websurfing (DOI: 10.4236/jbbs.2014.48037 ). Eye tracking gives information on where and how long a subject is looking at particular parts of an image or video. However, the interpretation is quite difficult. Now, neurocode tracking provides direct information on cognitive and emotional features of the subjects brain on the base of EEG recording(DOI: 10.4236/jbbs.2015.54014 ). Neurocode tracking has been developed by Hans Carlos Hofmann and myself (published open access:DOI: 10.4236/wjns.2014.42013  ) and provides this information in continuous steps of 364 milliseconds, just one thought long. You will find examples on youtube when searching for "neurocode", for example https://www.youtube.com/watch?v=Dgx4Nv2aqaM . This new combined technologies have also been used in the characterisation of the effect of a plant extract (Rhodiola rosea) on the brain (published open access). Thus, EnkephaloVision will help in the future to better understand the electric patterns of the brain during various cognitive and emotional challenges and by it decode the electric language of the brain.