Real-time Monitoring of Human Alertness

A recent study accurately classified 17 out of 20 individuals’ different stages of human alertness based on the spectral shape of brain activity.

Many accidents are associated with a driver or machine operator’s re-duced alertness level. Drowsiness often develops as a result of repetitive or monotonous tasks that are uninterrupted by external stimuli. In order to enhance safety levels, it would be most desirable to monitor the individual’s level of attention. Changes in the power spectrum of the electroencephalographic (EEG) signal are associated with the subject’s level of attention. At Santander and Madrid, Spain, our initial research was carried out in order to answer three questions. Does a trend exist, in the shape of the power spectrum, that will indicate a subject’s state of alertness? What points on the cortex are most suitable for use in detecting drowsiness and/or high alertness? What parameters in the power spectrum are most suitable for establishing a workable alertness classification in human subjects?

Figure 1. Temporal evolution of spectral power in three individuals during a chess game (high level of alertness).

Figure 2. Temporal evolution of spectral power in three individuals in a relaxed state (with open eyes).

We answered these questions by combining power-spectrum estimation and artificial–neural-network techniques to create a noninvasive, real-time system to classify EEG results as indicating a high attention level, relaxation, or drowsiness. The classification is made every 10 seconds or more, which is a suitable time interval for an alarm signal to be given if the individual’s alertness level is insufficient. This time span is set by the user.

Today, the development of a system to prevent accidents produced by drowsy driving remains an unsolved challenge. This work has, as its principal goal, the study of the behavior of brain waves using equipment that monitors brain activity in real time; this information is then applied to detect the alertness of car drivers.

Figure 3. Temporal evolution of spectral power during drowsi- ness (with open eyes) in three individuals.

Figure 4. Temporal evolution of power spectra in two indivi- duals during a rhythm (with closed eyes).

Human alertness is influenced by many factors. For instance, the circadian rhythm (in which alertness is naturally low at night) explains why truck drivers have as many as five times more vehicle accidents between 1 am and 6 am than at other times. Circadian rhythm also has a direct influence on workers at high-risk operations such as nuclear power plants.¹ Therefore, it is important to develop a system for monitoring alertness to avoid accidents produced by drowsiness (whether natural or disease related).² Mechanical sensors have been placed in vehicles to monitor speed and acceleration changes related to human drowsiness,³ but there were no results in real time.

Figure 5. Comparison of trends for average spectra by task. This is the a rhythm with closed eyes.

Many years ago, human alertness was found to correlate with particular EEG patterns. Portable equipment has been constructed to register this signal and carry out an off-line analysis,⁴ but the classification of alertness levels was not investigated.

There are many studies of the EEG signal using traditional methods⁵ or modern methods (such as neural networks),^6-8 but there have been no definitive results leading to the construction of equipment to monitor human alertness in real time. Electro-oculography has also been employed to monitor alertness.^9-11 EEG data have been registered in real vehicles for off-line analysis,^12-23 but alertness was not classified in real time.

Classification during a chess play Classification during relaxation Classification during drowsiness Figure 6. Alertness level classification performed off- line  for long records of different tasks, with evaluation every 20 seconds. Alertness levels: 1=drowsiness, 2=relaxation, and 3=high level of alertness.

Methods
To investigate the relationship between the EEG power spectrum and alertness, 20 volunteers (ranging in age from 25 to 35 years) were recruited. Their EEG readings were recorded in three different situations: during a chess game that required high levels of attention, when the subjects were relaxed, and when the subjects suffered from sleep deprivation.

EEG data were recorded from the C3 site; at this point, it was possible to detect both drowsiness and high alertness. The C3 channel was sampled at 100 Hz and bandpassed at 4 Hz to 45 Hz. Records with movement artifacts were rejected. The Welch method was used to estimate the spectrum: 100-point Han-ning windows with 50% overlap. Windowed 100-point epochs were extended to 512 points by zero padding.

Based on the trends for average spectra, 100% of the spectral shape was considered to be located between 0 Hz and 25 Hz. With this in mind, we chose suitable parameters for training a multilayer perception percent at 0 to 8 Hz, at 8.1 to 12 Hz, and at 12.1 to 25 Hz. A neural network captures complex linear and nonlinear input-output relationships, learning these relationships from the raw information itself. Average spectra were classified into three alertness levels: highly alert, relaxed, and drowsy. From long-term records of each alertness level, averages were calculated and used to train a neural network with a back-propagation algorithm. The algorithm produced a 2% root-mean-square error in 1,303 epochs.

Results
Figures 1 through 4 (pages 42 and 43) show partial records of three individuals in each state of alertness. When brain activity decreases, the EEG spectrum tends to be located in the a band (8 to 12 Hz). On the other hand, activity in t (4 Hz to 8 Hz) and b (12 to 25 Hz) bands increases during tasks that require high levels of attention.

If the EEG spectrum is viewed every second, it is highly variable, but if the average spectrum is extracted every 10 seconds or more, a trend appears (see Figure 5). As mental activity increases, the average spectrum tends to be enhanced both upward and downward. Figure 6 shows that the neural network was able to classify the state of the subject at all times. The trained network was used to classify the alertness level of the participating individuals in real time in other sessions. We showed the correspondence between the 18 sublevels of memory-level parallelism as an alertness bar. The final real-time system (Figure 7) shows the average EEG spectrum and the corresponding alertness level every 10 seconds.

Discussion
The proposed meth-od seems suitable for classifying alertness based on the spectral shape of brain activity. The system was tested with 20 individuals, of whom 17 had an alertness level that was classified properly. The rest of the cases did not fit the algorithm because they did not have a visible a rhythm (perhaps due to stress, hyperalertness, or anxiety). It is important to notice that the alertness scale generated by our neural-network engine is a continuum. Classification thresholds and criteria are defined experimentally. We do not assess the classification-error percentage because it is affected by the state of transition, but trends are shown clearly.

The results show no variation across subjects; furthermore, by training the neural network using the records of a single participant, it was possible to classify the others correctly. Therefore, we expect that, for subjects within the studied age span who have a visible a rhythm, behavior of the spectral shape for different alertness states is totally unaffected by age. Although these results are preliminary, new experiments are in progress to increase the number of subjects and expand their age range. The system could already be comfortably used; however, the integration of wireless sensors would make an important improvement for testing the system in clinical situations.

Robin Álvarez, MSc, is a doctoral student, Grupo de Bioingeniería y Telemedicina E.T.S. Ingenieros de Telecomunicación, Universidad Politécnica de Madrid, Spain. Francisco del Pozo is a full-time professor of the group. M. Elena Hernando, PhD, is a full-time professor of the group. Enrique Gómez, PhD, is a full-time professor of the group. Antonio Jiménez, MD, is Associate Professor of Neumología, Jefe de Sección de Neumología del Hospital Universitario Marqués de Valdecilla de Santander-Cantabria-España and assistant professor of the Universidad de Cantabria, Facultad de Medicina. Rosario Carpizo, MD, is jefe de la Unidad de Trastornos del Sueño and médico adjunto of the Neurofisiología Clínica del Hospital Universitario Marqués de Valdecilla de Santander-Cantabria-España.

References
1. Moore-Ede MC. Assuring human operator alertness at night in power plants. IEEE Trans Biomed Eng. 1991:522-524.
2. Terán-Santos J, Jiménez-Gomez A, Cordero-Guevara J. The association between sleep apnea and the risk of traffic accidents. N Engl J Med. 1999;340:847-851.
3. Díaz AS. Discriminating sensors for driver’s impairment detection. In: Proceedings of the 1st Annual International IEEE-EMBS Special Topic Conference on Microtechnologies in Medicine & Biology. October 12-14, 2000; Lyon, France. 578-583.
4. Califa KB. A portable device for alertness detection. In: 1st Annual International IEEE-EMBS Special Topic Conference on Microtechnologies in Medicine & Biology. Tokyo: Aoyama Gakuin University, Science & Engineering; 2000:584-586.
5. Wilson J, Bracewell TD. A time variation of professional driver’s EEG in monotonous work. In: IEEE Engineering in Medicine & Biology Society. 11th Annual International Conference. Sudbury, Mass: Raytheon Company; 1989:719-720.
6. Wilson BJ. Alertness monitoring using neural networks for EEG analysis. IEEE Trans Biomed Eng. 2000;2:814-820.
7. Kirk BP, LaCouse JR. Vigilance monitoring from the EEG power spectrum with a neural network. In: Proceedings—19th International Conference—IEEE/ EMBS. October 30-November 2, 1997; Chicago. 1218-1219.
8. Touretzky D, Mozer M. Using feedforward neural network to monitor alertness from changes in EEG correlation and coherence. Advances in Neural Information Processing Systems. 1996;8:931-937.
9. Ueno A, Ota Y, Takase M, Minmitani H. Parametric analysis of saccadic eye movement depending on vigilance states. In: Proceedings of the 18th Annual International Conference of the IEEE Engineering in Medicine & Biology Society. October 31-November 3, 1996; Amsterdam. 1782-1783.
10. Karl F, Van Orden JW, Jung TP. Combined eye activity measures accurately estimate changes in sustained visual task performance. Biol Psychol. 2000;52:221-240.
11. Hakkanen H, Summala H. Blink duration as an indicator of driver sleepiness in professional bus drivers. Sleep.1999;22:798-802.
12. Horne JA, Reyner LA. Driver sleepiness. J Sleep Res. 1995;4:S23-S29.
13. Miller JC. Quantitative analysis of truck driver EEG during highway operations. Biomed Sci Instrum. 1997;34:93-98.
14. Gillberg M, Kecklund G, Akerstedt T. Sleepiness and performance of professional drivers in a truck simulator—comparisons between day and night driving. J Sleep Res. 1996;5:12-15.
15. Miller JC. Batch processing of 10,000 h of truck driver EEG data. Biol Psychol. 1995;40:209-222.
16. Cabon P, Coblentz A. Human vigilance in railway and long-haul flight. Operation: Ergonomics. 1993;36:1019-1033.
17. Torsvall L, Akerstedt T. Sleepiness on the job: continuously measured EEG changes in train drivers. Electroencephalogr Clin Neurophysiol. 1987;66:502-511.
18. Nykamp K, Rosenthal L, Folkerts M. The effects of REM sleep deprivation on the level of sleepiness/alertness. Sleep. 1998;21:609-614.
19. Baulk SD, Reyner LA, Horne JA. Driver sleepiness—evaluation of reaction time measurement as a secondary task. Sleep. 2001;24:695-698.
20. Philip P, Taillard J, Guilleminault C. Long distance driving and self-induced sleep deprivation among automobile drivers. Sleep. 1999;22:475-480.
21. Lenne MG, Triggs TJ, Redman JR. Interactive effects of sleep deprivation, time of day, and driving experience on a driving task. Sleep. 1998;21:38-44.
22. Leger D.The cost of sleep-related accidents: a report for the National Commission on Sleep Disorders Research. Sleep. 1994;17:84-93.
23. Reyner LA, Horne JA. Evaluation of “in-car” countermeasures to sleepiness: cold air and radio. Sleep. 1998;21:46-50.