Electronic Device Control By Electrooculography(EOG) Gesture Recognition
B.E. Project Report
by Krishna Jajodia Amit Gudekar Amey Kadam under the guidance of Prof. Y.S.Rao Department of Electronics and Telecommunication Sardar Patel Institute of Technology, Andheri, Mumbai November 2010
Abstract This paper describes an eye-control method based on electro oculography (EOG) to develop a system for assisted mobility .One of its most important features is its modularity, making it adaptable to the particular needs of each user according to the type and degree of handicap involved .This paper describes the development of a neural networks gesture recognition system whereby one can control a electronic device by using the components of his brain wave bio-potentials .Such a system may be used as a control device through human eye-movements ,facial muscle and brain wave bio-potentials .Neural networks are trained to classify EOG data into one of two classes corresponding to two cognitive tasks performed by eight training segments.The operator’s forehead bio-potentials can be acquired and processed as electronic device control signals . Neural networks analyze user’s EOG signal in order to discern for the presence of a signal and then decide whether it corresponds to a valid command .The trained neural network can effectively recognize user intention, left or right based on EOG signal .The experimental results suggest that a electronic device can be operated by human brain wave bio-potentials with neural networks .This technique could be useful in multiple application such as mobility and communication aid for handicapped persons.
Contents 1 Introduction
1
2 Literature
4
3 EOG Electrodes 3.1 Ag-AgCl 4 MM TP Electrode - EL254 . . . . . . 3.2 AG-AGCL EL258S . . . . . . . . . . . . . . . . . 3.3 AG-AGCL+HOLE 8MM TP Electrode - EL258H 3.4 Ag-AgCL+Hole 8MM TP Electrode EL258RT . . 3.5 The ML317 EOG Pod . . . . . . . . . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
6 7 8 9 10 11
4 EOG gel
12
5 Design 5.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Requirement Specification . . . . . . . . . . . . . . . . . . . . . . 5.2.1 signal pick-up and amplification . . . . . . . . . . . . . . .
13 13 13 13
6 Electrooculography (EOG)
14
7 Implementation 7.1 Hardware Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Principles of EOG Bio-potential Measurement . . . . . . . . . . .
18 18 19
8 Future Scope and Applications 8.1 Eye Tracking Computer User Interference . . . . . . . . 8.2 EOG Based Eye Blink Detection System . . . . . . . . 8.3 Hospital Alarm System . . . . . . . . . . . . . . . . . . 8.4 Automatic Sleep Stage Classification . . . . . . . . . . 8.5 Tracking Facial muscle and Eye Motion For Computer 8.6 portable Clinical EOG . . . . . . . . . . . . . . . . . .
22 22 23 24 24 25 27
References
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
30
i
List of Figures 3.1 3.2 3.3 3.4 3.5 3.6
EOG electrodes . . EL254 . . . . . . . EL258S . . . . . . EL258H . . . . . . EL258H . . . . . . ML 317 EOG POD
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
7 7 8 9 10 11
4.1
EEEG/ECG/EMG/EOG PREP GEL 114 G - ELPREPEG
. . .
12
6.1 6.2 6.3 6.4 6.5
EOG electrodes . . . . . . placement of electrodes . . cornea-retinal potential . . Noise Reduction . . . . . . Eye movements by visually
. . . . .
. . . . .
14 15 15 16 17
7.1
eye movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
8.1 8.2 8.3
EOG measurement system . . Blink detection method . . . . Tracking facial muscle and eye mation . . . . . . . . . . . . .
22 23
. . . . . . . . . . . . . . . . . . . . . . . . inspecting
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . the spikes
. . . . . .
. . . . .
. . . . . .
. . . . .
. . . . . . . . . . . . . . . . . . . . . . . . motion for computer . . . . . . . . . . . .
ii
. . . . . .
. . . . .
. . . . . .
. . . . .
. . . . . .
. . . . .
. . . . . .
. . . . .
. . . . . .
. . . . .
. . . . . . . . . . graphics . . . . .
. . . . . .
. . . . . .
. . . . .
. . . . . . ani. . .
26
Chapter 1 Introduction Eye movements are the most frequent of all human movements. Eye movement research is of great interest in the study of neuroscience. Since eye movement can be controlled to some degree and track by modern technology with great speed and precision , they can now be used as a powerful input device and have many practical applications in human computer interactions. The EOG is one of the very few methods of recording eye movements that does not require a direct attachment to the eye itself. It is now accepted that the generated electrical potential arises due to the permanent potential difference 10 to 30mV that exists between the cornea and ocular fundus. An electrical field is set up in the tissues surrounding the eye and the rotation of the eye causes corresponding rotation of field vector. For this reason, it is possible to detect eye movement with the appropriate placement of electrodes on the skin surrounding the eyes. Information, though conservative, shows that there are 70 million disabled people in India. One in every ten children or 3of inter-disciplinary research projects. Despite the recent technological improvements, eye trackers remain very much a high cost research and academic tool requiring specially trained personnel to set-up and operate the systems. In the last years, there has been a significant increase in the development of assistive technology for people with disabilities, improving the traditional systems. Also, the growing use of the computer, both in work and leisure, has led to the development of PC-associated handling applications, mainly using graphic interfaces. This way, the traditional methods of control or communication between humans and machines (joystick, mouse, or keyboard), that require a certain control motor on the part of the users, they are supplemented with others that allow their use for people with severe disabilities.
1
Electro-occulography is recording technique that allows the standing potential between the cornea and the posterior pole of the eye tube recorded. Because this biological signal is proportional to the rotational angle of eye this technique permits a wide range of rotational angles to be recorded, allows documenting of eye movements when the eyelids are closed and unable the recording in the dark places. The EOG signal has the high signal to noise (S/N) ratio, therefore no electromagnetic shielding is required. The advantage of EOG eye gaze interface is its simple configuration : in recording, a pair of electrodes is placed on the right and left temples and potential is amplified using the DC or AC amplifier. Electro-occulography is widely used in ophthalmic research and clinical laboratories because it provides the method for recording with full range of eye movements. Electro-occulograph can be used in ophthalmology for diagnosis and prognosis of several eye ailments.The basic idea used in EOG is to determine the Arden-index of human eye from the deflections corresponding to the peak value and the dark trough value caused by changes of the resting potentials of the eyes. The resting potential is changed as the eye is moved and the movement of the eye is translated into electrical change of potential which is called an Electro-oculoghram. One of the most developing researches in engineering that utilizes the extensive research in medicine is Biomedical engineering. This area seeks to help and improve our everyday life by applying engineering and medical knowledge with the growing power of computers. The area of this project can be applied not only for helping disabled people but also in commercial use. Another area that will gain from Human-Machine interface is interactive computer games, testing subject’s responses and attention in simulators for training military and law enforcers. The system will get input from human tested subject and will act according to it. The human input is the electronic signals produced by moving eyes. There are many different ways to measure this signal and we will use the electro-occulography(EOG) to collect them. Generated electrical potential arise due to the permanent potential difference of between 10 to 30mV that exist between the cornea and ocular fundus. This is commonly referred to as cornea retinal potential, with the cornea being positive. An electric field is setup in tissue surrounding the eye and rotation of the eye causes a corresponding rotation of the field vector. For this reason it is possible to detect eye movement with the appropriate placement of the electrode on the skin surrounding the eyes. The EOG is one of the very few methods for recording eye movements that does not require a direct attachment to the eye itself. For this reason, the EOG technique is preferred for recording eye movements.
2
We propose to design and build and electro-occulography(EOG) bio-potential amplifier in order to obtain psychological signal due to eye movements and to use this signal to show directional discrimination. Our design can also be used as a model for future advancement in human computer interaction. The EOG bio-potenial amplifier should be capable of detecting frequencies between DC 10 Hz, the range at which most ocular movements operate. The EOG signal is in the micro volt range (50 to 3500 micro volt). Therefore, when the DC offset is removed, it will be challenging to obtain a strong, useable signal given the minute nature of the recorded signal. Our choice of an EOG over other possible methods was selected based on ease of usage and low cost of production. The electro-occulography(EOG) is a measurement a bio-potentials produced by changes in eye position. The fact that electrical activity could be recorded by placing electrodes on the surface of the skin in the eye region was discovered in the 1920s.It was realized that the electrical potential; induced corresponded to eye movement. Originally it was thought that the induced electrical activity caused by eye movement corresponded to action potentials in the above mentio0ned pairs of muscles. It is now accepted that the generated electrical potentials arises due to the permanent potential difference of between 10 to 30 mV that exists between the cornea-occular fundus. The recording of the eye movement in EOG does not require any direct attachment to the eye itself. For this reason the EOG signal is preferred for recording eye movements in sleep and dream research. Recently this technique has become popular for evaluating reading ability and visual fatigue of subjects.
3
Chapter 2 Literature The expression the literature typically refer to publish in books, journals and conference proceeding that relate to the failed of investigation within which a students project lies. Such literature also includes unpublished theses and dissertation. The idea of our project emerged when we saw human computer interaction for disabled and for more luxurious living. It enables the user to perform any activities without moving from there place. This application might be confused with the home automatic system but it is far most sophisticated then the usual home automation systems. This application is implied upon several disabled people who are paralyzed and even for thoughts who want to control various devices by the movement of their eyes alone without moving from their couches. This device is based upon human computer interface as in we tried to control a device from eye gesture. we have used simple neural analysis related to eye gestures and came to the conclusion that eye movements can produce signals oh certain microvolt which can be amplified to make an electronic device to work normally. Various websites and journals where enough to provide sufficint and the necessary details of the project related the literature and also helped enough to make correct implementation of the project. We also rendered help from various medical journals and from some of the medical practitioners . electroocculography(EOG) is the technique for measuring the resting potential of the retina. The resulting signal is called the electro-occulogram. The main applications are in ophthalmological diagnosis and in recording eye movements. Unlike the electro-retinogram, the does not represent the response to individual visual stimuli. Eye movement measurement: usually ,pairs of electrodes are placed either above and below the eye or to the left and the right of the eye. If the eye moved from the centre position towards one position, this electrode ”sees” the positive side of the retina and the opposite electrode ”sees” the negative side of the retina. Consequently potential difference occurs between the electrodes. Assuming that the resting potential is constant, the recorded potential is measure for the eye position. 4
Ophthalmological diagnosis: The EOG is used to access the function of the pigment epithelium. During dark adaptation, the resting potential decreases slightly and reaches minimum after several minutes. When light is switched on , a substantial increase of the resting potential occurs which drops off after a few minute when the retina adapts to the light. Te ratio of the voltages is known as the Arden ratio. In practice, the measurement is similar to the eye movement recording. The EOG signal is derived from the polarization potential, also known as the cornea- retinal potential(CRP), generated within the eyeball metabolically active retinal epithelium. The EOG signal is acquired though a bi-channel signal acquisitions systems namely, the horizontal and the vertical channels. Electrodes placed on the either side of the eyes or above and below them pickup the potential generated by the motion of the eyeballs. This potential varies approximately proportional to the displacement of the eyeballs within the conductive environment of skull. Saccades inherent in eye motion as well as the blinking of the eyelids can produce changes in EOG signals. The strength of the signal is 10 to 100 microvolt and the useful frequency component is DC 10 Hz. This necessitates the careful selection of the bio-potential amplifier. The recording of the EOG signal has traditionally been associated with several problems. The signal is seldom deterministic, even for the same person for different experiments. It is a result of the number of the factors including eye ball rotation and movements, eyelid movement. For this reason it is extremely essential to eliminate the shifting resting potential( mean DC value), because this value varies continuously.
5
Chapter 3 EOG Electrodes Silver-silver chloride (Ag-AgCl) electrodes provide accurate and clear transmission of surface biopotentials. EL250 series reusable lead electrodes are suitable for most applications (ECG, EEG, EGG, EMG, EOG, and ERS recordings.) Use EL258 (8 mm recording diameter) for most applications. Use EL254 (4 mm recording diameter) when closely spaced biopotentials are required. EL250 series reusable electrodes are permanently connected to robust and pliable leadwires (1 mm OD). The leadwires terminate in standard Touchproof connectors. Unshielded electrodes terminate in a single Touchproof connector. Shielded electrodes terminate in two Touchproof connectors, one connects to the Ag-AgCl disk and the other connects to the leadwire shield. The EL254 is unshielded and the EL254S is shielded. For best signal performance use shielded electrodes (EL254S or EL258S) as recording electrodes and unshielded electrodes (EL254 or EL258) as ground or reference electrodes. Generally, for each Biopotential amplifier module, two EL254S or EL258S and one EL254 or EL258 are required. Use reusable electrodes with: GEL103 conductive adhesive gel or see Electrode Acessories for adhesive collars and other gels.
6
Figure 3.1: EOG electrodes
3.1
Ag-AgCl 4 MM TP Electrode - EL254
Figure 3.2: EL254
Specifications: • Dimensions: 7.2 mm outer diameter, 4 mm recording diameter, 6 mm high. • Leadwire OD: 1 mm. • Leadwire length: 1 meter.
7
3.2
AG-AGCL EL258S
Figure 3.3: EL258S
Specifications: • Dimensions: 12.5mm outer diameter, 8mm recording diameter, 6mm high [EL258H-4mm high]. • Lead length: EL258, EL258S and EL258H - 1 meter, EL258RT - 1.5 meter.
8
3.3
AG-AGCL+HOLE 8MM TP Electrode - EL258H
Figure 3.4: EL258H
Specifications: • Dimensions: 12.5 mm outer diameter, 8 mm recording diameter, 6 mm high [EL258H-4 mm high]. • ead length: 1 m.
9
3.4
Ag-AgCL+Hole 8MM TP Electrode EL258RT
Figure 3.5: EL258H
Specifications: • Construction: Carbon fiber leadwire with integral Ag/AgCl electrode. • Leadwire Length: 1.5 m. • Leadwire Diameter: 1.0 mm. • Leadwire Resistance: 174 Ohms/m. • Dimensions: 7.2 mm outer diameter, 4 mm recording diameter, 6 mm high. • MRI compatible: Yes. • Radiotranslucent: Yes.
10
3.5
The ML317 EOG Pod
Figure 3.6: ML 317 EOG POD
Specifications: • Amplification Range 2 mV, 1 mV, 500 V, 200 V, 100 V. • CMRR common mode greater than 80 dB. • Cable Length Metric 1.5 m. • Certifications CE. • Filtering Low Pass Filter 500 (fixed) 2nd order Butterworth. • Frequency Response DC to 500 Hz. • Front Panel Control offset knob for initial zeroing of device. • Gain x1000. • Gain Error 5 • IMRR isolation mode greater than 110 dB. • Input Connection Type 3 shielded lead wire connectors. • Input Impedance less than 100 M?. • Model ML317. • Temperature Drift 3 mV/C. • Weight Metric 200 g. • Weight Metric 200 g. 11
Chapter 4 EOG gel EEEG/ECG/EMG/EOG PREP GEL 114 G - ELPREPEG/ECG/EMG/EOG PREP GEL 114 G - ELPREP Prepare skin and apply small amount to appropriate electrode site by squeezing near tube opening. Apply small amount to disc electrode and press into the paste that has been applied to the head. Clean with warm water.
Figure 4.1: EEEG/ECG/EMG/EOG PREP GEL 114 G - ELPREPEG Net weight: 114 g (4 oz)
12
Chapter 5 Design 5.1
Problem Statement
To implement such a device which would perform certain predefine actions according to the gesture patterns obtained by the actions performed by the human eye. The main hurdle of our project is to acquire the signals from the eye gestures or the eye movements. The next hurdle could be to make use of this signal and implement or involve them in a certain logic which would be predefine and make the electronic device (computer interface) to produce the result according to the logic defined by these gesture patterns.
5.2
Requirement Specification
In order to fulfil the above requirement there was a need to peak up the signals using sensors, filter these signals in the desired range and then amplify this signals so that these signals can be applied to the device being used(computer interface). Thus the hardware used for this project is as follows:
5.2.1
signal pick-up and amplification
• Signal pick-up and amplification: • Electrode used: Ag-AgCl electrodes • Lowpasssignal components: R-C network, • Fixed gain amplifier • Variable gain amplifier
13
Chapter 6 Electrooculography (EOG) EOG is a method for sensing eye movement and is based on recording the standing corneal-retinal potential arising from hyper polarizations and depolarization existing between the cornea and the retina; this is commonly known as an electrooculogram . This potential can be considered as a steady electrical dipole with a negative pole at the fundus and a positive pole at the cornea as shown below.
Figure 6.1: EOG electrodes The standing potential in the eye can thus be estimated by measuring the voltage induced across a system of electrodes placed around the eyes as the eye gaze changes, thus obtaining the EOG (measurement of the electric signal of the ocular dipole). The EOG value varies from 50 to 3500 V with a frequency range of about dc-100 Hz. Its behavior is practically linear for gaze angles of . It should be pointed out here that the variables measured in the human body (any bio potential) are rarely deterministic. Its magnitude varies with time, even when all possible variables are controlled. Most of these bio-potentials vary widely among normal patients, even under similar measurement conditions.
14
Figure 6.2: placement of electrodes This means that the variability of the Electro-oculogram reading depends on many factors that are difficult to determine: perturbations caused by other biopotentials such as EEG (electroencephalogram), EMG (electromiogram),in turn brought about by the acquisition system, plus those due to the positioning of theelectrodes,skin-electrode contacts, lighting conditions, head movements, blinking, etc. In various studies were made of the accuracy and precision of the EOG in tracking the eye gaze. To eliminate or minimize these defects, therefore, a considerable effort had to be made in the signal acquisition stage to make sure it is captured with the minimum possible perturbations and then during the study and processing thereof to obtain the best possible results.
Figure 6.3: cornea-retinal potential
The fact that electrical activity could be recorded by placing electrodes on the surface of the skin in the eye region was discovered in the 1920s. It was realised that the electrical potentials induced corresponded (almost linearly) to eye movement. Initially, it was believed that the induced electrical activity caused by eye movement. Nowadays, it is accepted that the induced electrical potentials 15
arise due to the permanent potential difference of between 10 and 30mV that exists between the cornea and the ocular fundus (left) as shown in the Fig below. This is commonly referred to as the cornea-retinal potential (CRP) with the cornea being positive. 1. The Noise Reduction. The eye movement signals are band limited due to the fact that there is a speed limit on eye movements. Thus, a low pass filter with 20Hz cutoff could remove most of the high frequency noises. The largest noise we observed was the 60Hz noise from the power line. We compared the standard deviation of the signal in order to discriminate meaningful signals from noise (see figure below). It turned out that the whole signal is within 2 times of the deviation, and the base level noise was mostly within the deviation. This led to our calibration strategy to choose the threshold parameter to be above the deviation.
Figure 6.4: Noise Reduction 2. Characteristics Of Eye Movements. In this experiment, we tried to characterize each type of eye movements by visually inspecting the spikes. The result is summarized in the following decision table. ’+’/’-’ indicates positive/negative peak, ’0’ means below certain level,and N/A means does not matter. Blink is characterized as either a consecutive ’+’ and ’-’, or ’+’, ’0’, ’-’. Each pair of graphs is a result of aligning and overlaying 10 trials for each movement. The left and right graph corresponds to horizontal and vertical bipolar measurement, respectively. You can see the trials are overlapping 16
Figure 6.5: Eye movements by visually inspecting the spikes strongly, which means that the shape and strength of signals are stationary.When the eye is moving fast towards left or right, we get a strong peak in the horizontal bipolar measurement, and in the case of up or down movement, the peak was strong in the vertical bipolar measurement, as expected by physiological insights. Note that for the blink, a positive peak followed by a slight negative peak on the vertical bipolar measurement is observed. 3. Calibration. Since the EOG signal varies depending on several uncontrollable factors, such as placement and conductance of the electrodes, and also the amplitude pattern which differs across subjects, it is essential to have a process to calibrate the parameters of the detection and recognition system. We had a semi-automatic procedure for calibration.This is a typical calibration graph we generated. There are all five movements in the time frame. The subject was asked to move his or her eyes to the left, center, up, center, and then blink in one second after the cue was given. Both vertical and horizontal signals were plotted in one figure along with the calculated standard deviation. The deviations are used as a guideline for choosing threshold; it should be at least larger than the deviation. After a couple of stable calibration graph is obtained, we decided the parameters for correct and robust discrimination of each eye movement.
17
Chapter 7 Implementation As performance requirements increase, the implementation of control elements in embedded applications is moving from 8 bits to 32 bits. At the same time, the implementation vehicle of choice for embeddedapplications is changing from ASICs to FPGAs due to cost and time to market pressures. This paradigmshift is causing significant change in the choices designers are making for the execution of embeddedapplications. This is most evident in consumer, industrial, and automotive applications, which are thefastest growing segments of the FPGA market. The ARM7 processor is widely used in these segments.Implementing the ARM7 processor in Actel Flash-based devices allows users to take maximum advantageof this industry standard processor as well as the changes that are occurring in the embedded market.The ARM7 processor is an industry standardarchitecture with a huge ecosystem of tools, support, and embedded designer knowledge. It is the mostwidely implemented 32-bit processor, with billions in use. The ARM7TDMI-S is a general purpose 32-bit microprocessor, which offers high performance and very low power consumption.The ARM architecture is based on Reduced Instruction Set Computer (RISC) principles This simplicity results in a high instruction throughput and impressive real-time interrupt response from a small and cost-effective processor core
7.1
Hardware Design
The electrodes which can be placed around the eyes will give certain signals which would pass through the low pass filters giving output signals of range 0 to 10 Hz. These signals thus filtered and then passed on to the pre amplifier which would make the signals much powerful so that they become more prominent analogue signals and are directly digitized by an A/D converter. Thus these digitized signals can be used directly for signal processing and are given to the device so that the device can perform the desired task. 18
Figure 7.1: eye movement
7.2
Principles of EOG Bio-potential Measurement
The unifying principles of any bio-potential recording mainly consists of electrode design and attachments suited to the application, amplifier circuit design for suitable amplification of the signal ang rejection of noise and interference and finally good measurement practices to mitigate artifacts, noise and interference. • Ag-AgCl Electrodes and Electrolytic Gel. Electrodes for bio potential recordings are design to obtain the signal of interest selectivity while reducing the tendency to pick-up artifacts. The design should be pragmatic to reduce cost and allow for good manufacturing and reliable long term use. These practice consideration determine whether high quality but reusable electrodes made of silver or gold, or chipper disposable electrodes are to be used. Ag-AgCl Electrodes have been used which produce low level of junction potential, motion artifacts and drift in the DC signal. Additionally, an an electrolytic gel based on AgCl was applied to the skin since the upper layers of skins are poor conductor of the electricity. A gel concentration in the order of 0.1 molar concentration results in good conductivity and low junction potential without causing skin irritation.
19
• Minimizing Noise,Artifacts and Inter-Channel Interference. The frequency component of the EOG signal is very close to DC and hence the separation of Dc drifts from the useful signal content is a difficult task. Dual channel acquisition of the EOG signal has been employed and considerable inter channel interference was observed. This interference arises due to two factors ,namely, small deviation of the eyeball position towards the other channel during motion and improper alignment of the electrode pairs. In our system appropriate threshold hav been set to counter this effect and aid in correct determination of the eyeball postion. • The Pre-Amplifier. The design consideration of preamplifier ought to include proper amplification and bandwidth, high input impedance , high CMRR, low noise, voltage fluctuation and stability against temperature. An instrumentation amplifier used to meet these requirements. The required low frequency response might make the amplifier susceptible shifts in junction potential at the kin electrode interface. A drift cancelation may be necessary if required by the application. • Signal Acquisition. An instrumentatiomn amplifier has been used since it reduced the effect of common mode signals like power line interference, electrodes movement and skin muscles artifacts, which affect the electrodes the pairs, almost equally. • Filtering and Amplification. The pre amplifier After the EOG signal has been acquired and amplified, the next stage is the passive band pass filter and the second stage of amplification. The useful EOG signal contents varies between DC-10Hz. A bandpass filter with passband of 0.1 to 10 Hz is used to pass the reverent signal contain and attenuate the DC offset. The noise, as well as the power supply interference are also suppressed. A second stage of amplification follows a bandpass filter, since the gain of amplifier is not sufficient to amplify te EOG signal; to usable level. This is achieved by a non-inverting mp with an amplification of approximate 400. the attenuation provided by the first stage of high pass filtering is insufficient . hence we require a second stage of offset removal which is provided by a first stage passive high pass filter with a cut off frequency 0.1Hz. The inputs from Ag-AgCl electrodes 20
are applied to a passive low pass filter consisting of the R-C network as shown. The 10K-470K and 0.01 micro farad from a passive filter network then its output is applied to the active filter network formed by the op-amp OP-07.Circuit consists of three stages : – The filter circuit using OP-07. – Fixed gain amplifier with gain =100. – Variable gain amplifier with maximum gain=1. The filter circuit consists of active and passive filter to pass the desire frequency i.e 0 to 10Hz and attenuates the unwanted frequency and thus increasing the signal to noise ratio. The fixed gain amplifier amplifies the signal output from the filter circuitry and gives the amplification of 100 . The amplifier is used in inverting mode configuration and thus the gain of the amplifier is given by - R f / R1 . thus the gain is given is given by -100k/ 1k i.e 100. In order to enhance the total gain of the circuit and to get the output as required for the signal for further processing, the output from the fixed amplifier is applied to variable gain amplifier with maximum gain of 100. Thus the the maximum overall gain is 100*100=10000.
21
Chapter 8 Future Scope and Applications 8.1
Eye Tracking Computer User Interference
Our main goal is to develop an inexpensive hardware software system for use in the most challenging cases the estimated 150,000 disable persons able to control only the muscles of their eyes. This encompasses the construction of an EOG eye tracking hardware and its fine tuning in software, as well as the definition of acknowledgeable eye behavior and the establishment of basic protocols governing on screen object selection and manipulation, such device can also be used for many virtual reality systems and video games.
Figure 8.1: EOG measurement system
Electro-oculography depends on the corneoretinal potential that creates an electrical field in the front of the head. This field changes in orientation as the eyeballs rotate. The electrical changes can be detected by electrodes placed on the skin near the eyes. In clinical practice, the detected voltage changes are amplified and used to drive a plotting device, whereby a tracing of eye position is obtained. It is position is obtained is possible to obtain independent measure ments from the 22
two eyes. However, the two eyes move in conjunction in the vertical direction. Hence, it is sufficient to measure the vertical motion of only one eye and the horizontal motion of both eyes. If the orientation of the eyes is measured , it is possible to locate the 3D position of a fixated target object by triangulation. Recognizing blinks as legitimate actions distinct from eye movement also allow their use for rapid invocation of important global commands, such as calling an attendant, and in each module as context sensitive command shortcuts. The EOG system can potentially recognize ”eye gestures,” such as left and right winking and blinking, or any combination there of he eye gesture command language could even be extensible and programmable by the user himself. For example, during text entry or while scanning read-only text, a left blink rapidly followed by a right blink could be a page-up command; right followed by a left would be a page-down, etc.
8.2
EOG Based Eye Blink Detection System
The electrooculogram represents the electrical activity of muscles that control movements of eyes. The eye blinking is a natural protection system which defends the eye from environmental exposure. The spontaneous eye blink is considered to be a suitable indicator for fatigue diagnostics during many, different tasks of human being activity.
Figure 8.2: Blink detection method
The action of eye blinks covers a specific range of frequency and for that reason it is possible to construct a function which processes the signal and generates an artificial peak when blink occurs. This function is called the detection function. This function is used to detect the spontaneous eye blink action. Nonlinear and linear signal processing methods are applied to obtain the detection function waveform. On this base the position of an eye blink is estimated. The results 23
demonstrate that the measurement of an eye blink parameter provides reliable information for eye-controlled systems from human-machine interface.
8.3
Hospital Alarm System
we proposed an eye-movement tracking system. Based on electro- occulography (EOG) technology we detected the signal with different directions in eyemovements and then analyzed to understand what they represented about (e.g. horizontal direction or vertical direction). We converted the analog signal to digital signal and then used as the control signals for human computer interface (HCI). In order to make the system robust, several applications with EOG-based HCI had been designed. Our preliminary results revealed more than 90
8.4
Automatic Sleep Stage Classification
Sleep is a natural periodic state of rest for the body, in which the eyes are usually closed and consciousness is completely or partially lost. In this investigation we used the EOG and EMG signals acquired from 10 patients undergoing overnight polysomnography with their sleep stages determined by expert sleep specialists based on RK rules. Differentiation between Stage 1, Awake and REM stages challenged a well trained neural network classifier to distinguish between classes when only EEG-derived signal features were used. To meet this challenge and improve the classification rate, extra features extracted from EOG and EMG signals were fed to the classifier. In this study, two simple feature extraction algorithms were applied to EOG and EMG signals. The statistics of the results were calculated and displayed in an easy to visualize fashion to observe tendencies for each sleep stage. Inclusion of these features show a great promise to improve the classification rate towards the target rate of 100 Standard sleep stage classification is based on visual analysis of central EEG, EOG and EMG signals. Automatic analysis with a reduced number of sensors has been studied as an easy alternative to the standard. In this study, a singlechannel electro-oculography (EOG) algorithm was developed for separation of wakefulness, SREM, light sleep (S1, S2) and slow wave sleep (S3, S4). The algorithm was developed and tested with 296 subjects. Additional validation was performed on 16 subjects using a low weight single-channel Alive Monitor. In the validation study, subjects attached the disposable EOG electrodes themselves at home. In separating the four stages total agreement (and Cohen’s Kappa) in the training data set was 74
24
8.5
Tracking Facial muscle and Eye Motion For Computer
A motion tracking system enables faithful capture of subtle facial and eye motion using a surface electromyography (EMG) detection method to detect muscle movements and an electrooculogram (EOG) detection method to detect eye movements. An embodiment of the motion tracking animation system comprises a plurality of pairs of EOG electrodes adapted to be affixed to the skin surface of the performer at locations adjacent to the performer’s eyes. The EOG data comprises electrical signals corresponding to eye movements of a performer during a performance. Programming instructions further provide processing of the EOG data and mapping of processed EOG data onto an animated character. As a result, the animated character will exhibit he same muscle and eye movements as the performer. The present invention providing a motion tracking system that enables faithful capture of subtle facial and eye motion. The invention uses a surface electromyography (EMG) detection method to detectmuscle movements, and an electrooculogram (EOG) detection method to detect eye movements. Signals corresponding to the detected muscle and eye movements are used to control an animated character to exhibit the same movements performed by a performer. More particularly, an embodiment of the motion tracking animation system comprises a plurality of pairs of electromyography (EMG) electrodes adapted to be affixed to a skin surface of a performer at plural locations corresponding to respectivemuscles, and a processor operatively coupled to the plurality of pairs of EMG electrodes. The processor includes programming instructions to perform the functions of acquiring EMG data from the plurality of pairs of EMG electrodes. The EMG datacomprises electrical signals corresponding to muscle movements of the performer during a performance. The programming instruction further include processing the EMG data to provide a digital model of the muscle movements, and mapping the digital modelonto an animated character. As a result, the animated character will exhibit the same muscle movements as the performer. In an embodiment of the invention, a plurality of pairs of electrooculogram (EOG) electrodes are adapted to be affixed to the skin surface of the performer at locations adjacent to the performer’s eyes. The processor is operatively coupled tothe plurality of pairs of EOG electrodes and further includes programming instructions to perform the functions of acquiring EOG data from the plurality of pairs of EOG electrodes. The EOG data comprises electrical signals corresponding to eye movementsof the performer during a performance. The programming 25
Figure 8.3: Tracking facial muscle and eye motion for computer graphics animation instructions further provide processing of the EOG data and mapping of the processed EOG data onto the animated character. This permits the animated character to exhibit the same eye movements asthe performer. A more complete understanding of the motion tracking system that enables capture of facial and eye motion of a performer for use in producing a computer graphics animation will be afforded to those skilled in the art, as well as a realization ofadditional advantages and objects thereof, by a consideration of the following detailed description of the preferred embodiment. Reference will be made to the appended sheets of drawings which will first be described briefly. Above is a block diagram illustrating a motion tracking system in accordance with an embodiment of the present invention; a block diagram illustrates a motion tracking system 100 in accordance with an embodiment of the present invention. The motion tracking system includes a motion tracking processor adapted to communicate with aplurality of facial muscular electrode pairs and a plurality of eye motion electrode pairs through a suitable electrode interface . The motion tracking processor 108 may further comprise a programmable computer having a data storage device 106adapted to enable the storage of associated data files. As known in the art, one or more computer workstations may be coupled to the motion tracking processor 108 using a network to enable multiple graphic artists to work with the stored data files inthe process of creating a computer graphics animation. The motion tracking processor 108 may further include a computer graphics animation system such as provided by a commercial software package that enables the creation of 3D graphics and animationfor the entertainment industry, such as the Maya.RTM. software product line sold by Alias—Wavefront or other like products. It should be understood that the computer graphics animation system may comprise an entirely separate computer hardware andsoftware 26
system from the motion tracking processor 108, or alternatively, may be incorporated with the motion tracking processor 108 (e.g., as a ”plug-in”) as part of a common hardware and software system.
8.6
portable Clinical EOG
It is very important to study the EOG system in eye movement for both clinicians and basic scientists due to the abundant neuropathological information. However, the present commercial stimulation instrument only provides very few and fixed patterns. Therefore, techniques of computer animation, data acquisition, data analysis and database management are applied to implement an intelligent instrument that can stimulate and diagnose EOG system with patterns freely set by the doctors or the researchers, so that vision and nerve system illnesses can be studied efficiently. Instead of the expensive and huge commercial stimulation instruments in the present market, in this paper a personal computer is used due to its cheapness, popularity, multifunction and fast speed. A lot of interesting stimulation patterns can easily be created in shape, time sequence, and color under Windows 95. It is hoped that this invention can contribute to clinical diagnosis and basic medical science research for EOG. Monitoring eye movements is clinically important in diagnosis of diseases of the central nervous system. Electrooculography (EOG) is one method of obtaining such records which usesskin electrodes, and utilizes the anterior posterior polarization of the eye. A new EOG diagnostic system has been developed that utilizes two off-the-shelf portable notebook computers, one projector and simple electronic hardware. It can be operated under Windows 95, 98, NT, and has significant advantages over any other similar equipment, including programmability, portability, improved safety and low cost. Especially, portability of the instrument is extremely important for acutely ill or handicapped patients. The purpose of this paper is to introduce the techniques of computer animation, data acquisition, real time analysis of measured data, and database management to implement a portable, programmable and inexpensive contacting EOG instrument. It is very convenient to replace the present expensive, inflexible and large-sized commercially available EOG instruments. A lot of interesting stimulation patterns for clinical application can be created easily in different shape, time sequence, and colour by programming in Delphi language. With the help of Winstar (a software package that is used to control I/O and interrupt functions of the computer under Windows 95, 98, NT), the I/O communication between two notebook computers and A/D interface module can be effectively programmed. In addition, the new EOG diagnostic system is 27
battery operated and it has the advantages of low noise as well as isolation from electricity. Two kinds of EOG tests, pursuit and saccade, were performed on 20 normal subjects with this new portable and programmable instrument. Based on the test result, the performance of the new instrument is superior to the other commercially available instruments. In conclusion, we hope that it will be more convenient for doctors and researchers to do the clinical EOG diagnosis and basic medical science research by using this new creation.
28
Acknowledgment
We would like to express our deep sense of gratitude to Prof.Y.S.Rao for his invaluable help and guidance during the course of project. We are highly indebted to him for constantly encouraging us by giving critics on our work. We are grateful for having given us the support and confidence.
Krishna Jajodia Amit Gudekar Amey Kadam November 2010 Sardar Patel Institute of Technology
29
Bibliography [1] Hashimoto,Takahashi,Shimada ”Wheelchair control using an EOG- and EMG-based gesture interface”. [2] By Harun H., Mansor W ”EOG signal detection for home appliances activation”. [3] K. Schilling, H. Roth, R. Lieb, and H. Sttzle ”Sensors to improve the safety for wheelchair users”. [4] Rafeal Barea, Luciano Boquete, Manuel Mazo”System for Assisted Mobility Using Eye Movements Based on Electroculography”. [5] D. G. Evans, R. Drew, and P. Blenkhorn ”Controlling mouse pointer position using a infrared head-operated joystick”. [6] R. Barea, L. Boquete ”Accurate interaction with computer by eye movement tracking ”.
30
