The Use of Eye Tracking in Assessing Visual Attention

: In aviation, procedural errors persist despite pilots having the correct knowledge and ability to perform particular flight tasks. These errors can be attributed to diagnostic errors, inaccurate situational assessment and poor hazard awareness. Eye tracking has emerged as a technology that can be used to monitor pilot performance and provide insights into attentional distribution and the underlying decisional processes. The study used a systematic literature review based on the Preferred Reporting Items for systematic reviews and Meta-Analyses (PRIMA) Statement. Relevant publications were searched from 25 databases. To search the databases, the study used the keywords “eye tracking”, “eye tracking measurements”, “eye tracking aviation safety” and “eye tracking pilot flight performance”, as well as “eye tracking and pilot training” and “eye tracking and pilot assessment”. From these databases, 48 publications were retrieved for analysis. Out of the 48, 33 were abstracts and 15 were full-texts. The 15 full-texts proceeded to systematic literature analysis. These studies were further divided to studies reporting the use of eye tracking as a training tool (7) and studies reporting the use of eye tracking as an assessment tool (8). The literature review demonstrates that eye tracking is a valid and effective tool that can provide quantitative data for flight performance and safety assessment. Continued growth in aircraft cockpit automation means that eye tracking remains an urgent and promising development in aviation research, especially in real-time eye tracking to predict and prevent pilot errors and in the integration of eye tracking technology in intelligent cockpit design.


Introduction
The eye is the most important sensory organ for pilots. It processes 80% of all flight information. However, even though increased automation and sophistication in sensing has transformed since the early days of aviation and that pilots do not process as much visual information outside the cockpit like in the past, multiple systems exist which extract information from multiple sources, analyze it and present it to the pilot for decisional processes. The ability to process the visual data is fundamental to pilot performance and aviation safety. The examination of eye movement data has emerged as a reliable tool for mapping eye scanning behavior and determining correlations with cognitive and psychological functions (Peißl et al., 2018). One of the tools that have emerged is the eye tracking technology which can allow users to monitor personnel performance and provide insights into the attentional distribution and underlying decisional processes in pilots .

Eye Tracking
Eye tracking refers to the process and technique used to measure where one is looking, also called the point of gaze and the motion of the eye relative to the head. Eye tracking measures are generated by an eye tracking device that captures eye positions and eye movement. Eye trackers operate by directing nearinfrared light towards the pupil. This causes detectable reflections in the pupil and corner which are tracked by an infrared camera. The movement of the eye is basically the pupil center in relation to the corneal reflection (Muehlethaler and Knetch, 2016). Figure 1 show how the center of the eye (pupil center) is tracked in relation to the position of the corneal reflection. The relative distance between the two positions is used to calculate the direction of the gaze. Studies on eye tracking can be traced to the 1970s and 80s when researchers began investigating visual search behavior, characteristics of eye movements and how they are related to cognitive and psychological conditions. At the heart of these initial studies were Saccadic-Eye Movement system (SEM) performance as an indicator of motor and neurosensory functioning of pilots. Studies moved to how SEM could be improved through training. Additional studies focused on the coordination between head, hand and eye movements and the variation between visual performance and hand performance as indicators of situational awareness (Peißl et al., 2018).
Eye tracking issues have been found to be the most significant factor in human error incidents in aviation, particularly significant for general aviation pilots flying under Visual Flying Regulations (VFR) (Muehlethaler and Knetch, 2016). The implementation of eye tracking in the flight simulation research is promising as it offers immediate feedback that could diagnose future factors affecting pilot attention and situational awareness on the deck of the flight. More than 75% of pilot errors are caused by perceptual failures. Therefore, it is important to study visual data processing in the form of eye movements and gain insight into the perceptual characteristics (Li et al., 2017). Many studies demonstrate considerably that while flying, pilots do not understand where and when to look at the eyes. Aviators are looking for a manner to maximize the efficiency of all necessary actions while preserving a high level of surveillance. Pilots' capacity to monitor the systems of the aircraft and important factor in the flight route affecting flight safety (BEA, 2013).

Eye Trackers
There are different types of eye tracking devices; however, all the eye trackers can be categorized as either screen-based eye-trackers or eye tracking glasses. How the different types of eye trackers are employed in different fields of research differ. The difference between the two categories is that screen-based eye trackers employ a computer, which can be stationary or remote, while the glasses type, are head-mounted. In aviation, eye trackers are integrated with flight simulators in order to perform different flight scenarios. Figure 2 shows the implementation of an eye tracking into flight simulation. This review will assess eye tracking technology, measurements and how they are used to predict attentional distribution and subsequently the underlying decisional processes that affect flight performance and safety.

Methods
The study used a systematic literature review based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRIMA) Statement (Moher et al., 2009). Systematic literature reviews encompass the analysis of previous studies using an explicit method for identification, selection and critically appraising relevant literature, collecting data from the studies and analyzing them. The aim of a systematic literature review is to identify, critically evaluate and integrate the findings of relevant high-quality studies. There six steps in performing a systematic literature review: 1. Formulation of one or more research questions 2. Searching relevant data and identifying publications 3. Defining the inclusion and exclusion criteria 4. Selecting studies relevant publications that can produce data for analyzing the research questions 5. Critical evaluation of publications and assessment of quality 6. Data analysis and comparison of results A systematic literature review was carried out to assess the use of eye tracking as an objective assessment tool for cognitive and psychological parameters and training tool in the aviation industry.

Inclusion Criteria
To search the databases, the study used the keywords "eye tracking", "eye tracking measurements", "eye tracking aviation safety" and "eye tracking pilot flight performance", as well as "eye tracking and pilot training" and "eye tracking and pilot assessment". To be included the article must have been published over the past 10 years (2010-2019) and employed eye tracking metrics in determining performance of eye tracking as a training tool and an assessment tool. Only full-text publications will be included.

Exclusion Criteria
Publications that were not focusing on the aviation field, such as those focusing on the use of eye tracking in the medical field, were excluded. Peer reviewed journals that were older than 10 years, or did not explicitly employ eye tracking metrics in training or assessment investigations were not included in the study. Abstracts will be excluded.

Results
A preliminary search of the keywords "eye tracking" and "pilot training" and "pilot performance assessment" on Google Scholar, with the range specified to 2010 to 2019 yielded 17,700 results. A total of 25 databases yielded resources that were relevant to the study. From these databases, 48 publications were retrieved for analysis. Out of the 48, 33 were abstracts and 15 were full-texts. The 15 fulltexts proceeded to systematic literature analysis. These studies were further divided to studies reporting the use of eye tracking as a training tool (7) and studies reporting the use of eye tracking as an assessment tool (8). Figure 3 presents the PRIMA search strategy. Eye Tracking as a Training Tool Weibel et al. (2012) explored the integration of data that had been produced by various sensor technologies. The mobile eye tracking system was a wearable comfortable and unobtrusive model called the Tobii Glasses System to collect information on the eye gaze. The Tobii eye trackers have the ability to visualize eye tracking data by superimposing a representation of the eye gaze location on an in-built scene camera. High-fidelity Piper Seneca simulator and wide body airline simulator were used in executing the flight maneuvers. The findings revealed that integrated off-the-shelf eye tracking systems when integrated with the ChromoViz are effective in simultaneously visualizing multiple data streams. Reynal et al. (2016) used eye tracking to determine whether flight performance and eye movements differed when conducting four different approaches. The researchers selected 16 pilots (8 Captains and 8 First Officers, from different airlines. The mean flight experience was 7991 h. Two synchronized Pertech eye trackers were used to collect data from 15 different Areas Of Interest (AOIs), including airspeed, attitude behavior, altitude indicator, heading, flight mode annunciator, navigation display, electronic centralized aircraft monitor, engine indicating and crew alerting system, multipurpose control display unit, external view, auto flight control panel, flaps control panel, landing gears control panel, no zone and out of zone. The crew performed four approach procedures on full flight simulation. The results indicated that time spent on scanning primary flight parameters was higher than secondary flight parameters, which is consistent with Standard Operating Procedures (SOPs). Further, eye movement measurements for Pilot Flying (PF) and Pilot Monitoring (PM) also were found to exhibit the same ocular behavior, with the exception of the attitude indicator, which was glanced more in PF than PM, again consistent with SOPs (Reynal et al., 2016).
A follow up study compared pilot performance when faced with an unbalanced approach. A total of 10 airline captains participated in the experiment. They had an average flight experience of 11372.73 h. The flight simulations were executed on a Boeing 737 NG fullflight simulator (CAE 600 series). The flight scenario was an approach to Hamburg airport. Pertech eye tracker was used to collect data on eye movements. A total of areas of interest: Heading, attitude indicator, airspeed, flight mode enunciator, external view, flight control unit, landing gear panel, engine indicating and crew alerting system, navigation display and attitude indicator. The measurements of interest were ocular behavior measured based on the percentage of dwell time per area of interest during the decision sequence. Findings indicated that 50% of the pilots persisted in an erroneous landing decision. The decision whether to land or not to land was taken considerably long before task execution. The findings implied that eye trackers can be used to determine the factors that impair the performance of the pilot (Reynal et al., 2017).
In a separate study, the researchers investigated the relationship between visual scanning and performance during final approach and unexpected go-round maneuver. The researchers selected 12 participants, captains and first officers, with a mean flight experience Eye tracking as a performance assessment tool (n = 8) of 8743.5 h. Flight simulations were performed on a Boeing 777 full-flight simulator and Airbus 330 fullflight simulator. The scenarios introduced pilots to unannounced forthcoming go-around, in addition to increased workloads, flight at night and ATC requests to go around at the last minute. Eye trackers, two 50 Hz head-mounted Pertech eye trackers, were used to test the link between response time and errors and dwell times and eye movements on 14 AOIs. The findings showed that all participants successfully stabilized the aircraft and successfully performed the go-round procedure as per the ACT clearance. However, only 3 crew out of the 12 completed the go-round on the initial response, 2 remembered only one piece of information and had to ask ATC to repeat instructions, while 7 crew missed instructions and called ATC to repeat instructions on clearance (Dehais et al., 2017). Peysakhovich et al. (2018) sought to identify and refine a four-stage step-by-step integration framework for eye tracking systems in modern cockpits. The first stage is the pilot training/flight performance analysis phase is concerned with estimating the pilot's flight skills by using eye movement measurements to predict performance. The second stage is the on-board gaze recording and is focused on the use of eye tracking to record points of gaze. This data can be used in incident and accident investigations. The third stage is the gazebased flight deck adaptation where eye tracking can be used to detect degradation in the cognitive states of pilots, such as in cases of spatial disorientation. Such data can be used to infer intentions, since eye movement measures and pupil size are useful in predicting decision making. The final stage is the gaze-based aircraft adaptation, where eye tracking is integrated in the cockpit. In this scenario, eye-tracking provides crew with feedback on aircraft system changes using estimations of the attentional state of the pilot. Automation of eyetracking can help in making decisions, such as temporarily taking authority over the pilot in case of incapacitation. In cases of upcoming controlled flight into terrain, automated aircraft can execute an auto-pull maneuver in the presence of limited attentional and cognitive resources .
In a study using tracking behavior of pilots (Lounis et al., 2018) investigated how eye tracking integration, as part of intelligent cockpit, can improve pilot-aircraft interaction. The researchers selected 12 pilots with an average of 2163 flight hours. The ISAE-SUPAERO flight simulator was used to execute flight procedures. A smart eye tracker was used to measure eye movements in 12 areas of interest. Findings revealed that non-monitored and fixations frequency improved pilot performance (Lounis et al., 2018). The researchers went ahead and developed a Flight Eye Tracking Assistant (FETA) proof of concept, which was an embedded system with the capability of evaluating online visual cockpit monitoring. The concept was tested using 5 pilots and the results showed that FETA was efficient in redirecting attention towards the critical flight instruments (Lounis et al., 2019). Wang and Sun (2013) tested the use of eye tracking in fatigue measurement. The researchers noted that fatigue was a critical issue associated with increased risk of incident, accident, injury and death. Fatigue impairs operational safety. In the transport system, fatigue is not only limited to crew, whether flight or cabin, but extend to all personnel, including air traffic controllers, technician and mechanics, as well as dispatchers and ramp workers. As a real-time and remote fatigue measurement solution, the study proposed eye tracking. The eye tracker used was the FaceLAB Eye Tracker and the measurement was eye closure threshold which was tested in fatigue and non-fatigue scenarios. The statistic value, PERCLOS, meaning the percentage of the intervals with closed eyes in a fixed time window disregarding regular blinking, was used as the indicator for fatigue menstruations. Findings showed that PERCLOS value revealed a significant difference between non-fatigue and fatigue states in all the three eye closure thresholds (70, 80 and 90%) (Wang and Sun, 2013). Therefore, using PERCLOS value is a reliable index for determining fatigue.

Eye Tracking as an Assessment Tool
Scanning techniques can differ among experienced and inexperienced pilots. Maxi and Stein (2013) evaluated if there was a significant difference between novice and expert helicopter pilots in scanning depending on the mission demands. The participants selected for the study were 16 student pilots with an average of 76 h on the simulator and 76 h in real aircraft and 17 flight instructors with 301 h on simulator and 1501 h flight experience. The Eurocopter (EC) 135 flight simulator was used in addition to the Dikablis® head mounted eye tracker. The findings revealed that there was no significant difference in scanning performance under different mission demands scenarios for novice and expert helicopter pilots. On the contrary, the study showed that performance differed on the basis of flight experience (p = 0.013) for the missions (Robinski and Stein, 2013). Sibley et al. (2015) employed eye tracking as one of the techniques that could be used to capture real-time metrics in in adaptive training in Unmanned Aerial Vehicles (UAVs). The metric of interest was real-time workload. The researchers selected 15 volunteers from the Naval Research Laboratory. However, in the end, after inclusion criteria tests, only 13 proceeded to the study. The researchers used a head mounted (worn as a cap) Advance Brain Monitoring (ABM) wireless B-Alert system in collecting Electroencephalographic (EEG) data. The Tobii X120 eye tracker was worn off-the-head and used to collect data on pupil diameter and gaze position. To execute scenarios, the researchers used the high-fidelity 3-D virtual training system, the Virtual Battlespace 2, for UAV simulations. The scenarios were executed over four-hour sessions in 2 days. Results show that pupil dilation is a significant measurement of mental workload, as indicated by differences in pupil diameter across the four test blocks. Li et al. (2016) investigated the use of eye tracking techniques in investigating the cognitive processes of attention and their effects on pilot decision making. The study recruited 35 participants consisting of avionics engineers and pilots. The experimental set up was situated at the Cranfield University high-fidelity B747-400 Flight Simulator. A mobile head-mounted eyetracker, the ASL Series 4000, was USED to capture eye movement patterns. The scenario simulated was B-747 instrument landing and the eye tracker was to capture the pilot's eye movement from the Primary Flight Display, Navigation Display, Engine-Indicating and Crew-Alerting System, Mode Control Panel and Outside of Cockpit. A total of 17 eye movement measurements were employed in the study, which could be summarized into three key measures: Fixations, pupil area and gaze. The preliminary findings indicated that eye movement focused on specific areas of interest such as Primary Flight Display (PFD). To establish if eye movement was related to the attention distribution, situational awareness and decision making, descriptive analysis revealed that the highest attention distribution was in primary flight display and Navigation Display (ND). Regression analysis show that mental demand, main flight hours and instructor score influenced total time allocated for PFD, F (3, 31) = 1.00, p > 0.05. In conclusion, eye tracking was a reliable method for analyzing attention shifts, situational awareness and decision making among pilots and the fixations, pupil area and gaze can be used to predict pilot's cognitive processes (Li et al., 2016).
To encapsulate the developments in the field, (Peißl et al., 2018) carried out a systematic review to reveal eye-tracking measures and how eye tracking is applied in the aviation industry. The study reviewed 79 papers from peer-reviewed journals drawn from the PsychINFO and Science databases. The review noted that the most common use of eye tracking is in improving performance of pilots in terms of investigating situational awareness, workload and attentional processes. Eye tracking systems can also be integrated as part of displays in aircrafts and unmanned aerial vehicles. Eye tracking movements can also be used as indicators of health and psychological factors such as fatigue, sleep and stress; motion sickness, hypoxia and spatial disorientation. Guo et al. (2018) noted that eye tracking devices can be used to improve aviation safety. Eye trackers can be used to collect data on eyeball movements, data which could be used to monitor pilot performance and detect potential incidents and accidents. Guo et al. (2013) notes that Synthetic Vision Systems (SVS) can provide 3D images of the future routine of the aircraft hence enhancing awareness of potential Controlled Flight into Terrain accidents. Such a design must have capabilities for terrain representation and fatigue, while the eye tracking component can be used to detect pilot fatigue my identifying facial expressions such as eyelid motions, staring, gaze, yawning and other visual cues of fatigue. The researchers state that there are a number of commercial fatigue detection systems that can be incorporated together with the terrain visualization system for greater utility. In the study, the researchers selected 20 participants and set up an integrated SVS-eye tracking system in the cockpit. The participants executed 8 experimental scenarios, each lasting between 80-100 min. The scenarios were designed so as to expose pilots to challenging situations that increase the probability of risk. The findings indicated that the proposed SVS-eye tracking system can be integrated in the cockpit and the data submitted to a server to be analyzed through a data prediction algorithm and the results communicated as a fatigue warning mechanism capable of reduce the risks associated with fatigue (Guo et al., 2018). Noelle et al. (2018) reiterate that eye tracking presents a low-cost alternative for collecting pilot data that can be used to improve performance and safety. The researchers note that eye tracking hardware can be obtained at a cost of $700 or less. To establish whether such low-cost software was effective in achieving the desired objectives, the researchers sampled 79 Navy and Marine Corps student pilots. Their ages ranged from 21 to 30 and all were enrolled at the Naval Air Station in Pensacola, Florida. 14 workstations were set up. Each station was equipped with Gazepoint GP3 eye trackers and a 24-inch monitor. The results confirmed that low cost eye tracking was effective in collecting pupil size data in all the participants, as depicted in the relationship between pupil size and Operation Span (OSPAN) scores .
Further developments in eye tracking is illustrated in the development of real-time eye tracking systems that integrate multiple sets of activity data in a highly dynamic and synchronous manner, hence capturing both special and time frames. Researchers have come up with the "Dynamic Data Alignment Model" which matches timestamps of different data streams so as to align them to the same spatial reference frame before they are fused together as a data activity register frame. In this scenario, the timestamp of eye data and dynamic display data can be synchronized and aligned spatially and the knowledge obtained used to improve aviation outcomes (Wee et al., 2019).  Reynal et al. (2016) 16 pilots (8 Captains and 8 First Officers) Pertech Reynal et al. (2017) 10 airline captains Pertech Dehais et al. (2017) 12 participants (captains and first officers) Pertech Peysakhovich et al. (2018) -- Lounis et al. (2019) 12 pilots Smart Assessment Wang and Sun (2013) -FaceLAB Maxi and Stein (2013) 16 student pilots Dikablis Sibley et al. (2015) 15 volunteers from the Naval Research Laboratory Tobii Li et al. (2016) 35 participants (avionics engineers and pilots) ASL Peißl et al. (2018) 79 papers from peer-reviewed journals - Guo et al. (2018) 20 participants SVS Noelle et al. (2018) 79 Navy and Marine Corps student pilots Gazepoint Table 1 summarizes some key features of the studies investigated in this review. These include the model of the eye tracker utilized and the number of participants used in the experimental study. Where possible further information is provided about the participants.

Conclusion
The systematic review provides a summary of the current empirical evidence on eye tracking assessment. In a majority of the studies, the eye tracking measurements fixations, pupil dilation and gaze positions. The dwell time in different Areas Of Interest (AOI) were used as markers of distributional attention and the relationship with flight performance and the probability of risk of error interpreted based on Standard Operating Procedures (SOPs). Since flight profiles as dynamical as environmental conditions, further studies, employing different flight scenarios offer the potential for the use of eye tracking technologies to assess pilot ocular performance, while integration of eye tracking technology in intelligent cockpit design can serve as a proactive intervention in cases of pilot psychological and cognitive impairment.

Ethics
This article is original and contains unpublished material. The corresponding author confirms that all of the other authors have read and approved the manuscript and no ethical issues involved.