Soft Computing based HUD Brightness Switching System for Mitigating Tunneling Effect

(1)

Soft Computing based HUD Brightness Switching

System for Mitigating Tunneling Effect

Vinod Karar

CSIR-Central Scientific Instruments Organisation, Chandigarh [email protected]

Smarajit Ghosh

Thapar University, Patiala

Abstract - The main role of head-up displays (HUDs) is to provide key flight, navigation, guidance, aircraft, weapon and target information to the pilot in his forward field of view on a see through screen known as beam combiner (BC). Hypothetically, it allows for optimal control of the aircraft events through the concurrent scanning of HUD symbology and the outside world scene. While the HUD has been shown to improve flight performance and efficiency of the pilot, there are perceptual and cognitive issues associated with its usage. It is found that in case of HUD usage the pilot’s attention in an aircraft is driven by display salience, his/her mission and his/her expertise. The physical properties of the display like color, salience, brightness, orientation, size, and clutter; pilot’s mission and expertise; and task requirements determine the level and division of attention and tunneling between the aircraft and the outside events. Based on the study results, it has been established that the HUD display brightness plays a key role in affecting pilot’s event detection capability. While the brightness of the HUD display can make the features embedded in the symbology significant, it can also force the pilot’s attention to be focused on the aircraft or the outside event depending on the level of the ambient brightness, HUD display brightness and the contrast ratio. The experimentation conducted under varying ambient brightness conditions resulted in varying responses from the participants depending on the ambient brightness, HUD display brightness and the display contrast ratio. The results showed the effect of these parameters on the capability of pilot to detect unintentional attention fixation to the display and to detect changes in aircraft events shown on HUD, or the outside environment. The behaviour of the participants was thus studied to simulate the likely behaviour of pilot under such conditions. The results were translated into a neuro-fuzzy based system such that the HUD display brightness could be dynamically altered to maintain adequate contrast ratio to minimize the tunneling effect due to the HUD brightness and resulting salience factors, thus also optimizing the attention between the aircraft (HUD symbology) and the outside event.

Keywords - Ambient brightness, ANFIS, Artificial neural network, Contrast ratio, Display brightness, Head-up display.

I. I

NTRODUCTION

With today‟s high technology, high-speed aircraft operating in ever increasingly crowded airspace, split second decisions are required from the pilot. It is difficult for him to do continual eye adjustments for focusing on aircraft events, instrument panel, outside world and the variation due to ambient brightness simultaneously. A pilot must take time to read the displays as well as instrument panels and integrate all the information presented on them. The pilot is thus forced to split his

attention between the outside world and the cockpit displays. This results in longer reaction time, pilot fatigue and decreased efficiency. This is not only of concern in modern passenger aircraft where the lives of hundreds of passengers depend on pilot decisions but also in fighter aircraft where pilot has to accomplish his mission apart from flying the aircraft. The pilot cannot afford to divert his attention from the target ahead to gather critical flight information and data like altitude, airspeed, angle of attack, artificial horizon, navigation, radar display etc. displayed in different formats on separate instruments panels in the cockpit display suite. Hence, the modern era aircrafts are fitted with intelligent instruments and the conventional cockpit has been replaced with glass-cockpit in which all the analog meters, indicators and gauges are replaced with digital display systems. In order to facilitate the view of all these displays without having to divert attention, the display systems like head-up display (HUD) and helmet-mounted display (HMD) have been developed which are functionally far superior to similar system like head-down displays (HDD) [1-6].

Aircraft pilots need to have information based on the physical parameters received via sensors of the aircraft. HUD displays this information in front of the pilot enabling him to fly the aircraft “Head Up” thereby reducing workload and enhancing his weapon aiming capability. The flight information, navigational and target/weapon release cues are generated on a CRT face which is collimated by an optical module and projected on the outside scene onto a small „see-through‟ screen positioned just in front of the pilot‟s line of sight looking ahead out of the aircraft. It optically projects the image of the flight information at infinity so that the pilot can see all flight parametric information superimposed on the image of the real world. This effective overlaying of the CRT display on the outside scene provides a simultaneous view to the pilot who is thus freed from the strain otherwise produced, as the pilot would need to refocus his eyes from the instrument panel to the outside scene/target.

II. C

APTURE OF

P

ILOT

’

S

A

TTENTION DUE TO

HUD

(2)

of the displays like HUD which can divide his/her attention between the aircraft and the outside events. Physical characteristic of the HUD display including color, brightness, orientation, size, and motion, can be manipulated to render critical display components more important. The effectiveness of a manipulation to increase the salience of an individual display component is dependent on the visual characteristics of the entire display of which the display brightness and the resultant contrast ratio are very important parameters [7, 8].

III. ANFIS

T

HEORY

Adaptive Neuro Fuzzy Inference System (ANFIS) is an adaptive network. The acronym ANFIS derives its name from adaptive neuro-fuzzy inference system. An adaptive network is a network of nodes and directional links. Associated with the network is a learning rule - for example back propagation. It‟s called adaptive because some, or all, of the nodes have parameters which affect the output of the node. These networks are learning a relationship between inputs and outputs. Since it integrates both neural networks and fuzzy logic principles, it has potential to capture the benefits of both in a single framework. Its inference system corresponds to a set of fuzzy IF–THEN rules that have learning capability to approximate nonlinear functions [9].

The ANFIS architecture is shown below in Fig. 1. The circular nodes represent nodes that are fixed whereas the square nodes are nodes that have parameters to be learnt.

Fig.1. ANFIS layer structure

For the training of the network, there is a forward pass and a backward pass. The forward pass propagates the input vector through the network layer by layer. In the backward pass, the error is sent back through the network in a similar manner to back propagation.

IV. M

ATERIALS AND

M

ETHODS

The HUD proves to be extremely useful when the ambient environmental conditions are not favorable such as rain, fog, low light conditions etc. as the high-quality HUD symbology and display collimated in the aircraft

application make the pilot focus further out reducing visual accommodation problems [10-12].

Still there exist some human factor issues of attention capture and tunneling with the use of head-up displays. The pilots or other users using HUD experience inefficient attentional switching from HUD to primary task. This may result in missing external targets, delayed responses to external events, and/or asymmetrical transition times (longer to switch from HUD-to-external visual processing than vice versa). The studies have demonstrated that the HUD can work as an attentional trap that draws information processing resources to the HUD and degrades response to external events. It can lead to longer switching time from external to HUD display. Also, deterioration in peripheral performance is due to a reduction of the attention focus.

Unintentional attention fixation to the display can happen while using HUD in an aircraft. The effects like difficulty in monitoring expected events, delayed and degraded response to unexpected events, narrowing of attention to exclude unexpected events due to unintentional fixation to display etc. happening when work load is high, due to symbology type and characteristics suggest tunneling effect associated with HUD usage and its linkage with the HUD symbology characteristics including salience [10, 13].

A. Experiment

It has been observed in various studies as well as during the course of head-up display testing and evaluation that the HUD display brightness plays a key role in affecting pilot‟s event detection capability. In order to optimize the attention capture between the aircraft and the outside event so as to minimize the tunneling effect, a set of experiment was conducted for varying ambient brightness conditions. Following that an ANN based system for automatic adjustment of HUD display brightness was developed. In the process, effects of display brightness and contrast ratio with varying ambient brightness, on the capability of pilot to detect changes in events taking place on HUD/outside environment were observed.

By ambient brightness we mean available light in an environment. To simulate the range of ambient brightness expected during the course of day and night mode of operation, the ambient brightness was varied in the range 1 cd/m2to 40,000 cd/m2 to simulate the lighting conditions possible during the entire day and night time. The contrast ratio is a property of a display system, we define contrast ratio as:

𝐶𝑜𝑛𝑡𝑟𝑎𝑠𝑡𝑅𝑎𝑡𝑖𝑜 = 𝑆𝐵 + 𝐴𝐵 𝐴𝐵

where, 𝑆𝐵 is Symbol brightness

𝐴𝐵is Ambient brightness

Contrast ratio ranges were varied from 1 to 18 as beyond this range, contrast ratio parameter results in uncomfortable brightness on the HUD display which may not be desired by the pilot at all. The results obtained are as shown in the Tables I, II and III.

(3)

HUD, an experimental study was conducted. In the study, a group of 14 persons were made to distinguish the outside events and aircraft events in varying possible brightness and contrast ratio conditions with objects on the HUD symbology and the outside varying dynamically in a predetermined fashion. The observations made were recorded and were used as the training and testing data for the ANFIS structure.

The data was collected extensively for the whole range of brightness. However, the brightness range was divided in three domains: high brightness, medium brightness and low brightness.

The extensively collected experimental data was very large. For every domain of brightness ten set of readings were taken. Two set of readings for all three ranges of ambient brightness is as shown in the Table 1, 2 and 3. From the collected experimental data, 60% data was used for the training, 20% for testing and 20% data was used for validation.

The Pilot‟s Attention in case of HUD usage in an aircraft is driven by display salience, his/her mission and expertise. The physical properties of the display, mission and expertise of the pilot, and task requirements determine the level and division of attention and tunneling between the aircraft and the outside events [7, 8, 14-26].

Table I: Experimental data for high brightness

Ambient Brightness (cd/m2)

Contrast Ratio

Aircraft event detection

(%)

Outside event detection

(%)

32000 1.1875 61 97

32000 1.2125 63 97

32000 1.2375 64 97

32000 1.163025 60 98

32000 1.184775 61 97

32000 1.2065 63 97

32000 1.164725 60 98

32000 1.1841 61 97

32000 1.14456 59 98

32000 1.16155 60 98

27000 1.25 65 97

27000 1.2833 66 97

27000 1.3166 66 96

27000 1.1884 62 98

27000 1.2173 63 97

27000 1.2463 64 97

27000 1.2753 66 96

27000 1.1937 62 98

27000 1.21963 63 97

27000 1.245 64 97

27000 1.1927 62 98

27000 1.2154 63 97

Table II: Experimental data for medium brightness

Contras t Ratio

(%)

18000 1.275 66 96

18000 1.325 67 96

18000 1.375 68 95

18000 1.425 70 95

18000 1.475 73 95

18000 1.2826 66 97

18000 1.32605 67 96

18000 1.36955 68 95

18000 1.43 71 95

18000 1.2519 65 97

18000 1.29065 66 96

18000 1.32945 67 96

18000 1.3682 68 95

18000 1.2551 65 97

18000 1.2891 66 97

18000 1.32315 67 96

9000 1.55 75 95

9000 1.65 77 95

9000 1.75 79 95

9000 1.85 80 95

9000 1.95 81 91

9000 1.5652 76 95

9000 1.6521 77 95

9000 1.7391 78 95

9000 1.826 80 95

9000 1.5813 77 95

9000 1.6589 77 95

9000 1.7364 78 95

9000 1.5102 74 95

9000 1.5782 76 95

9000 1.6463 77 95

Table III: Experimental data for low brightness

Contras t Ratio

(%)

175 2 82 95

175 3 90 92

175 4 92 88

175 5 94 84

175 6 95 83

175 7 96 81

175 2.71 88 93

175 3.56 91 89

175 4.42 93 86

175 5.27 94 84

175 6.13 94 82

175 6.98 96 81

175 2.55 85 94

175 3.33 90 91

175 4.1 92 88

(4)

175 5.65 94 83

175 6.43 95 82

175 7.2 96 80

175 2.36 84 95

175 3.04 90 92

175 3.72 91 89

175 4.4 93 86

175 5.08 94 84

175 5.76 94 83

175 6.44 95 82

175 7.12 95 81

60 2.6 87 94

60 4.2 92 87

60 5.8 94 83

60 7.4 96 81

60 1.1708 94 97

60 2.3674 84 95

60 3.74 91 89

60 5.1 94 84

60 6.48 95 82

60 2.24 83 95

60 3.48 91 89

60 4.72 93 86

60 5.96 95 83

60 7.2 95 80

60 2.08 82 95

60 3.18 90 93

60 4.26 92 87

60 5.36 94 84

60 6.44 95 80

60 7.528 96 80

25 4 92 88

25 6 95 83

25 4.4 91 89

25 6.1 94 84

25 7.8 96 81

25 3.3 80 95

25 4.85 93 85

25 6.4 95 82

25 1.7 74 95

25 3.05 90 92

25 4.4 93 83

25 5.75 94 81

B. Adaptive Neuro-Fuzzy Inference Systems

(ANFIS)

From the results obtained, a set of data was accumulated to train an Adaptive Neuro-Fuzzy Inference System so that the brightness of HUD symbology could be modulated according to the ambient brightness. During different times of day the environment lighting is varying thus the value of ambient brightness keeps on changing and as contrast ratio is a function of ambient brightness we observe a change in this parameter as well. As the pilot need to detect changes in event occurring on HUD as well as outside environment concurrently, it is required to have an optimum display symbology brightness depending on

the current ambient brightness to maintain adequate contrast ratio.

The experimental data collected thus helps in finding out the possible combinations which will help in having better detection of HUD event as well as outside event. Using the experimental data we have trained an ANFIS model. The neuro-adaptive learning method works in similar way as the neural networks. Neuro-adaptive learning techniques provide a method for the fuzzy modeling procedure to learn information about a data set. Using a given input/output data set, a fuzzy inference system (FIS) is constructed whose membership function parameters are tuned using either a back propagation algorithm alone or in combination with a least squares type of method. An ANFIS only supports Sugeno-type systems.

The ANFIS was constructed using the MATLAB platform. The input function chosen were: ambient brightness (20, 40000) cd/m2 and contrast ratio (1, 18). ANFIS using these two input functions gives the display symbology brightness (10, 9500) cd/m2 as output. Each input was distributed into three membership functions i.e. Low, Medium and High respectively. Fig. 2 and 3 gives the input membership function for the ambient brightness and input membership function for contrast ratio respectively. Fig. 4 shows ANFIS structure generated using the experimental data.

Fig.2. Ambient brightness - input membership function

Fig.3. Contrast ratio – input membership function

(5)

V. R

ESULT AND

D

ISCUSSION

The experiments were carried out simulating various background lighting conditions corresponding to high, mid-range and the low range.The detection rates for aircraft and outside event was low and high respectively for the brightness values greater than 27000cd/m2. This was due to poor display contrast ratio and a well illuminated background.There was marked improvement in aircraft event detection depicted through HUD symbology while the outside event detection was still high for the middle range of background lighting between 9000cd/m2- 18000cd/m2. The best and the worst results for aircraft and outside event detection were (81, 91) and (65, 97) corresponding to the display contrast ratios of 1.95 and 1.2419 respectively.

The improved display contrast against the lower background lighting corresponding to experiment carried

with background lighting between 25cd/m2-

175cd/m2improved aircraft event detectionat the cost of outside event detection. The best aircraft and worst outside event detection was 96% and 81% respectively for a contrast ratio of 7.8, while the combination of worst aircraft and the best outside event detection was 74% and 95% respectively for contrast ratio of 1.7. The best optimized response for both events varied from 88% to 93% for contrast ratios of 2.1 - 4.5.

The extensive data collected while the experimentation time was then used to train the ANFIS. After training the ANFIS, the system generates its output membership function itself. Following to this the system was subjected to the checking data and error was minimized. Once the training and testing is completed, system is ready to use. The system output was validated by checking its output for six different ambient brightness ranges with varying contrast ratio conditions. The brightness value calculated by the ANFIS was used for display symbology and then the HUD/outside event detection was again observed. The output results are as shown in Fig. 5, 6, 7, 8, 9 and 10. The developed ANFIS based system results in the automatic brightness adjustment of the symbology according to the ambient brightness. The resultant graph suggests that the brightness levels calculated by the system resulted in Aircraft as well as the Outside event detection in the expected range of values.

Fig.5. Comparison of Aircraft event detection with Outside environment event detection for display brightness output calculated by ANFIS at ambient

brightness 35,000cd/m2

(6)

Outside environment event detection for display brightness output calculated by ANFIS at ambient

brightness 750cd/m2

brightness 75cd/m2

Resultant ANFIS based system gives a decent reproduction of the experimental results as apparent from the above graphs. For higher brightness range as depicted in graphical representation of the result for ambient brightness of 35,000cd/m2, the aircraft event detection shown through HUD varies between 62% - 66% while the outside event detection varies between 98% - 96% which is similar to one obtained through experimentation. For middle range results shown through graph for ambient brightness of 15,000cd/m2, 8,000cd/m2, and 2,000cd/m2, the figures for aircraft event detection and the outside event detection variation is 68% - 91% and 96% - 90% respectively. Here the best optimized performance for both events is for display contrast ratio value of 2.1 – 4.5. For lower ambient brightness range depicted through graphs shown for values 750cd/m2 and 75cd/m2, the aircraft event detection improves with increase in contrast ratio at the cost of outside event detection. Here, again the results are in agreement with what were obtained through experimentation. Slight deviation at few places would improve with more training employed to optimize the error.

VI. C

ONCLUSION

The head-up display plays a significant role in the optimal control of an aircraft. However, flying an aircraft in daytime as well as in low light conditions requires the display brightness and the resultant contrast ratio to be adjusted accordingly. High brightness during twilight conditions will also lead to stress and poor outside event detection. Although, there is a manual control available to the pilot to switch the operation of HUD from DAY mode to NIGHT mode, it is not enough. To add to some intuitiveness to the HUD operation, an ANFIS based system was presented in this study. The ANFIS based system will help in the automatic brightness adjustment of the symbology according to the lighting conditions at the time of flight operation. The system after training was tested, and the outputs as discussed earlier were found to be satisfactory. The brightness level calculated by the system gave the Aircraft event detection and Outside event detection in the desired range of values. The results found were 95% in agreement with the established standards.

R

EFERENCES

[1] S. Pope, "The Future of Head-Up Display Technology," ed: Aviation International News, 2006.

[2] D. R. Tufano, "Automotive HUDs: The overlooked safety

issues," Human Factors, vol. 39, pp. 303-309, 1997.

[3] Y. A. V. Houten, "Attentional effects of superimposing flight instrument and tunnel-in-the-sky symbology on the world," NLR1999.

[4] Y. Zheng, M. Brown, C. M. Herdman, and D. Bleichman, "Lane

Position Head-Up Displays in Automobiles: Further Evidence for Cognitive Tunneling," in 14th International Symposium on Aviation Psychology, 2007. , pp. 1-3.

[5] G. L. Calhoun, M. H. Drapera, M. F. Abernathyb, F. Delgadoc,

and M. Patzeka, "Synthetic vision system for improving unmanned aerial vehicle operator situation awareness," in SPIE

Enhanced and Synthetic Vision, 2005, pp. 219-230.

[6] J. Y. C. Chen, R. V. N. Oden, L. Eifert, and A. L. Rodriguez, "Effectiveness of Head-Up Display for Driver Performance," US Army Research Laboratory-Human Research & Engineering Directorate, Orlando, FL.

[7] B. Hooey, B. Gore, C. Wickens, S. Scott-Nash, C. Socash, E.

Salud, and D. Foyle, "Modeling Pilot Situation Awareness," in Human Modelling in Assisted Transportation, P. C. Cacciabue, M. Hjälmdahl, A. Luedtke, and C. Riccioli, Eds., ed: Springer Milan, 2011, pp. 207-213.

[8] V. Karar, Y. Singh, P. P. Bajpai, and H. Garg, "Study of

Attention Capture Aspects with respect to Contrast Ratio for Wide Background Luminance Range in Head-up Displays," in International Multi Conference of Engineers and Computer

Scientists (IAENG-IMECS-2012), Hongkong, China,, 2012.

[9] C. D. Wickens and A. L. Alexander, "Attentional Tunneling and

Task Management in Synthetic Vision Displays," The

International Journal of Aviation Psychology, vol. 19, pp. 182-199, 2009/03/27 2009.

[10] V. Karar, S. S. Saini, H. Garg, S. Sharma, and P. P. Bajpai, "Critical Design Review of HUD for LCA," CSIR-CSIO2010. [11] L. J. Prinzel and M. Risser, "Head-up displays and attention

capture," NASA – Langley Research Center, Hampton, VA2004.

[12] C. D. Wickens and J. G. Hollands, Engineering Psychology and

Human Performance, 3rd ed. NJ: Prentice Hall, 2000.

[13] C. D. Wickens, S. Fadden, D. Merwin, and P. M. Ververs,

"Cognitive factors in aviation display design," in Digital Avionics Systems Conference, 1998. Proceedings., 17th DASC.

(7)

[14] I. Moir, A. Seabridge, and M. Jukes, Military Avionics Systems.

Wiltshire: John Wiley & Sons, 2007.

[15] I. Moir, A. Seabridge, and M. Jukes, Civil Avionics Systems.

Cornwall: John Wiley & Sons, 2006.

[16] M. Jukes, Aircraft display systems. London: Professional

Engineering publishing, 2004.

[17] D. N. Jarrett, Cockpit Engineering vol. Hampshire, England:

Ashgate Publishing limited, 2005.

[18] V. Karar, Y. Singh, P. P. Bajpai, and H. Garg, "Performance Comparison of Head-Up Displays with Head-Down Displays in Avionics applications," in National Conference on Advances in

Video, Cyber Learning and Electronics, Chandigarh, 2012.

[19] Y. Lee, J. Lee, and L. Boyle, "Visual Attention in Driving: The

Effects of Cognitive Load and Visual Disruption," Human

Factors: The Journal of the Human Factors and Ergonomics Society, vol. 49, pp. 721-733, 2007.

[20] L. Itti and C. Koch, "A saliency-based search mechanism for overt and covert shifts of visual attention," Vision research, vol. 40, pp. 1489-1506, 2000.

[21] H. C. Nothdurft, "Attention shifts to salient targets. ," Vision

Research, vol. 42, pp. 1287-1306, 2002.

[22] J. M. Wolfe and T. S. Horowitz, "What attributes guide the deployment of visual attention and how do they do it? ," Nature Neuroscience, vol. 5, pp. 1-7, 2004.

[23] M. I. Nikolic, J. M. Orr, and N. B.Sarter, "Why Pilots Miss the

Green Box: How Display Context Undermines Attention Capture.," The International Journal of Aviation Psychology, vol. 14, pp. 39-52, 2001.

[24] J. Jonides and S. Yantis, "Uniqueness of abrupt visual onset in capturing attention," Attention, Perception, & Psychophysics, vol. 43, pp. 346-354, 1988.

[25] S. Yantis and J. Jonides, "Abrupt visual onsets and selective attention: evidence from visual search," Journal of Experimental

Psychology: Human Perception and Performance, vol. 10, p.

601, 1984.

[26] E. M. Stelzer and C. D. Wickens, "Pilots Strategically

Compensate for Display Enlargements in Surveillance and Flight

Control Tasks," Human Factors: The Journal of the Human

Factors and Ergonomics Society, vol. 48, pp. 166-181, Spring 2006 2006.

A

UTHOR

’

S

P

ROFILE

Vinod Karar

was born at Gwalior, India. He completed his ME (electronics engineering) from Punjab Engineering College (presently known as PEC University of Technology), Chandigarh, India and BE (electronics

engineering) from Maulana Azad Regional

Engineering College (presently known as Maulana Azad National Institute of Technology), Bhopal, India. He is presently pursuing his doctoral degree at Thapar University, Patiala, India.

He joined CSIR-CSIO Chandigarh in July 1993 and is presently working as PRINCIPAL SCIENTIST in the area of Avionics and Optical Instrumentation.

Mr. Vinod Karar is member of various professional bodies like IEEE, IETE, AeSI, ISC, OSI, EMC Society for Engineers, Instrument Society of Indiaand was recently awarded with Hariramji Toshniwal Gold Medal -2011 by IETE, India. He is a key member of the team which received CSIR National Technology Award 2002 and SIATI Award 2011 in his area of research. A Certificate of Merit in Leadership Development Programme of CSIR-HRDC, Ghaziabad and Best Paper Award in a National Conference are also to his credit. He has published over 25 papers, 80 Technical Reports, 02 Technology Transfers and has guided number of Master‟s thesis. In addition to R&D work, he is also involved in teaching of M.Tech students (CSIR-PGRPE programme) on advanced instrumentation engineering.