Third parties interacting with AIs they perceive to be human operators may make false assumptions about the AI’s actions (Rodríguez Palmeiro et al., 2018). In this event, an autonomous system must take action or request intervention, and the abilities and limitations of the system could determine the outcome of the critical incident.
As human-AI partnership becomes more common, there may be a need to understand how interaction between human-AI teams and other humans might occur. Certainly in the field of AVs, there is a need to understand the way in which people respond to AVs; Would they behave differently if they knew that the car coming down the road has a complacent driver who may or may not be watching the road? Would those findings also have implications for the design of human-AI partnerships?
REFERENCES
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Man\’e, D. (2016). Concrete problems in AI safety. arXiv preprint arXiv:1606.06565.
Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A. et al. (2018). The Moral Machine experiment. Nature, 563(7729), 59-64.
Bainbridge, L. (1983). Ironies of automation Analysis, design and evaluation of man--machine systems. In (pp. 129-135). Elsevier.
Bazilinskyy, P., Petermeijer, S. M., Petrovych, V., Dodou, D., & de Winter, J. C. F. (2018). Take-over requests in highly automated driving: A crowdsourcing survey on auditory, vibrotactile, and visual displays. Transportation Research Part F: Traffic Psychology and Behaviour, 56, 82 - 98.
Bond, Raymond R., et al. "Automation bias in medicine: The influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocardiograms." Journal of
electrocardiology 51.6 (2018): S6-S11.
California DMV. (2019, November 20). Report of Traffic Collision Involving an Autonomous Vehicle (OL 316). Retrieved from
https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous/autonomousveh_ol316 Clark, H. H., Brennan, S. E., & others. (1991). Grounding in communication. Perspectives on
socially shared cognition, 13(1991), 127-149.
de Visser, E. J., Krueger, F., McKnight, P., Scheid, S., Smith, M., Chalk, S. et al. (2012). The world is not enough: Trust in cognitive agents Proceedings of the Human Factors and Ergonomics Society Annual Meeting.
Dormehl, L. Apple Watch may have saved yet another person’s life | Cult of Mac. (2018, May 11). Retrieved from https://www.cultofmac.com/547479/apple-watch-saves-another-life
Endsley, M. R., & Garland, D. J. (2000). Theoretical underpinnings of situation awareness: A critical review. Situation awareness analysis and measurement, 1, 24.
Floridi, L. (2014). The fourth revolution: How the infosphere is reshaping human reality. OUP Oxford.
Fridman, L., (2019, November 18) Tesla Vehicle Deliveries and Autopilot Mileage Statistics. Retrieved from https://lexfridman.com/tesla-autopilot-miles-and-vehicles
Gkartzonikas, C., & Gkritza, K. (2019). What have we learned? A review of stated preference and choice studies on autonomous vehicles. Transportation Research Part C: Emerging
Technologies, 98, 323-337.
Goddard, K., Roudsari, A., & Wyatt, J. C. (2011). Automation bias: a systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association, 19(1), 121-127.
Gold, C., Damböck, D., Lorenz, L., & Bengler, K. (2013). “Take over!” How long does it take to get the driver back into the loop? Proceedings of the Human Factors and Ergonomics Society Annual Meeting.
Gold, C., Körber, M., Hohenberger, C., Lechner, D., & Bengler, K. (2015). Trust in Automation – Before and After the Experience of Take-over Scenarios in a Highly Automated Vehicle. Procedia Manufacturing, 3, 3025 - 3032.
evasive manoeuvres. Accident Analysis & Prevention, 106, 211 - 222.
Jarrahi, MH. 2018. “Artificial Intelligence and the Future of Work: Human-AI Symbiosis in Organizational Decision Making.” Business Horizons 61 (4).
doi:10.1016/j.bushor.2018.03.007.
Kalra, N., & Paddock, S. M. (2016). Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?. Transportation Research Part A: Policy and Practice, 94, 182-193.
Körber, M., Gold, C., Lechner, D., & Bengler, K. (2016). The influence of age on the take-over of vehicle control in highly automated driving. Transportation Research Part F: Traffic
Psychology and Behaviour, 39, 19 - 32.
Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human factors, 46(1), 50-80.
Libert, B., Beck, M., & Davenport, T. H. (2019). Self-Driving Companies Are Coming.
Lu, H., Li, Y., Chen, M., Kim, H., & Serikawa, S. (2018). Brain intelligence: go beyond artificial intelligence. Mobile Networks and Applications, 23(2), 368-375.
Lu, Z., Happee, R., D.D. Cabrall, C., Kyriakidis, M., & de Winter, J. C. F. (2016). Human factors of transitions in automated driving: A general framework and literature survey. Transportation Research Part F: Traffic Psychology and Behaviour, 43, 183 - 198.
Melcher, V., Rauh, S., Diederichs, F., Widlroither, H., & Bauer, W. (2015). Take-Over Requests for Automated Driving. Procedia Manufacturing, 3, 2867 - 2873.
Merat, N., & Jamson, A. H. (2009). How Do Drivers Behave in a Highly Automated Car? Iowa Research Online, 5. doi: 10.17077/drivingassessment.1365
Mosier, K. L., Skitka, L. J., Burdick, M. D., & Heers, S. T. (1996, October). Automation bias, accountability, and verification behaviors. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 40, No. 4, pp. 204-208). Sage CA: Los Angeles, CA: SAGE Publications.
Müller, V. C., & Bostrom, N. (2016). Future progress in artificial intelligence: A survey of expert opinion. In Fundamental issues of artificial intelligence (pp. 555-572). Springer, Cham. Musk, E. [@elonmusk]. (2018, May 14). @Kasabiii It certainly needs to be better & we work to
improve it every day, but perfect is enemy of good. A system that, on balance, saves lives & reduces injuries should be released. [Tweet]. Retrieved from
https://twitter.com/elonmusk/status/996133894893129728?lang=en
Nader, R. (1965). Unsafe at any speed: The designed-in dangers of the American automobile. New York: Grossman.
NTSB (2017). Collision Between a Car Operating With Automated Vehicle Control Systems and a Tractor-Semitrailer Truck Near Williston, Florida May 7, 2016.
NTSB (2018). PRELIMINARY REPORT HIGHWAY HWY18MH010.
NTSB (2019 June). Low-Speed Collision Between Truck-Tractor and Autonomous Shuttle, Las Vegas, Nevada, November 8, 2017.
NTSB (2019 July). Rear-End Collision Between a Car Operating with Advanced Driver Assistance Systems and a Stationary Fire Truck, Culver City, California, January 22, 2018.
NTSB (2019 November). Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian, Tempe, Arizona, March 18, 2018.
NAVYA. Navya Safety Report. (2019, November 02). Retrieved from https://navya.tech/safety-report
NHTSA (2016, December 19). NHTSA - A Drive Through Time. Retrieved from https://one.nhtsa.gov/nhtsa/timeline/index.html
Crash Causation Survey. Retrieved from
https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812115
O’Neill, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. Nueva York, NY: Crown Publishing Group.
Parasuraman, R., & Manzey, D. H. (2010). Complacency and bias in human use of automation: An attentional integration. Human factors, 52(3), 381-410.
Reason, J., Hollnagel, E., & Paries, J. (2006). Revisiting the Swiss cheese model of accidents. Journal of Clinical Engineering, 27(4), 110-115.
Rodríguez Palmeiro, A., van der Kint, S., Vissers, L., Farah, H., de Winter, J. C. F., &
Hagenzieker, M. (2018). Interaction between pedestrians and automated vehicles: A Wizard of Oz experiment. Transportation Research Part F: Traffic Psychology and Behaviour, 58, 1005 - 1020.
Russell, S. J., & Norvig, P. (2016). Artificial intelligence: a modern approach. Malaysia; Pearson Education Limited.
Schaefer, K. E., Chen, J. Y. C., Szalma, J. L., & Hancock, P. A. (2016). A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems. Human factors, 58(3), 377-400.
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., ... & Lillicrap, T. (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419), 1140-1144.
Singh, I. L., Molloy, R., & Parasuraman, R. (1993). Automation-induced” complacency”: Development of the complacency-potential rating scale. The International Journal of Aviation Psychology, 3(2), 111-122.
Skitka, L. J., Mosier, K., & Burdick, M. D. (2000). Accountability and automation bias. International Journal of Human-Computer Studies, 52(4), 701-717.
Stake, R. E. (2013). Multiple case study analysis. Guilford Press.
Swartout, W. R. (2016). Virtual Humans as Centaurs: Melding Real and Virtual Virtual, Augmented and Mixed Reality. Cham.
Taiebat, M., Brown, A. L., Safford, H. R., Qu, S., & Xu, M. (2018). A Review on Energy, Environmental, and Sustainability Implications of Connected and Automated Vehicles. Environmental Science & Technology. Environ. Sci. Technol., 52(20), 11449-11465. Thimbleby, H. (1994). Technological hubris. Computing & Control Engineering Journal, 5(5),
229-234.
Waymo. Mission. (2019, November 21). Retrieved from https://waymo.com/mission Wachtel, J. By what hubris?–the readiness of the human operator to take over when the
automation fails or hands over control.
Yoon, S. H., Kim, Y. W., & Ji, Y. G. (2019). The effects of takeover request modalities on highly automated car control transitions. Accident Analysis & Prevention, 123, 150 - 158. Zeeb, K., Buchner, A., & Schrauf, M. (2016). Is take-over time all that matters? The impact of
visual-cognitive load on driver take-over quality after conditionally automated driving. Accident Analysis & Prevention, 92, 230 - 239.
Zeeb, K., Härtel, M., Buchner, A., & Schrauf, M. (2017). Why is steering not the same as braking? The impact of non-driving related tasks on lateral and longitudinal driver interventions during conditionally automated driving. Transportation Research Part F: Traffic Psychology and Behaviour, 50, 65 - 79.