Supporting Human-AI Teaming: Transparency, Explainability, and Situation Awareness

- (Brussels)

Supporting Human-AI Teaming: Transparency, Explainability, and Situation Awareness

Speaker(s)
Mica Endsley

Dr. Mica R. Endsley

President of SA Technologies and a former Chief Scientist of the U.S. Air Force

Dr. Endsley is president of SA Technologies and a former Chief Scientist of the U.S. Air Force. She has also held the positions of Visiting Associate Professor at MIT in the Department of Aeronautics and Astronautics and Associate Professor of Industrial Engineering at Texas Tech University. Dr. Endsley is a recognized world leader in the design, development and evaluation of systems to support human situation awareness (SA) and decision-making, as well as methods for improving SA through training of individuals and teams.

System autonomy and AI are being developed for a wide variety of applications where they will likely work in tandem with people, forming human-AI teams (HAT). Situation awareness (SA) of autonomous systems and AI has been established as critical for effective interaction and oversight of these systems. As AI capabilities grow, and more effective teaming behaviors are expected of AI systems, there will also be an increased need for shared SA between the human and AI teammates. Methods for supporting team SA within HAT are discussed in terms of team SA requirements, team SA mechanisms, team SA displays and team SA processes. A framework for understanding the types of information that needs to be shared within HAT is provided, including a focus on taskwork SA, agent SA, and teamwork SA. AI based on learning systems creates new challenges for the development of good SA and mental models. AI transparency and explainability are discussed in terms of their separate roles for supporting SA and mental models in HAT. The SA Oriented Design (SAOD) process is described as a systematic methodology for developing transparent AI displays for HAT and an example of its application to automated driving in a Tesla is provided. Situation awareness (SA) is critical for effective interaction with AI systems. Human-AI team performance requires taskwork SA, agent SA, and teamwork SA. SA is best supported by AI display transparency that is current and prospective. Explainable AI is primarily retrospective and directed at building mental models.

SKYbrary Partners:

Safety knowledge contributed by: