This article introduces core concepts of human performance modelling (HPM) in the field of aviation human factors research. Today’s HPM software tool sets derive from academic research conducted in the 1990s — and earlier military research — in the fields of experimental psychology and engineering. By the mid-2000s, HPM results were stimulating collaborations among diverse types of safety specialists.
SKYbrary editors reviewed two HPM-related books to compile this overview — Human Performance Modeling in Aviation and Integrated Models of Cognitive Systems — cited in References.
In Human Performance Modeling in Aviation, David C. Foyle and Becky L. Hooey described the potential for aviation safety breakthroughs to be enabled by integrating software tool sets. They said, for example, “HPM can suggest the nature of likely pilot errors, as well as highlight precursor conditions to error such as high levels of memory demand, mounting time pressure and workload, attentional tunneling or distraction, and deteriorating situation awareness.
“Fast-time modeling permits the generation of very large sample sizes from which low-rate-of-occurrence events are more likely to be revealed. An additional advantage associated with the use of human performance modeling includes the ability to propose and evaluate [flight deck/air traffic control] display and procedural changes. This is especially useful in that these evaluations of proposed changes can be done early in the design cycle, without the need to fabricate expensive prototype hardware. Finally, the careful characterisation and formal thinking of the assumptions and processes involved in the problem can, by themselves, lead the modeler to gain new insights into system development and usage.”
- Single-focus models of cognitive functions — These models study discrete human factors such as control of eye movements, visual attention, categorization, decision-making or memory. “Such single-focus models are necessary but not sufficient for understanding human cognition,” according to Wayne D. Gray (see References). “Although single-focused models are not usually created to be part of a larger, more integrated system, if cast in the right form, they can play strong roles in building integrated models of cognitive systems.”
- Human performance modelling (HPM) — In contrast to single-focus models, HPM has demonstrated since the mid-2000s the ability to “model the complex environment of aviation, including multiple operators, vehicle dynamics, environmental cues and constraints, and both nominal and off-nominal operations,” according to Human Performance Modeling in Aviation.
HPM Core Concepts
In his chapter of Integrated Models of Cognitive Systems, Richard W. Pew said that experimental psychologists during the early 1940s summarized results of their research with verbal-analytical models, physical models and mathematical models. “They began capturing their theories in computer models almost as soon as digital computers became available,” he said. By the early 1960s, some began to pursue “the idea of computer models in the context of simulations of human information processing and cognition more generally.
“Three major threads to [HPM history since 1992] are manual-control models of human control in closed-loop systems; task networks models that fundamentally predict probability of success and performance time in human-machine systems; and, cognitive architectures that typically capture theories of human performance capacities and limitations — and the models derived from them tend to be more detailed in the representation of the substance of human information processing and cognition. …
Cognitive architectures, evolved from psychological theory, are now the predominant HPM concept and well suited to integration with engineering models and data.
Pew said, “It is in the cognitive architectures and hybrid models that modelers have sought to extend the range of applicability to situations where there are potential choices of ‘what to do next’ that are process-constrained rather than time-constrained, that elaborate alternative strategies, and that deepen the models to be more realistic with respect to internal perceptual and cognitive processes for which external-environment constraint is less useful.” In his chapter, Frank E. Ritter said, “Because cognitive models increasingly allow us to predict behavior and explain the mechanisms behind behavior, they have many applications. … Researchers in psychology and cognitive science are interested in them as theories. Researchers in human factors, in synthetic environments, and in intelligent systems are interested in them for applications and design.”
In their chapter, Jerome R. Busemeyer, Eric Dimperio and Ryan K. Jessup said that the emergence of compute information processing in the 1950s enabled what HPM practitioner call the “cognitive revolution.” “Cognitive scientists realigned their attention on mental-processing mechanisms,” these authors said. “Short-term and long-term memory storage and retrieval were postulated and serial or parallel processes controlled flow of information.… However, motivation and emotion was foreign to computer systems, and it was eschewed by information processing [and decision-making] theorists.”
In their chapter, Vladislav D. Veksler and Michael J. Schoelles also explained why involving human emotional states as a variable in cognition has been resisted. “The paradigm has shifted; the issue is not how to avoid affect in accounts of cognition but how to account for behavior as emerging from a cognitive–affective control system. … This may well represent the beginnings of the next generation mental architectures — fully embodied and more capable of modeling a wider range of the human experience,” they said. …
“One of the main advantages of the integrated [HPM] approach … is that we have a formal or computational model that is able to derive precise predictions for cognitive and emotional interactions.
Examples of NASA Results
In the early 2000s, the U.S. National Aeronautics and Space Administration Human Performance Modeling (NASA HPM) project applied five cognitive modeling tools to predict human error and behaviour. The project’s modeling teams primarily worked to identify needs for changing or intervening in aviation system designs, procedures and operational requirements.
“The NASA HPM project focused on modeling the performance of highly skilled and trained operators (commercial airline pilots) in complex aviation tasks,” said David C. Foyle and Becky L. Hooey, the editors of Human Performance Modeling in Aviation.
One of the project’s two research problems focused on contemporary taxi navigation errors during airport surface operations.
“Project efforts resulted in both design solutions and procedural recommendations to enhance the safety of aviation systems,” they said. “Significant advancements to the state of HPM were achieved by broadening the scope of the five models to include the aviation domain and through the augmentation and expansion of specific modeling capabilities. … The modeling tools were used to develop a deeper understanding of the causes of taxi errors and to explore potential mitigating solutions.”
The second research problem focused on future synthetic vision system (SVS) operations, studying aspects of the conceptual design and the concept of operations development using a Boeing commercial jet simulator and data sets representing pilot performance and taxi navigation errors. (SVS displays create a visual virtual representation of the airport environment from a digital database via computer-generated imagery.)
The NASA HPM project’s modelers documented their challenges in model selection, model development, model interpretation and model validation. They also reached these overall conclusions:
- HPM effectively addressed real aviation-related safety issues;
- HPM addressed increasingly complex aviation behaviors, tasks and environments;
- HPM can be applied at every phase of design to eliminate latent error and improve system design;
- Their main interests in applying HPM were aviation safety; pilot performance, pilot error and error recovery; and system latent error;
- Rather than testing only the potential success of the proposed system, NASA project included HPM tests of “plausible conditions of failure”; and,
- Coupling human-in-the-loop (HITL) simulation and HPM yielded “insight into the underlying causes of pilot performance and error and, more importantly, … procedural or system designs that mitigate these problems, ultimately leading to increased aviation safety.”
In their chapter, Michael Byrne, Alex Kirlik and Michael Fleetwood said, “These actual aviation problems — as opposed to laboratory test problems developed for model evaluation or theoretical development — represent a class of real-world problems that are of sufficient complexity that the simple application of these existing models was not possible without significant extensions to the HPM state of the art [including] the emergence of new cognitive modeling architectures and the exploding field of artificial intelligence. …
“Beginning in the 1990s, the importance of the role of the human operator in the development of new concepts of air traffic operations in the United States [Federal Aviation Administration] and in Europe [EUROCONTROL] has prompted the modeling and simulation community to tackle increasingly complex real-world problems and situations using integrated cognitive modeling architectures.” In their chapter, Kevin M. Corker, Koji Muraoka, Savita Verma, Amit Jadhav and Brian F. Gore described HPM’s maturity and suitability for analyzing human factors in air traffic management systems and flight management systems. “This level of careful insight provides information important to the understanding of air traffic and flight management as it is actually performed — as opposed to how it is presumed to occur or how it is prescribed to occur,” they said.
These authors realised that, as they documented their NASA work in 2008, HPM seemed mature enough to generate unprecedented discoveries. They specifically credited HPM with introducing “important tools in the arsenal of design, analysis and evaluation tools used by aviation safety researchers and system designers.”
- Human Performance Modeling in Aviation, edited by David C. Foyle, NASA Ames Research Center, and Becky L. Hooey, San Jose State University Research Foundation at NASA Ames Research Center; CRC Press, Taylor & Francis Group, LLC; 2008.
- “An ACT-R Approach to Closing the Loop on Computational Cognitive Modeling: Describing the Dynamics of Interactive Decision Making and Attention Allocation,” by Michael Byrne, Alex Kirlik and Michael Fleetwood.
- The five modeling tools used in the NASA HPM project were: Adaptive Control of Thought–Rational (ACT-R); Improved Performance Research Integration Tool/ACT-R hybrid (IMPRINT/ ACT-R); Air Man–Machine Integration Design and Analysis System (Air MIDAS); Distributed Operator Model Architecture (D–OMAR); and Attention–Situation Awareness (A–SA).
- “Air MIDAS: A Closed-Loop Model Framework,” by Kevin M. Corker, Koji Muraoka, Savita Verma, Amit Jadhav and Brian F. Gore.
- Integrated Models of Cognitive Systems, edited by Wayne D. Gray, Series on Cognitive Models and Architectures; Oxford University Press; 2007.
- “The Rise of Cognitive Architectures,” by Frank E. Ritter.
- “Composition and Control of Integrated Cognitive Systems,” by Wayne D. Gray.
- “Cognitive Control in a Computational Model of the Predator Pilot,” by Kevin A. Gluck, Jerry T. Ball and Michael A. Krusmark.
- “Some History of Human Performance Modeling,” by Richard W. Pew.
- “Integrating Emotions, Motivation, Arousal into Models of Cognitive Systems,” by Vladislav D. Veksler and Michael J. Schoelles.
- “Integrating Emotional Processes into Decision-Making Models,” by Jerome R. Busemeyer, Eric Dimperio and Ryan K. Jessup.