Derive objective HF/E criteria instead of relying on user opinions.
Principle 4: Objective HF/E Criteria Icon: EUROCONTROL © (All rights reserved)
Opinions and anecdotes are not an adequate basis for professional system design, because they tell us little about underlying needs and mechanisms.
Translate user feedback into meaningful requirements and validate with the help of objective measures, which can be found within HF/E, but also other disciplines.
When developing a new product, system or interface under the user-centred design process, one has to determine how to involve users. This is can become a delicate matter. Type and extent of involvement significantly impact the final product, efforts and costs.
The easy way out is to pass on the responsibility for fixing the front-end design to the controllers themselves, since “they know best”. While this statement holds some truth, it is a grotesque distortion of the idea of user-centred design. In short, user-centred design strives for user integration into the design process but in a structured manner with clearly defined roles. Although controllers are experts in their field, they lack knowledge about engineering design, requirements elicitation and the state of the art in terms of interface and workplace design. The resulting products tend to be overly conservative and more of the same – most often reproductions of the current system with a slightly different look and labels.
A second misleading approach is to ask about user opinions (“how do you like it”). Opinions are highly volatile. Involving different groups of users at different times during the development might lead to completely different views on supposed “facts”. All in all, opinions and anecdotes are barely adequate for professional system design.
The elicitation of user opinions originally comes with good intentions: An incorporation of user opinions into design is supposed to increase controllers’ acceptance. Controllers’ acceptance, however, can become a misleading criterion, especially if it is purely opinion based. There is a fundamental difference between a professional domain, such as ATC, and consumer products. Websites and apps face a lot of competition and the users are free to choose those they like best. Ease of use and joy of use may trump functional scope and sophistication. Safety, reliability and productivity are not the main concerns. The opposite is true in a work domain, especially in ATC. Any software here has to provide a specific set of crucial functionalities in a reliable way to enable the controllers to handle traffic in a safe and efficient manner.
In short: Users can tell the HF/E experts what they think they want but rarely what they actually need.
It does not mean that one should avoid users’ opinions completely. Opinions can be a suitable starting point. However, they should be treated as symptoms of underlying needs and mechanisms. What helps in digging for the actual needs and mechanisms is translating all the user input into objective constructs and criteria. These can be used during an engineering approach that relies on measurable, quantifiable requirements. The two examples below illustrate this idea.
The translation of user feedback into HF/E design criteria is difficult. It requires broad knowledge, not only about the state of the art of the scientific research in the own discipline but also in related fields. If HF/E wants to contribute meaningful requirements for design, it needs to interlink with other disciplines and adapt existing methodologies into the HF/E toolbox. The challenge is to make the connection. Inter-disciplinarity has always been a key virtue of HF/E.
Example 1: Rework of Colour Set on Radar Screen
From incident investigations, there were indications that controllers were overlooking certain targets on the radar screen. The controllers themselves demanded more and more vibrant colours for certain states (warnings, alerts). They were very vocal about the need for more highlights and visual cues.
There were a large number of different warnings and alerts. Analysis showed that there was no concept behind the selection of colours. Additional colours had been added based on controller suggestions. There were large discrepancies in colour reproduction between screens. A complete redesign of the colour set was necessary.
In order to come up with a new colour set, the habit of suggesting and discussing colours needed to be broken. Instead, only the function of the colour-coded items and their relative priorities were discussed. This led to an ordered list of items. With some additional comparisons, this list was amended based on their relative importance (“distances” between the entries). Based on recommendations from HF/E literature, this list was divided into groups that justified the use of different information coding options – not only colours. The construct used to actually assign colours to the remaining items was the CIELUV colour space (Schanda, 2007). This 3D representation of colours is meant to have perceptual uniformity, i.e. the Euclidian distance of colours in the colour space represents the contrast as perceived by humans (Schader, Perott, Heister, Leonhardt, & Bruder, 2012). Working from the starting point that the STCA alert (short-term conflict alert) should have the largest contrast to the background colour, the relative importance could be mapped to the Euclidian distances. This resulted in colours that represented the required visual prominence of the colour-coded states.
Instead of discussing colours, the importance of the colour-coded items was established. This importance could be mapped to contrasts, which led to a colour set accurately representing the items’ importance, thus fulfilling the users’ needs. The discussion was shifted away from a design solution to needs and an adequate, objective construct (colour space) was used to come up with the colour set.
Example 2: Acoustic Optimisation of Control Room
Controllers complained about high noise levels in the control room. They maintained they understood the conversations of other controllers in the distance better than the conversations of their own neighbouring coordination partners. After some measurements, several local measures were taken with dubious results.
A new control room was being built and the acoustics were to be optimised. The conditions were supposedly going to be quieter thanks to modern air-conditioning and the lower thermal loads of the computers. It was unclear what the ultimate perception of the controllers would be if the problems were solved and whether optimisations were necessary.
Prototypically, reverberation time and sound pressure levels (SPL) are the fundamental constructs for the acoustic design of rooms used by engineers. However, they did not capture the problems described. Research yielded another psycho-acoustic construct, the speech transmission index (STI). It describes how well speech is intelligible. As such, it was the measurement of choice. Controller input was utilised to understand the communication needs (who talks with whom) and to define scenarios, i.e. which working positions are staffed during daytime peaks and during night-time lows. SPLs of typical controller conversations were measured. With all these variables known, simulations of the room and the resulting STIs for the different scenarios could be run. The result was a STI map of the room representing which working position was audible for which working position. Corrective actions could be taken.
Instead of relying on conventional engineering constructs and reworks after the initial operational use of the control room, literature research led to a new psycho-acoustic construct. This enabled the connection of the subjective impressions of the controllers with measurable physical quantities. Controller input was used to generate requirements and simulation scenarios. Potential areas of optimisation could be identified.
Source: White Paper on Human Factors Integration in ATM System Design, EUROCONTROL, 2019
The White Paper is available on Bookshelf here: White Paper on Human Factors Integration in ATM System Design