NOSS Operating Characteristics
NOSS Operating Characteristics
Note: This article is entirely based on the draft ICAO NOSS Manual.
Description
Normal Operations Safety Survey (NOSS) can be distinguished from other methods intended to collect safety data from normal operations based on the following ten operating characteristics that are unique to NOSS:
1. Over-the-shoulder observations, with clearly defined stop rules, during normal shifts
NOSS observations are performed by an observer situated close to or behind the controller working the operational position where the observation is taking place (direct observation). This set-up is similar to on-the-job training (OJT) in ATC, where the instructor sits close to or behind a trainee. The NOSS observer will make short personal notes on a small notepad about certain operational situations, which will enable the observer to later reconstruct the situation when writing a narrative about the observation.
One of the attributes that distinguishes NOSS from other safety data collection mechanisms is that NOSS captures data from normal operations only, i.e. the successful daily operations for which no safety occurrence reports are generated. This implies that if, during a NOSS observation, a safety occurrence takes place, that particular observation will be terminated and the data will not be included in the NOSS report.
From a systems safety perspective, it is important to realise that in such an event the safety data are not lost to the organisation, but are captured by the other mechanisms the organisation has in place. For the NOSS project, however, that particular observation session is by definition no longer being conducted in normal operations, and consequently the data from that session will not be used for the NOSS report.
The point in the operations of an organisation where normal operations are interrupted by a reportable occurrence or, in other words, the point where a NOSS observation session has to be terminated, is referred to as the “stop rule” for the NOSS observations. Stop rules will differ from one ATS provider to the next, depending on the existing safety data collection mechanisms in the organisation and indeed in the State concerned. The stop rule for use in a NOSS project shall be discussed and defined during the training of the observers.
NOSS observations are conducted only at operational positions where no OJT instruction or checks are taking place. The reason for this is that NOSS is designed to provide an organisation with a profile of its systemic strengths and weaknesses in managing threats, errors and undesired states during normal operations. Whilst an organisation may spend a considerable amount of time training people to become qualified air traffic controllers, the organisational expectation should be that normal operations are provided by fully qualified controllers. It is thus only fair to consider the systemic performance from that perspective.
Note. There is no reason that an organisation cannot monitor safety in daily operations while observations are taking place or during checks and/or OJT sessions. For a NOSS, however, checks and OJT situations must be excluded.
2. Joint management/controller association support
Allowing observers to sit in during normal operations is not something that is easily accepted by aviation professionals. If the purpose of the observer's presence however is adequately explained to them, most aviation professionals will accept the presence of an observer whilst they are working. A proven way to communicate the purpose of NOSS observations, and indeed of the overall NOSS project in an organisation, is to seek support for the project from both the management of the ATS provider and the controllers' association for the unit or area concerned.
When the management of the ATS provider and the controllers' association are both seen to be supporting the NOSS project, the potential for acceptance by the controller workforce is greatly enhanced. One way to make the bilateral support visible is through a letter, signed by the highest appropriate executive in the ATS provider and the president or chairman of the controller’s association, which outlines the purpose and provides an overview of the NOSS project.
ATS provider organisations considering conducting a NOSS must realize that the support of the controllers' association is crucial to the success of the NOSS. If, during the preparatory stages for a NOSS, a change were to occur in the leadership of the association or in the management of the ATS provider, it would be advisable to reaffirm that support through a new joint letter.
3. Voluntary participation
Participating in a NOSS observation, be it as an observer or as an observed controller, shall be strictly on a voluntary basis. If a person is a suitable candidate to become a NOSS observer, that person must have the option to accept or decline that responsibility. Similarly, controllers must have the option to allow or refuse a NOSS observer to be present during their shift. Refusal of a candidate to observe or of a controller to participate in a NOSS observation must be non-jeopardy for those individuals. No record of the person's identity must be made; the only thing that the NOSS observer may communicate to the project manager is the reason (if known) why the person declined to participate.
Note. The NOSS trials conducted in 2005-2007 suggest that normally the number of controllers who refuse to have a NOSS observer present can be expected to be low. There were a few cases where controllers declined the presence of an observer because they had already been observed once or twice before in the same NOSS period. Other than that, the preparedness of controllers to be observed for NOSS purposes was universally high.
4. De-identified, confidential and non-disciplinary data collection
The identity of the controllers who are on duty during a NOSS observation is not recorded. The only information recorded is the position where the observation takes place and the time when the observation starts and ends. The date on which the observation takes place is not recorded. The identity of the observer is not registered on the observation form with the narrative that is submitted by the observer.
All data from NOSS observations must be considered confidential by the organisation. Data collected during a NOSS programme must not be used for disciplinary purposes under any circumstances. Any breach of confidentiality or trust can mean the end of using NOSS in an organisation.
5. Systematic observation instrument based on the Threat and Error Management (TEM) framework
The target for NOSS is the operational context in which air traffic controllers do their work. NOSS is designed to allow an observer to see the threats, errors and undesired states that are dealt with during normal operations in an organisation through the eyes of the controllers. The observers are trained to recognize threats, errors and undesired states and how they are linked. The observers take minimal notes during the observations and fill out predesigned observation reporting forms after the observation has been completed. These forms are structured to help elicit the threats, errors and undesired states from the narratives provided by the observers and also aim to capture how the identified threats, errors and undesired states were managed and what countermeasures were used by the controllers.
It is not expected that the observers will capture 100 per cent of all threats and errors made during the observation period. Emphasis is on the thoroughness of the data that is captured, even though some threats and errors are not captured.
6. Trained and standardised observers
NOSS observers receive training in which the application of the Threat and Error Management (TEM) framework to ATC operations is explained. They are furthermore trained in using the NOSS observation forms and applying the appropriate codes from the NOSS coding tables. The training includes guidelines for the conduct of NOSS observations in the specific workplaces that will be observed, as well as guidelines on how to act if a safety occurrence happens during an observation (“stop rule”).
After the classroom part of the training, the candidate observers must perform at least two practice observations and fill out the associated reporting forms. The training facilitator will provide feedback to individual observers on the reports they have submitted. This one-on-one interaction between the facilitator and the observers helps ensure that all observers have the same view of what is expected of them (standardization) before they perform an actual NOSS observation.
7. Trusted data collection sites
Even though the data collected in a NOSS are de-identified and confidential, they still possess a certain degree of sensitivity for the organisation concerned, and it is therefore of great importance that a trusted site to store the data be selected and assigned.
The premise is that the data from NOSS belongs to the ATS provider organisation where the NOSS is conducted. It is therefore the responsibility of the organisation to determine where the data will be stored.
The data from most of the airlines that performed a line operations safety audit (LOSA), a similar method as NOSS for use in the flight deck environment) is kept by a body called The LOSA Collaborative (TLC) that was specifically created for that purpose. The airlines concerned agreed that their LOSA data are to be kept there for security purposes. For NOSS, a similar body has been created but detailed information on that body was not available at the time the ICAO NOSS manual was produced.
Factors that can help determine whether or not an ATS provider should store NOSS data in-house comprise - inter alia - the national legislation on freedom of information (i.e. to the press and the public) and the status of the air traffic.
8. Data verification process
After the NOSS data collection period, a “data verification round table” is held. This is a data verification process, which typically involves four or five key persons from the NOSS project in an organisation and may last up to five days, depending on the amount of data that has to be processed. The purpose of the data verification process is to ensure that all data from the observations are coded correctly and consistently before they are analysed. To that end the participants in the data verification process review all observation reports to verify the threats, errors and undesired states that the observers have entered and coded. In the verification process extensive use is made of the applicable ATC procedures for the unit(s) observed in the NOSS.
9. Data-derived targets for safety enhancement
The final report that is produced as the result of a NOSS presents an analysis and interpretation of the data collected during the organisation's normal operations. The report must contain clear indications for the SMS of the organisation of where the systemic strengths and weaknesses are vis-à-vis the management of threats, errors and undesired states in the operational environment. The report assists the SMS to determine the effectiveness of the organisation’s existing safety strategies and countermeasures and, at the same time, enables the SMS to identify specific areas where safety improvements can be made.
10. Feedback of results to the controllers
After the NOSS report has been made available to the organisation, the results of the NOSS should be communicated to the controllers in the organisation, including the controllers of the unit(s) where the observations were performed as well as other units, when feasible. Items the controllers typically will be interested in include the findings of the report and the action that the organisation proposes to take as a result of it.
Related Articles
Further Reading
ICAO
Categories
