Description

States have a responsibility, under the Chicago Convention, to ensure that acceptable levels of safety are established and maintained. States are further required to ensure that Safety Management Systems (SMS) are implemented by the providers of Air Traffic Services (ATS), which include the monitoring of overall safety levels.

At both national and organisational levels, therefore, Standards and Recommended Practices (SARPs) established by the International Civil Aviation Organisation (ICAO) require the regular monitoring and assessment of safety levels in air traffic management (ATM).

Applicable ICAO Standards

Annexes 1, 6, 8, 11, 13 and 14 include the requirement for States to establish a State Safety Programme (SSP), in order to achieve an acceptable level of safety in civil aviation. An SSP is a systematic approach to the management of safety by the State. An SSP is defined as an “integrated set of regulations and activities aimed at improving safety”, and comprises specific safety activities that must be performed by the State in order to fulfil its responsibilities concerning safe and efficient delivery of aviation activities in the State, including:
a) safety oversight;
b) safety data collection, analysis and exchange; and
c) safety-data-driven targeting of oversight of areas of greater concern or need.

Further, the same Annexes establish that States shall require, as part of their SSP, that aircraft operators and service providers (including an ATS) implement an SMS which shall “include the continuous monitoring and regular assessment of the safety performance”.

The ICAO Safety Management Manual (ICAO Doc 9859) clarifies that safety performance measurement is “a non-stop activity, involving continuous monitoring and measurement, by an organization, of selected operational activities that are necessary to deliver the services the organization was constituted to deliver (provision of aerodrome services, air traffic control, training, etc.)”.

Applicable European Requirements

At national level, the monitoring of safety performance falls within the State’s responsibility for safety oversight, as required by Regulation 2017/373 - Requirements for providers of ATM/ANS and other ATM network functions and their oversight and by EUROCONTROL Safety Regulatory Requirement ESARR 1.

At organisational level, the monitoring and management of safety performance falls within the operation of the service-provider’s SMS, as required by Regulation (EU) No 1035/2011 laying down common requirements for the provision of air navigation services and by EUROCONTROL Safety Regulatory Requirement ESARR 3.

Monitoring Metrics

The ways in which safety performance is monitored is determined partly by international agreement, and partly at the discretion of the States and organisations concerned.

At the international level, ICAO Annex 13 requires States to report to ICAO information on all aircraft accidents which involve aircraft of a maximum certificated take-off mass of over 2250 kg. Annex 13 also requires that States establish a mandatory incident reporting system and additionally recommends the establishment of a voluntary reporting mechanism as an accident prevention measure.

In Europe, the Regulation (EU) No 376/2014 on the reporting, analysis and follow-up of occurrences in civil aviation on occurrence reporting in civil aviation and EUROCONTROL Safety Regulatory Requirement ESARR 2 contain similar requirements but have an increased legal effect.

Measurements of safety in terms of undesirable events, such as accidents and incidents, are examples of “Lagging indicators”, which can capture safety performance “a posteriori”. Such indicators give valuable signals to all involved in an air traffic service - provider, regulator and recipient - of the levels of risk being experienced, and the ability of the organisations concerned to take appropriate mitigation action. However, other types of measurement - “Leading indicators”- can give a wider perspective of the safety “health” of the ATM system, and focus on systemic issues, such as safety maturity and SMS performance.

A holistic approach to performance monitoring is an essential input to safety decision-making. It is important to ensure that good safety performance is attributable to good performance of the safety system and not simply to a lack of incidents. It is also essential that the metrics chosen match the requirements of the stakeholders and decision-makers involved in safety improvement.

As shown in the diagram, stakeholders in the wider aviation industry and the general public require a relatively small number of safety indicators (safety performance indicators or key performance indicators) which can give an instant ‘feel’ for the overall position regarding safety performance. Conversely, those involved in the management of the services concerned need a more detailed set of metrics on which to base decisions regarding the management of the services and facilities being reviewed.

Periodicity of Reporting

The monitoring of safety performance should be a non-stop activity. Information gathered within the monitoring process must be comprehensive and timely enabling appropriate action to be taken in a timescale, and with sufficient responsiveness, that the safety performance of the ATM system is not impaired or threatened.

However, there is a further need to publish the results of safety performance monitoring for the benefit of those who may not be close to the management and decision-making processes, but nevertheless seek appropriate assurance on the current state and trends of the indicators and/or metrics being tracked.

The need for such information determines timing of publication, but it is common for States and major organisations who publish safety reports containing ATM safety performance information and data, to do so on an annual basis.

Further Reading

EUROCONTROL

• Regulatory Framework for Safety KPIs and Associated Metrics presentation by R. Cioponea, 2012
• What Is the Link Between SMS Day-to-Day Activities and ATM Community Wide Indicators?

EUROCONTROL annual safety reports posted on SRC’s webpage: http://www.eurocontrol.int/tags/safety-regulation-commission

Good examples of national safety monitoring and reporting:

• CAP 780 - Aviation Safety Review 2008, published by UK CAA
• “Flight Safety” magazine published bi-monthly by Civil Aviation Safety Authority (CASA), Australia, containing a non-statistical review of aviation accidents. It is available at: http://www.casa.gov.au
• Air Traffic Organization 2012 Safety Report, published by the FAA
• ICAO Global Aviation Safety Plan: 2014 - 2016
• ICAO Global Aviation Safety Plan: 2017 - 2019

Others

• Safety KPI : Level of EoSM, Application of RAT, Just Culture presentation by INAC, AESA, 2014