ICAO Methodology for Accident Rate Calculation and Trending
ICAO Methodology for Accident Rate Calculation and Trending
Article Information
Category:
Content source:
Content control:
Definition
The accident rate is defined as the ratio between the number of accidents which happened in a given year and the number of flights conducted during that same year. The accident rate gives an estimate of the accident risk per flight. It is generally expressed in accidents per million flights.
The accident rate can be calculated for any set of States like ICAO regions, civil aviation committees or the whole world. When applied to a set of States other than the world, ICAO estimates the number of flights by twice the number of departures in that set of States which is the most favourable situation in terms of accident rate.
Contents
- 1 Definition
- 2 Parameters
- 3 Analysis tools
- 4 ICAO Adaptive Moving Average (IAMA)
- 5 Interpretation
- 6 Other applications
- 7 Related articles
- 8 Further reading
Parameters
For data availability and quality issues, International Civil Aviation Organisation (ICAO) limits itself to flights and accidents with the following parameters:
- Commercial scheduled flights only and
- Flights conducted with aircraft with a maximum take-off mass above 2250 kg only.
Accidents are events as defined by the ADREP Occurrence Class Taxonomy.
Analysis tools
ICAO’s Global Aviation Safety Plan (GASP) uses the accident rate as a safety indicator target to measure efficiency of global safety initiatives. One of the targets defined in the GASP calls for is a significant decrease in the accident rate, to be measured using a 5-year moving average.
Accidents are random events by nature. The accident rate is therefore an indicator which has a strong random component making it move up or down constantly, making it difficult to judge whether a decrease is significant or not. 5-year simple moving averages alone are not filtering out enough of these random variations to allow easy detection of significant changes.
ICAO has therefore defined a trend indicator in form of an adaptive moving average using analysis tools based on the well established theory of Bollinger Bands. Bollinger bands are a technical analysis tool invented by John Bollinger in the 1980s. Having evolved from the concept of financial trading bands, Bollinger bands can be used to measure the highness or lowness of safety indicators.
Bollinger bands consist of:
- a middle band being a moving average, in our case the ICAO adaptive moving average (IAMA)
- an upper band (upperB) at one 5-year simple moving standard deviation above the middle band (IAMA + σ)
- a lower band (lowerB) at one 5-year simple moving standard deviation below the middle band (IAMA − σ)
There are two indicators derived from the bollinger bands, %b and BandWidth.
%b, pronounced 'percent b', is derived from the formula for Stochastics and indicates where the current accident rate is in relation to the bands. %b equals 1 at the upper band and 0 at the lower band. Writing last for the last accident rate:
%b = (last − lowerB) / (upperB − lowerB)
BandWidth tells us how wide the Bollinger Bands are on a normalized basis. Writing the same symbols as before:
BandWidth = (upperB − lowerB) / IAMA
Figure 1: World accident rate control chart
The ICAO GASP 2013 sets out a continuous improvement strategy for States to implement over the next 15 years through the establishment of core, and then more advanced, aviation safety systems. The target dates and the broad objectives are set out below:
Target Date Broad Objective
- (a) Near-Term (by 2017) Implementation of an effective safety oversight system
- (b) Mid-Term (by 2022) Full implementation of the ICAO State safety programme framework
- (c) Long-Term (by 2027) Advanced safety oversight system including predictive risk management
The near- , mid- and long-term objectives reflect an evolution of the targets set out in the 2007 edition of the GASP. The current targets and objectives of GASP 2013 have been developed to provide a detailed path for globally coordinated safety improvements. The previous targets of the GASP to reduce the number of fatal accidents and fatalities, to significantly decrease the global and regional accident rates and to improve cooperation between regional groups and safety oversight organizations remain inherent to the GASP 2013 objectives.
ICAO Adaptive Moving Average (IAMA)
Moving averages are commonly used to filter out volatility which is contained in an indicator and allow identifying trends. However, due to the random component of the accident rate, simple 5-year moving averages are smoother but still do allow to decide whether a change is significant or not. ICAO has therefore chosen an adaptive moving average for the accident rate indicator.
The ICAO Adaptive Moving Average (ICAO) becomes sensitive only during periods when the accident rate is moving in a certain direction and is insensitive to an accident rate movement when the accident rate is volatile.
The IAMA is initially based on a 5-year simple moving average calculated at a given date. For the High Level Safety Conference in 2010, ICAO set this date to 1st of January 2005.
At the beginning of each year, the IAMA is updated with the most recent 5-year simple moving average of the accident rate, provided that during the 2 last consecutive years:
- the accident rate exceeded the upper band (significant increase) or
- the accident rate fell below the lower band (significant decrease).
If at the beginning of the year, none of the 2 conditions above is met, the IAMA is not changed as accident rates exceeding the upper or lower bands during a single year are not considered to be significant.
The IAMA is not a continuous curve like a line but a step-function. Each step indicates a significant change from one risk level to another one and allows easy tracking of significant changes. Figure 1 shows the control chart for the global world accident rate. The IAMA is constant. There has been one burst of the upper band in 2008 (%b=101.67) but the burst was not confirmed in 2009.
Interpretation
Lower and Upper Band - The lower band can be seen as the minimal improvement effort needed in order to achieve a significant decrease if the effort is maintained over 2 consecutive years. It can be seen as a short term safety target relative to the current situation.
On the other side, the upper band defines the limit above which accident rates become significantly higher than the current average and may indicate the existence of an undermining safety issue.
BandWidth - The bandwidth is representative of the quality of the indicator and the measurability of improvement. The higher the bandwidth, the greater the volatility or fluctuations of the accident rate. A bandwidth close or over 100% would mean that the lower band is at 50% or less of the IAMA. The accident rate would need to be improved by at least 50% to be called significant, which is unrealistic.
A bandwidth of 30% is the limit above which the accident rate becomes an unreliable indicator. Figure 2 shows the accident rate control chart for the ICAO ESAF region. The bandwidth is 189%. The lower band is at 0.29, which is a totally unrealistic target. This indicator can therefore not be used to measure safety in that region.
Accident rate indicator unreliability may be caused by different factors among which:
- Inappropriate grouping of States indicated by a high disparity of traffic volume among the States of the group, or;
- Inappropriate measurement period indicated by a far too low number of events happening during that period.
Figure 2: ESAF accident rate control chart, with bandwidth=189%
%b - %b indicates where the accident rate is in relation to the upper and lower band. It is an alert for an upcoming IAMA change. If %b is negative, a significant decrease may be called if %b stays negative for an additional year. If %b is greater than 100%, a significant increase may be ahead. Figure 3 shows the accident rate control chart for the ICAO APAC region. %b is over 100% for 3 consecutive years, which has called 2 consecutive increases in the IAMA.
Figure 3: APAC accident rate control chart, with %b=153%
Other applications
The described methodology to calculate and trend accident rates can be applied to any other event counting safety indicator, either rated or not. One may for example count level busts in a given FIR and rate them by the number of aircraft handled during a given time period to create a level bust rate safety indicator using the defined methodology.
Rating is not always necessary. If the number of exposure events like flights, landings etc. is fairly stable from one period to another, the raw number of events may be taken as safety indicator and controlled using this same methodology.
Related articles
Further reading
Categories
