Definition

Safety auditing is a core safety management activity, providing a means of identifying potential problems before they have an impact on safety.

Safety regulatory audit means a systematic and independent examination conducted by, or on behalf of, a national supervisory authority to determine whether complete safety-related arrangements or elements thereof, related to processes and their results, products or services, comply with required safety-related arrangements and whether they are implemented effectively and are suitable to achieve expected results (Regulation 2017/373 Requirements for Providers of ATM/ANS and other ATM Network Functions and their Oversight)

Objective

Safety audits are conducted in order to assess the degree of compliance with the applicable safety regulatory requirements and with the procedural provisions of a Safety Management System if one is in place. They are intended to provide assurance of the safety management functions, including staffing, compliance with applicable regulations, levels of competency and training.

Description

Safety auditing is an element of safety management which subjects the activities of airline operators/service providers to a systematic critical evaluation. An audit may include one or more components of the total system, such as safety policy, change management, SMS as a whole, operating procedures, emergency procedures, etc. The aim is to disclose the strengths and weaknesses, to identify areas of non-tolerable risk and devise rectification measures. The outcome of the audit will be a report, followed by an action plan prepared by the audited organisation and approved by the regulator/supervisory authority. The implementation of the agreed safety improvement measures shall be monitored by the supervisory authority.

Safety audits are used to ensure that:

• Organisation’s SMS has a sound structure and adequate staffing levels;
• Approved procedures and instructions are complied with;
• The required level of personnel competency and training to operate equipment and facilities, and to maintain their levels of performance, is achieved;
• Equipment performance is adequate for the safety levels of the service provided;
• Effective arrangements exist for promoting safety, monitoring safety performance and processing safety issues;
• Adequate arrangements exist to handle foreseeable emergencies.

Safety audits are carried out by a single individual or a team of people who are competent (adequately qualified, experienced and trained) and have a satisfactory degree of independence from the audited organisation or unit. The frequency of the audits depends on the regulatory/management policy. For example some State authorities may conduct annual safety audits; others may consider that a full safety audit is only necessary at a few years interval. Ad-hoc safety audits may be conducted to verify the compliance of a particular system component or activity, or may be initiated following an incident. In accordance with ICAO Standards and Recommended Practices (SARPs) safety audits are to be conducted on a regular and systematic basis. Usually the frequency and scope of safety audits is fixed in a dedicated annual safety audit (inspection) programme of the responsible authority/organisation. Safety audits are one of the principal methods for fulfilling the safety performance monitoring requirements. Often audits are integrated, i.e. they include not only safety but also other business processes and performance areas, such as quality, capacity, cost efficiency etc.

All audits should be pre-planned and supporting documentation (usually in the form of checklists) of the audit content prepared. Among the first steps in planning an audit will be to verify the feasibility of the proposed schedule and to identify the information that will be needed before commencement of the audit. It will also be necessary to specify the criteria against which the audit will be conducted and to develop a detailed audit plan together with checklists to be used during the audit.

The conduct of the actual audit is essentially a process of inspection or fact-finding. Information from almost any source may be reviewed as part of the audit. The techniques for gathering the information include:

• Review of documentation;
• Interviews with staff;
• Observations by the audit team.

The results from the safety auditing present evidence of the performance and the general condition of the organisation’s SMS. Audits which limit observations to items of regulatory non-compliance are of limited value, because they will not encourage the audited organisation to act proactively. The audit report should be an objective presentation of the results of the safety audit. The key principles to be observed in the development of the audit report are:

• Consistency of observations and recommendations;
• Conclusions substantiated with references;
• Observations and recommendations stated clearly and concisely;
• Avoidance of generalities and vague observations;
• Objectivity of observations;
• Avoidance of criticism of individuals or positions.

According to ICAO Doc 9859 - Safety Management Manual safety auditing is a proactive safety management activity which provides means for identifying potential problems before they have an impact on safety. Therefore, safety auditing has the characteristics attributed to both the safety assurance domain of SMS, and the hazard identification element of risk management.

Safety audits may be conducted externally - by the designated State regulatory authority, internally - by the aviation services provider organisation, or by a qualified external safety auditor, for example a consultancy agency. Regardless of the driving force behind the audit, the activities and output from both internal and external audits are similar.

Regulatory Safety Audits (external auditing)

Under the Chicago Convention, States are required to put in place a safety oversight system to promote aviation safety by observing and assessing the compliance of aircraft operators/service providers with the applicable regulations, procedures and recommended practices. This is to be achieved through a mix of activities, including safety audits. Such audits conducted by a safety regulatory authority should take a broad view of the safety management procedures of the audited organisation. The key issues in such audits should be:

• Surveillance and compliance - the authority needs to ensure that international, national and local standards are complied with prior to issuing any licence or approval and continue to be complied with afterwards;
• Areas and degree of risk - the audit should assess how risks are identified and how any necessary changes are made to ensure that all safety standards are met;
• Competence - the audited organisation should have adequately trained staff for all safety related positions
• Safety management - ensure that the organisation’s SMS is based on sound principles and procedures, and that the organisation is meeting its safety performance targets.

ICAO Document 9734-A - The Establishment and Management of a State’s Safety Oversight System defines Eight Critical Elements of a State’s Safety Oversight System in - as shown in the diagram below:

Ideally the State regulatory authorities should have established procedures and criteria to focus inspections, audits and surveys (in an annual audit programme) on those areas of greater safety concern or need, as identified by the analysis of operational hazard data and risk areas.

Regulatory audits are independent of internal auditing activities undertaken by the organisation concerned within the framework of its safety management system.

Safety audit is an essential safety oversight tool for international and national regulatory and supervisory authorities. In 1999 ICAO established the Universal Safety Oversight Audit Programme (USOAP) with the objective to oversee the effective application of ICAO standards regarding the development of safety regulatory frameworks by Member States. In the area of Air Traffic Management the EUROCONTROL Permanent Commission approved in November 2002 the establishment of the ESARR Implementation Monitoring and Support (ESIMS) Programme. The ESIMS audits are focused on the States’ overall safety oversight, including safety audit capabilities. In Europe, the two safety oversight programmes have been coordinated to achieve an efficient use of avialable resources. ESIMS audits had been carried out for a decade and were replaced by the European Union Aviation Safety Agency (EASA) standartisation inspections.

Third Party Audits (external auditing)

The organisation’s management or the regulator may decide to have an external agency carry out an independent safety audit. ICAO Doc 9859 specifies that: “External audits of the SMS may be conducted by relevant authorities responsible for acceptance of the service provider’s SMS. Additionally, audits may be conducted by industry associations or other third parties selected by the service provider. These external audits enhance the internal audit system as well as provide independent oversight.”

An organisation which possesses the necessary expertise and technical experience to verify on behalf of a State authority the compliance of an air navigation service provider with the applicable regulatory requirements is called a qualified entity. An organisation wishing to become qualified entity must be certified by a State authority in accordance with the provisions of the SES Service provision regulation.

Internal Safety Audits (self auditing)

Internal safety audits and safety surveys should be used by the aviation service providers to assess the level of compliance with the applicable regulatory framework and the organisational SMS processes and procedures, to verify the effectiveness of such processes and procedures and to identify corrective measures if needed. Planning of the audits should take into account the safety significance of the processes to be audited and the results of previous audits. An annual audit program should include:

• Definition of the audits, in terms of criteria, scope, frequency, and methods;
• Description of the processes used to select the auditors;
• The requirement that individuals shall not audit their own work;
• Documented procedures for assignment of responsibilities, planning and conduct of audits, reporting results and maintaining records;
• Audits of contractors and vendors.

According to ICAO Doc 9774 - Manual on Certification of Aerodromes, an aerodrome operator should arrange for an audit of the aerodrome SMS, including an inspection of the aerodrome facilities and equipment. For conducting such a large scale safety audit “the aerodrome operator should also arrange an external audit for the evaluation of aerodrome users, including aircraft operators, ground handling agencies and other organizations” operating at the aerodrome.

Related Articles

• ESARR Implementation Monitoring and Support (ESIMS)
• ESIMS Cooperation with ICAO
• ESIMS Results
• Management System Assessment Tool (MSAT)

Further Reading

ICAO

• ICAO Doc 9735 - Safety Oversight Audit Manual;
• ICAO Doc 9806 - Human Factors Guidelines for Safety Audits Manual;
• ICAO Doc 9859 - Safety Management Manual, Fourth Edition - 2018;
• ICAO Doc 9774 - Manual on Certification of Aerodromes;
• ICAO Integrated Safety Management website;

European Commission

• Regulation 2017/373 Requirements for Providers of ATM/ANS and other ATM Network Functions and their Oversight

EUROCONTROL

• European Action Plan for Prevention of Runway Incursions, Annex G;

Others

• FAA FAA Advisory Circular No.120-92, Introduction of Safety Management Systems for Air Operations.