A388, en-route Batam Island Indonesia, 2010
A388, en-route Batam Island Indonesia, 2010
On 4 November 2010, a Qantas Airbus A380 climbing out of Singapore experienced a sudden and uncontained failure of one of its Rolls Royce Trent 900 engines which caused considerable collateral damage to the airframe and some of the aircraft systems. A PAN was declared and after appropriate crew responses including aircraft controllability checks, the aircraft returned to Singapore. The root cause of the failure was found to have been an undetected component manufacturing fault. The complex situation which resulted from the failure in flight was found to have exceeded the currently anticipated secondary damage from such an event.
Description
On 4 November 2010, a Rolls-Royce Trent 900 powered Airbus A380-800 being operated by Qantas Airways on a scheduled passenger flight from Singapore to Sydney Australia was passing 7000 feet in the climb in day VMC when an uncontained failure of the No 2 engine occurred. The climb was stopped and a ‘PAN’ call was made to ATC and preparations for a return to Singapore initiated whilst the flight crew collectively responded to the engine failure and to multiple secondary effects. Initially, in order to accomplish the minimum necessary actions, a holding pattern in the vicinity of the airport was flown. A challenging but uneventful approach and landing were subsequently accomplished and the aircraft was stopped on the runway. It then became apparent that the No. 1 engine could not be shut down which delayed the precautionary disembarkation of the passengers using stairs brought to the right hand side of the aircraft. None of the 469 occupants were injured but some of the debris which had fallen from the aircraft at the time of the engine failure caused minor injuries to two people on the ground. There was obvious evidence of extensive damage to the failed engine and lesser damage to the adjacent airframe structure.
Investigation
After formal delegation from the State of Occurrence, the Indonesian NTSC, an investigation was carried out by the Australian Transport Safety Bureau (ATSB). The Investigation was able to access recorded data from the FDR and from a wireless digital aircraft condition monitoring system recorder also fitted. However, after landing, despite attempts by both flight and ground crew to isolate the CVR and protect its 2 hour recording, this proved impossible and event recording data was overwritten. The Investigation noted that “at the time of the occurrence, the operator did not have, and was not required by Australian aviation regulation to have a procedure that would enable the CVR to be isolated in conditions of this nature. In contrast, Singapore legislation stated that reasonable measures may be taken to preserve any object or evidence deemed necessary for the purposes of the investigation.”
The aircraft flight path, reproduced from the Official Report, is shown below - a departure from runway 20C at Singapore, then a left turn over Batam Island to take up a racetrack hold to the east of Singapore prior to a final approach to land back on the same runway.
It was established that about four minutes after take off, in the climb through about 7000 feet with the aircraft commander as PF, there had been two ‘bangs’ and a number of warnings and cautions had been displayed on the Electronic Centralized Aircraft Monitor (ECAM). The first of these was an alert of No 2 engine turbine overheat but this was then followed by multiple other messages relating to a number of aircraft system problems, creating a complex situation which could be resolved simply by sequentially following all the ECAM annunciations. Following a transient engine fire warning for the No 2 engine, it was shut down and discharge of the first fire bottle selected. When no indication that the selected fire bottle had actually discharged was seen, the second fire bottle was selected with a similar outcome. Then, after review, the remainder of the engine failure procedure, including selection of an automated process for fuel transfer between tanks, was actioned before turning to the other ECAM system alerts displayed.
Fortuitously, in addition to the operating crew of a Captain as pilot in command, a First Officer acting as co-pilot and a routinely-rostered Second Officer, there was also a trainee Check Captain performing a routine Line Check on the aircraft commander whilst being overseen by a Supervising Check Captain. This exceptional crew resource was able to be deployed to handle the exceptional prevailing circumstances. A ‘PAN’ call was made to ATC and an intention to return to Singapore advised and upon crew request, the aircraft was then cleared to enter a holding pattern to the east of Singapore in order to work through the procedures relevant to the messages displayed by the ECAM and establish the state of the aircraft and its systems before making an approach. It became clear that there had been some impact damage caused by debris from the failed No 2 engine and fuel could be seen leaking from the damaged left wing fuel tanks. Since the calculated landing performance at high weight was close to limits - indications were that the aircraft would stop with 100 m of runway remaining with reverse thrust only available from one instead of the usual two engines - the cabin crew were given a precautionary brief to prepare the cabin for a possible runway overrun and evacuation. Prior to beginning the approach, the controllability of the aircraft was verified by a number of manual handling checks at holding speed. The approach was initially made with the AP engaged but, after some un-commanded disconnects, the last 1000 feet was flown manually to a successful landing back at Singapore about 1 hour 45 minutes after the engine failure. The aircraft came to a stop about 150 metres from the end of the 4000 metre runway used for landing with fuel continuing to leak from the left wing. The attending Rescue and Fire Fighting Services applied large quantities of water and foam below the left wing whilst shut down of the remaining three engines was attempted but No. 1 engine continued to run. Passenger disembarkation was eventually commenced about 50 minutes after landing using steps brought to the right hand side of the aircraft and had then taken about 50 minutes to complete. Even after passenger disembarkation, numerous attempts to shut down the No. 1 engine by the flight crew, maintenance engineers and the airport emergency services were all unsuccessful and final shutdown was only achieved about 3 hours after the aircraft landed by means of the pumping of fire fighting foam directly into the engine inlet.
The Investigation found that “the flight crew and cabin crew managed the event as a competent team in accordance with standard operating procedures and practices”.
Debris from the failed engine was found to have affected both the structure of the aircraft and a number of its systems. The Investigation found that whilst some debris from the failed No. 2 engine had been lost overboard and subsequently recovered by residents of Batam Island, a large fragment of the engine turbine disc had penetrated the left wing leading edge, passed through the front spar and entered the left inner fuel tank before exiting through the top skin of the wing. Although the passage of this fragment through the fuel tank had initiated a short-duration low-intensity flash fire within the tank, the Investigation “determined that the conditions within the tank were not suitable to sustain the fire”. Another short-lived fire which also self-extinguished was found to have occurred within the lower cowl of the No. 2 engine as a result of oil leaking into it from a damaged oil supply pipe. The ejected large turbine disc fragment had also severed wiring looms inside the wing leading edge that were connected to a number of aircraft systems. A separate disc fragment was found to have severed a wiring loom located between the lower centre fuselage and body fairing which included wires that provided redundancy for some of the systems already affected by the severing of wires in the wing leading edge so that some of those systems were rendered inoperative. The aircraft’s hydraulic and electrical distribution systems were also damaged and this then indirectly affected other aircraft systems.
The No. 2 engine failure sequence was found to have been initiated by an oil leak from a crack in an oil feed pipe which had lost its integrity as a result of fatigue over some time. During the occurrence flight, it was determined that the crack in the feed pipe involved had grown to a sufficient size for oil within it to be released in the form if an atomised spray into the buffer space between the bearing chamber and the hot air surrounding the Intermediate Pressure (IP) turbine disc and the air temperature was sufficiently hot for the oil to auto-ignite. It was established that “the resulting fire propagated through the bearing chamber buffer space and eventually impinged upon the IP turbine disc drive arm, resulting in the separation of the disc from the drive shaft”. Following this, the engine had behaved in a manner different to that anticipated by the manufacturer during engine design and testing with the disc accelerating to a speed in excess of its structural capacity and bursting into three main segments which exited the core at high speed and with sufficient force to puncture the engine outer casing.
The faulty pipe was in a section of the High Pressure/Intermediate Pressure (HP/IP) bearing chamber oil feed pipe known as the oil feed stub pipe and contained an area of reduced wall thickness which it was concluded had been the consequence of misalignment of a counter bore machined into the end of the stub pipe during manufacture. A detailed engineering analysis found that the stresses generated in this oil feed stub pipe were sensitive to the wall thickness and that this had had a significant effect on the fatigue life of the pipe. The Investigation found that the mis-aligned counter bore had not conformed with the corresponding design specification and noted that following the event, Rolls-Royce had found that a significant number of HP/IP bearing support assemblies in service on a number of other Trent 900 engines had been produced with oil feed stub pipes that did not conform to the design specification. As a result of these findings, a Safety Recommendation was issued on 1 December 2010 as follows:
- that Rolls-Royce plc address (the identified critical safety issue - the risk to the integrity of the failed oil feed stub pipes) and take actions necessary to ensure the safety of flight operations in transport aircraft equipped with Rolls-Royce plc Trent 900 series engines. [AO-2010-089-AR-012]
The Investigation noted that Rolls-Royce had taken action to fully address this Recommendation, which was included in a factual Preliminary Report released by the Investigation on 3 December 2010.
An Interim Factual Report was issued on 18 May 2011. This noted that a simulation of the flight following the engine failure based on the available recorded evidence and flight crew accounts had confirmed that the aircraft had remained in Normal Law thus ensuring that flight envelope protections had remained in place throughout.
The remainder of the Investigation had continued to concentrate on the effectiveness of the applicable manufacturing Quality Assurance processes and the effect they may have had on the circumstances which had led to the manufacturing fault involved. It was found that “the misalignment of the counter bores was the result of movement within the HP/IP bearing support assembly during manufacture and that a number of opportunities existed during the design and manufacture processes where the misaligned oil feed stub pipe counter bores could have been identified and managed. Those opportunities were missed for a number of reasons, but generally because of ambiguities within the manufacturer’s procedures and the non-adherence by a number of the manufacturing staff to those procedures”. It was noted that because the design datum for the oil feed stub pipe counter bore was no longer accessible once the pipe was fitted to the HP/IP bearing support assembly and so a corresponding manufacturing datum had been introduced to specify the location for the bore. However, “the manner in which the manufacturing datum was represented on the manufacturing drawings resulted in its position not being constrained to the location of the oil feed stub pipe”. This had resulted in it being difficult to detect the misaligned counter bores and the effect that they had had on the thickness of the oil feed stub pipe wall.
It was further found that “during an inspection of the first HP/IP bearing support assembly manufactured in 2005, the manufacturing drawings were referenced rather than the design definition drawings”. It was considered that “the manufacturer’s procedures for the inspection contained ambiguity that may have influenced the inspector’s decision to use the manufacturing drawings” and that as a result, “the lack of constraint on the manufacturing datum was not identified and the HP/IP bearing support assembly entered into production without having properly shown compliance with the design specification”. The Investigation found that, in the absence of some of the relevant inspection records relating to the HP/IP bearing support assembly fitted to the failed engine, it was unable to determine exactly why the manufacturing non-conformance was not reported. It was also noted that “records for HP/IP bearing support assemblies from other Trent 900 engines produced around that time had (also) not been retained by the manufacturer” with similar consequences for the Investigation. It was concluded that “a culture existed within the manufacturer’s facility that produced the HP/IP bearing support assemblies where it was considered acceptable to not report what were considered to be ‘minor’ non-conformances”.
It was noted that in 2007, Rolls-Royce had “identified that a number of components had left the facility with unreported non-conformances and carried out a major quality investigation. After that investigation, a number of newly manufactured non-conforming oil feed stub pipe counter bores were identified and reported by manufacturing personnel. However, due to a difference between the reference datum used by the manufacturer’s automated measuring machines and the datum specified on the drawings, the engineers assessing the effect of the non-conformance misunderstood how the non-conformance would affect the wall thickness of the oil feed stub pipe”. Then, in March 2009, Rolls-Royce had found that oil feed stub pipe counter bores had been misaligned in previously manufactured and released HP/IP bearing support assemblies and, for the first time, the effect which such a misalignment of the counter bore would have had on the thickness of the pipe wall. The engineer who had made the discovery had applied for an engineering assessment to establish the suitability of the non-conforming oil feed stub pipes for continued use through the manufacturer’s non-conformance management system. The Investigation found that there was a degree of uncertainty in the statistical analysis method used in the subsequent assessment and that this “was not effectively communicated to, or understood by, the engineers that assessed the application” and retrospective approval was subsequently given to allow the non-conforming HP/IP bearing support assemblies to remain in service. The procedure for such ‘retrospective concessions’ at Rolls Royce required that applications be approved by both the Chief Engineer and the Business Quality Director. For a reason that could not be positively identified by the Investigation, “neither the Chief Engineer’s nor the Business Quality Director’s approval for the oil feed stub pipe counter bore retrospective concession were sought”. The Investigation found that there had been “no other opportunities where the potential for cracking in the oil feed stub pipes could have been identified and the non-conformances managed” prior to the investigated failure the following year.
The formally stated Findings of the Investigation were as follows:
- Contributing Safety Factors in respect of the disc failure during the occurrence flight, the manufacture and release into service of the failed engine and the opportunity to manage the non-conforming oil feed stub pipes in the Trent 900 fleet. [Three “Safety Issues” identified].
- Other Safety Factors in respect of the release of Trent 900 engines with non-conforming oil feed stub pipe counter bores, the minimisation of hazards resulting from an uncontained engine rotor failure and the application of the landing distance performance calculation. [Nine “Safety Issues” identified]
- Other Key Findings on six points [No “Safety Issues” identified]
Safety Actions in response to the identified Safety Issues by Qantas Airways, Rolls-Royce, Airbus, EASA and CASA and Proactive Action taken and planned by Airbus in respect of software enhancements to both the fuel trim and electrical systems are been fully documented in the Investigation Report.
The only remaining observation from the Investigation was the opportunity to use information gathered to review current guidance on how to minimise hazards from uncontained engine failures since the damage caused had exceeded the parameters of the existing model in the relevant advisory material.
As a result, two further Safety Recommendations were issued:
- that the European Aviation Safety Agency, in cooperation with the US Federal Aviation Administration, review the damage sustained by Airbus A380-842, VH-OQA following the uncontained engine rotor failure overhead Batam Island, Indonesia, to incorporate any lessons learned from this accident into the advisory material. AO-2010-089-SR-039
- that the US Federal Aviation Administration, in cooperation with the European Aviation Safety Agency, review the damage sustained by Airbus A380-842, VH-OQA following the uncontained engine rotor failure overhead Batam Island, Indonesia, to incorporate any lessons learned from this accident into the advisory material. AO-2010-089-SR-040
The Final Report of the Investigation AO-2010-089 was published on 27 June 2013