Pipeline Investigation Report P05H0061

At 0829 mountain standard time on 18 October 2005, a programmable logic controller failed at the Foothills Pipe Lines Ltd. decompression/recompression facility owned by TransCanada and located within the BP Canada Energy Company Empress natural gas liquids facility, near Empress, Alberta. When the programmable logic controller failed, the "A" recompressor began to surge. At 0841, a nominal pipe size 2 section of piping on the pressure-up line for the "A" recompressor broke apart, releasing a negligible amount of natural gas. The control room operator immediately initiated an emergency shut-down of the facility. As a result of the emergency shut-down, approximately 23 x 10³ cubic metres of gas was released to atmosphere and approximately 11 x 10³ cubic metres of gas was released to flare. The "A" recompressor motor continued to run for another 20 minutes at which point an explosion occurred in the central area of the motor. The motor continued to run until it was manually shut down at 0911. There were no injuries.

The Foothills Pipe Lines Ltd. decompression/recompression facility, located within the BP Canada Energy Company (BP Canada) Empress natural gas liquids (NGL) facility, is owned by TransCanada but operated on its behalf by BP Canada. The function of the decompression/recompression facility is to decompress and condition high-pressure gas from the TransCanada Foothills Zone 6 pipeline to pressures and temperatures consistent with the NGL stripping facilities of the Empress area facilities. Residue gas from plants 2 and 5 of the BP Canada Empress NGL facility is then recompressed in the decompression/recompression facility to pipeline standards and re-injected into the TransCanada Foothills Zone 6 pipeline. TransCanada's supervisory control and data acquisition (SCADA) system receives data from the decompression/recompression facility for information purposes only through a data highway. TransCanada's SCADA system cannot control any functions within the decompression/recompression facility.

At 0829 mountain standard time,^{Footnote 1} data communications were lost between the decompression/recompression facility's "A" programmable logic controller (PLC) and the BP Canada Empress NGL facility control room (control room). At this time, the control room operators received an alarm indicating "Device box or communication error" as well as a flood of other alarms and noticed that most communications were lost with the decompression/recompression facility. The control room operators radioed the outside operators to go to the decompression/recompression facility to determine the reason for the inconsistent messaging on the control room screens. Although the decompression/recompression inlet and outlet valves did not show any indication of being opened or closed, the plant pressure indicated that the valves had closed and the control room operator believed that the failure in communication had triggered an emergency shut-down (ESD) of the facility.

At 0841, as the outside operator approached the decompression/recompression facility, the nominal pipe size (NPS) 2 pressure-up piping for the "A" recompressor broke apart at the connection between the NPS 24 suction header and a control valve. This connection had heavy components on top and a small diameter neck at the weld-o-let weld as well as a valve actuator mounted horizontally to the piping. The outside operator radioed the control room operator who immediately initiated an ESD of the facility.

Although the isolation valves between the decompression/recompression facility and Plant 2 were configured to close on the ESD, they did not close as expected. The control room operator therefore gave each valve a close signal from the operator screens in the control room. At this time, the control room operator noticed that the "A" recompressor was still showing some power. Following the ESD, the control room operator continued to monitor and depressurize the facility and verify its isolation. At 0856, an explosion occurred in the central area of the "A" recompressor motor. The motor continued to run after the explosion until it was shut down at 0911 by a manual shut-down switch in the control room.

An examination of the "A" PLC indicated that it failed due to a processor card failure. When the "A" PLC failed, its outputs were de-energized. Some of the "A" recompressor functions controlled by the "A" PLC that required a de-energized signal to close or shut down included the main and auxiliary lube oil pumps, the purge/damper valves and the suction and discharge valves. The "A" recompressor motor, however, required an energized signal to shut down and therefore kept running until site personnel manually shut it down. All unit protection including high vibration, low lube oil pressure, surge, and compressor and gearbox temperature trips were inhibited due to the de-energized signal from the failed "A" PLC. The examination of the "A" PLC also indicated that the ESD signal to the isolation valves between the decompression/recompression facility and Plant 2 was serially connected through the PLC to the data highway.

When the "A" recompressor discharge valve closed, the compressor began to surge. The surge controller detected the loss in flow and opened the recycle valve but the surge event continued. During the surge event, the NPS 2 pressure-up piping failed between the NPS 24 suction header and a control valve. The TSB Engineering Laboratory reviewed the metallurgical report of the failed connection and concurred with the findings that the failure was caused by low cycle fatigue under the influence of high cyclic stress and that the stress probably occurred during the high magnitude vibrations associated with the surge event.

An examination of the "A" recompressor and motor indicated the following: the lubrication system was shared between the compressor, the gearbox and the motor; the compressor bearings had failed; the compressor seals were damaged; the insulation on parts of the stator windings had degraded; partial discharges had occurred where the insulation had degraded; the insulation at those locations had been further damaged by exposure to the partial discharges; the explosion occurred in the central area of the motor and resulted in some parts of the motor housing striking the ceiling and interior of the building.

In April 2002, a gas leak occurred at the NPS 2 pressure-up piping between the NPS 24 suction header and a control valve on the "B" recompressor. This branch connection was the same design as the connection that failed on 18 October 2005. The April 2002 leak occurred as gas was venting through the control valve between the decompression/recompression facility and the inlet to Plant 2. The control valve had opened following a total loss of power to the control systems of the decompression/recompression facility and Plant 2.

Following an analysis by BP Canada of the events surrounding the April 2002 occurrence, BP Canada determined that the loss of power resulted from inappropriate job procedures and work practices being followed during the installation of some electrical equipment at the BP Canada Empress NGL facility. A metallurgical examination of the failed branch connection attributed the failure to low cycle fatigue cracking under the influence of high cyclic stress. The metallurgical report indicated that the cracking was probably related to a period of unusually high amplitude bending stresses combined with loose support brackets. BP Canada attributed the high magnitude vibration event to the venting of gas through the control valve.

Following that occurrence, BP Canada replaced the connection with Schedule 80 pipe, reinforced the support brackets to limit movement of the NPS 2 pressure-up piping but still allow thermal growth, and implemented a regular maintenance program to ensure that the support brackets remained secure. BP Canada also took measures to ensure that procedures for the installation of electrical equipment as well as the inspections following those installations would be adequate and would be followed.

The control room operators were unaware that the loss in communications with the "A" PLC was due to a PLC failure and that some of the functions controlled by that PLC, including the shut-down of the motor for the "A" recompressor, had been compromised by the failure.

The "A" recompressor went into surge when the de-energized signal generated by the failed "A" PLC shut down many of the primary functions of the "A" recompressor but not the motor.

Although the surge controller detected the loss in flow and opened the recycle valve, the discharge valve blocked the flow of gas and the compressor continued to surge. During the surge event, the piping arrangement of the NPS 2 pressure-up piping between the NPS 24 suction header and a control valve could not withstand the high magnitude vibrations generated by the surge event and subsequently failed.

Although measures had been taken following the April 2002 occurrence to reinforce the support brackets, the vibrations generated by a surge event were beyond the capabilities of that design.

Because the lube oil pumps had shut down, the compressor bearings eventually lost oil and failed, resulting in damage to the compressor seals. Gas was then able to travel from the damaged compressor seals through the lube oil system piping to the motor bearings. Once the gas in the motor housing reached the lower explosive limit, it ignited due to sparking from partial discharges at parts of the stator windings where the insulation had degraded.

Findings as to causes and contributing factors

1. The de-energized output from the failed "A" programmable logic controller did not result in a fail-safe shut-down of the "A" recompressor and motor.
2. Because many of the primary functions of the "A" recompressor shut down but the motor did not, a surge event occurred.
3. The piping design of the nominal pipe size 2 pressure-up piping could not withstand the high magnitude vibrations generated during the surge event.
4. Because of damage to the compressor seals during the surge event, gas migrated through the lube oil system piping to the motor housing where there was an ignition source.
5. Sparking from partial discharges had occurred in the motor housing at locations on the stator windings where the insulation had degraded.
6. The emergency shut-down of the decompression/recompression facility was compromised by the configuration of the emergency shut-down signal through the failed "A" programmable logic controller.
7. The potential impact of a programmable logic controller failure on the safe shut-down of the decompression/recompression facility had not been identified as a concern.

Safety action taken

Following an analysis by BP Canada Energy Company (BP Canada) and TransCanada of the events surrounding the October 2005 occurrence, TransCanada made many changes at the decompression/recompression facility including changes to the programmable logic controller (PLC) and wiring and the emergency shut-down (ESD) sequencing to ensure that a PLC failure would not compromise the safe operation and shut-down of the facility. TransCanada also redesigned the nominal pipe size (NPS) 2 pressure-up piping on both the "A" and "B" recompressors to ensure that the piping would be more resistant to vibration but still allow thermal growth. In addition, TransCanada installed partial discharge sensors on both the "A" and "B" motors to contribute to better monitoring and diagnostics.

In early December 2005, the Transportation Safety Board of Canada (TSB) sent a safety information letter to the National Energy Board (NEB) advising that a failure of a PLC may not result in a fail-safe condition. In late December 2005, the NEB issued a safety advisory to all oil and gas companies under its jurisdiction informing them of the possible hazards of a failed PLC.

This report concludes the Transportation Safety Board’s investigation into this occurrence. Consequently, the Board authorized the release of this report on 12 July 2006.