Pipeline Investigation Report P02H0052

• Table of contents

  1. Factual information
  2. Analysis
  3. Findings
  4. Safety action
  5. Appendices

Summary

At approximately 1053 eastern standard time, on 07 December 2002, Trans-Northern Pipelines Inc. was delivering refined petroleum products into the company station at Cornwall, Ontario, when a high-pressure situation occurred, resulting in rupture of the mainline pipeline between the Como pump station and the Lancaster pump station, near Saint-Clet, Quebec. Approximately 32 cubic metres of low sulphur diesel was released to the surrounding area and made its way into the Saint-Emmanuel drainage system. Most of the released product was recovered by the company. There were no injuries.

Factual information

On 07 December 2002, Trans-Northern Pipelines Inc. (TNPI) was delivering refined petroleum products from the Montréal, Quebec, refining basin into the Ottawa, Ontario, terminal storage facilities. This movement of product had originated from the company's Montréal pump station, with the pipeline pressure then being boosted at the Como, Quebec, and the Lancaster and Farran's Point, Ontario, pump stations (see system schematic in Appendix A).

At 1043 eastern standard time,^{Footnote 1} the pipeline control centre operator (CCO), located at the company's mainline control centre (MCC) in Richmond Hill, Ontario, initiated a series of operations to start a scheduled full stream delivery of products into Cornwall, at Kilometre Post (KP) 119.09 (Mile Post [MP] 74.5). The CCO began by setting the Cornwall inlet control valve (ICV) to 3585 kilopascals (kPa) or 520 pounds per square inch gauge (psig) to maintain mainline pipeline pressure and avoid an overpressure situation. Two minutes later, the CCO commanded the opening of the Cornwall take-off valve (TOV) and then the shutdown of the Farran's Point pump station (see the Cornwall Station schematic in Appendix B), followed by the shutdown of the Ottawa lateral and the Ottawa terminal valves.

At 1052, the CCO commanded the Cornwall mainline block valve (MLV), which is located immediately downstream of the Cornwall delivery point, to move to the fully closed position. It became fully closed at 1053, resulting in the start of the full stream delivery of refined products into Cornwall. At that moment, without being commanded by the CCO, the Cornwall TOV moved to the fully closed position in 23 seconds. The Cornwall meter manifold had just experienced a high-pressure condition, which prompted an alarm signal. The mainline pressure at Cornwall rose to 7019 kPa (1018 psig) and the Lancaster pump station (see the Lancaster Station schematic in Appendix C) then automatically shut down on a high pipeline pressure condition. The Supervisory Control and Data Acquisition (SCADA) system data showed that an operational upset had occurred. In addition, the CCO observed that the pressure in the mainline pipeline suddenly dropped while the inlet pressure at Lancaster surged from 875 kPa (127 psig) to 7460 kPa (1082 psig) and then receded to 193 kPa (28 psig).

At 1054, the mainline pressure at Cornwall had reached 6047 kPa (877 psig). The Cornwall ICV was reset by the CCO to a mainline pressure of 8964 kPa (1300 psig) and the flow of product out of Montréal was commanded to shut down. At this point, the CCO suspected that a line rupture had occurred and initiated the isolation of the pipeline system by taking the Como pumping unit off line, at KP 51.17 (MP 32).

At 1055, the Cornwall mainline pressure was observed to rise to 7143 kPa (1036 psig) and then immediately drop to 4937 kPa (716 psig). The shutdown of the Como and Lancaster pump stations was completed, thus isolating the pipeline section and preventing any further product from entering that pipeline section. The CCO notified field services personnel of the event and that the probable location of the occurrence site was between Cornwall and Como. The field services response included the dispatch of TNPI's aerial patrol.

At 1056, a line balance alarm was issued to the CCO indicating a loss of product from the pipeline, which was later determined to be approximately 32 cubic metres of low sulphur diesel released from the pipeline.

At 1255, TNPI's aerial patrol identified a product stain on TNPI's right-of-way, in a field east of the Saint-Emmanuel drainage system, just east of Saint-Clet, at KP 63.57 (MP 39.5). There was a hole in the soil over the pipeline at the point of rupture, measuring 91 cm in diameter. Approximately 180 m (588 feet) east of the Saint-Emmanuel drainage ditch, which was partially covered by snow, an area 18 m by 12 m had been covered with fuel. The rupture occurred in the middle of a farmer's field with access available through a bridge that crosses the Saint-Emmanuel drainage ditch. The drainage ditch and its associated drainage tiles system were visible from the highway adjacent to the occurrence site. Some 20 years before the occurrence, the same segment of pipe that ruptured had been struck by a third party, inflicting mechanical damage to the pipeline. The former landowner had previously admitted to TNPI to having installed the drain tiles some 20 years before that rupture.

By 1340, the first TNPI personnel arrived on scene and determined that the pipeline had ruptured. TNPI local response personnel were aware that drainage tiles were present at the occurrence site, but performed only a general reconnaissance of the surrounding area. They did not perform a detailed inspection of the drainage system. At 1422, TNPI's field services personnel arrived at the occurrence site with TNPI's oil spill containment and recovery trailer and established a work site adjacent to the occurrence location. They began the process of containment, isolation, and clean-up of the occurrence site.

At 1610, directly downstream of the occurrence site, a manually operated MLV at Saint-Polycarpe, Quebec, was closed by TNPI field personnel. Unlike other pipeline companies, TNPI does not have a mutual-aid arrangement with another pipeline company in the vicinity or with a fully qualified and competent local person to manually close valves on its pipeline system during an emergency.

At 1632, while en route from Toronto, Ontario, to the occurrence site, the Engineering Manager of Field Services for TNPI advised the Transportation Safety Board of Canada (TSB) telephone hotline of the company's concern about the presence of an agricultural drain tile to the west of the pipeline rupture, which could possibly provide a conduit into the larger drainage system. The Engineering Manager indicated that some product could potentially enter the drain tile and that the company was responding according to its emergency response plan.

Excavation of the pipeline started the evening of December 7. At 0315, on December 8, the field crews exposed a 10 cm (4-inch)plastic drain pipe in the immediate vicinity of the ruptured pipe. The crews quickly determined that the released refined product had seeped through the drainage tiles and had made its way into the Saint-Emmanuel drainage system, travelling approximately two kilometres downstream toward the St. Lawrence River. Field personnel erected a flume and installed an oil boom to collect spilled product with a vacuum truck. Contaminated soil was excavated and removed from the site. The pipeline was returned to service at 1944 on December 8.

Representatives of the National Energy Board (NEB) and the Quebec Ministry of the Environment were present at the occurrence site.

Weather and refined product temperature

At the time of the occurrence, the daytime temperature was −2°C,with moderate snowfall and strong westerly winds. The temperature of the product being transported was 13°C.

Mainline operations

The day-to-day operations of the pipeline system were remotely controlled by a CCO located at the MCC. Each newly appointed CCO received training in pipeline operations including how to determine and establish set-point pressures on the system. The CCO who was on duty at the time of the incident received training as a junior CCO in the operational aspects of the TNPI system. However, he did not receive any new training to enhance skill sets or develop new methods of managing and controlling pipeline operations until he reached the full CCO level (approximately six months later). While the CCO had access to reference material on the SCADA system, this material was predominately software reference material (help menus) in the computer-based system. TNPI stated that this training approach conformed to the requirements of both the NEB Onshore Pipeline Regulations and the Canadian Standards Association (CSA) standard CAN/CSA Z662, Oil and Gas Pipeline Systems (CSA Z662). While CCOs had access to various manuals and material, which was freely available in the control centre, they were not provided with manuals outlining corporate policies, practices, and standards for pipeline operations.

The maintenance operations for the Montréal, Ottawa, and Kingston, Ontario, sections of the pipeline were supervised from Lancaster, which is located at KP 97.85 (MP 61.2). Lancaster and Como are the only two stations on the TNPI system with no check valve on the discharge side of the station. The company had installed several process flow media isolators together with high-pressure sensor transmitters that were set at 8274 kPa (1200 psig) and a pressure control valve with discharge set points set to 8619 kPa (1250 psig). When a rise in pipeline pressure to maximum levels is detected, the valve will automatically close if the maximum pressure set-point pressure of 8619 kPa (1250 psig) is exceeded. In this instance, the station is automatically relegated off line, but the transient pressure surge wave can be carried past the station because there is no check valve after the station is taken off line. At Como and Lancaster, the station control valves have variable discharge set points that range from a low set value of 2069 kPa (300 psi) to a maximum value of 11 032 kPa (1600 psi), but are usually set at 8619 kPa (1250 psi). The CCO cannot set the valve above 8274 kPa (1200 psi). These valves will only move to a fully closed position when the actual pipeline pressure is higher than the set-point value of 8619 kPa (1250 psi). In the event the control valve is unable to react to the increase in pressure in a manner that maintains pressure below its set point, the pressure set points will trigger a shutdown of the pump units to remove the pressure source. An independent consultant determined that the installation of check valves at both the Como and Lancaster stations could partially prevent a transient pressure surge wave from travelling upstream from Lancaster.

The NEB-approved maximum operating pressure (MOP) between Cornwall and Como is 8274 kPa (1200 psig). Section 4.14 of CSA Z662 requires that pipeline systems be equipped with pressure control and overpressure protection systems to ensure that the MOP is not exceeded. At Cornwall, the meter station has its own manifold protection system, complete with relief valve. However, there was no overpressure protection on the mainline. Instead, the MOP of the pipeline is controlled by the line pressure set point at Como and Lancaster, which are set at 8618 kPa (1250 psig). For the TNPI system, pipeline pressure protection is normally installed at the source of pressure, that is, at the pump stations. Since Subsection 4.14.1.2 of CSA Z662 states that "overpressure protection shall be installed to ensure that the maximum operating pressure is not exceeded by more than 10% or by 35 kPa (.005 psig), whichever is greater," TNPI is obliged to provide overpressure protection at a level of 9100 kPa (1320 psig). TNPI believes that, by providing this protection at 8618 kPa (1250 psig) and at the local station level where the response would be instantaneous, it is conservative in its overpressure protection when compared to the CSA requirements.

Cornwall operations

Before the system was automated, the Cornwall ICV was operated manually within a narrow range of operating pressures. An upper operating pressure of 965 kPa (140 psi) on the meter station piping was in place to protect it and associated equipment at the station. Lancaster was taken off line prior to making a full stream delivery into Cornwall off the mainline to reduce the likelihood of Lancaster packing the pipeline and developing a transient pressure surge in the mainline pipeline, if Cornwall was suddenly shut down. There was also a requirement to restrict opening the Cornwall TOV when the mainline pressure was higher than 2413 kPa (350 psi).

Since the system was automated in 1993, the shutdown of the Lancaster pump station has not been required, but the company has retained the ICV set point of 965 kPa (140 psi) at Cornwall. A pressure control was installed to protect the station. When the pressure rises up to or above 965 kPa (140 psi), the local automated logic control is programmed to rapidly (in less than 10 seconds) close the ICV. After Cornwall was automated in 1995, the CCOs received no specific instructions about line opening pressures.

TNPI also developed a mathematical algorithm within its SCADA system to address the situation if the mainline pressure at Cornwall increased to 6895 kPa (1000 psig). Lancaster would shut down and go off line to protect against the transient pressure surge wave, even if the discharge pressure was less than the set-point value of 8619 kPa (1250 psi). However, Como would not have reacted to the shutdown of Lancaster and would continue pumping, resulting in increases in line pack between Como and Lancaster being driven by Como.

The Cornwall Station's ICV is a Fisher-type, model 1007, purchased by the company in 1954. It was equipped with a Fisher-type 350 electro-hydraulic operator purchased by the company in 1978. The ICV received annual maintenance, which was last performed on 16 October 2002. The maintenance work included changing the hydraulic oil and oil filter, as well as calibrating the ICV against the output signal from the local computer interface. After the occurrence, TNPI checked all the equipment settings at Cornwall, and nothing was found to be out of range with respect to set points that could trigger the combined uncommanded TOV move and a high manifold pressure shutdown. All systems operated as expected.

During 2002, TNPI experienced eight pipeline shutdowns while delivering refined products into Cornwall; two resulted in the shutdown of Lancaster. For the rest of federally regulated pipelines in the industry, there was only one reported shutdown during the same period. On the days in question, the CCO was attempting to maximize the flow of product into Cornwall and close in the delivery of product into Ottawa when a high-pressure event was triggered by the uncommanded closure of the TOV. The shutdowns were due to malfunctions related to a variety of operational conditions, including switching between storage tanks, the malfunction of shipper valves, and unintended valve closures. During all these unscheduled shutdowns, the SCADA system recorded a pressure of 8274 kPa (1200 psig).

During an unscheduled shutdown of a pipeline and with the sudden blockage of fluid in motion, a hydraulic surge is created. The kinetic energy, released as a transient pressure surge wave, can spike up the system's operating pressure, destroying system instrument recorders, pumps, pipes, fittings, and valves. The resulting hydraulic shock wave, moving at the speed of sound, travels the length of the pipeline until it encounters a closed MLV or check valve, then rebounds back to the point of origin. The hydraulic shock wave reverses again, oscillating back and forth until friction dissipates the pressure spike or a system component fails. TSB's mandatory reporting requirements call for pipeline companies to report overpressure situations.

Supervisory control and data acquisition

TNPI's SCADA system operates similarly to other SCADA systems, with respect to display and recording of data. However, TNPI's SCADA data, for purposes of pressure trending, are recorded at 15-secondintervals, and event data are gathered and recorded at 5-secondintervals, which TNPI believes exceeds the requirements of both CSA Z662 and the NEB. Just prior to the occurrence, the mainline pressure at Cornwall was recorded by the SCADA as 3585 kPa (520 psi). However, the hydraulic gradient for the product in the system showed a pressure of 4668 kPa (677 psi). A discrepancy between the recorded SCADA data and the observed data was also noted during another unscheduled shutdown. On 09 May 2003, while TNPI technicians were testing equipment at Cornwall, an anomaly resulted in the shutdown of Lancaster. During this shutdown, the technicians measured a pressure surge that reached 9660 kPa (1401 psig), the CCO observed a pipeline pressure of 9308 kPa (1350 psig), while the SCADA system recorded only 8274 kPa (1200 psig). Pipeline data were being retrieved locally by the TNPI technicians from a programmable logic controller (PLC) at a high sampling rate of scans at 2-secondintervals, through a personal computer gathering mechanism. TNPI indicates that Cornwall has overpressure protection set at 140 psi and that the mainline between Cornwall and Lancaster has overprotection and is set to never rise above 6895 kPa (1000 psig). As well, TNPI has indicated that the overprotection at Lancaster is set to protect at 8619 kPa (1250 psig) to remove or negate the source of any overpressure situation. However, during TNPI's testing, the installed pressure protection devices failed to protect the system from the overpressure situation.

History of the pipeline

The section of pipeline that ruptured was manufactured in 1952 by Stelco, using the electric resistance welding process and a pipe grade of steel of 317 megapascals (American Petroleum Institute 5LX, pipe grade X-46).The pipe section had an outside diameter of 273.1 mm (10 inches) and a wall thickness of 7.8 mm (0.307 inch).

The pipeline was installed with 0.60 m (2 feet) of minimum cover. At the occurrence site, the pipe depth of cover was 0.61 m. In 1987, TNPI lowered the pipeline across the drainage ditches located at either end of the field in which the occurrence took place. The nearest field bends (sags and over-bends) were located on either side of the property at the ditch crossings.

The pipe was externally coated with a coal tar enamel coating and was protected against corrosion with a cathodic protection system. The pipeline was hydrostatically tested for 24 hours to a minimum pressure of 10 342 kPa (1500 psi). In 1952, the Board of Transport Commissioners granted TNPI leave to operate with an MOP of 8274 kPa (1200 psig). The elevation difference between the rupture site in Lot 11, which is estimated at 53.3 m above sea level (asl) and the Saint-Polycarpe block valve, which is estimated at 52.5 m asl, is nominal. Lancaster pump station to the west is estimated at 53 m asl.

The TNPI pipeline system stretches from Montréal to Ottawa and Toronto, and into southern Ontario. The pipeline is a principal supplier of refined petroleum hydrocarbons to cities and airports. In 1952, the pipeline right-of-way was located in rural areas away from the main population base of cities located along the route. However, over the past 53 years, the population base of every city along the route has grown to the point that TNPI has population encroachment issues. These include illegal third-party access over the entire length of its pipeline system, which has resulted in damage to and failures of the pipeline. In some cases, the company was unaware of these third-party activities until well after they had occurred. Prior to the accident, the last line walk of the mainline between Montréal and Cornwall was on 02 May 2002. No abnormalities on the surface of the right-of-way were reported from KPs 61.14 to 64.20 (MPs 38.2 to 40.1). The last weekly flight patrol was performed on 02 December 2002 and reported "No reportable observations as per TNPI Aerial Patrol Procedure."

Integrity management program

As part of the company's ongoing integrity management program, TNPI has internally inspected the pipeline system six times since 1980 (in 1980, 1988, twice in 1993, and twice in 1998) using various types and models of in-line inspection (ILI) tools. At the completion of an ILI tool run, the vendor performed an initial data quality analysis of the raw data to ensure that the data were "usable." If there were problems with the recorded data, the vendor would then rerun the ILI tool. Once the vendor confirmed that the raw recorded ILI data were usable, the raw data were forwarded to TNPI for analysis and confirmation digs at potential sites on the pipeline system. Initial detailed analysis of the ILI raw data was usually performed by two full-time TNPI employees. They identified selected sites and requested further analysis from the tool vendor.

These ILI data analysts received four weeks of training in 1982 at the University of Texas. This course focused on interpreting integrity testing results produced by the ILI system available at that time. While TNPI data analysts have participated in vendor-sponsored training programs and conferences over the years, there has been no additional training to keep them current with the new generations of ILI tools.

During the 1980 survey, the ILI logs revealed a metal loss due to corrosion of between 15 and 30 per cent at the occurrence site, with a variance value of plus or minus 10 per cent. The feature was graded at level "1" by the tool vendor meaning that "the defect would remain buried and would not be excavated." The tool was not capable of discriminating between metal loss features, dents, and deformation. Arc burns, cracks, and longitudinal grooves were not detectable. In spite of a 1982 company policy requiring that all indications be correlated with the company's Plans, Profiles, and Books of Reference for the pipeline system, no correlation was made.

During the 1988 survey, the tool identified a feature at the occurrence site that was not graded by the vendor and not selected by TNPI for further investigation. The 1988 tool was more sophisticated than the earlier 1980 version. Data interpretation enabled fairly accurate discrimination between metal loss features and dents. However, metal loss in a pipe bend or a dent could not be identified by the 1988 tool. Longitudinal features, such as grooves, were not detectable.

During 1993, the company inspected the pipeline system using the Pipetronix ultrasonic ILI tool, which records more parameters. Prior to the inspection, a sizing survey of the internal diameter of the pipeline identified a feature at the occurrence site (KP 63.57), as a 1.27 cm (0.5-inch)ovality. The log confirmed the feature as a dent measuring 9.8 mm (0.4 inch) deep and 3986 mm (13 feet) long. As the depth represented 3.5 per cent of the outside pipe diameter, which is less than the repair threshold of 6 per cent cited in CSA Z662 (applicable at the time to perform field remediation work), the company decided that no further action was necessary. A consultant, retained by TNPI to assist with interpretation, examined the 1993 and 1988 data and interpreted the feature as a field bend, based on its length and its similarity to known bends. Neither TNPI, the consultant, nor the ILI vendor recommended examining the feature. The company did not perform a field verification dig.

In 1998, the company once again elected to run a sizing survey tool before using the Pipetronix magnetic flux leakage (MFL) ILI tool. The sizing survey tool identified a feature on the surface of the pipe at the occurrence site (KP 63.57), as being a 1.27 cm (0.5-inch)ovality spanning approximately 3.81 m (12.5 feet). The inspection and an examination by TNPI of the C-scanand sensor channels revealed a distortion typical of a pipe deformation. The company believed that the indication was the same feature previously identified as a field bend. It also believed that the absence of a metal loss signal in both ultrasonic and magnetic flux ILI surveys concealed the nature and seriousness of the feature. Despite the company's 1982 policy that all indications be correlated with the company's Plans, Profiles, and Books of Reference from the original construction, this indication was not so recorded. From the 1993 and 1998 Kaliper survey reports, the only noteworthy issue identified in the two final reports were the dimensions of the feature at the occurrence site.

Field bends are commonly installed along the length of the pipeline to follow ground contours or to provide changes in pipe elevation on either side of roadways, railways, ditches, and other obstructions. Field bends have a maximum length depending on the diameter, grade, and yield strength of the pipe. For the pipe involved in this occurrence, the maximum length of a field bend would be approximately 2.13 m (7 feet), based on the standards in place at the time of construction in 1952. The geometry of the pipeline, including the location of the field bends, is plotted in the company's Plans, Profiles, and Books of Reference right-of-way alignment sheets. These documents did not show the presence of a field bend at the occurrence site.

A review of the records from the six ILI runs revealed that several zones of corrosion were identified over the length of the pipeline system. During the reconditioning era of TNPI's bare 10-inchmainline between 1962 and the mid-1970s,some of these corrosion zones were repaired using puddle welding. While pipeline companies used this method at that time, puddle welding is known to produce microcracks in the parent metal of the pipe under the deposited weld material. CSA Z662 does not authorize its use for liquid pipelines. From TNPI's crossing of Rivière-des-Prairies (Mile Post 5.4) to Farran's Point (Cornwall Station), there are 754 puddle weld locations.

Unauthorized third-party activities

While repairs were being carried out, TNPI personnel noted that two types of coating had been installed on the pipe in the vicinity of the ruptured section: the original coal tar enamel coating applied in 1952 and a second form of coating referred to as "hot tape," which is a traditional tape coating used by TNPI to repair damaged coating. The coating was patched longitudinally on either side, over a total distance of 3.66 m (12 feet). At the two ends of the hot tape coating, the coal tar coating was tapered for a smooth transition. A review of TNPI's records of pipeline damage and coating repairs did not show data related to this coating repair, nor to the mechanical scoring and damage to the ruptured section of the pipeline.

According to TNPI's records, the previous landowner of the occurrence site had conducted unauthorized activities over TNPI's pipeline in 1976, 1981, and 1983. Near the occurrence site, there had been two unauthorized excavations, and the pipeline was damaged once. On 30 July 1981, an unauthorized ditch cleaning was conducted by a landowner approximately 523 m (1715 feet) west of the occurrence site. On 06 October 1983, another unauthorized ditch cleaning was conducted by a landowner approximately 180 m (588 feet) east of the occurrence site. No damage to the pipe was reported in either case. On 03 November 2000, a tile drainage contractor struck the mainline approximately 497 m (1628 feet) east of the occurrence site, which damaged the pipeline at the three o'clock position, over a distance of 2.01 m (6.58 feet). The previous landowner had admitted to TNPI staff that he had installed the tile drain "over twenty years ago."

Metallurgical testing and hydraulic analysis

A section of the ruptured pipe, approximately 8 m (26 feet) long, was shipped to the TSB Engineering Laboratory. The analysis (report LP 113/02) revealed that the rupture occurred at the three o'clock position with a rectangular "fish mouth" opening tear measuring approximately 1 m (40 inches) long. The fracture ran parallel to the pipe's longitudinal seam, and was located 9 to 10 cm (3.5 to 4 inches) away. The fracture occurred within a deformed and mechanically scored area. The pipe was deformed approximately 10 mm inward, which represents 3.5 per cent of the outside diameter. On its outer surface, longitudinal dents and grooves extended 2.7 m (9 feet) beyond the fracture zone.

The fracture face appeared fresh, with the exception of a series of semi-circular areas along the initiating fracture surface. Pre-existing cracks extended to a maximum depth of 40 per cent of the nominal pipe wall thickness, and initiated in the region of the noted mechanical scoring or damage on the pipe outer surface.

At the 10°C operating temperature of the pipe, Charpy impact test^{Footnote 2} results averaged 7 Joules (5 foot-pounds). These values are consistent with the materials used in 1952, but today's standards require a minimum of 27 Joules (19.9 foot-pounds) for a pipe with an outside diameter smaller than 457 mm (18 inches). The purely theoretical analysis using generalized assumptions, conducted by the TSB Engineering Laboratory, calculated that the static pressure required to rupture the damaged pipeline would have been between 11 032 and 15 169 kPa (1600 and 2200 psig).

In a letter dated 07 March 2005, TNPI provided a copy of a consultant's hydraulic report, dated 28 February 2005, on this occurrence. Based on the simulated hydraulic results, which utilized the company's SCADA records from the occurrence, the following conclusions were drawn:

• Starting from the given steady-state condition, the maximum pressure at the rupture site could reach 9032 kPa (1310 psig) when the downstream valves at Cornwall were closed and all other facility events were ignored at the upstream pipeline.
• With the actual SCADA events, the maximum pressure at the rupture site could reach 8039 kPa (1166 psig) before the rupture at 1054:38, when the downstream valves at Cornwall station were closed.
• The installation of check valves at the Como and Lancaster stations could partially prevent the pressure wave travelling upstream from the Lancaster pump station, resulting in the maximum pressure at the rupture site being 7364 kPa (1068 psig).
• The check valves at Como and Lancaster could have prevented fluid leakage from downstream of the Lancaster Station through the rupture when that incident occurred. However, the upstream check valve at the Como pump station would be less effective in this case because of the positive flow from upstream of the rupture site.

Analysis

As the delivery to Cornwall started, all pipeline operations appeared to be functioning normally. All valves were performing as commanded and no unusual valve activity was noted. While the CCO was attempting to maximize flow rates into Cornwall and to close in the delivery of product into Ottawa, a high-pressure event was triggered by the uncommanded closure of the TOV. The Cornwall TOV moved suddenly, without any instruction from the CCO, from the fully opened position to the fully closed position, as a result of a high manifold pressure condition that prompted the alarm signal to the CCO and a shutdown of the system. A pressure surge was generated in the mainline, travelled up the pipeline toward Montréal and ruptured the pipe at a zone of third-party damage.

After the occurrence, TNPI checked all the equipment settings at Cornwall, and nothing was found to be out of range with respect to set points that could trigger an uncommanded TOV move and a high manifold pressure shutdown. The cause of the TOV move and the subsequent high pressure resulted from CCO attempts to maximize flow rates into the station. The analysis will focus on the operations at Cornwall, and the inspection and maintenance of the pipeline.

Pipeline operations

For the year 2002, TNPI experienced eight pipeline shutdowns while delivering refined products into Cornwall. This is excessive when compared to the rest of the pipeline industry and would suggest shortcomings in the delivery procedures and the operational configuration of the Cornwall Station. For instance, the absence of overpressure protection devices on the mainline, even though required by Section 4.14 of CSA Z662, results in pipeline operations that are very sensitive to minor pressure fluctuations and might explain the relatively high number of shutdowns at the station.

When the system was automated in 1993, the company retained the ICV set point of 965 kPa (140 psi) at Cornwall to protect the meter station. The pressure set point is controlled through a local automated logic that prompts the TOV to close as soon as the set point is exceeded. When the set point was exceeded, the ICV moved to the fully closed position in seven seconds, generating a pressure surge on the mainline. These pressure surges result in system shutdowns, and often exceed the MOP of the pipeline. The company did not provide the CCO with guidance notes for operating the ICV at Cornwall. Since there is no check valve at Lancaster, nor overpressure protection on the mainline, these shutdowns produce a pressure surge that is carried upstream past Lancaster, proceeding toward Como. As a result, the mainline pipe is subjected to high stresses and fatigue loading, which can lead to pipe failure, as in this occurrence.

Supervisory control and data acquisition

A key component of the SCADA system is the monitoring of the internal pressure to ensure that pipeline operations are safe and efficient. While monitoring pressure conditions, the SCADA system responds quickly to situations outside of a pre-established safe operating range, more specifically pressures in conformity with Section 4.14 of CSA Z662 and approved by the NEB.

Records from TNPI's SCADA system showed a maximum pressure that never exceeded 8274 kPa (1200 psi), even though the maximum shutdown pressure was routinely set at Como and Lancaster at 8619 kPa (1250 psi), which exceeds the NEB-approved MOP. A discrepancy between the recorded SCADA data and the observed data was also noted during another unscheduled shutdown, when pressures observed by TNPI technicians and by the CCO were higher than recorded by the SCADA system. Because the SCADA system was gathering system events data at 5-secondintervals, but recording events data for trending purposes only at 15-secondintervals, key events were not being recorded.

The SCADA system was unable to capture the moment when the upper limit of 8274 kPa (1200 psi) was exceeded because of the setting and calibration by the company in combination with the scan times. The reasons for the differences between recorded and observed pressure during normal operations could not be determined. Therefore, while the SCADA system indicates that the operations are conducted in a safe manner and in accordance with the applicable regulations and standards, the pipeline is actually being exposed to higher operating stresses that increase the risks of pipeline rupture. The SCADA system does not consistently identify and record unsafe events that arise periodically on the operating pipeline system. Therefore, the SCADA system is not optimally configured to reliably support well-managed and safe pipeline operations.

Control centre operator training

CCOs received initial training designed to address both normal and emergency operations. However, this training was not updated to ensure that the overall operational efficiency of the MCC and the safety of the system were enhanced. TNPI did not provide guidance notes to the CCO on the Cornwall ICV, and there were no manuals available that clearly outlined corporate policies, practices, and standards for pipeline operations under normal and emergency situations. Consequently, the CCO set some pipeline operating parameters that exceeded the MOP. For instance, resetting Cornwall to a pressure of 8964 kPa (1300 psi), or setting the maximum shutdown pressures for both Como and Lancaster at 8619 kPa (1250 psi), does not conform with Section 4.14 of CSA Z662 and exceeds the NEB-approved MOP, causing overstress and fatigue of the pipeline.

Metallurgical testing and hydraulic analysis

The pipe failed from an instantaneous overstress extension of pre-existing cracks, which had initiated on the outer surface of the pipeline in the region of the mechanical scoring and damage on the outer surface. The pre-cracks were dormant for some time and severely oxidized. The failure mechanism could not be confirmed as either fatigue or environmentally assisted cracking.

Based on the size of the pre-cracks and the pipe tensile strength, the theoretical static pressure required to rupture the damaged pipeline would have been between 11 032 and 15 169 kPa (1600 and 2200 psig).

The low values obtained for the Charpy testing are consistent with the brittle nature of the fracture overstress crack extension observed on the failed pipe, as indicated by the shiny appearance of the fracture and the presence of chevron markings in the fracture area.

The March 2005 hydraulic analysis revealed overpressure at the occurrence site in excess of the NEB-approved MOP when the valve closed at the Cornwall Station. This analysis was performed using the limited SCADA data collected by TNPI, which does not record all events. The report determined that the maximum pressure at the rupture site could reach 9032 kPa (1310 psig) when the downstream valves at Cornwall were closed and all other facility events were ignored at the upstream pipeline. However, the consultant's report did not address the situation with all these upstream facility events taken into consideration, especially the fact that Como was continuing to pump into the section, thus increasing the line pack in the lead-up to the occurrence. The analysis is limited to the steady-state condition on the pipeline. During steady-state conditions, the normal operating pressure is approximately 5930 kPa (860 psig). However, the transition to full-stream deliveries into Cornwall was initiated, and the pressure at the occurrence site was determined by the report to be approximately 8039 kPa (1166 psig). The combination of the pressure transient and an increase in line pack being driven from the Como pump station, which had not reacted to the rise in line pressure by shutting down, resulted in non-steady-state conditions. While the investigation determined that the product was flowing in the pipeline at 13°C, the consultant used a temperature of 10°C to calculate pressures. Further, the report used a homogeneous value for the density of the flowing refined product, when the actual density was non-homogeneous. Each of these assumptions would have had a direct effect on the final results of the hydraulic study, especially under line pack conditions.

Both the TSB Engineering Laboratory study and the company's hydraulic study produced values that are consistent with the range of pressures that would have resulted at the occurrence site following the shutdown at Cornwall and the subsequent transient pressure surge wave. While the hydraulic analysis calculated an estimated value of 9032 kPa (1310 psig) under steady-state conditions and ignoring upstream events, the TSB's purely theoretical analysis, using generalized assumptions, calculated a value of between 11 032 and 15 169 kPa (1600 and 2200 psig).

The lack of check valves at Lancaster and Como increased the maximum pressures at the rupture site and increased the amount of product drain-down off the pipeline from Lancaster and exiting through the occurrence site into the Saint-Emmanuel drainage system.

In-line inspection program

While the ILI program identified anomalies at the occurrence site on several occasions, the methods employed by the company to interpret the anomalies were deficient. Starting in 1980, and relying on the interpretation results from the first ILI run, which occurred in 1980, the company, the ILI vendors, and a consultant consistently concluded that the ILI indication at the occurrence site did not present a threat. After receiving the 1993 and 1998 ILI results, the company did not give sufficient credit to the newer technology used, even though the detection and interpretation capabilities of the ILI tools had improved considerably since 1980. The results of the 1993 and 1998 ILI runs clearly showed that there was only one defect of importance on the pipeline system at the occurrence site. The defect was not excavated. Had the company reviewed route alignment sheets, the company would have likely realized that there was no field bend at this location. Furthermore, had the company performed a standard engineering calculation to determine the length of pipe needed for a field bend, it would have concluded that the feature detected at the occurrence site did not conform to the standard dimensions of a field bend.

In 1980, the ILI tool was not very reliable and the ILI analysts had not yet received formal training in interpreting ILI pipeline integrity testing results. It is therefore possible that the initial conclusion drawn in 1980 was erroneous and that a dent did exist at that time. Starting in 1982, the ILI data analysts received brief training on interpreting ILI pipeline integrity testing results. Due to the technological limitations of the tool at this time, the interpretation methods were very subjective. In addition, the ILI data analysts had very few opportunities to practice interpreting ILI integrity testing results because this type of testing was only performed once every five years.

Results from subsequent testing in 1988 also required subjective interpretation. Again, the ILI data analysts concluded that there was no threat. When the results from a more advanced ILI integrity testing tool, upon which the ILI data analysts had not received any formal training, were presented in 1993, they reviewed the 1988 results. It is likely that the ILI data analysts reviewed the 1988 results using the same subjective methods that they had learned in 1982. They concluded that no threat was indicated by the 1993 ILI integrity testing results, but regarded those results as confirming their previous conclusion that there was no threat.

Starting in 1998, pipeline integrity testing technology required much less subjective interpretation. However, without training on the reliability of this newer technology, it would have been difficult for the ILI data analysts to treat the advanced 1998 integrity testing results as more conclusive than all the previous results. It would have been especially difficult since their conclusions had been confirmed three times and had been held for 18 years without any occurrence. It is therefore likely that the ILI data analysts erroneously treated the results from the older technologies (1980, 1988, and 1993) and from the newer technology (1998) as if they were equally reliable. In so doing, the 1998 results would have been grouped with the previous results and would not have been salient enough for the ILI data analysts to realize that a threat did exist. The repetitive confirmation that integrity testing results from 1980, 1988, and 1993 posed no threat likely led to a confirmation bias^{Footnote 3} that, when coupled with a lack of training on the reliability of new integrity testing methods, resulted in the conclusion that the 1998 integrity testing results also indicated no threat.

Corrosion repair

Puddle welding as a corrosion repair method is not authorized by CSA Z662 and is known to produce microcracks in the parent metal of the pipe. The existence of microcracks can lead to pipeline leaks and failures over time, especially when the pipeline is operated at or near the MOP. Therefore, the identification and monitoring of locations where this method of repair has been performed is paramount to ensure public safety near the pipeline system.

Unauthorized third-party activities

The company maintains detailed records of pipeline damage and coating repairs for known or identified locations. However, the mechanical scoring and damage to the ruptured pipe and the coating repair noted in the immediate vicinity of the ruptured section were not recorded. They were most likely caused by a third party, even though the installation of the repair coating had been professionally installed and is the traditional type of tape coating used by TNPI to repair damaged coating.

The company has a history of unauthorized, third-party activity along the entire length of its pipeline system, which has resulted in damage to and failures of the pipeline. In some cases, the company was unaware until much later that the pipeline had been damaged despite the weekly flight patrol of the pipeline right-of-way.

Emergency response

A key objective of the emergency response plan is the containment of released hydrocarbons. Once on the scene, TNPI response personnel immediately began the process of containment, isolation, and clean-up. However, TNPI personnel did not perform a thorough site inspection and monitoring upon arrival. They did not realize that the spilled product had seeped into the drainage tiles, even though local field personnel were aware that drainage tiles were present in the Saint-Clet district. Moreover, they did not notice spilled oil while they drove over the farmer's bridge that crosses the Saint-Emmanuel drainage ditch where the product had migrated down toward the St. Lawrence River. Consequently, the release of product at the occurrence site was not adequately contained and caused unnecessary environmental damage. This is indicative of a significant shortcoming in the company's emergency response plan. Had there been a thorough site inspection, personnel would have found the drainage tiles. Then the standard procedure would have been to block off the end of the drainage tiles as a precaution against refined products entering into the Saint-Emmanuel drainage system.

From the MCC, the CCO commands the closure of the automated valves to limit the volume of released product. However, this volume can be further reduced by closing manual valves that are located between the automated valves. While TNPI's emergency response was quick, the closure of manual valves by company employees took longer.

In this occurrence, the volume of spilled product was relatively low because of the flat terrain and low gradient of the pipeline. However, other locations on the TNPI system do not have such favourable geometry conditions. Therefore, mutual-aid arrangements, with either another pipeline company or with local individuals, to manually close valves on its pipeline system during an emergency situation would ensure that product spillage and environmental damage would be minimized across the entire TNPI system. The installation of check valves at Lancaster and Como would have prevented fluid flow after the occurrence, and a check valve at Lancaster would have prevented fluid leakage downstream of Lancaster from exiting through the rupture site.

Findings

Findings as to causes and contributing factors

1. The uncommanded operation of the automatic Cornwall take-off valve resulted in an unscheduled shutdown of the station at a time when the line pack was increasing from Como operations, along with a pressure surge of between 11 032 and 15 169 kPa (1600 and 2200 psig), causing the rupture of the pipe.
2. Since there was no check valve at Lancaster, nor flow control and overpressure protection on the mainline, the high-pressure surge was carried upstream past Lancaster, subjecting the mainline to high stresses that led to the failure of the pipe.
3. The pipe failed from an instantaneous overstress extension of pre-existing cracks, which had initiated in the region of mechanical scoring and damage on the outer pipe surface and had been caused previously by unauthorized third-party construction activity.
4. It was consistently concluded that the in-line inspection (ILI) indication at the occurrence site was a field bend and did not present a threat.
5. Even though the reliability of the ILI tool has been significantly enhanced since 1980, the lack of training on new ILI technology and the infrequent opportunities to practice methods of interpreting integrity inspection results likely influenced the ILI data analysts to conclude in 1993 and in 1998 that no threat existed at the occurrence site.
6. Had the company performed a field excavation or reviewed route alignment drawings for the pipeline or performed a standard engineering calculation, it would likely have discovered that the feature detected by the ILI tool was not a field bend.

Findings as to risk

1. The absence of both overpressure protection devices and flow control devices on the mainline results in pipeline operations that are very sensitive to minor pressure fluctuations when attempting to maximize flow rates into Cornwall and might explain the relatively high number of shutdowns at that station.
2. The discrepancy between Supervisory Control and Data Acquisition (SCADA) data and observed data means that the SCADA system indicates that operations are conducted in a safe manner while, in fact, the pipeline is being exposed to higher operating stresses that increase the risks of pipe rupture.
3. The absence of refresher training and reference manuals with corporate policies, practices, and standards means that some pipeline operations parameters do not conform with the Canadian Standards Association standards and exceeded the National Energy Board–approved maximum operating pressure, resulting in overstress and fatigue of the pipeline.
4. Not identifying and monitoring locations where puddle welding has been performed is a risk to public safety near the pipeline system.

Other findings

1. The cause of the sudden take-off valve closure and the subsequent shutdown and high-pressure surge were due to the line pressure exceeding the pressure set point established by the control centre operator at a time when he was attempting to maximize flow rates into Cornwall and close in deliveries into Ottawa.
2. Company personnel did not perform a thorough site inspection and monitoring upon arrival to the site. Consequently, the spillage was not adequately contained and caused unnecessary environmental damage.
3. Mutual-aid arrangements with either another pipeline company or with local individuals to manually close its pipeline system during an emergency situation would ensure that product spillage and environmental damage are minimized across the entire Trans-Northern Pipelines Inc. system.
4. Despite the weekly flight patrol of the pipeline right-of-way, the company has a history of unauthorized, third-party activity that has damaged the pipeline.

Safety action

Safety action taken

The historical in-line inspection (ILI) data on the pipeline segment between Sainte-Rose Junction and the Farran's Point pump station were reviewed by Trans-Northern Pipelines Inc. (TNPI). Four additional features, which were determined to be dents in the pipeline, were identified and excavated. In the vicinity of the occurrence site, two additional sites were also identified as being associated with unauthorized activities. The company excavated one of these two sites.

The company indicated that it had undertaken a review of operating procedures. Subsequently, the mainline pressure at Cornwall was lowered from a set point of 3585 kPa (520 psi) to 3310 kPa (480 psi), prior to recommencing deliveries.

During the spring of 2003, the company performed a sizing survey followed by an ILI inspection. The company advised that eight additional excavations of the pipeline were performed. Upon receipt of the ILI reports, TNPI initiated a comprehensive review of the data and conducted a "joint-by-joint" analysis of the entire pipeline system.

The company has changed its analytical procedure to ensure that pipe deformation indications interpreted as field bends actually correlate to known topography or alignment features. It is planning to re-examine all ILI inspection records to ensure that no critical features or dents similar to the Saint-Cletfeature exist anywhere on the pipeline system. Outside consultants have been retained and have completed an examination of all the historical ILI data from Hamilton, Ontario, to the Farran's Point pump station. Further excavation work is being planned.

When TNPI technicians are on site at Cornwall testing station equipment, and as an assurance against inadvertently shutting down the facility, TNPI has changed its operation to have Lancaster taken off line prior to making Cornwall direct deliveries.

The National Energy Board (NEB) is aware of inconsistencies in the interpretation of the overpressure protection requirements on liquid pipelines, as outlined in Canadian Standards Association (CSA) standard CAN/CSA Z662, Oil and Gas Pipeline Systems (CSA Z662). Overpressure control is intended to protect the pipeline in the event that pressure control fails. However, overpressure control is not intended to limit the pressure to the approved maximum operating pressure (MOP). Rather, overpressure control is designed so that the pipeline pressure never exceeds 10 per cent more than MOP. The NEB is currently reviewing the requirements and will be seeking clarification from the CSA Z662 standard committee and from the industry at large. Once clarification has been obtained from all parties, the NEB will continue to monitor compliance with the standard during its ongoing overview of pipeline operations when conducting audits, inspections, and application reviews.

The NEB is aware of the potential integrity concerns related to improperly applied puddle welds. NEB staff have had ongoing discussions with TNPI regarding the existing puddle welds on the TNPI system. The company indicated that these puddle welds were placed on the pipelines over a period of years starting in the 1960s and that, since CSA Z183 prohibited puddle welds in 1977, it has not placed additional puddle welds on its pipelines.

The company recently provided the NEB with a report, dated 04 January 2005, that discusses puddle welds on the TNPI system, and that NEB staff is currently reviewing. The NEB will continue to work with TNPI to ensure that it has a program in place that monitors the integrity of the existing puddle welds. The NEB will monitor the progress of TNPI's puddle welding program during its ongoing overview of pipeline operations when conducting audits and inspections.

Since the occurrence, the NEB has worked with TNPI to identify areas where safety could be improved as a result of lessons learned from this event. The NEB continues to work with TNPI to ensure that it has an effective emergency response plan, and that it works toward reducing the number of incidents resulting from third-party damage. The NEB has had numerous discussions with TNPI to ensure that it has qualified staff working on integrity evaluations, such as ILI analysis. The NEB concurs that TNPI has performed an analysis of previous ILI tool run data and has re-evaluated and changed its procedure for evaluating defects, such as pipe deformation identified from ILI data.

The NEB has had discussions with TNPI regarding MOP compliance requirements to ensure that TNPI has an appropriate pressure control system on the pipeline system. The NEB will continue to have discussions with TNPI and will seek clarification from industry and the CSA Z662 standard committee regarding the question of overpressure protection requirements on liquid-filled pipelines.

Over the long term, the NEB will continue to review and monitor TNPI's management of its pipeline integrity, which is similar to the NEB ongoing monitoring of the pipeline industry's management of potential integrity issues such as puddle welding, pipe body manufacturing defects, and corrosion. The NEB is committed to taking the lessons learned from this occurrence and applying them to other companies during its ongoing overview of pipeline operations when conducting audits, inspections, and application reviews.

This report concludes the Transportation Safety Board's investigation into this occurrence. Consequently, the Board authorized the release of this report on 10 May 2005.

Appendices

Appendix A - Schematic of the Trans-Northern Pipelines Inc. pipeline system from the Montréal pump station to the Ottawa terminal

Appendix B - Schematic of the Cornwall meter station

Appendix C - Schematic of the Lancaster pump station

Appendix D - Glossary

asl

above sea level

CCO

control centre operator

cm

centimetres

CSA

Canadian Standards Association

CSA Z662

standard CAN/CSA Z662, Oil and Gas Pipeline Systems

DSV

downstream straightening vane

HP

horsepower

ICV

inlet control valve

I.D.

interface detector

ILI

in-line inspection

KP

Kilometre Post

kPa

kilopascals

m

metres

m³

cubic metres

MOP

maximum operating pressure

mm

millimetres

MCC

mainline control centre

MFL

magnetic flux leakage

MLV

mainline block valve

MP

Mile Post

NEB

National Energy Board

PLC

programmable logic controller

PMI

process media isolator

PP/I

pressure transmitter

psi

pounds per square inch

psig

pounds per square inch gauge

RTD

remote temperature detector

SCADA

Supervisory Control and Data Acquisition

TNPI

Trans-Northern Pipelines Inc.

TOV

take-off valve

TSB

Transportation Safety Board of Canada

USV

upstream straightening vane

V

volts

°C

degrees Celsius