TSB Recommendation R14-05

TC will fully implement this recommendation.

Proposed Railway Operating Certificate Regulations were published in the Canada Gazette, Part I, on 15 March 2014. A target date of 01 April 2015 has been set by which these regulations will be brought into force. These proposed regulations provide for the suspension or cancellation of the Railway Operating Certificate for non-compliance with safety requirements or SMS Regulations.

TC also proposed changes to its Railway Safety Management Systems Regulations and published them in the Canada Gazette, Part I, on 05 July 2014. TC has plans to bring them into force in the fall of 2015. The proposed changes improve how railway companies develop, implement, and assess their SMS. These changes include more detailed requirements for each SMS component. For example, the risk assessment component now prescribes specific circumstances used to trigger a risk assessment; lists those elements to be included in each risk assessment; and includes a requirement that remedial action be taken to mitigate identified risks and the results be evaluated for effectiveness. Another SMS component specifies that railways continually evaluate their SMS procedures to ensure that they remain effective and prescribes elements to be included in the evaluation plan. Moreover, risk assessments supporting significant operational changes, at the request of the Minister, must be filed with TC. Beginning in fall 2015, each risk assessment will be subjected to a more rigorous review, and TC will follow up on any resulting risk mitigation measures. This risk information will be incorporated into industry risk profiles and used to adjust priorities for inspections and audits.

TC is in the process of bringing into force additional enforcement capabilities. The Railway Safety Administrative Monetary Penalties Regulations were published in the Canada Gazette, Part II, on 22 October 2014, and come into force 01 April 2015. These regulations introduce fines as an additional enforcement tool to improve railway safety. Under the authority of the Railway Safety Act, TC will be able to fine railways for contraventions of the Act, or regulations or rules made under the Act.

Because the revised regulations are more prescriptive than the original ones, TC will be able to more easily take regulatory action should a railway be in non-compliance with the regulations.

TC has reassessed the required number of inspections and audits. The number of planned inspections will remain relatively stable, while the number of planned audits will increase. Audits will now be completed on at least a 3- to 5-year cycle based on risk. In support of the revised auditing plan, additional auditors with specialized training will be recruited by June 2015. TC will be developing training for auditors on several key factors of the new SMS Regulations, including the revised audit requirements, more stringent follow-up procedures and the use of new enforcement provisions (including administrative monetary penalties), by summer 2015. TC is currently developing a standardized auditing tool (SMS Compliance Assessment Tool) to improve the quality and consistency of the audits, which should be completed by spring 2015. As well, TC has developed, and is implementing, a Management Action Plan to improve its oversight of railway safety, including SMS.

Regional risk information will be shared with relevant internal stakeholders and, beginning in 2016, new safety data from leading indicators will be incorporated into departmental systems to ensure the department stays ahead of trends and relevant operational changes in the industry. A national review process to closely monitor all operators with compliance or on-going safety issues has also been established.

These actions will support TC in conducting more frequent and thorough SMS audits. Audits will now involve more rigorous review and timely follow-up tied to enforceable penalties for non-compliance. However, TC does not intend to increase the scope of audits conducted every 3 to 5 years.

TC has accepted the recommendation and is bringing into force additional regulations and enforcement capabilities. The Railway Operating Certificate Regulations are planned to come into force on 01 April 2015. Revised SMS Regulations that better describe the processes that are to be included in a railway SMS are planned for the fall of 2015. These include explicit expectations that the effectiveness of those processes in advancing safety will be monitored by the railways and continually improved. In addition, the new Railway Safety Administrative Monetary Penalties Regulations, to be in force on 01 April 2015, will enable TC to more easily take enforcement action in the event of non-compliance with the SMS Regulations.

TC has modified its oversight regime to include minimum requirements regarding SMS audit frequency and has developed plans to use data submitted by the railways to adjust its inspection and audit schedule as risks become better understood. TC is also hiring more auditors and strengthening its training programs. These actions may address many of the deficiencies that were noted in its inspection and audit programs.

TC has not committed to auditing every SMS component within a given time period. If a company does not do something to trigger a focussed audit, TC may only conduct an audit once every 3 to 5 years based on risk. In such circumstances, the audit may be of very limited scope which would not cover every component of a railway's SMS.

While significant progress has been made, TC has not yet demonstrated that it has put in place an effective oversight regime that ensures that all railways are audited with sufficient scope and at a frequency to confirm that the required processes are effective and that corrective actions are implemented to improve safety. As a result, deficiencies within a railway's SMS may not be identified and addressed in a timely manner. Therefore, the Board assesses the response to Recommendation R14-05 as being Satisfactory in Part.

Both CN & CP have a comprehensive SMS that has been audited by TC on several occasions. VIA has recently undergone a targeted SMS audit with Transport Canada. Although no non‑compliances were highlighted, opportunities for improvement were identified and corrective action plans are underway. Many shortline railways have been audited as well.

The RAC has mandated that membership requires enhanced attention to improving safety culture within their organizations. It is well recognized that good safety culture ensures optimum performance of SMS. RAC has delivered conferences and workshops to members. A new position was announced in January 2015 for an RAC Chief Safety Culture Officer who will oversee a program for the industry governing related training, assessments, outreach, tools and association standards for members.

Transport Canada  has committed to conducting thorough SMS audits of every SMS component for all railways under its jurisdiction on a three to five year cycle or more frequently, if required. A risk based approach will be used to determine whether a company, which is on a three to five year schedule, should be audited sooner. This will ensure that audits have sufficient scope to ensure that deficiencies within a railway's  SMS are identified and addressed in a timely manner.

On 20 February 2015, new Railway Safety Management System (SMS) Regulations, 2015 were announced. The SMS Regulations require companies to develop and implement a formal framework that integrates safety into their day-to-day operations. The new SMS Regulations were published in the Canada Gazette, Part II on 25 February 2015, and are effective on 01 April 2015.

Also, on 20 February 2015, TC proposed amendments to the Railway Safety Act to strengthen oversight, with a focus on communities, accountability, safety management systems and authorities. The amendments will give the Minister the power to order corrective measures if it is believed that a company is applying its safety management system in a way that compromises railway safety. This new authority will encourage companies to more effectively manage the risks that exist in railway operations.

The Board is encouraged that the RAC has identified the need for enhanced attention to safety culture amongst its membership and the Board anticipates a positive impact from the efforts of the new RAC Chief Safety Culture Officer.

As the railway industry continues to make progress towards improved safety culture, TC must demonstrate that its oversight regime will ensure that all railways are audited with sufficient scope and at a frequency to confirm that the required processes are effective and that corrective actions are being implemented.

The Board is pleased that TC has now made a clear commitment to auditing every component of a railway's SMS on at least a 3 to 5 year cycle. New SMS Regulations come into effect on 01 April 2015, requiring railway companies to implement a formal framework that integrates safety into their day-to-day operations. In addition, proposed amendments to the Railway Safety Act will give the Minister the power to order corrective measures if it is believed that a company is applying its safety management system in a way that compromises railway safety. Therefore, the Board considers the response to the recommendation to have Satisfactory Intent.

Our Class 1 railways have a comprehensive SMS that has been audited by TC on several occasions. New SMS regulations came into effect on 01 April 2015 and all RAC members have recently undergone targeted SMS inspections by Transport Canada. The RAC has mandated that membership requires enhanced attention to improving safety culture within their organizations. It is well recognized that good safety culture ensures optimum performance of SMS. The RAC delivered a number of conferences and workshops to members in 2015. A new position was established in January 2015 for an RAC Chief Safety Culture Officer who oversees the program for the industry governing related training, assessments, outreach, tools and association standards for members.

A three phase transition plan has been established to help companies be aware of and progressively implement the new requirements of the SMS regulations.

Phase I was conducted between April and October 2015. Initial inspections were conducted of all federally regulated railways companies to ensure that:

• Processes had been developed according to the 2015 safety management system (SMS) regulatory requirements for:
  • Designating an executive accountable for the railway company's SMS;
  • Notifying the Minister of, and furnishing upon request, related risk assessment documentation for:
    • Whenever a railway begins to transport dangerous goods or begins to transport dangerous goods different from those it already does,
    • Whenever a proposed change to its railway operations may affect the safety of the public or personnel or the protection of property or the environment
• Railways were continuing to conduct risk assessments using their existing SMS processes while progressively transitioning to the new risk assessment requirements of the 2015 SMS Regulations.

All federally regulated railway companies were visited and at the same time, additional education and awareness regarding the requirements of the 2015 SMS regulations was offered.

Phase II started immediately following the completion of Phase I. Targeted inspections are underway to ensure that all companies are compliant with the development of their SMS according to the regulatory requirements. Phase II is expected to be completed by 31 March 2016.

Phase III will begin as of April 2016 and will involve audits of company Safety Management Systems. These audits will verify if companies are continuing to meet the requirements of the regulations and are striving to achieve the highest level of safety. Training and tools for rail safety inspectors are being developed to support the conduct of these audits. 

The RAC has taken steps to enhance attention to improving safety culture within their member organizations. The RAC delivered a number of conferences and workshops to members in 2015. A new position was established in January 2015 for an RAC Chief Safety Culture Officer who oversees the program for the industry governing related training, assessments, outreach, tools and association standards for members.

TC established a three phase plan to assist in implementing the new requirements of the SMS regulations which came into effect on 01 April 2015. Between April and October 2015, initial inspections were conducted as part of Phase 1 to ensure the regulations were being acted upon, including the establishment of various processes. For Phase 2, targeted inspections are being conducted to ensure that railway companies are developing their SMS according to regulations. Starting in April 2016, Phase 3 will involve auditing railway companies to verify if they are continuing to meet the requirements of the regulations and are striving to achieve the highest level of safety. In addition, training and tools for Rail Safety Inspectors are being developed to support the conduct of these audits.

The Board acknowledges that good progress has been made on TC's three phase transition plan to help companies be aware of and progressively implement the new requirements of the SMS Regulations. However, it is not yet complete. Therefore, the Board considers the response to the recommendation to have Satisfactory Intent.

Transport Canada has put in place a significant number of measures to improve railway safety including improved regulations regarding companies' safety management systems, regulations prescribing fines for contraventions to the Railway Safety Act, and a new liability and compensation regime for federally regulated railways.

The Railway Safety Management System Regulations, 2015 (SMS Regulations) came into force on 01 April 2015. The new SMS Regulations are more comprehensive than their predecessor (e.g., new provisions for consultation and communication with bargaining agents and/or employees, new processes for employee-reporting of contraventions and hazards without reprisal, and strengthened processes requiring companies to identify risks, outline how these risks will be remediated and to evaluate the effectiveness of remedial actions), provide clear expectations and requirements, and have an expanded scope by capturing local railways operating on main and non-main federal track. 

The objective of the SMS Regulations is to foster the creation of a strong safety culture where safety is embedded throughout a company's operations and on a day-to-day basis. The design of the SMS Regulations achieves this by requiring companies to establish a suite of mutually reinforcing and complementary processes that form their safety management systems.

The SMS Regulations are a hybrid of prescriptive, management and performance based requirements, which facilitate more effective implementation by companies and enforceability by Transport Canada:

• The prescriptive component means a company must have a certain number of processes in place, depending on the breadth of their operations on federal track, and within those processes, may be required to also have a procedure, method or plan.
• The management based component means that the SMS Regulations do not tell a company what the process, procedure, method or plan needs to look like, but rather what is to be achieved.
• The performance based component means that a company must regularly assess the effectiveness of its safety management system through mechanisms like an annual monitoring report and an internal audit, and address identified deficiencies.

This mix of the prescriptive, management and performance based components support continuous improvement and is evident in the 12 processes that are central to the SMS Regulations.

Transport Canada's commitment to oversight

TC undertook a 3 phase implementation plan for the new SMS Regulations. The first 2 phases occurred between 01 April 2015 and 31 March 2016. These phases were designed to help all regulated companies become aware of and progressively implement the new requirements. To support companies in transitioning to the new regulations, information sessions were held, a guide was created for industry to assist in understanding the intent and expectations of the SMS Regulations, and Safety System Oversight (SSO) inspectors conducted initial and targeted inspections at each regulated company. Assessment tools and instructions were created and training was provided to SSO inspectors to assist them in conducting nationally consistent inspections. 

The third phase of implementation began as of April 2016 and involves audits to assess the development and implementation of companies' safety management systems. Transport Canada is committed to enhancing oversight of the SMS Regulations by conducting comprehensive audits of railways under its jurisdiction on a 3 to 5 year cycle, or more frequently if required. For larger, more complex railways, a comprehensive audit may not be completed within a single year, but would be conducted over multiple years with several processes audited annually. The completion of a comprehensive audit does not mean that a railway would not be audited for another 3 to 5 years. The selection of railways to audit is informed by risk and a railway can be audited at any time that Transport Canada deems appropriate. 

Audit planning

To evaluate risk and to appropriately monitor railway operations, TC plans and carries out oversight using the Rail Safety risk-based business planning methodology. This methodology is designed to identify issues for which a possible intervention may be required as well as to aid in the prioritization of the department's inspection regime. Oversight activities that the department conducts can take different forms, such as, on-site inspection, audits, document review, and data collection and analysis.

Audits consist of 4 distinct phases: planning, conduct, reporting, and follow-up. All phases are conducted according to internal audit instructions. Audit activities are planned and conducted according to a plan, and all findings linked to the audit scope are outlined in the Audit Report provided to the railway company. Companies are required to submit a Corrective Action Plan subsequent to an audit that identifies corrective actions for each audit finding as well as the person responsible for the corrective action and timing of implementation. SSO inspectors will verify whether corrective actions have been implemented.

If a company fails to address non-compliance found during an audit, TC may take additional enforcement action, which could include an administrative monetary penalty, prosecution, or cancellation of the railway operating certificate. If the safety management system established or implemented by a company has deficiencies that could risk railway safety, a Ministerial Order may be issued under the Railway Safety Act to compel a company to take corrective action.

Subsequent to conducting audits, teams will meet to share their lessons learned which will be taken into account to improve guidance material for SSO inspectors and the conduct of future audits. A total of 26 safety management system audits were conducted in 2016-2017. A similar number of audits are planned for 2017-2018.

The following is a breakdown of the 26 audits conducted in 2016-2017:

Strengthened auditing capacity within Transport Canada

In support of the oversight of the new SMS Regulations, Transport Canada has taken measures to improve its capacity to conduct audits through increased human resources and through the creation of tools and guidance for Inspectors.

Additional SSO Inspectors were hired in the 5 Transport Canada regional offices. Staffing levels in headquarters also increased with the addition of expertise in auditing and in risk analysis to improve best-practices in conducting audits. Additional guidance has been developed to better ensure that the conduct of audits and other oversight activities are nationally consistent by informing inspectors on planning, conducting, reporting and following up on audits, and in determining whether a railway is compliant to regulatory requirements. New material also provides in-depth guidance on the interpretation and application of the SMS Regulations and conveys the linkages between processes, procedures, plans and methods, and linkages to the Railway Safety Act and other regulatory instruments.

The SMS Regulations improve the clarity and quality of safety management systems, which supports better implementation by the rail industry and more effective oversight by Transport Canada. The enforcement provisions in the Railway Safety Act address non-compliances and safety concerns, and the enhanced capacity to conduct audits due to additional personnel, expertise, and new tools will better ensure that companies' safety management systems are effective.

The RAC has mandated that their membership requires enhanced attention to improving safety culture within their member organizations. It is well recognised that good safety culture ensures optimum performance of SMS. The RAC and industry continue to drive a safety culture committee overseeing assessments, outreach, tools, training, and standards for members.

Railway companies have elaborate and comprehensive safety management systems (SMS) that are audited regularly by Transport Canada (TC).

TC undertook a 3-phase implementation plan for the new SMS Regulations. The first 2 phases occurred from 01 April 2015 to 31 March 2016. The 3^rd phase of implementation began in April 2016 and involves audits to assess the development and implementation of companies' safety management systems.

TC remains committed to enhancing oversight of the SMS Regulations by conducting comprehensive audits of railways under its jurisdiction on a 3 to 5 year cycle, or more frequently if required. In 2016–2017, 26 safety management system audits were conducted. These companies were required to submit a Corrective Action Plan subsequent to the audit to identify corrective actions for each audit finding. Safety System Oversight inspectors then verified whether the corrective actions were implemented. 

In support of the oversight of the new SMS Regulations, Transport Canada implemented measures to improve its capacity to conduct audits through increased human resources. Staffing levels at TC headquarters were also increased with expertise in auditing and in risk analysis to improve best practices in conducting audits. New material was developed for inspectors to provide in-depth guidance on the interpretation and application of the SMS Regulations and to convey the linkages between processes, procedures, plans and methods, and linkages to the Railway Safety Act and other regulatory instruments. The enhanced capacity to conduct audits due to additional personnel, expertise, and new tools will better ensure that companies' safety management systems are effective.

Continued progress has been made on TC's 3-phase plan to help companies be aware of and progressively implement the new requirements of the SMS Regulations. The Board looks forward to the completion of the first cycle of TC's SMS audits, which is expected to enhance its on-going oversight of the SMS Regulations. The Board considers the response to the recommendation to have Satisfactory Intent.

The RAC and industry continue to drive attention to improving safety culture. To this end, we continue to maintain a safety culture committee and offer industry support. Working with industry members on overseeing assessments, outreach, tools, training, and standards.

Railway companies have elaborate and comprehensive SMS that are audited regularly by TC. Throughout 2017, the RAC led an industry SMS working group to share best practices and offer support on application. In October 2017, TC and RAC hosted an SMS workshop with industry and other stakeholders. The purpose of the workshop was to share ideas and best practices.

One of the results of the working group and workshop is that industry and TC will maintain the commitment to work collaboratively to improve SMS. The regular TC audits and the successful company application of the SMS responds to all aspects of Recommendation R14-05.

Transport Canada has put in place a significant number of measures to improve railway safety including more stringent requirements for the securement of unattended railway equipment, improved regulations regarding companies' safety management systems, regulations prescribing fines for contraventions to the Railway Safety Act, improved tank car standards, emergency response plans, and a new liability and compensation regime for federally regulated railways. A detailed list of measures can be found at http://www.tc.gc.ca/eng/mediaroom/infosheets-menu-7564.html.

The Transportation Safety Board's recommendation R14-05 specifically addressed the auditing of companies' safety management systems and recommended that “Transport Canada audit the safety management systems of railways in sufficient depth and frequency to confirm that the required processes are effective and the corrective actions are implemented to improve safety”.

The Railway Safety Management System Regulations first came into force in 2001, and established within the rail industry a more comprehensive way of managing safety. These regulations supplemented the existing legislative framework, and required railway companies to take responsibility for managing the safety of their operations, identifying hazards, assessing and mitigating risks, while building a safety consciousness into their day-to-day operations.

In 2008, the conclusions of both the Railway Safety Act Review and the Standing Committee on Transport, Infrastructure and Communities study on rail safety included several recommendations for improving the implementation of safety management systems in the rail industry. Those recommendations resulted in amendments to the Railway Safety Act which came into force on May 1, 2013. These amendments, as well as Transport Canada's more than 10 years' worth of lessons learned from regulatory oversight of railway safety management systems, resulted in the need to revise and modernize these early regulations.

The Railway Safety Management System Regulation, 2015 (SMS Regulations) came into force on April 1, 2015. The new SMS Regulations are more comprehensive than their predecessor (e.g., new provisions for consultations and communications with bargaining agents and/or employees, new processes for employee-reporting of contraventions and hazards without reprisal, and strengthened processes requiring companies to identify risks, outline how these risks will be remediated and to evaluate the effectiveness of remedial actions), provide clear expectations and requirements, and have an expanded scope by capturing local railways operating on main and non-main federal track. 

The objective of the SMS Regulations is to foster the creation of a strong safety culture where safety is embedded throughout a company's operations and on a day-to-day basis. The design of the SMS Regulations achieves this by requiring companies to establish a suite of mutually reinforcing and complementary processes that form their safety management systems. 

The SMS Regulations are a hybrid of prescriptive, management and performance based requirements, which facilitate more effective implementation by companies and enforceability by Transport Canada.

The prescriptive component means a company must have a certain number of processes in place, depending on the breadth of their operations on federal track, and within those processes, may be required to also have a procedure, method or plan. The management based component means that the SMS Regulations do not tell a company what the process, procedure, method or plan needs to look like, but rather what is to be achieved. The performance based component means that a company must regularly assess the effectiveness of its safety management system through mechanisms like an annual monitoring report and an internal audit, and address identified deficiencies.

This mix of the prescriptive, management and performance based components support continuous improvement and is evident in the following twelve process that are central to the SMS Regulations:

1. Process for accountability^* – Designating an executive who is responsible for the operations and activities of the company to be accountable for the extent to which the regulatory requirements for SMS are met.
2. Process with respect to a safety policy – Creating a policy that reflects the company's commitment to promoting railway safety.
3. Process for ensuring compliance with regulations, rules and other instruments – Providing a framework for identifying legal obligations, monitoring changes to them, and verifying compliance with them.
4. Process for managing railway occurrences^** – Establishing procedures for reporting and reviewing railway occurrences.
5. Process for identifying safety concerns – Conducting analyses to identify safety concerns, including trends or repetitive situations.
6. Risk assessment process – Conducting risk assessments to identify risks and required remedial action.
7. Process for implementing and evaluating remedial action – Requiring that the remedial action chosen to treat an identified risk be implemented and that the effectiveness of the action be evaluated.
8. Process for establishing targets and developing initiatives^* – Establishing targets and developing related initiatives to achieve those targets each calendar year.
9. Process for reporting contraventions and safety hazards^** – Providing a framework for the reporting of contraventions and safety hazards by employees to the railway company without fear of reprisal for having reported.
10. Process for managing knowledge^** – Ensuring employees and non-employees have the requisite knowledge, and that employees have the required skills and qualifications, to carry out their duties or activities safely.
11. Process with respect to scheduling^** – Applying the principles of fatigue science when scheduling employees who work certain schedules.
12. Process for continual improvement of the SMS^* – Conducting internal monitoring and audit activities to monitor and evaluate the implementation and effectiveness of the SMS.

• ^* Applicable only to federal railway companies and local railway companies operating on federal main track.
• ^** Applicable only to federal railway companies.

Transport Canada's Commitment to Oversight

Transport Canada undertook a three-phase implementation plan for the new SMS Regulations. The first two phases occurred between 1 April 2015, and 31 March 2016, and were designed to help all regulated companies become aware of and progressively implement the new requirements. To support companies in transitioning to the new regulations, information sessions were held, a guide was created for industry to assist in understanding the intent and expectations of the SMS Regulations, and Safety System Oversight (SSO) inspectors conducted initial and targeted inspections at each regulated company. Assessment tools and instructions were created and training was provided to SSO inspectors to assist them in conducting nationally consistent inspections. 

The third phase of implementation, which began as of April 2016 is ongoing and now part of normal oversight of SMS, and involves audits to assess the development and implementation companies' safety management systems. Transport Canada has committed to enhancing oversight of the SMS Regulations by conducting comprehensive audits of railways under its jurisdiction on a three to five year cycle or more frequently if required. For larger, more complex railways, a comprehensive audit may not be completed within a single year but conducted over multiple years with several processes audited annually. The completion of a comprehensive audit does not mean that a railway would not be audited for another three to five years. The selection of railways to audit is informed by risk and a railway can be audited at any time that Transport Canada deems appropriate. 

Audit Planning

To evaluate risk and to appropriately monitor railway operations, TC plans and carries out oversight using the Rail Safety risk-based business planning methodology. This methodology is designed to identify issues for which a possible intervention may be required as well as to aid in the prioritization of the department's inspection regime. Oversight activities the department conduct take different forms, such as, on-site inspection, audits, document review, and data collection and analysis.

Audits consist of four distinct phases: planning, conducting, reporting, and follow-up. All phases are conducted according to internal audit instructions. Audit activities are planned and conducted according to a plan, and all findings linked to the audit scope are outlined in the Audit Report provided to the railway company. Companies are required to submit a Correction Action Plan subsequent to an audit that identifies corrective actions for each audit finding as well as the person responsible for the corrective action and timing of implementation. SSO inspectors will verify whether corrective actions have been implemented. 

If a company fails to address non-compliance found during an audit, TC may take additional enforcement action, which could include an administrative monetary penalty, prosecution, or cancellation of the railway operating certificate. If the safety management system established or implemented by a company has deficiencies that could risk railway safety, a Ministerial Order may be issued under the Railway Safety Act to compel a company to take corrective action.

Subsequent to conducting audits, teams will meet to share their lessons learned which will be taken into account to improve guidance material SSO inspectors and the conduct of future audits. A total of 26 safety management system audits were conducted in 2016-17. A similar number of audits are planned for 2017-18.

Strengthened Auditing Capacity within Transport Canada

In support of the oversight of the new SMS Regulations, Transport Canada has taken measures to improve its capacity to conduct audits through increased human resources and through the creation of tools and guidance for Inspectors. 

Additional SSO Inspectors were hired in the five Transport Canada regional offices. Staffing levels in headquarters also increased with the addition of expertise in auditing and in risk analysis to improve best-practices in conducting audits. Additional guidance has been developed to better ensure that the conduct of audits and other oversight activities are nationally consistent by informing inspectors on planning, conducting, reporting and following up on audits, and in determining whether a railway is compliant to regulatory requirements.  New material also provides in-depth guidance on the interpretation and application of the SMS Regulations and conveys the linkages between processes, procedures, plans and methods, and linkages to the Railway Safety Act and other regulatory instruments. The SMS Regulations improve the clarity and quality of safety management systems, which supports better implementation by the rail industry and more effective oversight by Transport Canada. The enforcement provisions in the Railway Safety Act address non-compliances and safety concerns, and the enhanced capacity to conduct audits due to additional personnel, expertise, and new tools will better ensure that companies' safety management systems are effective. Further details on training and guidance include:

• Industry guide: March 31, 2015 first version sent to industry. June 14, 2016 revised version sent to industry.
• Conducted a workshop with industry in October 2017 to discuss lessons learned from both TC and industry perspectives with the first series of audits conducted under the new regulations.
• Working groups continue to meet intensively to develop inspector guidance and tools.
• These products were rolled out to inspectors via two all SMS inspector meetings held February 2016, November 2016 and October 2017 and via numerous Webex session.
• Pilot training courses were delivered in 2017. Based on feedback, adjustments will be made in 2018.

Building on the success of the workshop conducted with industry in 2017, Transport Canada will organize a second workshop to take place in 2018 to continue the dialogue on lessons learned from both the government and industry perspectives.

* The total number of audits for 16/17 has not changed; however, the classification as comprehensive/targeted has been updated from what was previously reported. The change was made when it was determined there was inconsistency in how the definition of targeted and comprehensive audits was being interpreted.

** Comprehensive Audits are defined as an overall assessment of a railway's development and implementation of some or all of a system to manage safety in day-to-day operations, and whether the system complies with the requirements of the SMS Regulations.

*** Targeted audits are defined as an opportunity to conduct an in-depth examination of a safety issue that links to one or more of the railway's safety management system processes and in most instances, has been identified by a functional group (e.g., Operations, Equipment or Engineering)

Transport Canada has committed to enhancing oversight of industry compliance with the SMS Regulations by conducting comprehensive audits of railways under its jurisdiction on a 3- to 5-year cycle, or more frequently if required. The selection of railways to audit is informed by risk. A railway can be audited at any time that Transport Canada deems appropriate.

For the new SMS Regulations, Transport Canada undertook a 3-phase implementation plan. The first 2 phases, completed between April 2015 and March 2016, ensured that all regulated companies were aware of and were progressively implementing the new requirements. Safety System Oversight (SSO) inspectors conducted some initial and targeted inspections at each regulated company. Assessment tools and instructions were created and training was provided to SSO inspectors to assist them in conducting nationally consistent inspections. 

The 3^rd phase, which was started in April 2016, is ongoing and now part of normal oversight of SMS. This phase involves audits to assess the development and implementation of companies' safety management systems. 

In addition to the previous training and guidance provided to industry, the following activities were completed in 2017:

• In October 2017, a workshop with industry was conducted to discuss lessons learned with the first series of audits conducted under the new regulations.
• Working groups met to develop inspector guidance and tools.
• Additional inspector guidance and tools were rolled out in October 2017.
• Other pilot training courses for inspectors were delivered in 2017. Based on feedback for this training, some of the courses were adjusted as required.

In 2018, TC will organize a 2nd workshop to continue the dialogue on lessons learned relating to SMS audits from both the government and industry perspectives.

For 2017–18, TC completed 27 SMS audits, consisting of 25 comprehensive audits and 2 targeted audits. All Class 1 railways have received at least one comprehensive audit. About half of the non-Class 1 railways have received at least one comprehensive audit. Audit activities were conducted according to a plan, and all findings linked to the audit scope were outlined in the audit report provided to the railway. The railways were required to submit a Corrective Action Plan, identifying the corrective actions, the person responsible, and the timing of implementation. TC inspectors have been following up with the railways to verify that the required corrective actions are being implemented.

For 2018–19, an SMS audit plan has been developed, consisting of 33 comprehensive audits and 4 targeted audits. Upon completion of these audits, more than 80% of the non-Class 1 railways will have received at least one comprehensive audit.

The RAC has continued to maintain a safety culture committee and to offer industry support relating to SMS. The RAC continues to work with industry members on overseeing assessments, outreach, tools, training, and standards. Throughout 2017, the RAC led an industry SMS working group to share best practices. One of the results of the working group (and the October 2017 workshop) is that industry and TC remain committed to work collaboratively to improve SMS.

The Board is encouraged with the continued progress to help companies with the implementation of the new requirements of the SMS Regulations. The first cycle of TC's SMS audits is being completed, and TC has met with industry to share lessons learned relating to these initial audits. The Board looks forward to the continued sharing of ideas and best practices, as well as the completion of at least one comprehensive audit for each federally regulated railway. Upon completion of the first cycle of SMS audits, the Board looks forward to a summary briefing on what TC has found and what corrective actions were required.

The Board considers the response to the recommendation to show Satisfactory Intent.

Railway companies have elaborate and comprehensive SMS that are audited regularly by TC. These audits are generally confirming that railway companies have the required processes, that they are effective, and that corrective actions are implemented to improve safety. The RAC continues to lead an industry SMS working group designed to share best practices and offer support on application. In October 2018, TC and RAC hosted its 2nd SMS workshop with industry and other stakeholders. The primary purpose of the workshop was to share ideas and best practices but it also included a discussion on how to address the Railway Safety Act Review Committee's recommendations dealing with SMS.

Industry and TC have agreed to maintain the commitment to work collaboratively to improve SMS.

In 2018, progress was made in the following areas relating to training and guidance:

• In April 2018, TC launched the pilot of a new audit management software intended to facilitate and improve the consistency of audits. This pilotis currently running in Head Office and Ontario Region.
• In April 2018, TC completed a “Way Forward” document that defines targeted audits to assist in assessing the effectiveness of railways' SMS going forward.
• In October 2018, TC conducted a second workshop with industry to continue thedialogue on lessons learned from both the government and industryperspectives and to identify areas that require furtherclarification.  As part of the workshop, TC evaluated lessons learned assessments and other suggestions for improving the tools, templates,guidance, and training materials that support the program. 
• In October 2018, new tools and processes were rolled out to inspectors via face to face meetings and a number of Webex sessions.
• In November 2018, the Revised National Internal Oversight Guidance forconducting SMS audits was rolled out.
• Safety Systems Oversight (SSO) National Engagement Audit Committee (SNEAC)established to share best practices, promote national audit consistency,develop and enhance audit tools and templates, and participate in thedevelopment of a national plan with key performance indicators, on anongoing basis.
• In November 2018, TC developed a new standardized approach to aligning auditfindings with the regulatory regime, and a new process for determiningwhen risk assessments will be requested from railway companies in responseto notifications of changes in railway operations.
• The SMS Regulatory and SMS Auditor courses were revised in 2018 based onfeedback from inspectors. The revised courses are scheduled forroll-out in early 2019.

* Comprehensive Audits are defined as an overall assessment of a railway's development and implementation of some or all of a system to manage safety in day-to-day operations, and whether the system complies with the requirements of the SMS Regulations.

As of February 2019, all Class 1 railways have received at least one comprehensive audit (5).

By the end of fiscal year 2018-19, 86% of federally-regulated non Class 1 railways (58 out of 67 companies) would have had at least one comprehensive audit.

It should be noted that a comprehensive audit may include some or all processes as identified in Section 5 of the Safety Management System Regulations, 2015. For the larger railways, Transport Canada risk-ranked the processes and conducts an audit of 2 to 4 processes each year with a plan to complete all processes by March 31, 2020.

Further, sections 30 and 58 of the Safety Management System Regulations, 2015 require that railway companies operating on main track and local railways operating on main track complete an internal audit every 3 years respectively. These audit reports were requested by Transport Canada and were reviewed by auditors to ensure that:

• they met the requirements of the regulation; and
• that any findings were addressed through a corrective action plan.

The findings from these reports were consistent with findings identified in the comprehensive audits completed by Transport Canada. Railway companies often had processes in place but had not fully implemented the processes or perhaps had not effectively communicated them to staff. Findings from company audits were considered in the environmental scan where TC had audits scheduled.

The quality of the audit reports provided some evidence that railway companies have a better understanding of the SMS Regulations. There continue to be areas where industry requires further direction. As such, Transport Canada is currently revisiting the SMS Industry Guidance document in order to provide additional clarification. This will be done in consultation with the Railway Association of Canada.

Overall, Transport Canada is identifying fewer non-compliances^{Footnote 1} and more deficiencies^{Footnote 2} in the audits completed. This can be interpreted to mean that railways are becoming more familiar with the requirements of the Regulations and have the required processes and procedures in place; they continue to work at fully implementing these requirements.

For 2019-20, Transport Canada will put more focus on determining the effectiveness of a railway company's SMS. This will be done through targeted audits. TC completed a “Way Forward” document that defines a targeted audit and is currently looking at developing the audit process for a targeted audit. Targeted audits will be done in conjunction with subject matter experts.

The SMS workshop planned for 2019-20 will look more to industry to present their progress with implementing the SMS Regulations and what impact they have seen on their operations.

TC conducts comprehensive audits of railways under its jurisdiction on a 3- to 5-year cycle, or more frequently if required. Comprehensive SMS audits, which are conducted according to a plan, are now part of TC's normal oversight. 

In 2018, a number of training and/or industry guidance initiatives were completed, including:

• In April 2018, TC launched the pilot of a new audit management softwareto help improve the consistency of audits. This pilot is running at TCHead Office and the Ontario Region.
• In April 2018, TC completed a “Way Forward” document that definestargeted audits to help assess the effectiveness of railways' SMS.
• In October 2018, TC conducted a workshop with industry to continue thedialogue on lessons learned from both the government and industryperspectives. Areas that required further clarification were also identified.During the workshop, TC evaluated lessons learned assessments and othersuggestions for improving the tools, templates, guidance, and training materials.
• In October 2018, new tools and processes were rolled out to the Safety Systems Oversight (SSO) inspectors.
• In November 2018, the Revised National Internal Oversight Guidance for conducting SMS audits was rolled out.
• The Safety Systems Oversight National Engagement Audit Committee (SNEAC) was establishedto share best practices, to promote national audit consistency, to develop and enhance audit tools and templates, and to participate in the development of a national plan with key performance indicators.
• In November 2018, TC developed a new standardized approach to align audit findings with the regulatory regime, and a new process to determinewhen risk assessments will be requested from railway companies in response to notifications of changes in railway operations.
• The SMS Regulatory and SMS Auditor courses were revised in 2018 based on feedback from inspectors. The revised courses are scheduled forroll-out in early 2019.

As of February 2019, each Class 1 railway had received at least one comprehensive audit.

By the end of March 2019, 58 of the 67 federally-regulated non-Class 1 railways (86%) would have had at least one comprehensive audit. In addition, railway companies operating on main track must complete an internal audit every 3 years. TC auditors reviewed these audit reports to ensure that they met the requirements of the regulation, and that any findings were being addressed.

For fiscal year 2019-20, TC will begin the work of determining the effectiveness of a railway company's SMS. This will be done through targeted audits. Targeted audits will be performed in conjunction with subject matter experts. In addition, an SMS workshop is planned for 2019-20 that will focus on the industry's progress with implementing the SMS regulation and the impact that railways have seen on their operations.

The Board is encouraged with the continued progress to help companies with the implementation of the requirements of the SMS Regulations. Upon completion of the first cycle of SMS audits, the Board looks forward to receiving a briefing on what non-compliances and deficiencies TC found and what corrective actions were taken by the railways. The Board considers the response to Recommendation R14-05 as having Satisfactory Intent.

We would like to reiterate that railway companies have elaborate and comprehensive SMS that are audited regularly by TC. These audits are generally confirming that railway companies have the required processes, that they are effective, and that corrective actions are implemented to improve safety. In October 2018, TC and RAC hosted its 2nd SMS workshop with industry and other stakeholders that shared ideas and best practices and included a discussion on how to address the Railway Safety Act Review Panel’s recommendations relative to SMS.

Industry and TC have agreed to maintain the commitment to work collaboratively to improve SMS.

In 2019, progress was made in the following areas relating to training and guidance:

• The pilot of new audit management software intended to facilitate and improve consistency of audits was completed in Headquarters. Training on this software is planned for 2020 so that all regions can begin using it in the 2020-2021 fiscal year.
• The Safety Systems Oversight (SSO) National Engagement Audit Committee’s (SNEAC) name was changed to the Safety Management Advisory Review Team (SMART) and began reviewing and updating auditing templates based on lessons learned from using them for a number of years.

March 2019

• Introduction to auditing course was delivered and will be delivered again in the fourth quarter of the 2019-2020 fiscal year so that all employees are trained on the audit manual, and auditing principles and process.
• Railways selected based on risk were asked to file with TC their annual report of their SMS, targets and initiatives, and index. These audit reports were reviewed and feedback was provided to railways.

September 2019

• The Guidance to Audit the Process with Respect to Scheduling was released to all auditors and an all staff meeting was held to ensure it was well understood.
• An all staff meeting was held to improve communication with employees and improve consistency across regions. Regular all staff meetings are being planned to provide updates.
• The Railway Risk Profiles were mapped to the SMS Regulations and analysis is currently underway (as of December 2019) to ensure they have the right balance of questions from each process and no gaps exist. Feedback on the risk profiles was also compiled and a plan on improving them developed.

October 2019

• The revised SMS Regulations course was delivered with a clear link to the internal oversight guide.

December 2019

• A new five year plan covering the years 2020 – 2025 was developed to ensure all railways continue to be audited at least every 3 – 5 years or more often if their risk level warrants it. The new audit cycle will move beyond determining that railways have developed and implemented their SMS and include the effectiveness of the SMS. To this end, internal guidance on effectiveness will be developed. The new five year plan also includes continuing to host Industry Workshops, reaching out and conducting Education and Awareness with all railways unable to participate in the workshops so all railways receive the information provided, and conducting targeted audits based on issues previously identified.
• Railways were surveyed to understand their thoughts on the previous two Industry Workshops. The responses indicated that these previous workshops were beneficial. As a result, a third Industry Workshop is being developed and will focus on the SMS processes that TC’s audits have determined are more problematic and processes that the railways have indicated they are having difficulty with.

* Comprehensive Audits are defined as an overall assessment of a railway’s development and implementation of some or all of a system to manage safety in day-to-day operations, and whether the system complies with the requirements of the SMS Regulations.

Most of the audits planned for this fiscal year are underway as of December 2019. This means they are in the planning, conduct, or reporting phase of the audit process.

As of December 2019, all Class 1 railways have received at least one comprehensive audit since the Regulations came into force on April 1, 2015.

By the end of fiscal year 2019-20, 100% of federally-regulated companies will have had at least one comprehensive audit and all required SMS process [es] will have been audited since the Regulations came into force in April 2015.

In 2019, progress was made in the following areas:

• New audit management software was successfully piloted at Transport Canada (TC) Headquarters. Training on this software is planned for 2020. TC regions will begin using the software in fiscal year 2020–2021.
• The Safety Management Advisory Review Team (SMART) has begun reviewing and updating auditing templates based on lessons learned from recent years.
• TC employees are being trained on the audit manual, and auditing principles and processes. This auditing course will be delivered again in the fourth quarter of fiscal year 2019–2020.
• Selected railways were requested to file with TC their annual report of SMS, targets and initiatives, and index. These audit reports were reviewed and feedback was provided to railways.
• The Guidance to Audit the Process with Respect to Scheduling was circulated to all TC auditors. An all staff meeting was held to ensure this guidance was well understood. Other regular all staff meetings were being planned.
• The Railway Risk Profiles were mapped to the SMS Regulations. Analysis was started to ensure that there is the right balance of questions from each process and no gaps exist.
• The revised SMS Regulations course was delivered with a clear link to the internal oversight guide.
• A new five year plan (2020–2025) was developed, ensuring that all railways will be audited at least every 3–5 years, or more often if their risk level warrants it.
• The new five year plan includes hosting additional Industry Workshops, conducting education and awareness with railways who are unable to participate in the workshops, and conducting targeted audits based on issues previously identified.

According to TC, by the end of fiscal year 2019–20, all federally-regulated companies (i.e., both Class 1 and non-Class 1 railways) will have had at least one comprehensive audit in the current 5-year cycle. Upon completion of this first cycle of SMS audits, the Board looks forward to receiving a briefing on what non-compliances and deficiencies TC found and what corrective actions were taken by the railways. The Board considers the response to Recommendation R14-05 to show Satisfactory Intent.

Transport Canada (TC) have been conducting regular audits of railway companies’ SMS. These audits are generally confirming that railway companies have the required processes, that they are effective, and that corrective actions are implemented to improve safety.

Railway companies are committed to continue to work closely, and collaborate, with TC on SMS matters and its improvement.

Overview: Transport Canada (TC) continues to make significant progress with the implementation of its SMS Regulations for railways subject to the provisions of the Railway Safety Act. In 2020, Rail Safety completed the audit of all railway companies’ safety management system processes to ensure that they are compliant with the regulation and that they have been implemented. With that work complete, attention has now shifted to auditing these processes to examine their effectiveness, which is consistent with previous commitments.

Regulatory surveillance: As of December 2020, TC has completed its planned comprehensive audits of all federally regulated companies that was started in the 2016/17 fiscal year and finished one year ahead of schedule, representing a total of 126 audits. TC requested corrective action plans as a result of these audits where necessary and continues to follow-up to ensure all railways have taken corrective action to address the findings.

Examples of the corrective actions that have been undertaken include:

• updating the list of instruments to include all rules and regulations applicable to a company's operations;
• implementation of a process to verify compliance with the rules/regulations;
• identification of a process to identify trends in and conduct analysis on hazards/contraventions;
• creation of a written procedure regarding risk assessments; and
• communication regarding the incorporation of the principles of fatigue science when creating employees schedules.

Safety management: During 2020, TC made significant progress on various initiatives to reinforce its oversight of safety management systems, especially by better leveraging data and technology:

• To ensure a more consistent audit methodology and documentation process around audits, TC's regions are now using TeamMate’s Electronic Working Papers—an audit management software—for all audits being conducted;
• Audit templates have been reviewed and updated by the Safety Management Advisory Review Team (SMART), which is made up of managers, inspectors and auditors, and are now in use for all audits being conducted;
• An SMS-specific risk-based business process was created and launched to identify potential targeted audits for systemic issues not easily addressed through the normal inspection processes;
• An ad-hoc evaluation of the results of audits conducted since 2016 under the new SMS Regulations was completed to identify areas where the industry may require more guidance and areas where greater consistency is required. In addition, a process for conducting similar evaluations based on audit findings, as well as audit lessons learned has been developed so that the program continues to improve. In December 2020, TC engaged the services of a consultant to develop an audit follow up procedure to provide consistent follow up on audit findings and railway corrective action plans.

The targeted audit framework was approved in June 2020 and has been implemented. Targeted audits to measure the effectiveness of SMS processes are now underway for shortline railway's track infrastrucure programs, specifically how they are managing knowledge, ensuring compliance and identifying safety concerns.

Due to COVID-19, the audit course scheduled for the fourth quarter of 2019-2020 fiscal year was offered virtually in December 2020.

In 2020, Transport Canada (TC) completed audits of all federally regulated railway companies’ safety management systems (SMS) to ensure compliance with the regulations and to ensure that they have been implemented. Corrective action plans, as a result of these audits, have been requested where necessary and follow-up continues to ensure that all railways have taken corrective action to address the findings.

TC has also made progress in the following areas:

• Adopted and implemented a consistent audit methodology and documentation process for audits in the regions;
• Introduced a new audit management software for improving the tabulation and tracking of results in the regions;
• Reviewed and updated audit templates under the stewardship of the Safety Management Advisory Review Team (SMART), which are now in use for all audits;
• Developed an SMS-specific risk-based business process which helps to identify systemic issues that are not easily addressed by regular inspections, and supports the execution of potential targeted audits;
• Completed an ad hoc review of the results of audits conducted since 2016 under the Railway Safety Management System Regulations (2015) to identify areas where the industry may require more guidance;
• Developed a new five-year plan for 2020–2025 that audits all railways on the effectiveness of their SMS at least every 3 to 5 years. This plan includes a continued commitment to host industry workshops, combined with awareness outreach to those railways unable to participate in the workshops.
• Targeted audits for measuring the effectiveness of SMS processes are currently underway for shortline railways’ track infrastructure programs, and specifically how they are managing knowledge, ensuring compliance and identifying safety concerns.

TC has confirmed that, with the initial comprehensive audit of all federally regulated railways completed, a targeted audit framework for measuring the effectiveness of SMS processes has been developed and it is in the early stages of implementation.

The Board is encouraged by TC’s progress and looks forward to receiving information on the findings related to the effectiveness of federal railways’ SMS. Therefore, the Board considers the response to Recommendation R14‑05 to show Satisfactory Intent.

Transport Canada (TC) have been conducting regular audits of railway companies’ SMS. These audits are generally confirming that railway companies have the required processes, that they are effective, and that corrective actions are implemented to improve safety.
Railway companies are committed to continue to work closely, and collaborate, with TC on SMS matters and its improvement.

Transport Canada has implemented robust procedures for auditing the railway companies’ safety management systems, with respect to both measuring effectiveness and strengthening oversight of the Safety Management System Regulations.

In broad terms, Transport Canada defines the effectiveness of a company’s SMS as “the degree to which the SMS processes (including procedures, plans and methods), which make up the SMS, produce the desired result”. For example, the process for ensuring compliance with regulations, rules and other instruments, would be effective if it can be determined that employees know the regulatory requirements and how to comply, the company knows when instances of non-compliance are occurring and, when non-compliance occurs, the company looks at the root cause and takes appropriate measures to improve.  This approach to evaluating effectiveness is consistent with the ISO 9001:2008 definition which is “a verification of the effectiveness of the management system operated by an organization through the examination of materials, processes, finished product, methods of test, records, systems, and services.”

Measuring Effectiveness

In June 2020, the department completed a framework for measuring effectiveness of safety management systems. Building on this progress, the department developed a methodology to support this framework, and in September 2020, successfully piloted this methodology. Drawing on the results of the pilot, Transport Canada has:

• Launched safety management system effectiveness audits (September 2021). The first audits will be completed by March 2022. As per the framework, the findings from these audits will be shared with inspectors from all functions to inform current and future oversight activities and will inform improvements to industry guidance materials.
• Completed the development of tools to measure baselines and targets for measuring the effectiveness of safety management systems (September 2021).
• Updated the rail safety performance indicator profile, which sets out improve performance measures to show the effectiveness of the oversight program (October 2021).
• As a key component of TC’s response to the recent OAG follow-up audit, an evaluation of the effectiveness of Rail Safety’s oversight, including SMS, is being undertaken by the Transport Canada’s Internal Audit and Evaluation Branch. This report is scheduled for completion in September 2022. The final report will be shared with TSB and will be published on the TC website.
• Establish and document guidelines to help auditors measure effectiveness of various SMS processes which is scheduled to be completed in December 2021.
• Established guidelines for timely follow-up to safety management system audits (November 2021).

Oversight of the Safety Management System Regulations

In 2020-21, Transport Canada implemented several measures to improve oversight of safety management systems, including:

• Implemented new audit management software in order to improve data collection and analysis;
• Developed industry guidance to provide better information for railway companies with respect to implementing a compliant and effective safety management system;
• Improved and implemented the risk-based business planning process to target safety management system audits to areas of greatest risk; and
• Completed and implemented an audit follow-up procedure to ensure timely and consistent follow-up procedures after audits.

In practice, these improvements are being implemented as part of the department’s oversight regime for safety management systems. For example, during the 2020-21 fiscal year, 14 targeted safety management system audits were completed, specifically looking at how railway companies are managing the identification of safety concerns, ensuring compliance with the Rules Respecting Track Safety and managing knowledge of track employees. Companies involved in the audits submitted corrective action plans and Transport Canada is following up with the companies to monitor progress.

Similarly, Transport Canada began targeted audits of two national freight railways in October 2021 to study occurrences related to movements exceeding the limits of authority. The finding of this audit will be incorporated into the broader rail safety oversight program. In addition, TC is continuing with its commitment to complete at least one comprehensive audit of every company’s safety management system every 5 years. Transport Canada uses a risk-based approach to determine the frequency of these safety management system audits, and conducts follow up with companies after each audit. The department completed nine comprehensive audits in 2020-21, and is planning to conduct 15 in 2021-22.

TC continued to work on the implementation of procedures for auditing  railway companies’ SMS in order to measure the effectiveness and strengthen its oversight of the SMS Regulations.

Between September 2021 and December 2021, TC completed the following activities, based on the results of the 2020 pilot on auditing for SMS effectiveness:

• SMS effectiveness audits were launched, the first of which will be completed by March 2022. The findings from these audits will be shared with inspectors from all functions to inform current and future oversight activities and will inform improvements to industry guidance materials.
• Tools were developed to measure baselines and targets for measuring the effectiveness of SMS.
• The rail safety performance indicator profile was updated. This sets out improved performance measures to show the effectiveness of the oversight program.
• Guidelines for timely follow-up to SMS audits were established.

During the 2020-21 fiscal year, TC completed 14 targeted SMS audits, specifically looking at how railway companies are managing the identification of safety concerns, ensuring compliance with the Rules Respecting Track Safety, and managing knowledge of track employees. TC is following up with the companies to monitor progress of the corrective actions plans submitted as a result of the audit findings.

Furthermore, in October 2021, TC started targeted audits of 2 national freight railways to review occurrences where movements exceeded limits of authority. TC is also continuing with its commitment to complete at least one comprehensive audit of every company’s SMS every 5 years. TC completed 9 comprehensive audits in 2020-21.

As a response to the OAG follow-up audit, TC undertook an evaluation of the effectiveness of its Rail Safety Directorate’s oversight, including SMS. The final report, scheduled for completion in September 2022, will be shared with the TSB and published on the TC website.

The Board notes that TC has developed guidelines to help auditors measure effectiveness of various SMS processes. The Board is encouraged by TC’s progress and looks forward to receiving information on the findings related to the effectiveness of federal railways’ SMS. Therefore, the Board considers the responses to Recommendation R14-05 to show Satisfactory Intent.

Transport Canada has developed and implemented a robust auditing approach that assesses compliance, implementation and effectiveness of railway company safety management systems (SMS).^{Footnote 3}

From a compliance perspective, TC audits whether a railway company’s SMS meets the requirements established in the Railway Safety Management System Regulations, 2015 (SMS Regulation). This includes requirements to develop certain processes, procedures, plans, and methods, as well as the requirement to conduct certain activities. Implementation is the act of putting the components of the SMS into action meaning that documented processes are being undertaken as part of day-to-day activities. It is not sufficient to be compliant with the regulation if the company has not fully implemented their SMS. When conducing an audit, auditors may determine that while a strict non-compliance does not exist, a deficiency to implementation is found.

TC defines effectiveness as the degree to which the SMS processes, procedures, plans and methods, which make up the SMS, produce the desired result and are, in fact helping the company achieve the highest level of safety.

Measuring effectiveness

In February 2022, Transport Canada (TC) completed development of a suite of tools to measure the effectiveness of rail companies’ SMS, including indicators to measure the effectiveness of each of the SMS processes in the regulation. In spring 2022, TC’s auditors were trained on the use of these indicators. Additionally, in September 2022 rail companies were provided the effectiveness indicators. A follow-up meeting with industry will take place in May 2023 to obtain further feedback which will allow TC to further assess and improve the indicators. Revisions to the indicators incorporating this feedback will be completed by December 2023.

In March 2022, TC completed the first two audits for effectiveness of two national railway companies, specific to occurrences of movements exceeding limits of authority (MELA). Among the SMS process assessed for effectiveness were the process for assessing safety concerns and process for verifying compliance. The decision to focus on MELA was determined by the Department’s Risk Based Planning, which demonstrated that these types of occurrences are a persistent safety concern and they are difficult for inspectors to observe directly while conducting oversight.

The two audits for effectiveness generated findings that required the railway companies to submit a corrective action plan that identified how they will make their SMS more effective. Findings with respect to effectiveness in these audits showed that:

• Processes to ensure that company SMS were effective were not in place;
• Staff did not have adequate direction in order to complete investigations of occurrences;
• Continuous analysis of occurrences was not effectively taking place;
• Companies were not evaluating the effectiveness of their procedures, plans and methods associated with their SMS nor have they developed their own criteria to evaluate the effectiveness;
• Ineffectiveness was identified in processes for verifying employees’ knowledge of skills and duties.

The corrective action plans for both audits were submitted by companies and accepted by Transport Canada. A follow up audit will be scheduled in the future to confirm implementation; however, it should be noted that the current audit is already seeing some improvement based on planned actions undertaken by rail companies.

In addition to the two targeted effectiveness audits conducted in 2021/22, TC also conducted:

• 14 comprehensive audits, which assess a company’s compliance and implementation of SMS processes as part of TC’s commitment to audit all company SMS at a minimum of every five years, and
• 4 follow-up audits

Following all audits, Transport Canada reviews the corrective action plans submitted by rail companies to ensure they respond to the audit findings and the department monitors progress through the conduct of follow-up audits.

In 2022-23, TC is planning to conduct 7 targeted audits of effectiveness (of processes, such as, the process for identifying safety concerns, process for managing knowledge, and risk assessment process); 16 comprehensive audits, and 4 follow-up audits. These effectiveness audits include functional inspectors, who are subject matter experts and provide operational expertise to the conduct of the audit.

Finally, TC Audit and Evaluation Services has completed its evaluation of the effectiveness of Rail Safety’s oversight, including SMS. An advance copy of the final report has been sent separately. The contents of the report cannot be discussed publicly until it is published by Transport Canada, in early 2023.

In summary, TC has developed and implemented a robust program to audit the effectiveness of railway SMS. In the absence of established indicators for measuring SMS effectiveness in the rail industry, TC has established its own, which it continues to refine and improve through the conduct of effectiveness audits, input from auditors, and consultation with railways. Through our longstanding and ongoing practice of receiving and assessing corrective action plans and the conduct of follow-up audits to ensure they are implemented, the department verifies that actions are implemented to improve safety.

In addition to the update provided in November, 2021, in summer 2022, Transport Canada launched a public consultation on a review of the Railway Safety Management System Regulations, 2015. The RAC submitted comments to the consultation document and Industry has been in consultation with TC.

Among the comments, the RAC highlighted that there needs to be sufficient time allotted between regulatory changes, as well as audits, to allow for improvements to be realized. The TSB has also acknowledged this concept by increasing the TSB Watchlist cycle to three years (ref. TSB Review, November 2022 which states in part, “Realizing that these issues are complex and take time, the Watchlist is moving to a three-year cycle, to allow time for meaningful progress to be made by industry and Transport Canada.”)

The current combination of TC audits and the Process for Continual Improvement of the Safety Management System does not allow sufficient time for railways to continuously improve effectively, and realize improvements, prior to further auditing or review. Some railways are audited multiple times per year. These railways must also conduct continual monitoring and internal audits of SMS every three years, which requires both a report and the handling of any corrective actions to resolution. In practical terms, this does not allow time to implement and evaluate the effectiveness of corrective actions before new audits are imposed.

Transport Canada (TC) continued to work on the implementation of an auditing approach that assesses compliance, implementation, and effectiveness of railway companies’ safety management systems (SMS).

In 2022, TC

• developed a suite of tools to measure the effectiveness of rail companies’ SMS, including indicators to measure the effectiveness of each of the SMS processes in the regulations;
• provided the effectiveness indicators to the railway companies;
• completed the first 2 targeted audits for effectiveness of 2 national railway companies, specific to occurrences of movements exceeding limits of authority;
• conducted 14 comprehensive audits and 4 follow-up audits; and
• completed, through its Audit and Evaluation Services, an evaluation of the effectiveness of its Rail Safety Directorate’s oversight, including SMS. TC expects to publish the final report in early 2023.

In 2022–23, TC is planning to conduct 7 targeted audits of effectiveness, 16 comprehensive audits, and 4 follow-up audits. Furthermore, TC will have a follow-up meeting with industry in May 2023 to further assess and improve the effectiveness indicators, and it will complete revisions to these indicators by December 2023.

The Board notes that the findings of the targeted audits completed in 2022 do not yet show that the railway companies’ SMS processes are effective. Based on the findings from these audits, the railway companies were each required to submit a corrective action plan that identified how they will make their SMS more effective. The corrective action plans for both audits were submitted by the companies and accepted by TC. TC indicates that the current audit is already seeing some improvement based on planned actions undertaken by rail companies. The Board also notes that the Railway Association of Canada indicated that sufficient time is required to allow railway companies to implement corrective actions and evaluate their effectiveness before new regulatory audits are conducted.

The Board acknowledges the progress made and appreciates TC’s efforts in auditing the effectiveness of railway companies’ SMS. However, until the TSB is able to determine the impact of TC’s revised SMS auditing process, the Board considers the responses to Recommendation R14-05 to show Satisfactory Intent.