Programme implementation, monitoring and evaluation

7.1 Introduction

The focus of this guide is to identify high-risk rural roads and develop countermeasures that reduce fatal and serious crashes along a route or at a site. Once these routes and measures have been identified a suitable programme of implementation is important, along with a system to monitor the effectiveness of these countermeasures: ‘The effectiveness of treatments guides investment in road safety programs and reliable and accurate information will be necessary to determine the effectiveness of treatments’. [ 127]

In this section we look at issues associated with developing programmes to treat high-risk rural routes, and then monitoring the effectiveness of those programmes to:

1. identify the benefits or rather the effectiveness of the various treatments 2. identify the most effective packages of treatments

3. assess the levels of funding that may be required to achieve various levels of crash reduction 4. ‘prove’ that funding has been spent wisely.

Figure 7.1 is a modified version of the safety management triangle. The foundation of this triangle is the identification and analysis of crash issues, which would include the means of identifying high-risk rural roads, corridors or sites (section 4).

Having identified our sites/routes and clarified our safety concerns, this guide discusses some possible treatments or strategies that could be used to improve the safety of our high-risk rural routes, and reduce the risk of death or serious injury, the primary outcome.

FIGUre 7-1

road safety management triangle

1 Death & serious injury

2 reduced personal

& collective risk 3 lead performance indicators 4 Safety measures and programs 5 Crash issues/analysis PrIMArY OUTCOMe SeCONDArY OUTCOMe IMTerMeDIATe OUTCOMeS

Programs of treatment for

• roads and roadsides and • speed

Identification of high-risk rural roads, sites and corridors

In an ideal world, the analysis of the effectiveness of each treatment or programme item would be assessed by applying only one specific treatment to a range of sites and monitoring the

performance of the treatment over time, before moving on to apply the next treatment. However, in New Zealand, the number of people killed or seriously injured in any one location is too small and the risk of doing nothing could be too severe – a purist approach is precluded by the delays associated with the post-implementation data collection and the immorality of ‘playing with people’s lives’. So in order to facilitate the necessary analysis, the road safety management triangle introduces the concept of intermediate and secondary outcomes.

In this section we begin by looking at the development of a programme of treatments, and how to establish the appropriate intermediate measures. We then look at the monitoring site-specific secondary and primary measures.

7.2 Programme development

While the focus of the guide is on high-risk rural routes – those typically located in the upper and right side parts of figure 7-2 – it is important to remember low-cost safety management treatments still apply to the bottom left quadrant.

The assessment of rural road risks in section 4 identifies the longer-term plan for a particular rural road. Some regions will have no rural road sections in the upper and right side portions of figure 7-2, but that does not mean a programme of ongoing safety improvement should be abandoned; it just needs to be tailored to fit the appropriate end game. Analysing the data and understanding the issues are important (section 5).

7.2.1 Programme prioritisation – focus on infrastructure transformation

countermeasures

The main focus for a Safe System approach to high-risk rural roads is to address fatal and serious crashes and the three key crash types. A report by Monash University [128] discusses

infrastructure versus fundamental improvements, specifically with regard to the pattern of key crash types on rural roads. The most strategically important transformational countermeasures found within the literature review were as follows:

• Crashworthy barrier system – when used over extended lengths of high-speed rural road,

barrier systems have the potential to reduce fatal and serious injuries to the occupants of errant vehicles by around 90%, with conservatively estimated benefit–cost ratios of around eight. Flexible barrier systems can address two major rural crash categories, namely single vehicle and head-on crashes, on straight or curved road sections, without the need for costly road

duplication and/or geometric improvements to rural infrastructure.

• Grade-separated interchange – can virtually eliminate intersection crashes (potentially 100%

effective) but the high cost of grade-separation makes them less attractive than some other alternatives.

• Roundabouts – can reduce casualty risk at intersections by between 70% and 80%, and crash

costs by around 90%. In addition, they have been found to result in benefit–cost ratios of around 19 when constructed at rural intersections with a high crash record.

Although these measures cost more, they have significant benefits. It is desirable to plan for their implementation in the long term.

7.2.2 Challenges to implementation

A Safe System report [129] identified the following challenges to implementing a Safe System:

• cost, particularly in relation to the main infrastructure type countermeasures (ie roundabout, grade separation, median barriers)

• construction timeframes

• community support: it would be particularly important to gain acceptance from the community with the introduction of lower speed limits or compliance would be minimal

• inter-agency planning: there is a strong need for organisations to work together to deliver a Safe System

• incompatibility of travel modes: specific countermeasures may produce incompatibility between different road users.

7.2.3 Programme implementation

Consideration of the types of countermeasures and planning is important in providing the best possible outcome in terms of reducing the number of fatal and serious crashes along a route, site or area.

Turner, Tziotis, Cairney and Jurewicz [129] state:

‘the timeframe for implementation of a Safe System infrastructure is an important consideration. A step process will most likely be required, and over a long term period (eg 20 years). Some initiatives can be implemented immediately, but others require longer. With a focus on longer term objectives, total cost can be divided over a larger number of years. The total costs per year may not be

substantially more than amounts currently spent on safety (including through maintenance and major projects budget) although likely costs still need to be determined.’

7.2.4 Focus on incremental improvements across network

The focus for a programme of works should concentrate on incremental improvements across networks to help achieve larger benefit–cost ratios. So what are incremental improvements? Having identified that a route requires larger infrastructure/capital projects to produce a Safe System transformation, the end result has to some degree been confirmed. However, given the limited funding and associated priorities, together with the lead time associated with getting major infrastructure projects to construction, doing nothing until that project eventuates continues to place drivers at an increased risk of death or serious injury.

As responsible road safety practitioners and network managers, we need to consider this risk. Incremental improvements are viable if they:

• help reduce the cost of the final project, ie provide incremental benefit and costs

• return an economic road safety benefit over the intervening period, ie between now and the realistic date for delivery of the major project.

For example, the Safe System transformation of a high-volume section of highway may be the construction of a physically separated dual carriageway in 10 years. If the long-term solution will be developed on essentially the same alignment, an incremental approach could be adopted.

Assuming the safety-related maintenance is being undertaken, an incremental solution could involve, in the first instance, widening sections of the carriageway where required, then installing a median barrier (1+1), improving roadside hazard management, installing passing lanes and progressively moving to a 2+1 lane arrangement, before moving to a 2+2 lane arrangement. The works undertaken at each step contribute, at least in part, to the overall Safe System

Transformation Works, reducing the costs associated with the final project.

If, however, the final solution involves a completely new alignment, any proposed works will have a reduced economic life and should be analysed over the pre-implementation period.

7.2.5 Consistency and road classification

The road environment should provide the road user with strong indications about what to expect, how to behave and safe operating speeds. The consistency of road environment messages along the road corridor is important. These messages are delivered through the carriageway width, alignment, access management, signs and markings standards and other traffic control devices. The basis for determining the service levels for both travel time and safety is the road hierarchy or, for the state highway network, the recently published state highway classification system (www. nzta.govt.nz/planning/process/state-highway.html#planning). Hence, in developing road safety programmes the road hierarchy needs to be considered and safety measures applied that are both appropriate and consistent with the road function and the traffic volumes it carries.

As well as determining the appropriateness of the safety measures, the road classification is likely to be a determinant in prioritisation for funding.

7.2.6 Driver awareness measures/self-explaining roads

Driver awareness measures for self-explaining roads provide clear direction and unambiguous information to all road users which drivers can use to make decisions and modify their behaviour depending on the design and function of a road and the associated risks. These measures are more likely on routes where there are higher levels of personal risk but low to medium levels of collective risk.

7.2.7 Communication and consultation

It is vital to engage with key stakeholders (community, affected and interested parties) when developing projects in order to create a common sense of purpose, draw on and learn from other’s perspectives, make better decisions, align mutual interests, identify and mitigate risks, and find shared solutions to challenges.

Relationship building, the basis for effective engagement, takes time. Many of the hallmarks of good relationships – trust, mutual respect and understanding – are intangibles that develop and evolve over time. Early engagement provides a valuable opportunity to set a positive tone with stakeholders from the outset of a project. The absence of established relationships and communication channels can put our project at an immediate disadvantage.

Establishing and maintaining good relationships requires a long-term view. Organisations that take this approachsee the value of consistently following through on their commitments to

stakeholders. They take grievances seriously and deal with them in a reliable and timely manner. They continually invest in communicating about their work in a way that makes sense to their stakeholders. Effective engagement and communication will ultimately ensure the project’s success. [131]

As stated within the Austroads research report ‘Community Consultation Process and Methods for Quantifying Community Expectations on the Levels of Service for Road Networks AP-R290-06’ [121]:

• An ideal consultation with road users and other stakeholders is one that:

- consists of a number of clearly defined stages, each with their own specific objectives - includes both external stages (ie those that include road users and stakeholders) and internal

stages (ie those that include employees of the road agency only)

- is iterative in nature (ie part of an ongoing and iterative cycle of learning, refinement and improvement embedded within the development process rather than an ‘isolated event’ that takes place externally to it).

• The development of levels of service and intervention criteria for maintenance and improvement activities through community consultation is complex and requires careful planning. The process consists of several iterative stages: listen, communicate, reflect and plan, implement, monitor and measure. The process alternates stages that involve the community with stages that require bi-internal agency assessment and evaluation. Each stage is conducted in a structured manner and requires specific techniques and specialised skills.

• The process begins with a two-way communication (‘listen’ and ‘communicate’) between the road agency and the community with the purpose of gaining a common understanding of community concerns, priorities, current road classification system and levels of service as well as agency issues, priorities and budget limitations. This part of the process also helps develop a common language and identify the most effective channels for further communication of road maintenance issues. The two-way communication establishes the foundation for a transparent and strong relationship between the road agency and the community.

7.3 Monitoring and evaluation

Monitoring and evaluation is important in gauging the effectiveness of different Safe System treatments. This is also important when developing types of countermeasures for specific issues and implementation procedures for future programmes. Specifically:

• Monitoring involves an assessment of progress and collecting information through the course of

a project, can be before, during and after to gather results for which to do an evaluation (section 7.3.1).

• An evaluation analyses the results of monitoring and determines the results and effectiveness

of the types of treatments used (section 7.3.2).

7.3.1 Monitoring

Monitoring and collection of data for evaluation will help to identify if road safety has been improved: ‘Systematic recording of data and analysis of trends from which goals and targets [section 7.3.3] can be calculated allows the most recent values of measures and their trends to be compared with target levels.’ [130]

7.3.2 Evaluation

As stated in Austroads Report ST 1571 [127] the role of evaluation is to:

• ensure that recently delivered programmes are effective and enable remedial action if they are not

• build up a reliable knowledge base about the effectiveness of different interventions, which will allow more effective programmes to be developed in the future.

There are effectively two levels of monitoring and evaluation:

• strategically monitoring and then evaluating the effectiveness of the overall programme or strategy, which is made up of various projects or initiatives

• individually monitoring and evaluating specific projects or initiatives that make up the overall programme or strategy.

While good monitoring and evaluation will support future road safety improvement programmes, the monitoring and evaluation effort should not consume excessive amounts of staff time or other resources that could be used to undertake more road safety initiatives. As a general observation,

In addition, ’Only by monitoring the effects of real treatments which have been applied in real traffic situations can a reliable picture of a countermeasure’s effects be obtained. Theoretical analyses form first principles and simulation provides valuable insights. The quality of an evaluation is measured by its "Validity"‘[4]. The Austroads document contains detailed information on validity and general trends, changes to traffic flows, regression to the mean, crash risk mitigation and adjustment periods, which is located in section 6 of that report. Sections 7.3.4–7.3.6 summarise section 6 of the Austroads document.

In the following sections we look at the monitoring and evaluation of individual initiatives or projects and then the monitoring of the overall strategy.