DAVID SHINAR – Traffic Safety and Human Behaviour
STOPHER & STECHER – Travel Survey Methods Quality and Future Directions HENSHER & BUTTON (eds.) – Handbooks in Transport
FULLER & SANTOS – Human Factors for Highway Engineers GAUDRY & LASSARE (eds.) – Structural Road Accident Models DAGANZO – Fundamentals of Transportation and Traffic Operations
SECOND EDITION
BY
Rune Elvik
Institute of Transport Economics, Oslo, Norway
Alena Høye
Institute of Transport Economics, Oslo, Norway
Truls Vaa
Institute of Transport Economics, Oslo, Norway
Michael Sørensen
Institute of Transport Economics, Oslo, Norway
United Kingdom – North America – Japan India – Malaysia – China
Second edition 2009
Copyright r 2009 Emerald Group Publishing Limited Reprints and permission service
Contact: booksandseries@emeraldinsight.com
No part of this book may be reproduced, stored in a retrieval system, transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise without either the prior written permission of the publisher or a licence permitting restricted copying issued in the UK by The Copyright Licensing Agency and in the USA by The Copyright Clearance Center. No responsibility is accepted for the accuracy of nformation contained in the text, illustrations or advertisements. The opinions expressed in these chapters are not necessarily those of the Editor or the publisher.
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library ISBN: 978-1-84855-250-0
Awarded in recognition of Emerald’s production department’s adherence to quality systems and processes when preparing scholarly journals for print
TABLE OF
C
ONTENTS
Preface . . . xi
PART I Introduction . . . 1
1. Background and Guide to Readers. . . 3
1.1. Purpose of the Handbook of Road Safety Measures . . . 3
1.2. Which questions does the book answer? . . . 5
1.3. Structure of the book . . . 6
1.4. Science and politics in road safety. . . 8
2. Literature Survey and Meta-Analysis. . . 15
2.1. Systematic literature search. . . 15
2.2. Criteria for study inclusion . . . 19
2.3. Study classification . . . 19
2.4. The use of meta-analysis to summarise study results . . . 20
2.5. Does a weighted mean estimate of effect make sense? . . . 25
2.6. Developing accident modification functions . . . 30
2.7. Specification of accident or injury severity . . . 32
2.8. Updated estimates of effect: Revision of the book . . . 33
3. Factors Contributing to Road Accidents . . . 35
3.1. A simple conceptual framework. . . 35
3.2. The scope of the road accident problem worldwide . . . 37
3.3. Incomplete reporting in official road accident statistics. . . 47
3.4. Exposure: Traffic volume. . . 53
3.5. Accident rates for different types of exposure. . . 56
3.6. The mixture of road users . . . 57
3.7. A survey of some risk factors for accident involvement. . . 59
3.8. A survey of risk factors for injury severity . . . 67
4. Basic Concepts of Road Safety Research . . . 81
4.1. Random and systematic variation in accident counts. . . 81
4.2. The use of accident rates to measure safety. . . 86
4.3. Explaining road accidents – the concept of cause . . . 87
4.4. Road accidents as a self-regulatory problem. . . 93
5. Assessing the Quality of Evaluation Studies. . . 99
5.1. The concept of study quality. . . 99
5.2. Assessing study quality. . . 99
5.3. The importance of study quality: Some illustrations. . . 106
5.4. The treatment of study quality in meta-analysis . . . 113
5.5. Can the findings of road safety evaluation studies be accounted for in theoretical terms?. . . 113
6. The Contribution of Research to Road Safety Policy-Making. . . 117
6.1. An idealised model of the policy-making process . . . 117
6.2. The applicability of cost–benefit analysis. . . 119
6.3. Monetary valuation of road safety in different countries. . . 124
6.4. Current monetary valuations of impacts of road safety measures in Norway . . . 125
6.5. The preventability of road accident fatalities and injuries. . . 127
6.6. Vision Zero . . . 130
References . . . 131
PART II Road Safety Measures . . . 143
1. Road Design and Road Equipment. . . 145
1.0. Introduction and overview of 20 measures . . . 145
1.1. Cycle lanes and tracks. . . 155
1.2. Motorways. . . 164
1.3. Bypasses . . . 169
1.4. Urban arterial roads . . . 172
1.5. Channelisation of junctions. . . 178
1.6. Roundabouts. . . 185
1.7. Redesigning junctions . . . 190
1.8. Staggered junctions (reconfiguring crossroads to two T-junctions). . . 195
1.9. Grade-separated junctions . . . 199
1.10. Black spot treatment. . . 206
1.11. Cross-section improvements. . . 212
1.13. Improving road alignment and sight distance. . . 233
1.14. Reconstruction and rehabilitation of roads. . . 248
1.15. Guardrails and crash cushions . . . 251
1.16. Game accident measures. . . 258
1.17. Horizontal curve treatments . . . 268
1.18. Road lighting. . . 272
1.19. Improving tunnel safety . . . 281
1.20. Rest stops and service areas . . . 287
References . . . 289
2. Road Maintenance. . . 335
2.0. Introduction and overview of nine measures. . . 335
2.1. Resurfacing of roads. . . 339
2.2. Treatment of unevenness and rut depth of the road surface. . . 344
2.3. Improving road surface friction . . . 348
2.4. Bright road surfaces. . . 358
2.5. Landslide protection measures . . . 360
2.6. Winter maintenance of roads . . . 363
2.7. Winter maintenance of pavements, footpaths, cycle paths and other public areas. . . 373
2.8. Correcting erroneous traffic signs. . . 376
2.9. Traffic control at roadwork sites . . . 380
References . . . 385
3. Traffic Control . . . 397
3.0. Introduction and overview of 22 measures . . . 397
3.1. Area-wide traffic calming. . . 403
3.2. Environmental streets . . . 408
3.3. Pedestrian streets . . . 412
3.4. Urban play streets . . . 415
3.5. Access control. . . 419
3.6. Priority control. . . 423
3.7. Yield signs at junctions. . . 427
3.8. Stop signs at junctions . . . 430
3.9. Traffic signal control at junctions . . . 433
3.10. Signalised pedestrian crossings . . . 440
3.11. Speed limits . . . 445
3.12. Speed-reducing devices . . . 452
3.13. Road markings. . . 458
3.15. Stopping and parking control. . . 474
3.16. One-way streets. . . 479
3.17. Reversible traffic lanes . . . 481
3.18. Bus lanes and bus stop design. . . 487
3.19. Dynamic route guidance. . . 492
3.20. Variable message signs . . . 495
3.21. Protecting railway–highway level crossings . . . 499
3.22. Environmental zones. . . 504
References . . . 507
4. Vehicle Design and Protective Devices. . . 543
4.0. Introduction and overview of 29 measures. . . 543
4.1. Tyre tread depth . . . 550
4.2. Studded tyres . . . 554
4.3. Antilock braking systems and disc brakes . . . 560
4.4. High-mounted stop lamps. . . 564
4.5. Daytime running lights for cars. . . 567
4.6. Daytime running lights for mopeds and motorcycles. . . 571
4.7. Improving vehicle headlights . . . 574
4.8. Reflective materials and protective clothing. . . 582
4.9. Steering, suspension and vehicle stability . . . 586
4.10. Bicycle helmets. . . 591
4.11. Motorcycle helmets. . . 596
4.12. Seat belts in cars. . . 600
4.13. Child restraints. . . 609
4.14. Airbags in cars. . . 615
4.15. Seat belts in buses and trucks . . . 624
4.16. Vehicle crashworthiness. . . 627
4.17. Driving controls and instruments . . . 635
4.18. Intelligent cruise control . . . 639
4.19. Regulating vehicle mass (weight). . . 642
4.20. Regulating automobile engine capacity (motor power) and top speed. . . 649
4.21. Regulating engine capacity (motor power) of mopeds and motorcycles. . . 656
4.22. Under-run guards on heavy vehicles . . . 661
4.23. Safety equipment on heavy vehicles. . . 663
4.24. Moped and motorcycle equipment. . . 668
4.25. Bicycle safety equipment. . . 671
4.27. Fire safety standards . . . 680
4.28. Hazardous goods regulations. . . 682
4.29. Electronic stability control . . . 687
References. . . 690
5. Vehicle and Garage Inspection. . . 733
5.0. Introduction and overview of four measures. . . 733
5.1. Vehicle safety standards. . . 737
5.2. Periodic motor vehicle inspections. . . 742
5.3. Roadside vehicle inspections. . . 749
5.4. Garage regulation and inspections. . . 753
References . . . 755
6. Driver Training and Regulation of Professional Drivers . . . 759
6.0. Introduction and overview of 12 measures. . . 759
6.1. Driving licence age limits . . . 763
6.2. Health requirements for drivers. . . 771
6.3. Driver performance standards. . . 779
6.4. Basic driver training. . . 785
6.5. The driving test . . . 793
6.6. Training and testing of moped and motorcycle riders. . . 797
6.7. Training and testing of professional drivers. . . 802
6.8. Graduated driving licences (GDLs) . . . 806
6.9. Motivation and incentive systems in the work place . . . 815
6.10. Regulation of driving and rest hours. . . 817
6.11. Safety standards for emergency driving. . . 827
6.12. Safety standards for school transport . . . 833
References. . . 839
7. Public Education and Information. . . 859
7.0. Introduction and overview of three measures. . . 859
7.1. Education of pre-school children (0–6 years) . . . 862
7.2. Education in schools (6–18 years old). . . 865
7.3. Road user information and campaigns. . . 867
References . . . 873
8. Police Enforcement and Sanctions . . . 879
8.0. Introduction and overview of 13 measures. . . 879
8.1. Stationary and manual speed enforcement. . . 885
8.3. Seat belt enforcement. . . 893
8.4. Patrolling. . . 899
8.5. Red-light cameras. . . 902
8.6. Demerit point systems and licence suspension. . . 907
8.7. Fixed penalties. . . 913
8.8. DUI legislation . . . 916
8.9. DUI enforcement . . . 930
8.10. Restrictions for DUI-convicted drivers. . . 935
8.11. Treatment of DUI-convicted drivers . . . 941
8.12. Fines and imprisonment . . . 945
8.13. Motor vehicle insurance . . . 949
References. . . 955
9. Post-Accident Care. . . 981
9.0. Introduction and overview of three measures. . . 981
9.1. Emergency medical services. . . 983
9.2. Rescue helicopters . . . 990
9.3. Automatic crash notification. . . 994
References. . . 998
10. General-Purpose Policy Instruments. . . 1005
10.0. Introduction and overview of 13 measures. . . 1005
10.1. Organisational measures . . . 1012
10.2. Information for decision-makers. . . 1017
10.3. Quantified road safety targets and road safety programmes . . . 1020
10.4. Safe community programmes. . . 1023
10.5. Exposure control. . . 1026
10.6. Land use plans (urban and regional planning) . . . 1031
10.7. Road plans and road construction. . . 1039
10.8. Road safety audits and inspections. . . 1043
10.9. Motor vehicle taxation. . . 1048
10.10. Road pricing. . . 1053
10.11. Changes in the modal split of travel. . . 1061
10.12. Road traffic legislation. . . 1069
10.13. Regulating commercial transport . . . 1075
References. . . 1079
PART III Vocabulary and Index. . . 1093
Definitions of Technical Terms. . . 1095
List of Abbreviations. . . 1115
PREFACE
The second, revised edition of The Handbook of Road Safety Measures, first published by Elsevier Science in 2004, gives a systematic overview of current knowledge regarding the effects of road safety measures. The book gives state-of-the-art summaries of current knowledge regarding the effects of 128 road safety measures. Since 2004, the introduction part and 65 chapters have been revised and 5 chapters have been added. Easily accessible knowledge on how to prevent traffic injury is in increasing demand, as the number of people killed or injured in road accidents continues to grow on a global basis. It is hoped that this book may serve as a reference manual for road safety professionals in every country. The 2004 edition of the book was published in Spanish in 2006. The book is based on the Norwegian edition of the book, first published in 1982 and continuously updated and expanded since 2001. Work on this book started as far back as 1980. During the whole period from 1980 until now, the endeavour to develop and update the book has been funded by the Norwegian Ministry of Transport and Communications and the Norwegian Public Roads Administration. In recent years, the Swedish Road Administration has been an important contributor as well. The Institute of Transport Economics (TØI) would like to thank these institutions for their financial support and their long-term commitment to this research effort. Without the original Norwegian edition, the current English version could never have been produced. The present edition is the result of the coordinated effort of Chief Research Officer Rune Elvik and researchers Alena Høye, Truls Vaa and Michael Sørensen – all belonging to the Institute of Transport Economics. The final preparation of the manuscript for publication was made by Unni Wettergreen. The points of view expressed in the book are those of the authors and do not necessarily reflect the positions of the funding agencies. Errors and omissions, if any, are the sole responsibility of the authors.
Oslo, May 2009
Institute of Transport Economics Lasse Fridstrøm
B
ACKGROUND AND
G
UIDE TO
R
EADERS
1.1 P
URPOSE OF THEH
ANDBOOK OFR
OADS
AFETYM
EASURESAs the title of this book is Handbook of Road Safety Measures, most readers will perhaps expect a handbook to give instructions or advice concerning its main topic, but not all readers will expect the same kind of instructions or advice. It is therefore appropriate to start the book by describing its background and purpose.
Although this book is called a ‘handbook’, it does not provide any instructions or advice of a general nature with respect to how best to design or implement road safety measures. The term ‘handbook’ rather denotes a reference manual, a catalogue or an encyclopaedia of road safety measures.
Why is this book written and what is its main purpose? The book is written in order to summarise and present in an easily accessible form what is currently known about the effects of road safety measures. A road safety measure is any technical device or programme that has improving road safety as the only objective or at least one of its stated objectives. Road safety measures may be directed at any element of the road system: patterns of land use, the road itself, road furniture, traffic control devices, motor vehicles, police enforcement and road users and their behaviour.
This book takes a broad view of what constitutes a road safety measure. It is not limited to a particular class of safety programmes, but tries to cover everything that is intended to improve road safety. A total of 128 road safety measures are included. Improving road safety is, unfortunately, not a concept that has a standard scientific definition. In this book, it refers to a reduction in the expected number of accidents, a
The Handbook of Road Safety Measures
Copyright r 2009 by Emerald Group Publishing Limited All rights of reproduction in any form reserved
reduction in accident or injury severity or a reduction in the rate of accidents or injuries per kilometre of travel.
The main purpose of the book is to describe, as objectively as possible, the effects of road safety measures on road safety. Some road safety measures influence not only road safety but also the ease of travel and the quality of the environment. Ease of travel is a broad concept that includes aspects such as accessibility (the availability of a certain destination for travel), out-of-pocket expenses (like motor vehicle operating costs) and travel time. In this book, the term mobility is used to denote the ease of travel in terms of accessibility, cost and travel time. Environmental impacts of road safety measures refer primarily to impacts on traffic noise and air pollution, but in some cases, other impacts are briefly mentioned, for example, impacts on the working conditions of professional drivers.
Some of the terms that have been used to describe the contents of this book, such as ‘current knowledge’ and ‘objective description’, require a more extensive discussion. This will be undertaken in later chapters of Part I (in particular, Chapters 4 and 5). Before describing the main questions, the book tries to answer, its structure and the role of research in promoting road safety, what this book is not intended to be needs to be explained.
This book is not a technical design handbook. It does not tell readers how to design a junction or how to build a car. This book does not offer a prescription for road safety policy. It does not tell readers which road safety measures ought to be taken, nor does it instruct policymakers in how to set priorities for the provision of road safety.Section 1.4outlines how the line separating road safety research from road safety policymaking is understood in the book.
This book does not tell you how to do road safety research; however, it tries to assess systematically the quality of current knowledge about the effects of road safety measures. In doing so, this book of course invokes widely accepted standards of technical rigour and quality in applied research. However, assessing the quality of what is known is not the same thing as instructing researchers about how to improve knowledge.
This book does not tell readers how to set up an accident recording system or how to investigate accidents, but discusses the concept of accident causation and briefly summarises what is known about factors that contribute to road accidents. Although this presentation may perhaps give readers some ideas about what they should be looking for when trying to find out why road accidents happen, it is highly deficient in acting a guide as to how best to investigate and record road accidents.
Some readers may take exception to the consistent use of the word ‘accident’ in the book, preferring perhaps other words like crash or unintentional injury event (Langley 1988). Hopefully, these readers will not be deterred from using the book. Some of the arguments for not using the word ‘accident’ are, we believe, based on misunderstand-ing. For example, it has been argued that the word ‘accident’ has traditionally been used to represent events that occur at random, and which are therefore unpreventable. This point of view is both correct and incorrect. It is correct in that there is an element of randomness in accident occurrence. However, the occurrence of accidents is never entirely random. Young male drivers are systematically over-involved in road accidents. The gender and age of drivers involved in road accidents are, therefore, not entirely a matter of chance. On the contrary, the occurrence of a specific road accident is random in the absolute sense that if it could have been accurately predicted, it would not have happened (assuming that accidents are not deliberate; that nobody wants to become involved in an accident).
Part of the nature of random events is that the precise time and place of their occurrence, as well as the precise nature of their impacts, are unpredictable. But unpredictability in this sense does not necessarily imply un-preventability. To illustrate this, imagine a 100-km-long road, chopped up into 100 consecutive 1-km sections. The number of accidents recorded on each 1-km section is counted, and the distribution of accident counts among the 100 sections is found to closely follow the Poisson probability law, which means that accident occurrence in these 100 road sections is random in the sense that it is not statistically possible to identify one road section that has a higher expected number of accidents than any other road section. Yet it hardly follows from this observation that the accidents occurring along the 100-km road cannot be prevented. Suppose, for example, that all drivers using the road slowed down by 10 km per hour. It is very likely that there would then be fewer accidents. Or, suppose road lighting is installed along the road. Again, it is likely that there would be a reduction in the number of accidents. ‘Accident’ is the right word for a road crash, precisely because it connotes randomness. It is a matter of fact that there is a large, but not always dominant, element of randomness in accident occurrence. It is, however, a serious misunderstanding to suggest that randomness as such implies that accidents cannot be prevented.
1.2 W
HICH QUESTIONS DOES THE BOOK ANSWER?
This book provides answers to the following questions:
Which measures can be used to reduce the number of traffic accidents or the severity of injury in such accidents?
Which accident problems and types of injury are affected by the different measures? What effects on accidents and injuries do the various road safety measures have
according to international research?
What effects do the measures have on mobility and the environment? What are the costs of road safety measures?
Is it possible to make cost–benefit evaluations of the measures?
Which measures give the greatest benefits for traffic safety seen in relation to the cost of the measures?
Not all these questions are equally easy to answer, and it is not always possible to give a precise or conclusive answer. For example, the effect of a measure on accidents may vary from place to place, depending on the design of the measure, the number of accidents at the spot, any other measures that have been implemented, etc. As a result, different studies of the same measure may provide different conclusions. An attempt has been made to identify sources of variation in study findings and to try to form as homogeneous groups as possible when presenting estimates of the effects of measures on road safety. This will be discussed more detail in Chapter 2.
1.3 S
TRUCTURE OF THE BOOKThe book consists of three parts, each of which can be read independently. The chapters in each part are also designed to be read independently.
Part I describes the purpose of the book and its structure, the method used in surveying and analysing the literature the book is based on, factors contributing to road accidents, basic concepts of road safety research, the quality of road safety evaluation research and scientific approaches to planning and policymaking.
Part II describes road safety measures in 10 different areas. Within each area, a number of different types of measures are described in individual sections. The 10 areas are 1. Road design and road equipment (20 measures)
2. Road maintenance (9 measures) 3. Traffic control (21 measures)
4. Vehicle design and protective devices (29 measures) 5. Vehicle and garage inspection (4 measures)
6. Driver training and regulation of professional drivers (12 measures) 7. Public education and information (4 measures)
9. Post-accident care (3 measures)
10. General purpose policy instruments (14 measures).
Part III contains a glossary of words, symbols and abbreviations, which are used in the book and a subject index.
In Part II, each chapter and each of the sections within each chapter has been written following the same structure. The first section in each chapter gives an overview of the amount of research available and summaries of the effects on accidents, environment and mobility, as well as an overview of costs and cost–benefit analyses. The sections that described specific types of road safety measures all consist of the same subsections, a short description of which is given in the following.
Problem and objective. This section describes the road safety problem, which the measure is designed to solve or reduce. A road safety problem can be described in terms of a high number of accidents, a high accident rate or a high proportion of serious injuries. For example, it is widely seen as a problem that pedestrians and cyclists are more often involved in injury accidents per kilometre travelled in traffic than car occupants, and that they tend to be more seriously injured than car occupants when involved in an accident. As far as possible, the size of the road safety problem which each measure is intended to affect is shown by means of accident figures or estimates of risk. However, not all road safety problems can be described exhaustively in numerical terms only. This applies, for example, to the feeling of insecurity that some road users experience.
Many road safety measures are intended to tackle local problems, having a fairly clearly limited scope in time and space. However, this does not apply to all measures. Some measures are directed towards more general problems, which may affect all road users and all places. In such cases, it is difficult to state precisely the number and nature of accidents which these measures are designed to affect. For some road safety measures, the concept of ‘target accidents’ is thus somewhat ill defined (Hauer 1997). Description of the measure. This section gives information concerning the design of a road safety measure and its intended function. Detailed technical descriptions are not given. Illustrations showing the measure are given in some cases.
Effect on accidents. This section deals with the effects on accidents, or on the severity of injury in accidents, which have been found in research. Whenever possible, effects are stated in terms of the percentage change of the number of accidents or injuries attributable to a certain measure. All estimates of effect presented in this book are uncertain. The most important sources of such uncertainty are identified for each
measure. Statistical uncertainty is stated in terms of a 95% confidence interval for the estimate of effect. For measures where no studies have been found that quantify effects on road safety, the effect is described in other ways.
Effect on mobility. In addition to the effect on accidents and injuries, many road safety measures also have effects on mobility. These impacts are briefly described, but not in as great detail as safety effects.
Effects on the environment. Effects on the environment are briefly described. Such effects include traffic noise and air pollution in a wide sense. Major incursions into the landscape and changes in land use should also be regarded as important environmental effects.
Costs. For the majority of measures, information is given regarding the cost of the measure. The information is taken partly from official budgets and accounts, partly from research reports and partly from producers or dealers in safety equipment. Good estimates of cost have not always been found. The cost figures presented are usually an estimate of the average cost for a ‘unit’ of a measure, for example, 1 km of track for walking and cycling, one roundabout, one signalised junction, one seat belt, one set of ABS brakes, etc. In addition, total costs are presented for measures whose extent of usage is sufficiently well known.
Cost–benefit analysis. Examples are given of cost–benefit analysis of most measures. It is important to bear in mind that the results of cost–benefit analyses depend strongly on the context to which they refer. Monetary valuations of impacts, which are a key element of cost–benefit analysis, vary substantially between countries. As a rule, one would therefore not expect the results of cost–benefit analyses made in one country to apply directly to another country. The context to which most of the analyses presented refer is the current situation in Norway. However, where cost– benefit analyses have been reported in other countries, they are quoted. The applicability of cost–benefit analyses to road safety measures is discussed in detail in Chapter 6 of Part I.
1.4 S
CIENCE AND POLITICS IN ROAD SAFETYRoad safety research, in particular road safety evaluation research, is highly applied. This type of research is carried out mostly to help reduce the number of road accidents and the injuries resulting from them. Can science and politics be kept apart in such a highly applied field of research? Where is the dividing line between science and politics in road safety?
A distinction can be made between three types of issues that arise in policymaking. The three types of issues can be stated in the following terms:
Normative: A is a good thing (or the right thing to do). Empirical: If action B is performed, A will be produced. Prescriptive: Therefore, we ought to take action B.
Normative issues are about deciding what we think is good or right and are ultimately matters of moral judgement. Most people would probably agree that reducing traffic injury is a good outcome. Hence, most people would probably also endorse a policy objective stating that traffic injuries should be reduced.
Formulating the ideals and objectives that policy should strive to realise clearly lies within the realm of politics rather than science. Policy objectives represent human value systems and seek to articulate these in an attractive way. Does this mean that science has nothing to say about normative issues? No. A scientific evaluation of the solutions proposed to normative issues can be made by relying on principles of logical consistency. For example, a policy objective stating that every road user has the right to safer travel than the average risk faced by road users can be rejected as logically inconsistent, since it is impossible for everyone to be safer than average.
A broader scientific analysis of human value systems belongs to ethics and moral philosophy, and is outside the scope of this book. The main topic of road safety evaluation research is to determine whether road safety measures are effective in improving road safety. This is entirely an empirical issue.
It was stated inSection 1.1that this book describes, as objectively as possible, what is known about the effects of road safety measures, in particular their effects on road safety. What does this statement mean? How can any description of knowledge claim to be objective? Objectivity is not something that can be meaningfully measured in numerical terms. It is, however, an ideal of science to which this book strives by seeking to present objective knowledge about the effects of road safety measures, assessing knowledge according to standards of validity that are independent of the
content of that knowledge, and depend solely on how it was produced, and refraining from advocacy.
Let us elaborate on each of these points.
Objective knowledge. In discussing what we mean by scientific knowledge, epistemology has traditionally relied on a subjective conception of knowledge, in which knowledge is
defined as justified true belief. Within this framework, knowledge cannot exist without a knowing subject. In short, a justified and true statement does not constitute knowledge unless someone is aware of the statement and believes it.
This conception of knowledge lies close to everyday usage of the term. Hauer, for example, in discussing the state of knowledge with respect to the effects of road safety measures, states (1988, 3): ‘My own critical views about the amount of factual knowledge that is available in the field of road safety delivery rest on years of study. As I moved from one inquiry to another and began to notice how shallow are the foundations of what passes for knowledge, I gradually realized that ignorance about the safety repercussions of the many common measures is not the exception.’ Three years later, he remarked (Hauer 1991, 135): ‘How little we know about the safety consequences of our road design decisions and about the repercussions of our traffic control actions is simple to demonstrate. One needs only to ask the engineer: ‘‘Approximately how many accidents per year do you expect to occur with design X?’’ While the engineer might venture an opinion, in truth, the arsenal of knowledge at the disposal of the North American engineer just does not suffice to give an answer.’ While conforming both to everyday usage and the traditions of epistemology, the subjective concept of knowledge creates a number of difficulties. Although it makes sense to say that person A knows more about a subject than person B, if person A can pass a more difficult examination about the subject than person B, it hardly makes sense to say that the amount of knowledge that is available to the general public concerning a subject is determined primarily by how much person A can remember when undergoing an examination in the subject.
Karl Popper introduced the concept of objective knowledge (Popper 1979), which he defines (1979, 73) as ‘the logical content of our theories, conjectures, guesses’. He adds that ‘Examples of objective knowledge are theories published in journals and books and stored in libraries; discussions of such theories; difficulties or problems pointed out in connection with such theories, and so on.’ Knowledge in the objective sense, according to Popper (1979, 109), is knowledge without a knower; it is knowledge without a knowing subject.
In short, the concept of objective knowledge can be defined as all results of research, theoretical or empirical, that are available to the general public by virtue of being written or otherwise stored in a medium that is accessible to anyone who wants to learn its contents. Knowledge in this sense exists, as pointed out by Popper, in the shelves of libraries and archives. This kind of knowledge is objective in the sense that it exists irrespective of whether anyone keeps it inside his or her head. It is, however, not necessarily objective in the sense that everyone who reads a certain paper in a journal
will find the results reported in the paper convincing and therefore believe them, as required according to the subjective conception of knowledge.
This book seeks to develop objective knowledge about the effects of road safety measures by relying on an extensive and systematic search of the literature, described in detail in Chapter 2, and by summarising this literature by means of formal techniques of meta-analysis that minimise the contribution of subjective factors that are endemic in traditional, narrative literature surveys.
Assessing the validity of knowledge. Can the results of road safety evaluation studies be trusted? Do these studies always show the true effects on road safety of the measures that have been evaluated? Regrettably, the answer to these questions is no. Hauer (2002, 3)laments: ‘By publishing many biased accounts on a variety of treatment, all giving inflated estimates of safety effect, one creates an entirely incorrect lore about what is achievable. . . . The publication of incorrect results is like the release of toxin into a pristine body of water. It does not take much to make an entire lake unfit for drinking. . . . The remedy to knowledge pollution is not reader education. While it is useful to educate potential readers to assess critically the results of safety studies, it is too much to hope that reader education can undo the damage done by publishing poorly done research.’ In this book, a systematic framework has been used to assess the validity of the studies that are quoted. This framework applies to published or at least written studies, and not to oral communications, personal beliefs, tacit knowledge or other forms of subjective knowledge.
Checking studies according to a set of criteria of validitymay be regarded as an overly restrictive and simplistic way of assessing the validity of knowledge. Three points can be made in defence of this approach. First, the set of criteria for assessing the validity of evaluation studies are intended as normative criteria, not as descriptive criteria. All too often, controversies about research revolve around the contents of the results, rather than the methodological rigour of the research, and are heavily influenced by vested interests, rather than a disinterested search for the truth (seeCrossen 1994, for some striking examples of these tendencies).
Second, it is conceded that a set of normative criteria is bound to be incomplete, in the sense that it does not exhaust the considerations that are regarded as relevant in assessing the validity of studies. Some considerations about study quality may apply just to one particular study and are thus not easily stated in general terms.
Third, while an informal and subjective assessment of the validity of research can reflect considerations that are difficult to formalise, it is nevertheless likely to be subject to more or less unknown biases. No matter how hard we try to be objective, there is
always a risk that we go by the rule that ‘bad studies are . . . those whose results we do not like’ (Rosenthal 1991, 130). By assessing validity in terms of formally stated, normative criteria, the role of personal prejudices in the assessment can be minimised.
Refraining from advocacy. Suppose an effective remedy for road accidents is found. Surely that is good news. Let us apply the remedy at once. Advocacy in research reports refers to statements recommending or calling for the use of specific road safety measures. To offer policy recommendations is to engage in advocacy. While advocacy may be tempting to many researchers (‘Hey look, I’ve found a wonderful solution to an important social problem! Please give me some applause’), it is a temptation that should be resisted. Let us explain why.
In the first place, advocacy will, at least in the long term, undermine the confidence in research. Many road safety measures are controversial. The fact that a certain road safety measure is effective does not always mean that people like it. A researcher who has repeatedly advocated lower speed limits to improve road safety will find his credibility greatly reduced next time he publishes a study that, once again, concludes that lowering speed limits is an effective way of improving road safety.
In the second place, there is nearly always more than one way of improving road safety. Treatment A may be effective for a particular accident problem, but so are treatments B, C, D, E and F. To choose between these treatments, policymakers need to know more than simply the fact that they are all likely to reduce the number of accidents. Perhaps costs differ greatly. Perhaps the impacts on mobility and the environment are different. Perhaps public opposition is strong to three of the measures, but not to the other three. And so on. In short, making road safety policy involves complex trade-offs that tend to be overlooked by those who advocate a particular road safety measure. In the third place, to advocate something one should really be sure that it works. If knowledge is not firmly established, one can get nasty surprises when introducing a treatment that was erroneously believed to be effective. Unfortunately, knowledge about the effects of road safety measures is not always very firmly established. Some readers may object to these arguments by saying that this book offers covert policy recommendations by presenting cost–benefit analyses of the road safety measures it covers. However, a cost–benefit analysis is not a policy recommendation. It is simply a way of showing, in terms of a common scale, the relative importance of various impacts of a programme. Trying to identify the practical implications of a cost–benefit analysis is not as straightforward as some people think. It is not the case that an action should always be adopted if the benefits of that action are greater than
its costs, and should never be adopted if the costs are greater than benefits. This point is made in virtually every textbook on cost–benefit analysis. Moreover, it is not obvious that road safety policy can or ought to be based slavishly on the results of cost–benefit analyses. To determine the weight that cost–benefit analysis should carry in road safety policy requires judgements that must be made outside the framework of cost–benefit analysis, and are not part of the analysis as such.
L
ITERATURE
S
URVEY AND
M
ETA
-A
NALYSIS
2.1 S
YSTEMATIC LITERATURE SEARCHA comprehensive survey of studies evaluating the effects of road safety measures has been made. These studies have been identified by means of a systematic literature search. This section describes how the literature search was done.
The literature search consists of a ‘fixed’ part and a ‘variable’ part. The fixed part is a comprehensive search for studies in a sample of sources. The variable part is based on the results of the fixed part of the search. This approach is sometimes referred to as the ancestry approach. The fixed part of the literature search is a systematic survey of the following main groups of sources:
Previous Norwegian editions of Handbook of Road Safety Measures Scientific journals
Reports issued by selected research institutes
Conference proceedings from a sample of regular conferences The library of the Institute of Transport Economics
Bibliographical databases.
The variable part of the literature search comprises references found in studies that were retrieved from these sources.
Previous Norwegian editions of Handbook of Road Safety Measures. Previous editions of this book have been published in Norwegian and in English. The previous editions
The Handbook of Road Safety Measures
Copyright r 2009 by Emerald Group Publishing Limited All rights of reproduction in any form reserved
of the book (Pedersen, Elvik and Berard-Andersen 1982,Elvik, Vaa and Østvik 1989, Elvik, Mysen and Vaa 1997, Elvik and Vaa 2004) have been examined, and we have tried to obtain studies to which references were made. No studies that have been referred to in the earlier editions of the book have been omitted. Even though the first edition of the book refers to many studies that by now are relatively old (over 30 years), none of these studies have been omitted. There are two main reasons for this. First, by keeping old studies, one has the opportunity of finding whether new and old studies reach the same conclusions. Second, the research is cumulative. This means that new studies are based on and add to the results of older studies, but attempt to refine, confirm, falsify, or develop these results by replicating studies or by applying better research methods.
Scientific journals. A number of scientific journals has been hand-searched and relevant papers have been identified.Table 2.1shows the journals that have been searched and the volumes included for each journal.
The journals that were judged to be the most important have been examined from around 1970 or from the first published volume. Less important journals have been searched from 1980. Highway Research Record ceased publication in 1974 and was replaced by Transportation Research Record.
Reports issued by research institutes. Reports issued by a number of research institutions and public agencies in different countries have been searched. Table 2.2 shows the institutions whose publications have been systematically surveyed in the literature search.
Volumes included for the different series of reports issued by these institutions largely cover the period for which the report series in question has been in existence. For report series that were regarded as less important, only volumes from after 1980 have been studied.
Conference proceedings. Every year, or at other fixed intervals, a number of international conferences or seminars are held that deal with the questions of road safety. Normally, conference proceedings, which contain the contributions to these conferences, are published. For conferences that are held regularly, the proceedings from conferences in recent years have been searched systematically.Table 2.3 shows the conferences concerned.
In addition to these regular conferences, a number of other conferences are held. Proceedings of these conferences have been obtained if there was reason to believe they might contain relevant papers.
Literature search in the library of the Institute of Transport Economics. Literature searches have been made in the library of the Institute of Transport Economics using subject words. These searches were done on a supplementary basis, designed to identify studies that were not found in the other sources that were searched systematically. Bibliographical databases. Literature searches have been carried out using several international bibliographical databases. These are ROADLINE at VTI (Swedish Road Table 2.1: Scientific journals surveyed as part of the literature search
Journal Volumes included
Accident Analysis and Prevention 1969–
Australian Road Research (ceased publication in 1991) 1970–91 Dansk Vejtidsskrift (Danish Road Journal) 1980–
Ergonomics 1980–
Highway Research Record (ceased publication in 1974) 1960–74
Human Factors 1980–
IATSS Research 1980–
ITE-Journal (formerly Traffic Engineering) 1970–
Journal of Risk and Uncertainty 1988–
Journal of Safety Research 1969–
Journal of Traffic Medicine 1974–
Journal of Transport Economics and Policy 1970– Journal of Transportation Engineering 1970–
Nordic Road and Transport Research 1989–
NTR-nytt (News from Nordic Research) 1992–
Policy Sciences 1980–
Public Roads 1980–
Recherche-Transports-Se´curite´ (RTS – INRETS Research Review) 1984–
Risk Analysis 1981–
Samferdsel 1970–
Safety Science (formerly Journal of Occupational Accidents) 1980–
Strassenverkehrstechnik 1980–
Traffic Engineering and Control 1970–
Transportation Research Part F 1998–
Trafikken og Vi 1970–
Transportation Research (series A and B) 1980–
Transportation Research (series C) 1993–
Traffic Injury Prevention 1999–
Transportation Research Record (replaced Highway Research Record) 1974–
Table 2.2: Institutions (listed alphabetically) whose publications have been searched in literature survey
Institution Period covered
Australian Road Research Board (ARRB, Australia) 1970– Beratungsstelle fu¨r Unfallverhu¨ting (BFU, Switzerland) 1980– Bundesanstalt fu¨r Strassenwesen (BASt, Germany) 1974–
Danmarks Transportforskning (DTF) 2001–
Kommunikationsforskningsberedningen (KFB, TFB, TFD, Sweden) 1977– Lunds Tekniske Høgskole (Lund Institute of Technology, Sweden) 1977– Nordisk Ministerra˚d (Nordic Council of Ministers, Nordic countries) 1973– Nordisk Vegteknisk Forbund (NVF, Nordic Road Federation, Nordic countries) 1970– Organization of Economic Cooperation and Development (OECD) 1970– Ra˚det for Trafiksikkerhedsforskning (Danish Council for Road Safety Research, Denmark) 1969–2001 SINTEF Samferdselsteknikk/NTH Samferdselsteknikk (Norwegian Institute of Technology, Norway) 1975– Society of Automotive Engineers (SAE, USA) 1980– Statens vegvesen (Public Roads Administration, Norway) 1980– Statens Va¨g- och Trafikinstitut (VTI, Swedish Road and Transport Research Institute, Sweden) 1975– SWOV (Institute for Road Safety Research, The Netherlands) 1970– TØI (Institute of Transport Economics, Norway) 1963– Transport Research Laboratory (TRL, TRRL, RRL, Great Britain) 1965–
US Department of Transportation (USA) 1980–
US Transportation Research Board (TRB, USA) 1960– Vejdirektoratet (Public Roads Administration, Denmark) 1980– Va¨gverket (National Roads Administration, Sweden 1980–
Table 2.3: Conference proceedings which have been studied as part of the literature search
Conference (frequency) Year
Alcohol, Drugs and Traffic Safety (every 3 years) 1971– Australian Road Research Board Conference (every second year) 1980– PTRC Summer Annual Conference (now: European Transport Forum, annual) 1985– Road Safety in Europe (VTI et al.) (every second year) 1985– Road Safety on Four Continents (VTI and TRB) (every second year) 1985–
TRB Annual Meeting (annual) 1985–
and Transport Research Institute), OECD’s database IRRD, the database TRANS-PORT (Silverplatter), Sciencedirect (the online database from Elsevier), PubMed (of the US National Library of Medicine) and the Cochrane Library.
A large number of road safety evaluation studies have been found in the sources listed above. Many of these studies refer to other studies, which were obtained if the references appeared to be relevant. Relevance was judged according to study titles and abstracts (if available). This approach to searching the literature does not guarantee 100% coverage. We do believe, however, that we have retrieved a large proportion of the best road safety evaluation research that has been published.
2.2 C
RITERIA FOR STUDY INCLUSIONThe main objective of the literature search was to find studies that have quantified, or at least have tried to quantify, the effect of one or more road safety measures on the number of accidents, accident rate and the number of injuries or risk of injuries. Studies that have evaluated the effects of road safety measures by relying on proxy measures for safety, such as conflicts between road users or changes in road user behaviour, rather than accidents or injuries, are less relevant. One reason for this is the fact that for many forms of behaviour, the relationship to accident occurrence is unknown. Another reason is that the ultimate objective of all road safety measures is to reduce the expected number of accidents or injury severity.
This does not mean that measurements of road user behaviour, for example, are not of interest. On the contrary, they can make a study more valuable by supplementing accident records. For example, the validity of a study is greater if it describes changes in both speed and accidents – and shows that these changes are closely related to each other – than if an otherwise similar study provides information only on speed or accidents by itself.
2.3 S
TUDY CLASSIFICATIONStudies have been classified according to the road safety measure whose effects they have evaluated. Some studies have evaluated several measures and are therefore included for each of the measures evaluated. However, the majority of studies evaluate the effects of just one road safety measure.
It has traditionally been regarded as a strength if a study tried to evaluate the effects of a particular road safety measure. However, as far as road safety policy is concerned,
several measures are usually combined in one programme. In that case, it is important to know not just the effects of each measure that goes into the programme but the combined effects of all measures put together. It is not obvious that the effects of a road safety programme will be equal to the sum of the effects of the individual measures that make up the programme. The effect of a measure will not necessarily be the same when it is implemented in combination with other measures, as when it is implemented on its own.
Another general limitation of road safety evaluation research is that it often requires that the measures are implemented fairly extensively to provide enough data to evaluate effects. This means that evaluation research does not always provide a good basis for predicting the effects of new measures. Those who develop new measures would like to be able to predict the effects of the measures before they are introduced. Such prediction is not always possible. In Chapter 5, the possibility of giving a theoretical account for the findings of road safety evaluation research will be discussed.
2.4 T
HE USE OF META-
ANALYSIS TO SUMMARISE STUDY RESULTSThe results of studies that have evaluated the effects on accidents and injuries of different measures are summarised by means of meta-analysis, provided it is applicable. Meta-analysis is a quantified synthesis of results of several studies that have evaluated the same road safety measure stated in the form of a weighted mean estimate of effect (Elvik 1999). As a part of the meta-analysis, moderating factors are investigated that influence the size of the effect of a road safety measure on accidents or injuries. There are a number of textbooks on meta-analysis (Cooper and Hedges 1994,Petitti 2000,Lipsey and Wilson 2001) that describe various techniques in detail. Here, only the main elements are described to help readers understand the results that are presented in the individual chapters.
Main elements of meta-analysis. The study unit in a meta-analysis is a result, or an estimate of effect. An estimate of effect has to be stated as a precise point estimate in order to be included in a meta-analysis. If a result is stated simply as: ‘No statistically significant changes in the number of accidents were found’, it cannot be included in a meta-analysis. Moreover, the standard error of an estimate of effect has to be known, at least if results are to be weighted according to their statistical precision. A single study can contain more than one result. In such cases, all results, or the most important results from studies with a very large number of results, have been included in the meta-analyses. Multiple results from the same study have been treated as statistically independent, although this assumption may not always be correct.
Study results can be summarised by means of meta-analysis if the studies
provide at least one numerical estimate of the effect of a road safety measure, or provide information that can be used to derive such an estimate and
state the number of accidents on which the estimate of effect is based or provide other information that allows the calculation of the statistical uncertainty of the effect estimate, such as the confidence interval.
Basics of the log odds method of meta-analysis. The log odds method of meta-analysis has been applied throughout (Fleiss 1981,Shadish and Haddock 1994). According to this method, a weighted mean estimate of effect is calculated on the basis of the estimates of effect found in the studies that have been retrieved. This method of meta-analysis was chosen because the odds ratio (OR) is the most commonly found estimate of effect in road safety evaluation studies. An example of how an OR is calculated is as follows: If a study finds that there were 75 accidents on road X before a measure was implemented, and 23 accidents afterwards, whereas on a comparison road, there were 67 before the implementation of the measure on road X and 25 afterwards (no measure was implemented on the comparison road), the OR is (23/75)/(25/67) ¼ 0.307/ 0.373 ¼ 0.822. This corresponds to an accident reduction of 17.8% (1 þ 0.822). In studies that employ multivariate techniques of analysis, effects are normally stated in terms of an OR that has been adjusted for confounding.
When applying the log odds method of meta-analysis, a summary effect is calculated as the weighted mean of the logarithms of the individual estimates of effect (ORs). Combining logarithms of ORs yields an unbiased estimate of the weighted mean effect of a set of studies. The steps in a log odds meta-analysis are
calculation of estimates of effect,
calculation of statistical weights and choice of the model of meta-analysis: Fixed effects when there is no systematic variation in the estimates of effect or random effects when there is systematic variation in the estimates of effect,
calculation of summary estimates of effect, and
confidence intervals: for each summary effect, a 95% confidence interval is calculated.
Calculation of estimates of effect. Estimates of effect are calculated as ORs. Some of the estimators of effect commonly found in road safety evaluation studies are listed in Table 2.4. The list is not exhaustive. Estimates of effect based on coefficients produced by multivariate analyses, which have the statistical properties of ORs, are not as common, but have increasingly been used in recent studies. The different estimators of effect should not be mixed up. Producing summary estimates of effect in meta-analysis
based on studies that employ different estimators of effect can be misleading because both the statistical properties and the substantive interpretations of the various estimators differ. When other estimates of effect other than ORs are reported, ORs are calculated as far as possible based on the available information.
Calculation of statistical weights and choice of model. There are two methods of combining estimates of effect in meta-analysis, the fixed effects model and the random effects model. The fixed effects model of analysis is based on the assumption that there is no systematic variation in effects in the set of studies considered, that is, all estimates of effect are samples of the same ‘true’ effect. When there is systematic variation, or heterogeneity, in the estimates of effect, the estimates cannot be regarded as representing the same ‘true’ effect. In this case, a random effects model is more adequate. In a random effects model, an account is taken of heterogeneity in the results and an underestimation of the uncertainty of the summary effect is avoided.
The differences between the fixed effects and the random effects models can be summarised as follows: The fixed effects model is adequate only if there is no heterogeneity in the results. Otherwise it will assign too much weight to results with large statistical weights and the confidence interval of the summary effect will be underestimated. The random effects model can be applied whether or not there is heterogeneity in the results. When there is significant heterogeneity, it assigns relatively less weight to results with large fixed effects weights, and confidence intervals of summary effects are larger than that in the fixed effects model. The less heterogeneity there is in the estimates of effect, the more similar will be the results from the two models.
When applying fixed effects and random effects models in meta-analysis, they differ with respect to how the statistical weights are calculated. In the fixed effects model, the Table 2.4: Commonly used estimators of effect in road safety evaluation studies
Name of dependent variable Formal definition
Odds Uat/Ubt
Odds ratio (simple or adjusted) (Uat/Ubt)/(Uac/Ubc)
Ratio of odds ratios [(Uati/Ubti)/(Uaci/Ubci)]/[(Uatj/Ubtj)/(Uacj/Ubcj)] Ratio of relative risk [Uati/(UatiþUbti)]{[Uatj/(UatjþUbtj)]
Accident rate ratio (Ua/Ta)/(Ub/Tb)
U ¼ number of accidents, T ¼ traffic volume, exposure to risk, a ¼ after, or with, some measure whose effect is evaluated, b ¼ before, or without, some measure whose effect is evaluated, t ¼ test group, c ¼ comparison group, i ¼ category I, j ¼ category j.
statistical weight of the natural logarithm of each effect estimate is the inverse of its variance:
wi¼
1 vi
The variance of the logarithm of the OR is vi¼ 1 Aþ 1 Bþ 1 Cþ 1 D
where A, B, C and D are the four numbers that enter the calculation of the estimate of effect. In studies that do not use comparison groups, the terms 1/C and 1/D drop out. The same applies to studies that state the effects of a road safety measures in terms of an accident rate ratio. Statistical weights are estimated on the basis of the recorded number of accidents. In case of zero accidents, 0.5 is added to all four (or two) numbers used in estimating the statistical weight of a result.
In a random effects model, the statistical weights are calculated as a function of the fixed effects weights and a measure of the heterogeneity in the estimates of effect. The more heterogeneity there is in the results, the more similar will the statistical weights of the estimates of effect become, that is estimates based on large fixed effects weights will have their weights adjusted more than estimates based on small fixed effects weights.
In order to test the amount of heterogeneity in the estimates of effect, the following test statistic, Q, is estimated: Q ¼X g i¼1 wiy2i Pg i¼1 wiyi 2 Pg i¼1 wi
where yiis the estimate of effect i and withe fixed effects weight of estimate i. This test
statistic has a w2distribution with g1 degrees of freedom, where g is the number of estimates of effect that have been combined. If this test statistic is statistically significant, a random effects model is more adequate than a fixed effects model. In a random effects model, the statistical weights are modified to include a component reflecting the systematic variation of estimated effects between cases. This component is estimated as follows (Shadish and Haddock 1994):
t2¼Q ðg 1Þ C
Qis the test statistic described earlier, g the number of estimates and c the following estimator: c ¼X g i¼1 wi Pg i¼1 w2i Pg i¼1 wi 2 6 6 4 3 7 7 5
The variance of each result now becomes vi ¼t2þvi
The corresponding statistical weight becomes the inverse of the variance.
Random or fixed effects?Most meta-analyses that are presented in the book have been calculated based on a random effects model. Fixed effects models have been applied only when too few estimates of effect are available for calculating a random effects model. In meta-analyses that have not been updated after 1997, the fixed effects model is the most commonly used model.
Summary effects. The weighted summary effect based on a set of g estimates is calculated as follows: y ¼exp Pg i¼1 wiyi Pg i¼1 wi 0 B B @ 1 C C A
where ‘exp’ is the exponential function (i.e., 2.71828 raised to the power of the expression in parenthesis), yi the logarithm of each estimate of effect and wi the
statistical weight of each estimate of effect.
Confidence intervals. A 95% confidence interval for the weighted mean estimate of effect is obtained according to the following expression:
95% confidence interval ðupper=lower limitÞ ¼ exp Pg i¼1 wiyi Pg i¼1 wi 0 B B @ 1 C C A 1:96 1 ffiffiffiffiffiffiffiffiffiffiffi Pg i¼1 wi s 2 6 6 6 6 4 3 7 7 7 7 5
The weights in this expression are either the fixed effects weights or the random effects weights, depending on the model of analysis adopted.
2.5 D
OES A WEIGHTED MEAN ESTIMATE OF EFFECT MAKE SENSE?
A concern that many people have about meta-analysis is the so-called apples and oranges problem. This refers to the fact that studies that may differ greatly among themselves are combined into an overall estimate of the average effect of a road safety measure. It is argued that this does not make sense if studies are very heterogeneous, for example, with respect to different versions of the measure, countries or methods used in the studies.
Fortunately, the relevance of this argument can to some extent be tested in a meta-analysis. By doing so, one gains an impression of how meaningful it is to generalise a set of findings of evaluation studies in terms of a weighted average result. A way of checking whether a weighted mean estimate of effect makes sense is to prepare a funnel graph plot. An example of such a graph is shown inFigure 2.1.
The graph shows 94 results of studies that have evaluated the effects of road lighting on the number of accidents. The horizontal axis shows the natural logarithms of the estimates of effect. Values below 0 mean that the number of accidents is reduced, the
0 100 200 300 400 500 600 700 -3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0
Effect estimate (natural logarithm; 0 = no effect)
Statistical weight (fix
ed eff
ects)
Summary effect (fixed effects): -0.194 Arithmetic mean: -0.292
Median: -0.319
Figure 2.1: Funnel graph of studies that have evaluated the effects of road lighting on the number of accidents (unspecified severity).
value 0 means that the number of accidents is unchanged and values above 0 mean that the number of accidents increases. The vertical axis shows the statistical weight (fixed effects) of the results. The greater the statistical weight, the more the accidents which form the basis of a result. The dots indicate the individual results. Furthermore, three measures of the main tendency of the results are shown: the median, the arithmetic (unweighted) mean and the summary effect that has been calculated with the fixed effects model.
By studying such funnel graphs, an informed opinion can be formed of how reasonable a weighted mean result is. Properties of the distribution of estimates of effect that are investigated based on the funnel graph are the modality and dispersion of the results, the skewness and the sensitivity to outliers.
Modality and dispersion of the resultsrefers to the shape of the distribution of estimates of effect and how many humps or peaks it has. Figure 2.1 shows a unimodal distribution, that is, a distribution where the data points gather round a single peak. In this type of distribution, the weighted summary effect lies close to the highest peak of the distribution and thus is representative of the centre of gravity of the distribution. A bimodal distribution is one that has two peaks. In this type of distribution, the average will often lie between the two peaks and thus will not really be very informative. If possible, bimodal distributions should be divided into two, and an average should be calculated for each mode.
There may also be distributions with no clear pattern at all, randomly scattered distributions. In these types of distributions, the results are highly dispersed, with no clear tendency in any direction. An average may then be arbitrary and any differences concealed as a result of arbitrary assignment would be important to highlight. Ideally, the distribution of the results should not only be unimodal but also exhibit a systematic pattern where the results that have the largest statistical weights are closest to the mean and results that are further away from the mean have smaller statistical weights. It is not always easy to see if the results follow an ideal distribution or not. There are statistical methods for investigating the distribution of the results and for treating results that are not ideal.
First, heterogeneity can be tested statistically as has been described earlier, and a random effects model can be applied that takes into account heterogeneity. A random effects model takes into account that there is heterogeneity, but does not explain it. Second, there are possibilities for explaining heterogeneity. The simplest way is to divide results into groups and to calculate new summary effects for each of the sub-groups of results. Results may be grouped, for example, according to injury severity or
variants of the measure. When summary effects differ between sub-groups, and when heterogeneity is reduced within the sub-groups, the sorting variable is likely to have contributed to the heterogeneity. It is then called a moderator variable.
Heterogeneity can also be explained by using meta-regression. In meta-regression analysis, regression models are developed on study level with the estimates of effect as dependent variable and characteristics of the studies as predictors. Characteristics of the studies may be the same variables as the sorting variables in the sub-group analysis (e.g., type of measure investigated, type of roads, methodological aspects, and so on). Thereby, it is possible to investigate which characteristics of the studies affect the outcome of the studies, while controlling for several factors at the same time. One restriction of meta-regression is that it requires quite large numbers of estimates of effect. As a rule of thumb, there should be at least 10 estimates of effect for each predictor included in the model. When there are few estimates of effect, the results may be arbitrary and highly sensitive to, for example, adding or omitting predictors or individual estimates of effect from the analysis.
A third possibility that should be considered in some cases is to refrain from calculating a summary effect. When the distribution of estimates of effect is highly heterogeneous without showing any signs of unimodality, a summary effect would not be meaningful. Indications for such a distribution are results that are highly different between the fixed effects and the random effects model and extremely large confidence intervals in the random effects model. This is illustrated by a numerical example in which six estimates of effect have been generated that have a highly heterogeneous and non-unimodal distribution. In this example, the result from the fixed effects model is a summary effect of 57% (95% confidence interval [58; 55]), and that from the random effects model is a summary effect of þ6% (95% confidence interval [62; þ195]).
Skewness in a distribution refers to how the data points are distributed around the average, that is, how the individual results distribute themselves around a weighted average result. Ideally, the distribution should be symmetrical around (the natural logarithm of) the summary effect. If a distribution is very skew, the mean will give a misleading impression of where the majority of the results lies. An indication of skewness is a large difference between the median and the arithmetic mean of the distribution. An unskewed distribution will have very similar median and arithmetic mean.
Publication biasis one possible source of skewness. Publication bias means that studies are more likely to be published when the results are in accordance with the expectation. In most accident studies, the expectation is that one will find accident reductions following the implementation of a safety measure. Publication bias leads to a skewed
