Offending Behaviour Programmes

(1)

Offending Behaviour Programmes

Source document

Author: Per-Olof Wikström and Kyle Treiber

www.yjb.gov.uk

(2)

Background

This review was commissioned to serve as a background source document to accompany guidance produced by the Youth Justice Board for England and Wales (YJB), identifying key elements of effective practice in interventions in the youth justice context. It has not been written primarily for an academic or research audience, but for managers and practitioners working in the youth justice field who are directly involved in providing, or brokering access to, services for young people who offend and their families. The review offers an accessible guide to the current state of the evidence base on effective interventions and services, helping youth justice practitioners and managers to be aware of and deliver more rigorously evidence-based services.

The review document is divided into sections structured around a number of key themes or headings relevant to practice in youth justice services. The source document is

structured to mirror the Key Elements of Effective Practice to facilitate cross-referencing between the two documents, and to ensure it is a useful document for the intended audience who may wish to explore the areas covered in the Key Elements of Effective Practice summary in more depth. These nine common sections therefore reflect what are considered to be core areas of consideration for practice and management within youth justice. The following Key Elements of Effective Practice titles and corresponding source documents are available from the YJB website (www.yjb.gov.uk):

Accommodation

Assessment, Planning Interventions and Supervision

Education, Training and Employment

Engaging Young People who Offend

Mental Health

Offending Behaviour Programmes

Parenting

Restorative Justice

Substance Misuse

Young People who Sexually Abuse.

(5)

Introduction

The ultimate aim for the youth justice system is to prevent offending and reoffending by children and young people. Achieving this goal requires the use of effective and

properly implemented and delivered prevention and intervention programmes, of which offending behaviour programmes are one chief example.

The main purpose of this source document is to present a systematic review of research from the past five years regarding the effectiveness of cognitive behavioural and multi- systemic offender-oriented crime prevention programmes for young people. The review was limited to these types of interventions to avoid overlap with other reviews in the series, and because previous reviews have shown that cognitive-behavioural and multi- systemic interventions tend to be the most promising for young people who offend (Izzo and Ross, 1990; Latimer et al, 2003; Lipsey, 1995; Lösel, 2001; MacKenzie, 2006). We systematically reviewed the literature related to these types of interventions, published or unpublished, which became available between 2001 and 2006, in order to undertake the most complete and up-to-date examination of the evidence base possible. The review identified more than 500 articles, including 300 evaluations of cognitive behavioural and multi-systemic interventions (see Appendix B for a description of the methodology employed). For the purpose of this review, we define a young person who offends as someone aged between10 to 17 years old.

The main objectives of this document are to review:

evidence on whether these kinds of programmes and interventions have proven effective or promising

existing knowledge about the extent to which these programmes’ effectiveness varies for different kinds of young people when introduced in different kinds of social circumstances

our knowledge about any (other) factors that have proven important for the most effective implementation and delivery of these programmes and interventions, and any factors that may contribute to their failure.

In the document we will also provide an initial overview of some key assumptions and problems regarding:

the assessment of the effects of prevention programmes and interventions based on reviews of conducted evaluation research

the use of such knowledge to inform policy and develop practice.

This will illustrate the limitations and problems youth justice managers and practitioners must take into account in order to apply this knowledge to best effect when developing policy and practice.

To ensure the best possible crime prevention and reduction outcomes for young people who offend, the fundamental (and equally important) main tasks for youth justice managers and practitioners are:

(6)

to make sure they only use programmes and interventions that have proven effective or promising¹

to ensure that such programmes and interventions are fully implemented, delivered and sustained as intended.

Achieving this requires that youth justice managers and practitioners have adequate user competence, that is (i) adequate knowledge about different kinds of programmes and interventions’ effectiveness or promise (managers); (ii) adequate skills and training to effectively implement, deliver and sustain such programmes and interventions as intended (managers and staff). User competence also involves adequate knowledge about (iii) various programmes and interventions’ cost-effectiveness, which enables informed judgements about whether a programme is cost-effective, or which is the most cost-effective among competing programmes and interventions, which target the same causal factors or processes (managers).

To build and ensure user competence among youth justice managers and practitioners, and to effectively implement and deliver promising prevention programmes or

interventions for young people who offend, requires adequate human and financial resources (e.g. for recruitment, training, supervision and continued assessment of outcomes). Without adequate resources and user competence, no prevention programme or intervention is likely to succeed.

1 With the obvious exception of those cases in which they are involved in demonstration projects to test new programmes or interventions.

(7)

Addressing the causes of young people's crime: some important basics about causation and prevention

Human behaviour (like acts of crime and disorder) is an outcome of the interaction between an individual’s characteristics and experiences and the features of the

environments in which he/she operates (Bronfenbrenner, 1979; Magnusson and Statin, 1998; Wikström, 2006). To change a young person’s behaviour one can, in principle, either change the individual or change the environments in which he/she operates (or both).

For some young people, individual characteristics and experiences may be the most important factor influencing their problematic behaviour; for others, it may be the environment in which they operate (Wikström and Butterworth, 2006). Therefore, to design an effective prevention programme or intervention, one needs to know what key individual and/or environmental factors to target, both generally and for specific groups of young people who offend, or in particular kinds of social contexts.

This requires:

an understanding of what causes (directly and indirectly) young people’s offending

knowledge about how we can manipulate these causes (or their causes) to reduce or eliminate the behaviour in question.

Direct causes move a young person to engage in an act of crime. Indirect causes affect the causes that move a young person to engage in an act of crime. For example, if a young person’s moral judgements and habits are factors that contribute to his/her engagement in an act of crime, they are examples of direct causes, while those factors that help mould that young person’s moral values and habits (such as his/her moral education and experience of certain moral contexts) are examples of indirect causes.

Indirect causes may be thought of as the ‘causes of the causes’ (see Wikström, 2007a for further discussion).

Treating the problems of prediction and causation as separate problems Anyone who implements a programme or intervention with the aim of preventing a specific behaviour assumes, or should assume, that the programme or intervention addresses (at least) some of the factors that (directly or indirectly) cause the behaviour in question. Without causation there can be no prevention (Wikström, 2007a). To prevent something is to make something not happen that would otherwise have

happened (the idea of preventing a young person’s offending is the idea that the young person would have committed crimes, or more crimes, if the intervention hadn’t taken place).

Prevention programmes or interventions are based on causal hypotheses (regardless of whether or not these are made explicit). Effective prevention occurs as a result of an intervention that:

removes or suppresses a (direct or indirect) cause

successfully interferes with, or alters, a causal process (i.e. the process that links the cause and the effect and produces the effect).

(8)

In the latter case, prevention of offending and reoffending is about counteracting the influence of a cause (or causes) in operation; an example would be interventions, which aim to counteract the criminogenic influences of an individual’s cognitive deficiencies.

Successful prevention thus implies that the prevention programme or intervention has effectively removed or suppressed a cause or interfered with or altered a causal process.

A test of a prevention programme or intervention may therefore, in principle, be viewed not only as a test of whether or not the particular programme or intervention works, but also a test of the correctness of the underlying causal hypothesis.

In this context it is important to highlight that what predicts is not necessarily what causes. The fact that we can use of range of variables (such as those collected by assessment and prediction tools like Asset²) to predict a young person’s crime

involvement, or future risk of crime involvement, does not mean that the factors that predict are also the factors that cause a young person’s criminality.

This problem is particularly common in some applications of the popular risk factor approach and the related strategy of risk-focused prevention. When a prevention strategy focuses on reducing as many risk factors as possible (risk-focused prevention) without properly addressing the extent to which the identified risk factors (stable predictors) are causes or mere correlates, there is a danger the prevention efforts will deal largely with factors that are markers or symptoms rather than causes, and waste valuable time and resources on their implementation and testing (Wikström, 2007a). It is therefore essential to take the problem of causation seriously when aiming to create effective prevention for young people’s involvement in crime.

Treating how best to establish causes, and how best to manipulate these causes, as two separate problems

Even if we have knowledge about the key causes of young people’s involvement in crime, this does not necessarily imply that we know how to successfully manipulate them in order to change a young person’s behaviour. Establishing that a factor causes (or contributes to) the occurrence of an undesired behaviour is not the same as establishing how we can effectively intervene to affect the particular cause.

If an intervention fails to work, this may be due to the fact that:

we did not target a cause (or causal process)

our intervention was not designed (or implemented, delivered and sustained) in a way that effectively influenced the cause (or causal process).

One implication of this is that even if we try one method of manipulating a putative cause (or causal process) and it fails, it does not necessarily mean that the targeted cause (or causal process) is unimportant. We might still be right about the significance of a cause, but mistaken about the potential of our particular intervention to affect the cause (or causal process). There may, for example, be other ways (perhaps by modifying our original approach) to influence the particular cause (or causal process). This is

2 It should be noted that in the case of Asset information on risk factors is based on assessments by practitioners concerning ‘the extent to which they think the problems they have identified are associated with a risk of reoffending’ (Baker, 2004:74, authors’ emphasis). Practitioners draw these conclusions using data collected about and from young people who offend, which is thought to relate to their offending.

(9)

important to bear in mind because there is always a danger that we may wrongly

conclude, on the basis of a failed intervention, that the targeted cause (or causal process) was not important.

Moreover, some causal processes may be irreversible (or difficult to reverse) so that when an effect has appeared, it cannot be reversed (or only reversed with great difficulty) by manipulating what caused it to appear in the first place – so-called, asymmetric causation (Wikström, 2007b). It seems likely that many developmental processes relevant to the characteristics or experiences that influence a young person’s current crime involvement may have the characteristic of asymmetric causation. For example, certain childhood experiences (e.g. malnutrition or receiving repeated severe blows to the head from a parent), may have influenced the emergence of cognitive deficiencies that affect a young person’s current propensity to commit crime. In such cases, effective intervention for these individuals may be a question of finding methods that can improve (already existing) cognitive deficiencies or limit the behavioural consequences of their expression rather than target the parent’s mistreatment of the child, which caused the deficiency in the first place (because the damage has already occurred and cannot be reversed by targeting its cause³).

An important implication of this is that for some young people (in the 10–17 age range), the individual characteristics contributing to their current problematic behaviour may have emerged before the age of 10, and crucially, these characteristics cannot always be reversed by addressing what caused them in the first place. For example, it may not always be possible to compensate for the effects of poor parental handling in childhood by aiming to improve parent behaviour later in the young person’s life.

A comprehensive strategy to address young people’s crime involvement therefore needs:

in the longer-term: to focus on preventing the emergence of frequent and serious crimes by young people by developing early prevention programmes and

interventions that target the childhood development of characteristics and experiences that later in life influence their propensity to engage in crime and disorderly conduct

in the short-term: to address the environmental and individual factors that cause young people to engage in acts of crime and disorder and influence their risk of continued prolonged offending.

In this source document we will only discuss the problem of eliminating or reducing offending (or the risk of continued prolonged offending) among young people who offend, but we feel it is important to stress the significance of early prevention since it may often (in the longer-term) be the most effective way to eliminate or reduce frequent and serious offending among young people (by preventing it from occurring in the first place).

We have already stated that human actions result from the interaction between

individuals’ characteristics and the features of the environments in which they operate.

This introduces a particular problem for assessing the effect of interventions that specifically target individual characteristics and experiences or environmental features,

3 This is of course not to say that in these cases one should not address parental mistreatment of their children, only that one should do so for different reasons.

(10)

but not the interaction between the two. For example, certain individually-oriented interventions may differ in their effectiveness for different individuals because those individuals operate in different environments, or certain environmentally-oriented interventions may differ in their effectiveness for different individuals because of those individuals’ unique characteristics and experiences.

Creating effective intervention programmes for young people’s offending requires identifying the types of interventions that work best for certain types of young people, and in what types of social circumstances.

It should be evident from the discussion so far that creating the most effective crime prevention for young people requires not only testing the effectiveness of existing programmes and interventions but, crucially, making sure we have developed

programmes and interventions that actually target the main causes (or the causes of the causes) of young people’s crime.

Because resources are limited, it is essential that the prevention programmes or interventions selected for implementation or testing have a strong theoretical rationale and empirical grounding. Otherwise:

we risk spending time and resources trying out and evaluating measures that are unlikely to produce the desired effects (or which may only produce marginal effects or even worsen the situation)

we may fail to develop and test programmes and interventions that have the potential to more effectively produce the desired effects.

The best way to avoid this problem is to make sure that the programmes and

interventions that are (i) in use or (ii) selected for implementation and testing (whether newly developed or existing) are only those based on the best available theory and empirical research regarding the problem of crime causation (and therefore have the greatest potential to be effective). Prevention programmes and interventions based on sound theory (which are properly implemented) more often prove effective or promising than those which are not. Izzo and Ross (1990:138) report from a meta-analysis of young offender programmes that ‘programs that were based on a theoretical principle were an average of five times more effective than those who had no particular

theoretical basis’.

When conducting a review of existing evaluations of interventions and programmes that meet a minimum scientific standard, it is important to bear in mind that the outcome of the review regarding the effectiveness of interventions and programmes is limited to those that have been properly evaluated. Some potentially effective programmes (or suggested improvements of existing programmes to make them potentially effective) may not yet have been properly evaluated.

Key points 1: Important basics about causation and prevention

Factors that predict offending do not necessarily cause offending.

Successful interventions address the causes of crime.

Identifying the causes of crime requires strong theoretical and empirical research.

Research indicates that interventions that are based on strong theoretical and empirical evidence are the most effective.

(11)

The causes of young people's involvement in crime

We have already highlighted that not all of the factors normally listed as risk factors (predictors) are causes of young people’s involvement in crime, and that most of them may only be symptoms or markers. This is a problem that has been clearly and

repeatedly acknowledged by one of the leading developmental criminologists (David Farrington), and needs to be taken seriously to avoid spending time and resources on programmes and interventions with little prospect of being effective.

…there is no shortage of factors that are significantly correlated with offending and antisocial behavior; indeed, literally thousands of variables differentiate significantly between official offenders and nonoffenders or correlate significantly with self-reported offending.

(Farrington, 1992:256)

…a major problem with the risk factor paradigm is to determine which risks factors are causes and which are merely markers or correlated with causes.

(Farrington, 2000:7)

…little is known about the causal processes that intervene between risk factors and offending.

(Farrington, 2003a:207) Thus, in order to better guide practice, we need to assess (to the best of our current knowledge) which among the identified risk factors (stable correlates) are likely to be causes, and which are likely to be mere symptoms and markers. It is commonplace in the offender-oriented prevention literature to talk about risk factors as criminogenic needs (but this difference in vocabulary does not change the point being made here).

We have further highlighted the importance of distinguishing between causes and the causes of the causes. When we understand the direct causes of young people’s crime involvement (i.e. what moves them to engage in acts of crime), we will be in a good position to develop and refine programmes and interventions that target these causes, and the causes of those causes, and concentrate our efforts in areas in which

interventions have the most potential to be effective.

We suggest that the current knowledge base indicates that key individual factors, which directly influence young people’s propensity to engage in crime, are their moral values (their conception of what it is right or wrong to do in given situations), and their ability to exercise self-control (to inhibit and redirect their actions). Researchers refer to these constructs by various names (e.g. pro-social values, norms, attitudes and beliefs or impulsivity, self-regulation and risk-taking, respectively) but they are widely accepted, and empirically supported, as primary individual causes of delinquent, anti-social and criminal behaviour. Moffitt, for example, has suggested, and gathered evidence to support the argument, that persistent offending that begins in early childhood arises from neurological deficits, which manifest as hyperactivity, impulsivity and low self- control (Moffitt, 1993; 1997; 2003). Gottfredson and Hirschi’s renowned general theory of crime posits low self-control as the key causal factor, individual or otherwise, that leads to offending, and a meta-analysis by Pratt and Cullen has shown this variable to be

‘one of the strongest known correlates of crime’ (Gottfredson and Hirschi, 1990; Pratt

(12)

and Cullen, 2000:952). Farrington has likewise stated that ‘hyperactivity and

impulsivity are among the most important personality or individual-difference factors that predict later delinquency’ (Farrington, 1996:8).

Farrington (1992:275) collated key risk factors for offending into six categories, three of which relate to moral values (‘internalized norms and attitudes favoring antisocial behaviour’; ‘weak conscience, low guilt or remorse, low internal inhibitions against antisocial behavior’; and ‘low empathy, emotionally cold, callous, egocentric, selfish’), and two of which relate to self-control (‘impulsivity, hyperactivity, sensations seeking, risk taking, poor ability to defer gratification’; and ‘poor ability to manipulate abstract concepts’), indicating that these two concepts are key predictors of offending (see Wikström and Sampson, 2003). In 2006, Wikström and Butterworth published evidence from a large UK cross-sectional study showing that self-control and morality were key individual factors implicated in adolescent offending.

Acts of crime are moral behaviours because they are are based upon what it is right or wrong to do in a given setting. Specifically, acts of crime are acts that break moral rules, individuals’ attitudes and beliefs about those rules, and their ability to control their actions in accordance with those rules, naturally play a key role in explaining why some individuals break moral rules.⁴

The most important environmental factor that concurrently (directly) and

developmentally (indirectly) influences young people’s engagement in acts of crime (whether or not they choose to break moral rules) is the moral context in which they operate (i.e. the characteristics of the settings they encounter that determine the rules about what is right or wrong to do in those settings, and the degree to which those rules are sanctioned) (Wikström, 2004; 2005; 2006; Wikström and Treiber, 2007). The fact that offending varies by moral context has been long established by the ecological tradition in criminological research, using concepts such as social disorder,

disorganisation and (poor) collective efficacy, to denote the varying expression of norms and their formal and informal enforcement in different contexts. Studies relating to these topics show clear evidence that differences in moral contexts may play an important role in offending (see Sampson and Raudenbush, 1999; Skogan, 1992 regarding social disorder and disorganisation; Sampson et al, 1997; Sampson, 2004; Wilson et al, 1997 on collective efficacy).

The involvement of a young person in crime may be regarded as the outcome of the interaction of his/her morality and ability to exercise self-control on the one hand, and his/her exposure to criminogenic moral contextson the other.⁵ A young person’s morality is important because it influences the extent to which he/she might consider breaking the moral rules of a setting and committing an act of crime. A young person’s ability to exercise self-control is also important because it influences the extent to which he/she can refrain from acting upon temptations and provocations in a setting to commit an act of crime. The moral contexts in which a young person develops are important

4For a review of research evidence regarding the role of self-control in crime involvement, see, for example, Pratt and Cullen, 2000. For a review of research evidence regarding the role of morality in crime involvement, see, for example, Stams et al, 2006.

5 This is not to deny the importance of broader social factors like inequality and segregation. However, we argue that the role of these factors in crime causation is best analysed in terms of, for example, how they contribute to the emergence of different kinds of moral contexts, and how they affect young people’s differential exposure to different moral contexts.

(13)

because they help to mould the development of moral values and self-control. The moral contexts to which a young person is contemporaneously exposed are also important because it is in these environments that he/she reacts to the temptations and provocations that interact with his/her morality and ability to exercise self-control to determine what action will follow (see Wikström, 2006 for further discussion on the role of moral contexts in crime causation).

A criminogenic moral context is one that promotes young people’s involvement in crime through the lack of effective communication and enforcement of moral rules consistent with the law. These are typically settings in which young people socialise, unsupervised, in environments that lack collective efficacy (environments characterised by weak social cohesion and poor informal social control) (see, for example, Osgood et al, 1996; Wikström & Butterworth, 2006).

If we are correct in our assessment of the knowledge base (i.e. our current analytical and empirical knowledge about crime causation), this means that programmes and

interventions, which aim to eliminate or reduce young people’s offending, should target those factors that influence:

the development and sustainability of morality (and supporting moral emotions), with the goal of promoting moral values (and associated moral emotions) consistent with, and supporting, abidance of moral rules defined by the law

the ability to exercise self-control, with the goal of strengthening the capability to exercise self-control (also including interventions that aim to reduce excessive alcohol and drug consumption, which can momentary affect the ability to exercise self-control)

the emergence and sustainability of criminogenic moral contexts, with the goal of reducing the prevalence of criminogenic moral contexts

young people’s exposure to criminogenic moral contexts, with the goal of minimising their exposure to criminogenic moral contexts.

Recent findings from the Peterborough Youth Study (Wikström and Butterworth, 2006) show that young people’s pro-social values, generalised ability to exercise self-control, and exposure to criminogenic moral contexts are all important factors that predict young people’s involvement in crime. Young people who lack stronger pro-social values have a poorer generalised ability to exercise self-control, often take part in criminogenic moral contexts and tend to be frequently involved in crime.

It is also interesting to note that of the existing offender-oriented programmes and interventions, those that target factors affecting people’s morality (particularly those reinforcing pro-social values) and ability to exercise self-control (those strengthening cognitive capabilities) generally show the greatest promise (see the ‘Offender-oriented programmes’ chapter).

Key points 2: The causes of young people’s crime involvement

Most risk factors are not causes of crime and therefore interventions based on the risk factor approach may not be the most successful.

Current evidence suggests that moral values (pro-social values, norms, attitudes, beliefs, etc.) and self-control (impulsivity, self-regulation, risk-taking, etc.) are the key individual factors that influence offending.

(14)

Offending is the outcome of an interaction between an individual’s moral values, ability to exhibit self-control and his/her exposure to criminogenic moral contexts.

Offender-oriented intervention programmes should target the ability to exercise self-control and factors that influence:

development and sustainment of moral values

emergence and sustainment of criminogenic moral contexts

young people’s exposure to such contexts.

(15)

The problem of demonstrating the effects of an intervention programme

A central task for youth justice managers and practitioners is to ensure the selection of offender-oriented programmes and interventions that have proven effective or

promising. The central questions for any assessment of what works have been well formulated by Bloom (2006):

What social programs, policies, and interventions work? For whom do they work, and under what conditions? And why do they work – or fall short?

(Bloom, 2006:ix)

How do we establish that an intervention has had a desired effect?

The fundamental principle of establishing causal effects is to manipulate the putative cause (or causal process) and observe what happens. If the outcome changes in

predicted ways as a result of our manipulation (intervention), we have some evidence of a causal dependency between the putative cause and the effect (and therefore some evidence that our intervention can prevent or contribute to the prevention of the undesired effect). However, successful manipulation alone is not enough to make sure that the outcome can be attributed to the manipulation. We also need to establish that without the manipulation, the observed change would not have happened (the

counterfactual question). This is why experimental designs are generally preferred.

Experimental designs, in which one compares young people exposed to an intervention with those not exposed to the intervention, and compares the outcomes (e.g. crime frequency), is the basic way to establish whether or not an intervention has an effect.

For example, if we believe that a particular intervention will reduce young people’s future involvement in crime and we compare young people who took part in the intervention with those who did not, only to find that both groups reduce their involvement in crime, we do not have much evidence that our intervention was effective. On the other hand, if those who received the treatment show reduced involvement in crime, but the controls do not, we have some evidence that our intervention may have been the cause.

However, since there may be individual differences in how young people respond to a particular intervention (e.g. depending on their individual characteristics and

experiences and/or the environments in which they operate), it is generally also necessary to randomly select (for sufficiently large samples) which young people are exposed to the intervention, and which are not, to ensure that what is measured is only the effect of the intervention (and is not confounded by any significant differences between the compared groups in any individual or environmental factors that influence the effectiveness of the intervention). In a prolonged intervention in particular, there is also a risk that the outcome may be influenced by other selectively operating causal factors than the intervention. Again, comparing randomised groups takes care of this problem because one can assume that these factors equally affect both groups (due to the randomisation). These are the main reasons why randomised experiments are generally regarded as ‘the gold standard’ of evaluation.

(16)

However, many current (in fact, most) evaluations do not constitute randomised experiments, and by including other types of evaluations we introduce more

uncertainties into our assessment of a particular intervention’s effectiveness. The crucial question for any review of evaluations is what should be the minimum scientific

criterion for their inclusion? Farrington argues (in our opinion correctly) that:

…it is not for a reviewer to attempt to review all evaluation studies on a particular topic, however poor their methodology, but rather to include only the best studies.

(Farrington, 2003b:50) The question is; what scientific threshold should we use for inclusion?

If we stick with evaluations that meet ‘the gold standard’, we can be reasonably sure (if we also verify that the programmes or interventions have been properly implemented with an adequate dosage, which may not always be the case) that our evidence has a solid foundation. The less stringent we are in selecting our criteria for inclusion, the higher the risk that our evidence is based on misleading findings and, subsequently, the higher the risk that our guidance to practice may be flawed. In an ideal world, we would base our conclusions about the effectiveness of a particular programme or intervention only on randomised experiments but since in many cases there is a shortage of

randomised experiments, we must compromise between the quantity and quality of available information.

Methodological quality scales have been developed to help address this problem. One example is the Scientific Methods Scale (sometimes also referred to as the Maryland Scale), which includes five levels (Sherman et al, 2002:16–17):

Level 1: Correlation between an intervention and an outcome at one point in time.

Level 2: A before-and-after intervention measure of the outcome.

Level 3: Comparison of a before-and-after intervention measure for one group and a before-and-after measure without any intervention for another (control) group.

Level 4: A before-and-after measure in multiple experimental and control units, controlling for other variables that influences crime.

Level 5: Randomised experiments.

The Maryland Scale does not include a common type of evaluation often used to guide practice and that is ‘evaluations’ based on the opinion of those who conduct an intervention as to whether or not it is effective (testimonials).

While it is easy to agree that evaluations based on opinion or Level 1 designs should not be considered as any evidence of the effectiveness of an intervention, and that Level 5 evaluations are the ‘gold standard’, in our opinion, the assessment of the relative scientific value of evaluations at Levels 2 to 4 is not as straightforward as the Maryland Scale implies. Sherman et al (2002:17) regard Level 3 as ‘the minimum design that is adequate for drawing conclusions about what works’, but acknowledge that this design does not solve the problem of selection effects because experiment and control groups are not randomised. The reason why Level 3 is considered a threshold is never made fully clear (but we assume it is because it includes controls).

(17)

Despite its problems, the Maryland Scale provides a useful, if approximate, guide to effective evaluation methodologies. However, see Appendix A for a more robust scale that takes intervention as well as evaluation design into consideration in valuing outcomes.

How do we determine if an intervention has been effective?

A common way to determine if an intervention has been successful is to calculate the statistical significance to see whether the differences obtained are greater than what would have been expected by chance. Although a statistically significant difference could be regarded as a minimum requirement for considering an intervention as potentially effective, it is not a strong independent criterion because statistical significance does not necessarily imply any greater effect (in large samples, even the weakest associations are significant).

To determine if a significant difference is also an important difference, one can calculate the effect size (ES)⁶. In principle, the effect size (which can be measured in different ways, such as Cohen’s d ⁷ or the correlation coefficient) is a measure of how much of the variation in the outcome can be explained by the intervention (e.g. the percentage reduction in crimes by young people as a result of the intervention).

There are no simple rules regarding what constitutes a large, medium or small effect size that can be used as a straightforward criterion for the selection of programmes or interventions. Sometimes rules of thumb are given in the literature, for example, referring to Cohen (1988), Butler et al (2006:18) states that, ‘effect sizes have been categorised along a continuum of no effect (ES < 0.2), low (0.2 ≤ ES < 0.5), medium (0.5 ≤ ES < 0.8) and high (ES ≥ 0.8)’. This translates approximately to a change of less than 15% constituting little to no effect, a change of less than 35% constituting a small to medium effect, and a change of more than 35% constituting a large effect (Cohen, 1988). However, the minimum effect size for considering an intervention worthwhile largely depends on what one regards as a useful effect, for example:

in terms of the intervention’s cost-effectiveness (e.g. one may be happier to accept smaller effects from an inexpensive, rather than an expensive, intervention)

in terms of the seriousness of the outcome targeted by the intervention (e.g. even a small effect on a serious outcome, such as homicide, may be regarded as reason enough to motivate an intervention, while a small effect on a more trivial outcome, such as littering, may not).

Since the effect sizes obtained in meta-analyses of offender-oriented programmes are generally small⁸ the question of whether these programmes are worthwhile has been raised. A common argument in defence of keeping programmes with small effect sizes

6 Please note that in studies with small samples the effect size may be relatively large without reaching statistical significance (which tends to occur particularly in some demonstration projects).

7 Cohen’s d compares the mean difference in outcome between experimental and control groups, taking into consideration their standard deviations (i.e. the mean change divided by the average standard deviation of the groups).

8 See, for example, Cunningham, 2002; Curtis et al, 2004; Farrington and Welsh, 2002; Landenberger and Lipsey, 2005; Lipsey et al, 2001; Littell et al, 2005; MacKenzie, 2006; Pearson et al, 2002; and Wilson et al, 2005. See also the ‘Offender-oriented programmes’, ‘Cognitive behavioural therapy’ and ‘Multi- systemic therapy’ chapters.

(18)

is that ‘small program effects should not be underrated in policy making and practice’

because they ‘are similar to those found for recognised methods of medical treatment’

and that ‘they may pay off from monetary perspectives as well’ (Lösel and Beelman, 2003:98). What is clear, however, is that offender-oriented programmes or interventions alone are far from enough of what is needed to achieve large reductions in offending by young people.

How do we best summarise the findings from many evaluations?

A central problem in assessing the effect of a particular type of programme or

intervention is how to summarise the findings from a number of different evaluations (with different designs, samples, etc). One way is to average the outcomes (mean effect sizes) from all evaluations that meet a minimum scientific standard (e.g. based on a certain level of a Scientific Methods Scale) and regard this as a summary measure of the evidence of the effectiveness of a particular programme or intervention. Such types of studies are generally called meta-analysis and have become an increasingly popular method to assess programme or intervention effectiveness (for an overview of the technique of meta-analysis see, for example, Durlak and Lipsey, 1991 or Wilson, 2001).

This practice may not be so much of a problem if the criteria for inclusion are narrow (e.g. if only Level 5 ‘gold standard’ evaluations are included in the meta-analysis) but become more problematic when the criteria for inclusion are much wider (e.g. if it is based on a mix of Level 3 to 5 evaluations according to the Maryland Scale).

Averaging the effects of well and less well-designed evaluations is obviously not fully satisfactory, and this method risks producing misleading findings. For example, in cases where the findings from well and less well-designed evaluations show different

outcomes, it may be advisable to only count the findings from the more rigorous ones rather than average all effect sizes. On a more general note, as pointed out by Wilson (2001:78), ‘the mean effect size is meaningful only if the effects are consistent across studies’, and therefore cases in which the mean effect size is based on summing highly- conflicting findings (e.g. evaluations showing a mix of positive and negative effects), the measure is at best uninformative (and at worst misleading). In principle, one can make a sound argument that, for example, one well-designed evaluation (e.g. a randomised experiment⁹) may outweigh the evidence from 10 weakly designed evaluations (e.g. 10 before-and-after measure studies).

However, even a well-designed evaluation does not always guarantee unbiased findings.

While a well-designed evaluation is of crucial importance for our ability to determine a programme or intervention’s effectiveness, it is not always enough to accomplish the task because there may be (in fact, often are) problems of attrition and

implementation/delivery that may distort the findings. For example, it is not uncommon:

that some people may refuse to participate in a programme or intervention, and that others may not fully participate or drop out

that those delivering the intervention may lack adequate skills and training or motivation to fully carry out the intervention as intended.

Dumas et al point out that:

9 Given the programme in question is properly implemented and delivered.

(19)

…traditional outcome research has paid considerable attention to other key methodological issues (e.g. experimental design, reliability of measurements and statistical power), but has more often assumed, rather than

demonstrated, that participants received the intervention(s) they were supposed to receive as designed.

(Dumas et al, 2001:39) Even a ‘gold standard’ randomised experiment may, in the worst case, be rendered more or less useless by problems of attrition or implementation/delivery failures. It is

therefore essential to consider not only the quality of the design of an evaluation, but also aim (as far as possible) to assess how well the studied intervention was

implemented and delivered.¹⁰

It is customary to differentiate between a programme or intervention’s (i) overall effect and (ii) effect on those who fully took part (e.g. Bloom, 2006). The overall effect is generally referred to as the effect of the intention-to-treat (ITT) and the effect on those who fully took part as the effect of treatment-on-the-treated (TOT). As may be expected, it is generally reported that the effect sizes are higher when calculated restricted to those who actually took part in the programme, for example, when excluding non-shows and drop-outs (e.g. Landenberger and Lipsey, 2005:462).

Measures of outcomes of ITT and TOT address two different, but equally, crucial questions; in the case of ITT, “What is the effect of offering a programme or

intervention?”, and in the case of TOT, “What is the effect of actually taking part in the intervention?”. If there is a big difference in effect when measuring ITT and TOT, it may indicate problems of engaging subjects or keeping them in the programme, and hence points to room for improved effectiveness by targeting subjects’ treatment motivation.

It is important to note that any evaluation that does not take into account

implementation and delivery is an evaluation of the effects of ITT rather than the effects of TOT. The effect of ITT includes the effect of the intervention on those who

10One way to address implementation and delivery when assessing a programme or intervention’s effect is to use instrumental variables (see, for example, Bloom, 2006:75–114; Angrist, 2006). In principle, instrumental variables are variables that measure important aspects of a programme or intervention other than the outcome. One key purpose of instrumental variables is to assess whether differences in outcomes between evaluations can be systematically linked to such things as differences in attrition, implementation rigour, dosage or type of outcome measure (e.g. differences in length of outcome observation period).

Instrumental variables can also help to provide information about the crucial question of whether a particular intervention is differently effective for different young people or when applied in different social circumstances. Instrumental variables, which have proven influential, are often referred to as moderators (i.e. factors that interact with the intervention in producing the outcome). Taking moderators into account may help improve interventions and their implementation and delivery (for example, by indicating how to modify or target the intervention so it better addresses the needs of different groups of young people who offend).

Instrumental variable approaches may also be particularly valuable in the evaluation of prevention programmes that consist of a range of interventions, where it can be very difficult to determine which components of a programme are effective (or ineffective, or even counterproductive) because they can be used to assess the contribution of the programmes’ various parts (interventions) to the outcome. It should be highlighted that it is often easier to select important instrumental variables for analysis if the

programme or intervention is based on strong theory.

(20)

participated fully in the intervention; therefore the effect of TOT provides an index of how effective the programme can be when participants are effectively engaged in the intervention and it is implemented and delivered as intended. This can illustrate, and highlight, the importance of effective implementation and delivery to effective practice, particularly under routine practice conditions.

A meta-analysis of randomised experiments combined with an instrumental variable analysis is probably the best method available to properly assess the overall

effectiveness of a programme or intervention. However, because such studies are rare, it is important to keep in mind that the evidence base is never better than the quality of the evidence upon which it is based.

Key points 3: Demonstrating the effects of an intervention programme

The evidence base is never better than the quality of the evidence upon which it is based.

Few interventions are evaluated using the best methods available, which means their conclusions may be based on misleading findings.

Intervention effectiveness is measured at a group level. The outcomes for individual cases may therefore vary strongly.

(21)

The problem of effective implementation and delivery

Proper implementation and delivery is not only a concern when evaluating a programme or intervention’s (potential) effectiveness, but also a key problem in the creation of effective practice. Even if a programme or intervention is in principle effective, if it is not properly implemented and delivered, there is no reason to expect it to have any greater effect. For example, meta-analytic research consistently shows that poorly- implemented and delivered cognitive behavioural programmes are associated with smaller (or no) effects than better implemented and delivered programmes and interventions (e.g. Lipsey and Landenberger, 2006).

A particularly interesting finding in this context is the research that indicates that

programmes and interventions tend to be more effective when researchers/evaluators are actively involved in their set up and implementation (see, e.g. Landenberger & Lipsey, 2005; Lipsey, 1995; Lipsey & Landenberger, 2006; Leschied et al., 2001; Lösel, 2001;

MacKenzie, 2006; van de Weil et al, 2002).

A main suggested reason for this is that many of those working in day-to-day practice may not always have the relevant skills and training or motivation to fully carry out the programme or intervention as intended.¹¹ Gendreau et al (2006:434–435) report, for example, that among the factors that contribute to whether a programme works are that the ‘staff are trained and hired based on their knowledge of effective relationship and therapeutic skills, and evaluators are involved in the programme’, but comment that

‘few practitioners and managers in correction have the requisite management skills and clinical training to establish and conduct treatment programs of therapeutic integrity’

(p.437). Latimer et al (2003:9) reports, in a meta-analysis of 195 youth offending programmes (of all kinds) targeting under-18s, that poor programme integrity was the rule rather than the exception; ‘only 11 programmes indicated the existence of a program manual, provided staff training and supervision, and monitored program compliance’¹².

The key problem appears to be the difficulty of transforming programmes or

interventions which have proven effective or promising when researcher/evaluator-led into effective everyday practice in the criminal justice area. This may be referred to as the problem of mainstreaming effective or promising demonstration programmes or interventions. The extent to which this problem can be effectively addressed has huge implications for the possibility of creating more effective routine practice among criminal justice organisations working towards the goal of preventing the further

criminal involvement of children and young people who offend. It is therefore important to consider what is known about the factors that contribute to successful programmes and interventions.

11 This may sometimes also have to do with the fact that in some demonstration projects, the initial recruitment to take part (before randomisation) may be selective (only those who agree to take part in the treatment are randomised), and therefore there is a risk that demonstration projects include more subjects favourable to treatment than would apply in a study of routine practice.

12 They report that of these programmes, 15% had a programme manual, 15% monitored programme compliance, 22% offered staff supervision, and 46% provided staff training (p.10), but only 6% (11 out of 195) offered all of these components of programme integrity.

(22)

Gendreau et al (2006) suggest seven principles of effective correctional intervention:

1. that the organisational culture is ‘receptive to implementing new ideas and has a code of ethics’ (p.425)

2. ‘that implementation of the program is based upon individual level survey data on the need for service and a thorough review of the relevant treatment literature’

(p.425)

3. that ‘the director of the program has an advanced degree in a helping profession with several years experience working in offender treatment programs’ (p.427) 4. that ‘offenders are assessed on a risk instrument that has adequate predictive

validities and contains a wide range of criminogenic needs’ (p.427), where criminogenic needs refer to ‘an offender’s way of thinking and behaving that supports offending behaviour’ (p.426).

5. that the programmes employ treatment methods which target behaviour (because they are proven to be the most effective types of offender-oriented programmes) as well as ‘the criminogenic needs of higher risk offenders’ (p.427) which, according to Gendreau et al (2006:428), involves the following key targets (depending on the particular subject’s identified needs):

a. change attitudes and feelings supportive of law violations and anti-criminal role models

b. reduce anti-social peer associations

c. reduce problems associated with alcohol/drug abuse

d. replace the skills of lying, stealing, and aggression with pro-social alternatives.

They also add the following targets as important only ‘when they are linked to any of the above’ (i.e. linked to a, b, c or d):

e. increase self-control, self-management and problem-solving skills f. enhance constructive use of leisure time

g. improve skills in interpersonal conflict resolution

h. promote more positive attitudes/increase performance regarding schoolwork and the workplace

i. resolve emotional problems associated with intra- or extra-familial child abuse j. promote family affection/communication/monitoring/problem solving

k. resolve deviant sexual arousal

l. alleviate the personal and circumstantial barriers to service (client motivation, background stressors).

These criminogenic needs (particularly points a–e) are highly consistent with the direct individual causes of young people’s involvement in crime, namely morality and self-control, which were discussed earlier in ‘The causes of young people’s involvement in crime’ chapter.

6. that ‘program therapists engage in the following therapeutic practices’ (p.427):

a. anti-criminal modelling

(23)

b. effective reinforcement and disapproval c. problem-solving techniques

d. structured learning procedures for skill-building e. effective use of authority

f. cognitive self-change g. relationship practices h. motivational interviewing

7. that ‘the agency establishes a system…whereby offenders are referred to other community agencies that can provide high quality services’ (p.427).

In their meta- and other analyses of the evidence for their suggested seven principles of effective correctional intervention, Gendreau et al (2006:436) report the strongest empirical support for principles 4 and 5 (which mainly concerns the importance of need assessment and particular programme content), ‘appreciable potential but less empirical support’ (p.437) for principles 2, 3 and 6 (which mainly concern the importance of proper implementation, management/staffing and particular practices), but no empirical support for principles 1 and 7 (which concern organisational culture and inter-agency co-operation) (p.437), the latter mainly due to a lack of proper research into their importance.

Gendreau et al (2006) are not alone in trying to establish principles for the development of effective practice. Other prominent examples include Andrews (1995), Lösel (1995), McGuire (1995) and Nation et al (2003). By and large, the content of the principles are much the same, so there appears to be some general agreement on the core features of effective practice.

In this context it is, in our opinion, particularly important to bear in mind that it is what we do rather than how we organise and manage what we do that is of fundamental importance, especially when politicians and policy-makers responsible for devising and implementing crime prevention often appear obsessed with issues of programme organisation and management rather than content.

Only when we do the right things (i.e. get the content right) do questions about how to most effectively organise and manage implementation and delivery become important.

In fact, the well-organised and well-managed delivery of ineffective (or even

counterproductive) intervention programmes may cause more problems than they solve (possibly making the situation worse).

Having dealt with the problems of how to establish whether programmes and

interventions are effective (and the general problems of effectively transforming such knowledge into routine practice), we now turn to the task of assessing the evidence for the effectiveness offender-oriented prevention programmes and interventions for young people, with a particular focus on cognitive behavioural and multi-systemic therapies.

(24)

Key points 4: Effective implementation and delivery

Implementation and delivery may significantly impact an intervention’s outcomes;

even effective interventions may prove ineffective if poorly implemented.

Interventions are often less effective when implemented in routine practice compared to research conditions.

If implementation and delivery in routine practice can match that of demonstration projects, intervention outcomes may be significantly improved.

However, the selection of intervention programmes is fundamentally more

important to the outcome than their organisation and management; a well-managed but ineffective intervention will still be ineffective, while a poorly-managed but effective intervention may still be effective.

(25)

Offender-oriented programmes

Overall, evaluations of offender-oriented programmes demonstrate a rather small effect on reoffending by young people. On the basis of his classic meta-analysis of offender- oriented intervention programmes for young people, Lipsey (1995) reports that for all types of programmes, the average recidivism for treated young people was 45%

compared to 50% for controls. This amounts to a 10% reduction in recidivism. Whether or not this should be considered a useful effect size can be discussed, but Lipsey argues that:

…a net 10% average reduction cannot be called trivial – it is, for example, within the range of effects viewed as significant in medical treatment and other such domains.

(Lipsey, 1995:67) However, there are many different kinds of offender-oriented intervention programmes and some are more effective than others (some tend to be ineffective and others appear even counterproductive; see Lösel, 2001; Mackenzie, 2006 and Redondo et al, 1997).

Moreover, the effectiveness of a particular programme depends on the rigour and quality of its implementation and delivery (and some programmes are poorly

implemented and delivered). One can expect (and indeed tends to find) stronger effects for certain kinds of offender-oriented programmes, particularly when they are well implemented and delivered. The trick to creating the most effective practice is thus to select the most promising programmes and make sure they are effectively implemented and delivered as intended.

As shown below, there seems to be quite a high consensus among evaluators that offender-oriented programmes that target (or include key components addressing) offenders’ ways of thinking and the moral content of their thinking are particularly promising (such programmes are generally referred to as cognitive behavioural

programmes). They also seem to agree that multi-modal programmes tend to do better than others, particularly programmes that combine a cognitive behavioural approach with interventions aimed at influencing criminogenic features of the offender’s immediate environment (e.g. criminogenic features relating to family, peer or school environments). These types of programmes are often referred to as systemic

programmes.

Lösel (2001) summarises the offender-oriented intervention research (on youths and adults) by saying that:

…nearly all research syntheses showed relatively consistent differences between modes of treatment: theoretically and empirically driven well- founded, multimodal, cognitive-behavioral and skill-oriented programs that address the offenders’ risk, needs, and responsivity had substantially larger effects than the overall mean”. He goes on to say that, “in contrast,

traditional psychodynamic and non-directive therapy and counselling, low- structured milieu therapy and therapeutic communities, merely formal variations in punishment had relatively weak or no effects.

(Lösel 2001:68)

(26)

MacKenzie (2006:333) concludes on the basis of her systematic reviews of different kinds of offender-oriented programmes (adult and youth) that ‘the effective programs are skill-oriented, based on cognitive-behavior/behavior models, and treat deficits simultaneously (i.e. are multimodal)’.

Most studies included in the above-mentioned meta-analyses are North American. In a meta-analysis of what works in correctional rehabilitation restricted to European studies, the authors conclude that ‘also in Europe the cognitive and behavioural

treatment modalities are the most effective in the treatment of offenders’ (Redondo et al, 1997:514).

Meta-analyses that have focused specifically on offender-oriented programmes applied to young people, by and large, report similar findings:

…effective programs differ significantly from ineffective programs in terms of their inclusion of techniques that can foster the development of the offender’s thinking and reasoning skills, social perceptions, and problem-solving /…/

Most programs that included a cognitive component worked; most that did not, failed.

(Izzo and Ross, 1990:139)

…at the high end/…/ are treatments that have a more concrete,

behavioural/or skills-oriented character. At the lower end of the continuum are treatments that are more oriented to psychological processes, e.g. the several forms of counselling.

(Lipsey, 1995:74) In general, positive programs that target anger management, academic skills, anti-social attitudes, cognitive skills and social skills demonstrate

improvements in recidivism compared to programs that do not target these needs. On the other hand, programs that target the psychological well-being of youth, leisure/recreation and substance abuse demonstrate diminished effects (i.e. increases in recidivism compared to programs that do not target these specific needs).

(Latimer et al, 2003:13) Addressing the effectiveness of selected offender-oriented programmes for young people, by comparing multi-systemic therapy (MST), residential programmes (e.g.

wilderness programmes) and community supervision (e.g. probationary and intensive supervision programmes), MacKenzie found that ‘only MST was found to be effective in reducing recidivism’ (2006:185).

Even though cognitive behavioural and systemic therapies tend to be the most promising kinds of offender-oriented programmes, there is great heterogeneity in the outcome of reviewed studies. Therefore one needs to consider carefully what elements of content or factors of application distinguish more successful programmes from less successful programmes. This is of crucial importance for effective practice because effective practice depends not only on selecting types of programmes that have proven effective, but also, within a group of programmes, on the ones that most effectively suit current purposes. Moreover, any knowledge of what other factors (than particular content) makes a programme more or less effective, in general, or for specific offender groups, is central knowledge for achieving best practice.

(27)

The kinds of intervention programmes we focus on in this source document (i.e.

cognitive behavioural and multi-systemic therapies) aim to influence (change) already existing individual characteristics that are important for young people’s criminal propensity (cognitive behavioural therapies), as well as some kinds of proximate environmental factors, which affect young people’s exposure to criminogenic moral contexts (multi-systemic therapies). An example of a criminogenic moral context is an area with weak formal and informal social controls where young people socialise unsupervised.

What these programmes do not target is (i) the broader social factors that cause the emergence and sustainability of criminogenic environments and young people’s

differential exposure to such environments, and (ii) the social and developmental factors that are responsible for the (early) development of the individual characteristics and experiences that influence later-life propensities to engage in acts of crime (although systemic therapies targeting children may address some of these).

It is important to point this out because it helps us to have realistic expectations of what offender-oriented intervention programmes can achieve in terms of crime reduction by highlighting the fact that offender-oriented intervention programmes only target some (and not necessarily the most important) of the many factors that need to be addressed by a comprehensive strategy to substantially reduce offending by children and young people.

Key points 5: Offender-oriented programmes

On average, offender-oriented intervention programmes for young people who offend reduce recidivism by 10%.

Promising offender-oriented programmes are those which address offenders’ ways of thinking and the moral content of their thinking (e.g. cognitive behavioural therapy [CBT] interventions) and multi-modal programmes, which combine a CBT approach with interventions influencing the criminogenic features of an offender’s immediate environment (MST interventions).

Programmes should be selected for implementation not only because research evidence supports their effectiveness, but also because they are the most appropriate for the current purposes.

Offending Behaviour Programmes