What Can This Study Tell Us? Strengths and Limitations

First, however, I must recognise how (and to what extent) the methods employed in this research directly affect the utility of the results for the critical discussion of penal severity and community punishment. I do so by discussing some of the inbuilt strengths and limitations of the

19 _{In this study, the first phase has been undertaken in the development of a theoretical}

framework in Chapter Three and the identification of research aims in section 4.1.1 above, whilst the second will be described in Chapter Five, and the third undertaken in Chapter Six.

20

Chapter Five is substantially developed from a Research Report that was disseminated to all interested participants, as well as to both participating Centres and the Trust. Offender- participants were also issued with a certificate of participation, partly as a memento of their involvement but also as a means of evidencing their engagement with the research (for the purposes, e.g., of seeking employment).

methodology I adopted, as well as practical difficulties that were encountered during data generation. I conclude on the effectiveness of this research design as a means of answering my research questions. 4.4.1 Sampling: Of Gatekeeper-Participants, Offenders and Institutions Access and recruitment are particularly difficult in the context of criminal justice, given the relative vulnerability of certain groups (especially offenders, as subjects of overt penal power) to abuse by the researcher or by others (Noaks and Wincup 2004: 37-52), not to mention the constant structural pressures on the willingness and ability of potential participants to engage with academic study (cf. Mair 2008: 404-408). The sampling method adopted attempts to counteract this by engaging directly with supervising officers to select the most diverse sample available under the circumstances. However, three limitations to this approach must be stressed from the start: the relative lack of demographic sensitivity of the data collected under the sampling criteria adopted; the dual role of staff as

gatekeeper-participants; and the likelihood that the sampling method will

reach a relatively confined group of offenders. Whilst the second and third of these issues can be more or less taken together, the first warrants individual examination.

Whilst the study sought to incorporate as broad a range of demographic backgrounds as possible, it did not attempt to account for such differences systematically.21 Indeed, in practice, the offender sample collected was overwhelmingly white and male (although in terms of age, offenders showed a greater amount of variation: see 5.1). As a result, female and black and minority ethnic (BME) viewpoints are under- represented by the sample, increasing the risk that the data generated will

21 _{I frame this discussion mainly in terms of offenders. A similar point should be borne in}

mind regarding the staff sample, although we should bear in mind Annison’s (2007) observation that, empirically, more OMs are female than male.

overlook (or distort) critical differences in offender experiences arising out of marginalised demographic statuses.

For instance, whilst we must be careful to avoid stereotypes, women tend to bear a greater social responsibility for domestic work and childcare than men (Weeks 2011: 1-36, e.g.), which one might expect to have an impact upon their experience of community punishment. Would the (stereo)typical woman, vested with primary childcare responsibilities, particularly feel the impact of community punishment where it interfered with her maternal duties? Or might she especially value her liberty in this context, feeling the impact of the punishment less because she is at least still in the family home? Absent a greater representation of marginalised voices in the research, it is impossible to do more than indicate avenues for further study when faced with these questions.

We must also bear in mind the related issues with gatekeeper-

participants and offender engagement. Both issues raise concerns that the

sampling method adopted was incapable of reaching every part of the (offender) population – so-called sampling bias, which is a particular feature of non-random sampling methods (Agresti and Finlay 2009: 19). The reliance of the study on probation staff to identify potential offender participants, as well as the use of probation centres as the site for interviews, increased the risk of sampling bias by limiting the pool of potential offender participants, and therefore the offender experiences that could be incorporated into the sample.

In the case of incorporating staff as gatekeeper-participants, the problem is that staff may have a vested interest in recommending certain types of offender. Given the historical antipathy between probation values and successive governments’ punitive and managerial aims, and the increasing emphasis of centralised government management of probation activities in terms of performance targets and inspections, Mair (2008:

406-407) identifies a particular reluctance on the part of probation staff to engage in research that they perceive as portraying them in a negative light. Given the opportunity to act as gatekeepers to offenders, the danger is therefore that supervisors only put forward those cases that portray their activities positively. In particular, it was probable that those offenders who were engaging well with their orders and were well on the way towards rehabilitation, as well as those who posed a lower risk (of reoffending or of serious harm: see Canton 2011: 129-145) to the public were suggested as potential participants far more commonly, and were therefore over-represented in the sample. Less cynically, staff were asked to propose potential offender participants precisely because they had unique insights into the likelihood that an offender would be willing and able to take part in the study: a group that would, once again, be likely to be engaging well with their orders and posing a lower risk to the public.

From the offender’s perspective, getting access to them through the Service raised issues about the visibility of my independence from the participating Trust and the Probation Service as a whole. If I were perceived as a mere stooge of the Service, then offenders with adverse experiences of community punishment would be less likely to be willing to talk to me, on the grounds that I would not fairly represent their positions (Noaks and Wincup 2004: 56-59). This group would, once again, be likely to include those less willing to engage with their orders, with more compliance issues, and who presented a higher risk, and who were therefore subject to a more intensive level of probation intervention (Canton 2011: 74-83). In sum, pressures in both populations under study render the sampling method adopted vulnerable to sampling bias.

Although this limitation is to some extent inherent in the adopted research design, steps were taken to minimise its potential effect upon the validity and reliability of the study’s findings. Staff were approached

explicitly with the aims of the study and informed repeatedly that the research was anonymous, and in any event not a review of their Centre’s effectiveness at implementing and executing community orders. They were asked to select as wide a variety of cases as possible within the sampling criteria, and kept abreast of how the offender sample in each Centre was composed. Finally, staff themselves seemed alert to the potential for sampling bias, mentioning it in interviews and email correspondence, and several intimated that they were putting forward more ‘difficult’ clients. The establishment of rapport therefore helped obviate this risk to some extent.

Offenders, on the other hand, were reassured of my independence during the consent and information meeting stage, and were encouraged to bring both positive and negative experiences to the table should they choose to take part. This could not obviate the implications of my evident working relationship with staff or my use of Centre resources, but did help to alleviate the supposition that I was unduly supportive of the Service.

Moreover, whilst this limitation has perhaps the greatest effect on the validity of the data generated, its impact on the utility of the findings is reduced by the use of the supervision requirement as a sampling nexus. Recall (from 1.3.3) that this requirement is closest to the traditional rehabilitation-oriented role of the Probation Service, and contains amongst the least (visibly) onerous requirements available under a community punishment, namely, to attend periodic one-to-one supervision sessions (s. 213 CJA03). As a result, the supervision requirement lies at the heart of the populist punitive sentiment that community punishment is ‘soft on crime’ (Newburn 2007).

This apparent ‘softness’ makes probation supervision a highly effective case study in the attempt to identify the true penal impact of community punishment. If we accept that supervision is (generally

speaking) the least onerous potential element of community punishment, then those cases that involve the supervision of effectively-engaged and compliant offenders, who are less likely to be under threat of enforcement action and therefore to feel the penal ‘bite’ of probation supervision, will demonstrate the least severe penal impact of community punishment (Canton 2011: 123-126). This minimum penal impact is ideally situated for evaluating the retributive credentials of community punishment as proportionate sentences, especially in the context of the ‘soft on crime’ populist punitive critique (cf. Flyvbjerg 2011: 305).

4.4.2 Methods: Interviewing as Deep, Narrow, Contextual and Constructed Turning to the methods deployed, the use of qualitative interviews was motivated by the well-established ability of the semi-structured interview to provide in-depth and contextualised information about participants’ subjective experiences and opinions (Mason 2002: 63-67).

The central strength of semi-structured interviewing is that it enables us to explore individuals’ subjective experiences in a reflective and critical way (Byrne 2004: 182). This is particularly useful to a study that conceives of (community) punishment as a process of pain infliction (cf. Christie 1981), given that pain is ultimately a subjective phenomenon. What matters for this study’s purposes is not that hard treatment has been inflicted but that pain has been experienced. Accordingly, this research method focusses more on the mental state of the participant (that is, her opinions, views and perceptions) than it does upon the corporeal reality she experiences (the processes by which community punishment is imposed and executed; Silverman 2006: 113-114).

A further strength of the research design is its high degree of

contextualisation. By approaching the question of penal impact through

offenders’ experiences of their daily lives, it incorporates a variety of potential topics that may have more or less to do with the process of

community punishment than others. Case-files provided a large amount of contextual information (regarding, for instance, the effects of the order upon work, family and personal life) can be gathered about the offender and her situation, especially when supplemented by interviews with the participating offender’s supervisor. Given the complexity of day-to-day life, any attempt to discern the penal impact of community punishment requires this context, in order to identify when pains are being felt and which pains, if any, are related to the phenomenon under study.

However, despite these strengths, semi-structured interviewing is as susceptible as any other methodology to weaknesses that will limit the data generated by it. In particular, it is important to emphasise the

narrowness of scope of the data generated, as well as the constructed

nature of qualitative interviewing.

Interviews are an inherently work-intensive approach to data generation, requiring substantial time and energy to arrange, prepare for, and indeed conduct. As a result, interview research tends to be reliant on relatively small samples when compared to quantitative (and some other qualitative) approaches (Mason 2002: 67). Qualitative interviewing may be effective at producing in-depth knowledge, but it nevertheless trades that depth for a relative narrowness of scope.

This is really an issue regarding the generalisability of data: the extent to which the experiences of the sample can be taken as representative of (and therefore used to make inferences about) those of the broader population (Silverman 2006: 303-310). However, it is not always necessary to acquire the level of statistical representativeness required in quantitative study, as certain research questions can be answered on the basis of few (or even only one) case (ibid). Indeed, in the past the natural and social sciences have both been advanced significantly by small, well-designed case studies (Flyvbjerg 2011: 304-305).

Nevertheless, these data do not purport to represent an entire class of people (offenders subject to community punishment, and/or the staff responsible for their supervision). They only represent the experiences of the participants. The findings should not be read as a universal understanding of the penal impact of community punishment, but rather as a detailed, exploratory examination of the penal impacts that have arisen in some implementations of those sentences imposed on the participating offenders. The pains attending community punishment at the societal level may well be broader than the results of this study, or may be dependent upon other factors, contexts and practices than those observed herein.

These cases do enable us to develop a broader understanding of sentence severity, however, one which can take more account of subjective factors and is therefore less vulnerable to the distortion and partiality that has characterised so many other approaches to sentence severity (recall 3.1). In developing a novel approach to reconciling the subjectivity of pain with the objectivity required of consistent retributive sentencing, this study requires the depth of focus provided by a qualitative methodology. It will not provide a comprehensive overview, but it will provide an exploratory examination of the pertinent issues in the penal impact of community punishment.

A second limitation arises from the constructed nature of qualitative interview data. In comparison to other forms of research (and indeed other approaches to interviewing), the archetypal qualitative, semi- structured method is explicitly concerned with generating rather than

discovering data (Mason 2002). This is to say that it recognises the active

role that both the interviewee and the interviewer play in developing the data that are recorded in the interview transcript: the interviewee answers questions about her experiences, but those questions are formulated by

the interviewer, who also interprets the answers, and her expression of her opinion is likely to be affected by the context within which the questions were asked, such as the language used, her perceptions of the interviewer, and the impressions she has of the antecedent questions (Silverman 2006: 112).

This is usually thought of as one of the strengths of qualitative interviewing, since it does not subordinate the research participant to being a mere subject, from whom data are harvested before the researcher moves on (Noaks and Wincup 2004: 75-77). However, the particularly active role of the researcher (i.e. me) raises potential concerns in terms of prejudice. I use the word in its literal sense: I run the risk of interpreting the interviewee’s comments in a biased way because of conceptions and expectations arising before the fact. This risk of skewing the data generated is most commonly associated with ‘confirmation’ bias, that is, seeing what one expects to see (Flyvbjerg 2011: 309-311).

Whilst the risk of confirmation bias in small-scale research is often overstated (ibid.), it cannot be ignored, and I have taken measures to minimise it. In particular, offering the opportunity for member validation ensures that it is the interviewees’ perspectives, rather than my own interpretations thereof, which are privileged in the data. In the final instance, however, the capacity for confirmation bias inherent in small-N research is impossible to wholly eradicate, and this should be borne constantly in mind when reading the data generated.

The shadow-side of confirmation bias is that it raises questions about the impact of the researcher upon the data generated. Whilst my perspectives could be counterbalanced by data triangulation and member validation, the fact is that the data generated were constructed by me in, as well as after, interview sessions. The possibility that my conduct, and even my appearance, affected the data developed, cannot be discounted.

I have already identified some means that I took to attempt to ensure that my presence had no effect upon the data provided by participants, including my efforts to assert independence from the Probation Service when approaching potential offender participants. Beyond this, I also took a number of practical steps to attempt to be as encouraging of free and honest discussion on the participant’s part. Firstly, I attempted to dress in a ‘smart casual’ way that conveyed respect, without being overly formal or intimidating. Secondly, I avoided discussing the broader aims of the research after recruitment (unless specifically asked by participants), to prevent participants from thinking that I was expecting them to respond to questions in certain ways.

Ultimately, however, it is impossible to entirely avoid the possibility that my personal characteristics impacted upon the behaviour of research participants, and therefore upon the data generated. The data must therefore be read as limited in this respect, as well.

4.4.3 Practical Limitations: Research in a Time of Crisis

The vicissitudes of research threw up two additional practical limitations attending the research methodology adopted: firstly, poor attendance of group interviews; and secondly, limited recruitment of offenders (and therefore of staff) in IC.

Both restrictions draw from a common root, which might be summed up in two words: Transforming Rehabilitation. The scale and scope of probation privatisation reforms, which commenced during the research period and significantly intensified towards its end, meant that staff were increasingly unable (if not unwilling) to continue to support the study as their roles as offender managers were placed under increasing stress, and their professional futures made increasingly uncertain. At the same time, the central bureaucracy of the Trust became less and less supportive in terms of supplying case-files expeditiously. Since IC was in

any event a larger and busier Centre than OC, and the SPO liaison exerted less direct control over her staff than her OC counterpart, this meant that it became increasingly impracticable to recruit new participants in IC. Given that data saturation appeared to have been reached, in that participants’ statements at interview were more repetitive than productive of new themes and concepts, I decided to end the research at an earlier stage than initially anticipated. As a result, there was a numerical imbalance between IC and OC, as noted in Table 4.1 above. Whilst I attempt in the next chapter to correct for the possibility of privileging the more rural Centre by ensuring that every participant’s voice is heard at least once, I may have either over-represented OC in relation to IC, or overemphasised the opinions of IC participants, to at least some extent.

The consequence of the increasing unwillingness of the central Trust (and inability of the participating Centres) to support the research, coupled with the limited time-frame and funding of the study, led me to set a relatively tight conclusion date for the study. This limited window of opportunity meant only a three-week notice period of the time and date of the group interview. Whilst this was agreed with the support of participating staff, who scheduled mandatory meetings with offenders on the days in question, offender attendance at these second interactions was severely limited. Of the six OC offenders, four were expected to attend and only two actually did, whilst in IC two were confirmed attendees and only one was there on the day.

To some extent, this was to be expected, as high attrition rates and flexibility are stressed in the relevant research design literature (e.g. Ellis,