Strengths and limitations of the methods to evaluate the Processes

An initial and follow-up cross-sectional survey was chosen because it is a study design that could be feasibly conducted, at low cost, in the time-frame available. Furthermore, electronic surveys were easily disseminated as everybody in the target group had PHE or NHS email addresses. Additionally, as no information was available prior to the first survey it was a particularly appealing method to capture data about the initial phase of the TB-STS; it was possible to design, pilot and disseminate the survey quickly. It was also easy to repeat the survey at the second time point to assess any changes that may have taken place between two periods.

However, the study has a number of limitations, which are discussed below.

Firstly, the survey was developed after the initiation of the TB-STS so baseline information could not be collected. As a result, we may have underestimated the

129 difference between the surveys. However, the initial survey was conducted before the roll-out of any training for the TB-STS and prior to the employment of all national staff to coordinate cluster investigations and therefore it seems unlikely that there would have been any major changes prior to the first survey.

Secondly, the target population for the survey consisted of all public health staff, physicians and nurses working in TB control in England. It was not possible to enumerate the sampling frame because no register of clinical and health protection staff working in TB could be identified making it impossible to calculate a response rate and any associated potential biases. Response bias may have occurred if only those with a vested interest in the TB-STS (either positive or negative) responded to the survey; however, given that there was neither overwhelming support nor objection to the TB-STS, this is unlikely to have occurred or these responses may have cancelled each other out. In addition, the survey may have over-represented those with a smaller workload (as people with a greater workload may not have had time to respond to the survey); there were more responses from low TB incidence areas (45%, 27% and 28% from low, medium and high TB incidence areas) but this reflects the distribution of TB across England.

Thirdly, the 50% retention rate between the surveys is quite low which could bias the results if there was a systematic difference between responders and non-responders. It is possible that the opinions and experiences of a particular group of people were omitted. However, the validity of our results is supported by the fact that non- responders to the follow-up survey did not differ significantly compared to those that responded to both surveys based on profession or burden of TB in their geographical area.

Finally, the survey was not powered to look for differences between professions or geographical areas and when testing multiple hypotheses one might find a significant result due to chance. Therefore, it is especially important to consider the findings of the surveys in the wider context of the evaluation and the other methods used, and interpret them accordingly.

130 Interviews were used to explore the TB-STS user experience. This was appropriate given that the findings were used in a health services research context to supplement the quantitative work conducted around the implementation and perception of the TB-STS. They provide useful insights into the use and perceptions of the TB-STS from the perspective of the main service user (health protection staff). This was useful for understanding the broader findings of the evaluation. Semi-structured interviews were chosen as the method of data collection because they provided an opportunity for more in-depth information on issues identified through describing the service and the initial and follow-up survey, to be gathered and further explored. Data from these interviews also inform the interpretation of the thesis findings (see discussion, page 191).

All the people who were approached by the interviewer accepted to participate; the interviews were representative of all HPUs. Interviews potentially result in a number of biases associated with the interviewee, in this case the strain typing leads in the HPUs and their perception of the interviewer. The strain typing leads had very different roles depending on the burden of TB in their HPU which may have influenced the way they approached the interview: for some, strain typing was a large part of their job so they may have been keen to highlight its importance and utility to justify their role; for others, strain typing was a small part of their overall job so they may have understated its usefulness in order to justify not using it, creating more time for other parts of their job.

The role of the interviewer may also be important. The interviewer presented herself as an external evaluation scientist, who had no conflict of interest. However, people may have assumed that the interviewer worked for PHE. This may have led to a social desirability bias whereby the interviewees say what they think the interviewer wants to hear – if the interviewer was perceived to be from PHE, this could have led to exaggerated support for the TB-STS. Finally, the interviews were all conducted over the telephone, increasing the distance between the interviewer and interviewees, and excluding non-verbal cues. Not being able to see who was conducting the interview may have helped interviewees to speak openly, or it could have made them

131 feel more guarded as there was less opportunity to build a rapport. Given the service delivery nature of the interviews, rather than those covering personal issues, these biases are likely to have minimal impact.