Data collection methods

The final study design involves a Time Driven Activity Based Costing analysis, which includes a process evaluation, alongside an analysis of organisational sensemaking within each of the included practices. Three types of data require collection at each practice for the TDABC analysis:

i) Information on activities undertaken within each practice: collected through interviews with practice staff.

ii) Activity data from all practice staff involved in vaccinations, including activities undertaken and time spent: collected using activity logs.

iii) Cost data from practice management: collected using a costing survey.

For the sensemaking evaluation of organisational management, data will also need to be collected through interviews with practice staff. Therefore, to reduce the time and resource burden within each practice, the data collection for this evaluation was combined with that for the evaluation of activities (described in (i) above) into a single semi-structured interview format.

• Interviews

For the reasons described in 5.4.3 semi-structured interviews were used at each practice to generate data for use in the implementation evaluation and analysis of organisational sensemaking. A topic guide was developed, with topics and questions derived from important components of the process of implementing vaccinations, as well as factors identified in the literature related to organisational management and performance. The guide is presented in appendix 9.4. GP practices are very busy, high pressure environments, so great flexibility was allowed in terms of structure and staff

participation in the interviews, depending on the availability of staff, in order to maximise

participation. The main point of contact for the practice was the practice manager, who was requested to arrange the interviews. The request was to arrange for a cross-sectional profile of staff from within the practice who are involved in the organisation and delivery of the vaccination programme to

106 participate. Interviews could be with individual staff, or groups of clinical or administrative staff, or mixed groups of staff, depending on what was convenient for the practice. The total data collection activity took place over a period of two to three weeks and so interviews could be conducted on one or more days at the practice. Due to the variation in organisation no prescription was given as to which staff should participate, only that they should cover experience of both clinical service delivery and administration and management and be able to provide a representative view of the practice.

At the start of each interview I briefed participants on the nature and the scope of the research and requested written consent, which was provided individually by all staff members. The interviews were recorded, and, in addition, I took contemporaneous notes in the interviews and in the following experiences of vising each practice to put the interview in context. Once completed the interviews were transcribed verbatim by a professional transcription service (www.transcriptdivas.co.uk). The coding method employed for the process evaluation is presented in 6.2.1 and for the organisational sensemaking analysis in 6.3.2.

• Activity log and cost survey

I requested the tools used in the previous ABC study conducted on the vaccination programme in New Zealand from the study team (Turner et al., 2009). The activity log from this study was used by both clinical and administrative staff and comprised a daily log for the time spent on vaccination activities, within specified categories and a column for staff to enter their estimated time taken in minutes. To develop the activity log, I expanded the data points collected by the New Zealand team to allow much greater flexibility for staff to record their activity without being constrained by pre- defined categories. I also created separate forms for clinical and administrative staff. The forms were divided into three categories for the clinical staff:

i) Activity undertaken during clinics (i.e. giving vaccinations), where appointment start and end time, alongside vaccinations delivered and consumables used, were recorded. ii) Activity undertaken outside of clinic time (administrative tasks, reading etc.)

iii) Activity undertaken regularly, but less than once per month, that was not captured in ii)

For the administrative staff, only activities ii) and iii) were included. The final activity log forms used during the study are presented in appendix 9.5. The logs were kept by all practice staff with a role in vaccination for a 2-week period. The list of staff working during the nominated 2-weeks was provided by the practice manager and logs provided to each staff member, and all logs had to be returned for the voucher to be provided, even if they were blank (i.e. the staff member did not undertake any vaccination activity). Staff were trained to use the logs on the same day that the interviews took place and so the activity logs were completed after the interviews in all cases. In the very small number of

107 cases where a staff member could not be trained directly, a colleague was nominated to ensure they understood how to use the forms. Training was simple and took around 10 minutes.

I showed the costing survey to two practice managers of non-participating practices (one in Cumbria and one in London) to get feedback on its suitability and usability. The facility cost data requests were largely kept the same, with minor modifications to make the categories suitable to the England

context, including separating out the staff salary costs more clearly. An additional section was added for costs specific to the vaccination programme, i.e. consumables, cold-chain and fridges. Practice managers were asked to fill it out as completely as possible using routine available accounting data for the last complete 12 months (usually the preceding financial year April to April) that they had available. Practice managers had concerns about the confidentiality of the data, so part of the agreement was to ensure that none of the costing data was reported separately, nor in a way that the practice could be identified. The costing survey is presented in appendix 9.6.

5.5.1 Piloting and modification

The sampling method is described in section 5.7, however the first practice recruited was used as a pilot practice to evaluate the data collection methods and modify these based on experience and feedback before being delivered in the other recruited practices.

At the pilot practice all the interviews were conducted on a single day, with a GP, the practice manager, the practice nurse and a receptionist. From the interviews, the most interesting information came from times within the interviews where the discussion was allowed to flow, and therefore in future interviews a loose structure of questioning was employed to encourage free flow of

conversation and enable interesting ideas to emerge. No other modifications were made to the topic guide.

The practice nurse (PN), practice manager (PM) and reception staff were left with the activity logs to complete for 2 weeks, as the GP had no role in vaccination. There were only minor changes to the activity log, which were primarily to do with clarifying the instructions of what to include in which section and modifying headings for sections to more closely match language used within GP

practices. Otherwise the data recorded were clear and matched the expectations of each section. One particular challenge however was recording activity from receptionists. Overall, vaccination activity formed a relatively small amount of time for each of the five reception staff, who all worked less than full time. Thus, they had only one or two vaccine encounters each, so recording was sparse and dropped off as the week went on. As a result, I decided to not collect individual data from all

reception staff if their only role was to book appointments and check patients in. The mean time spent per patient on checking in and booking was 1.8 minutes and this will be correlated with data from the other practices where receptionists did fill out activity forms (if they had wider roles in

108 reminder/recall for example), to determine an average time spent per patient on booking/checking in patients for appointments.

Minor modifications were made to the costing survey, particularly the naming of some of the facility level categories and adding in additional options around rent or facility payments, as there is wide variation in how this cost is described at practice level.