CHAPTER III: METHOD
Phase 5: Post-Intervention Questionnaire
The post-intervention questionnaire was sent to staff 1 month following the conclusion of the short cycle feedback intervention phase. The post-intervention questionnaire was not
immediately provided to staff following the conclusion of the short cycle feedback intervention phase in order to mitigate a ‘halo effect’, where participants may have provided exaggerated responses due to the immediacy between the intervention and post-intervention questionnaire phases of the study. Additionally, the timing regarding the provision of the post-intervention staff survey was determined by the principal investigator and supervisor, on the basis that memory retention may be compromised if the staff survey is administered beyond a 1 month period of the relevant phase (intervention phase). Incidents reported in IIMS continued to be provided to the researcher. These incident reports were reviewed to determine any stability of effect from the study, particularly the short cycle feedback intervention.
Description
The post-intervention questionnaire had 18 questions (in contrast to the pre-intervention questionnaire which featured 91 questions), which focused on satisfaction regarding the short
cycle feedback intervention, utility of the feedback received and whether the participant personally observed improvements in patient safety. Feedback was measured to determine if reported incidents were reduced due to the learning and development aspects of the short cycle feedback intervention. Data specific to the post intervention staff survey was used to identify the acceptance of the intervention and staffs’ belief on the effect the intervention had on patient safety.
Distribution
The post-intervention questionnaire was provided to staff members a month after the conclusion of the intervention phase. As per the pre-intervention questionnaires, the
distribution of post-intervention questionnaires was primarily via pay-slips, with a few being hand-delivered to Allied Health staff. Informal feedback from staff members was that the number of question in the pre-intervention questionnaire was “excessive” and proved difficult for staff to complete in a timely fashion during their shift. Furthermore, staff anecdotally advised that they would only be able to complete the questionnaire during either their 20- minute tea break or 30-minute meal break; hence, they would not be completing the questionnaire in their own time.
Collection
A follow-up reminder was provided to staff, a fortnight following the initial distribution of the post-intervention questionnaire, in addition to the researcher informally meeting with staff on both wards to confirm whether participants have either received or completed the post-
intervention questionnaire. Similarly to the pre-intervention questionnaire, responses were accepted 2 weeks after the conclusion of phase 5; hence, a 6 week time period was allowed for responses to be returned. Thereafter, responses dissipated and no further follow-up reminders were provided.
Response Rate
The overall response rate for the post-intervention questionnaire was 38.38% (38 / 99). Furthermore, the response rate from the study ward and control ward was 56.36% (31 / 55) and 15.55% (7 / 45), respectively. The highest number of participants was nursing staff (N = 24). A summary of participants, by profession, is indicated in the table below:
Table 8: Post-intervention Questionnaire Response Rate, by Profession
Profession Ward Number Distributed Participants
Administration
Study 2 2 2
Control 2 2 0
Combined 4 4 2
Allied Health Study Control 9 7 9 7 4 3
Combined 16 16 7
Medical Study Control 16 11 11 9 1 0
Combined 27 20 1 Nursing Study 34 33 24 Control 25 23 2 Combined 59 55 26 Support / Hotel Study 2 2 1 Control 2 2 2 Combined 4 4 3 Blank Study 0 0 1 Control 0 0 0 Combined 0 0 1
Total Study Control 63 47 55 45 31 7
Combined 110 99 38
Analysis of Questionnaires
Comparison of Pre- and Post- Intervention Questionnaires from Study Ward
Responses in the form of nominal values for both questionnaires were tabulated in aspreadsheet. These nominal values were converted to numerical values, to allow for further analysis. Averages for each response were calculated, as well as the percentage of responses according to each possible nominal response. Nil responses were manually deleted from the spreadsheet, in order to be excluded from aggregated calculations.
Feedback Intervention
There were 2 questions, unique to the post-intervention questionnaire, which specifically sought the participant’s opinion regarding the feedback specialist. These questions were made available to participants from the control ward, despite that the feedback specialist
intervention was not featured within this ward. The 2 questions focused on the participant’s “satisfaction” regarding the “availability” of the feedback specialist, and the “overall
experience” with “learning from feedback” from the feedback specialist. It is noted that the former question allowed participants to select a “no opinion” response; however, the latter question did not feature a “no opinion” response, which was a deliberate strategy employed by the researcher in order to force a measured response. Despite this, there were 4 participants that did not provide a response, which was considered a “nil response” and was omitted from the aggregated data.
Comparison of Pre- and Post- Intervention Questionnaires from Control ward
Participant data from the control ward, in relation to both questionnaires, were entered into the same spreadsheet as per the study ward. This data recording method is listed in the previous section.Comparison of Pre- and Post- Intervention Questionnaires between Study and
Control Wards
Common Questions
There were 7 questions that were featured in both the pre-intervention and post-intervention questionnaires. These common questions pertained exclusively to feedback and were featured
under this section heading in both questionnaires accordingly. A comparison was made
between an individual participant’s responses to these common questions, per the participant alias used in both questionnaires. Of the 15 participants that completed these common
questions on either questionnaire, there were 6 participants that had finished both. Furthermore, there were 3 participants each from the study ward and the control ward. A spreadsheet was used to tabulate these results and nominal responses were converted to the respective numerical value. Variances between the pre-intervention and post-intervention numerical value was calculated, to determine whether there was a change in an individual’s response. It was determined that the participants from the study ward had a positive average improvement of 1.5 across all responses to the 7 common questions. This calculation was based on a conversion of the participant’s response. The 5-point Likert scale used was converted from a nominal score to a numerical score, to which averages were calculated and compared.
Conversely, there was a fractional positive average improvement of 0.2 in relation to the participants from the control ward.
Responses by Profession
Asides from the unique alias that each participant self-generated the individual was required to indicate their profession. This participant information was recorded in a spreadsheet. Nursing staff participants improved in the post-intervention questionnaire, compared to the
responsiveness to the pre-intervention questionnaire. Anecdotally, nursing staff verbally advised the researcher that the pre-intervention questionnaire was burdensome, due to the number of questions. In contrast, there was a decrease of Allied Health staff responses from
the pre-intervention questionnaire to the post-intervention questionnaire (N = 4). The responses by profession is summarised in the table below:
Table 9: Comparison of Response Rate, by Profession, between the Pre-Intervention and Post-Intervention Questionnaires
Profession Ward Pre Post Variance
Administration Study Control 2 1 2 0 -1 0
Combined 3 2 -1
Allied Health Study Control 7 4 4 3 -3 -1
Combined 11 7 -4
Medical Study Control 1 3 1 0 -3 0
Combined 4 1 -3 Nursing Study 12 22 10 Control 2 2 0 Combined 14 24 10 Support / Hotel Study 2 1 -1 Control 2 2 0 Combined 4 3 -1
Blank Study Control 0 0 1 0 1 0
Combined 0 1 1
Total Study Control 24 12 31 7 -5 7