In section four, participants were asked to provide information about the decision making process and their opinions of it.
They were firstly asked to indicate how long the decision making process had taken and how many meetings had taken place: Again as table 31 shows, findings are broadly as might be expected with IFR decisions requiring the least time in terms of number and timings of meetings and larger strategic decisions the most.
Table29. Decision making process
Decision making process IFR Organisational Strategic
n = 30 n = 189 n = 83
n (%) n (%) n (%)
Time taken to reach a decision:
1 - 3 months 20 (67) 57 (30) 26 (32)
4 - 6 months 6 (20) 72 (38) 19 (23)
More tha n 6 months 4 (13) 54 (29) 31 (37)
No a ns wer 0 (0) 6 (3) 7 (8)
Number of meetings at which the decision was discussed:
1 -5 26 (87) 77 (41) 27 (33)
6 - 10 4 (13) 57 (30 ) 16 (19)
11 or more 0 (0) 46 (24 ) 33 (40)
No a ns wer 0 (0) 9 (5) 7 (8)
Length of time for which the decision was discussed at each meeting: 0.5 hours or l es s 14 (48) 25 (13) 11 (13) 1 hour 9(30) 61 (33) 26 (32) 1.5 hours 4 (13) 47 (25) 16 (19) 2 hours 0 (0) 38 (20) 15 (18) 2.5 hours 1 (3) 6 (3) 2 (2) 3 hours or l onger 1 (3) 4 (2) 3 (4) No a ns wer 1 (3) 8 (4) 10 (12) N = 302
Participants were asked to what extent they agreed with a number of statements about the decision making process, ranking their opinions on a Likert scale; (Q16)
Figure 9. Participants views of aspects of the decision making process
Figure 10 gives numbers and percentages of participants responding to each of the statements about the decision making process. (Numbers refer to question numbers. ‘Strongly agree’, and ‘agree’ are grouped and
‘neutral’, ‘disagree’, and ‘strongly disagree’ are grouped). These items were later used to construct the co-production scale. Respondents were positive about the decision making process. More than 80% of participants agreed that they had ‘a sense of involvement’, ‘a variety of knowledge and
experience’ and the ‘ability to reach agreement.’ Just over 20% however, considered that the ‘outcome of the decision was unexpected.’
Respondents were asked which professional and stakeholder groups were involved in the decision making process and to rank their level of influence on the decision making process.
Figure 10. Types of people involved in the decision making process and their level of involvement by number and percentage in decision studied with 95% confidence intervals
Figure 11 shows that commissioning staff were involved in almost all the decisions described. The voluntary sector was involved much less (although still in just under half of the decisions).
Respondents were asked for their opinions about a range of practical issues encountered during the decision making process. Categories were derived from the qualitative work. Figure 12 shows number and percentage
response to the 5 statements again dichotomised into ‘agree’, ‘strongly agree’ and ‘strongly disagree’, ‘disagree’ or ‘neutral.’ A majority agreed that that the ‘formal process for arriving at a decision was understood’ that ‘there was a lot of time pressure,’ and that ‘the people and materials
needed for the decision making process were available.’ Most disagreed or were neutral that the ‘problem was difficult to frame’ or that the work was ‘interrupted by cancelled or poorly attended meetings’. However, 34% agreed with the statement that that the work was interrupted by ‘reorganisation, restructuring or change of personnel.
Figure 11. Participants views of aspects of the decision making process
Participants were asked to rank the importance of different factors in the decision making process, using the Gallego scale (104)(Bracket 1 in Figure 13) and a list of factors developed from our qualitative work (Bracket 2 in Figure 13) .
Figure 12. Perceived importance of different factors in the decision making process
Bracket 1:Gallego scale (86) Bracket 2:EMD qualitative work
As can be seen from Figure 13, in this context factors considered important by our respondents included items both from the Gallego scale and derived from our own qualitative work. Respondents were asked to select the single most
important factor from the combined list or to suggest one themselves. Figure 14 demonstrates the importance to our respondents of the factors introduced by our own qualitative work – for example “available budget” which was selected as the single most important factor in the decision by the highest number of 16% of the respondents. Complying with national guidelines was also regarded as an
Figure 13. Perceived single most important factor in the decision making process (percentage indicating item)
In addition to the important factors, respondents were asked to rate different
sources of information and evidence in the decision making process. (Qs25 & 26 in questionnaire: see Appendix 1). Again these questions were derived both from a previously used scale (103) and from items derived from our qualitative work. Figure 15 shows the percentage for each item identified as the single most
important source of information or evidence. As can be seen, respondents expressed a clear preference for locally based information and evidence over nationally available sources such as NICE guidance for example.
Figure 14. Perceived single most important source of information or evidence (percentage indicating item)
Respondents were asked questions concerning the availability of desired evidence at meetings. The majority of respondents (57%) indicated that the required evidence or information was available at all or most of their decision meetings. When discussion identified that more evidence was required however, only (45%) said that this could mostly be made available in time for the next meeting. And in general, although sourcing of evidence and information did not appear to be a particular problem for most
respondents, (16%) identified that if further information were needed - it would rarely be available for the next meeting.
Respondents were next asked to rate how they felt about the outcomes of the decision making process. The percentage of participants who ‘agree,’ or ‘strongly agree’ and the percentages, who ‘strongly disagree,’ ‘disagree’ or who are ‘neutral’ about each statement is indicated in Figure 16. The first four items reflect questions from a subscale of the Decisional Conflict scale (117).
Figure 15 Views on the outcome of the decision making process.
As can be seen from Figure 16 there were very high levels of satisfaction with decisions taken, for example with the statements: ‘I expect the decision to be implemented,’ ‘I feel we have made an informed choice’; ‘the decision reflects what is most important for the organisation,’ and ‘I am satisfied with the decision’. Forty per cent of respondents believed that ‘we should have made better use of information in the decision.’ And very few respondents concurred with the statement that ‘it was a purely
financial process.’
Participants were asked which of a number of formal decision making tools had been used as part of the decision making process.
Figure 16. Proportions of respondents who used seven well known formal decision making tools in their decision
Although these more traditional medical/health economics tools are
developed for commissioning decision making, many are not well used. As Figure 17 demonstrates, less than 50% of respondents indicated that they had used cost/QALY comparisons or data, National centre for Health
Outcomes Development (NCHOD) data, a balanced scorecard or the hierarchy of evidence. All 345 respondents were then asked to provide details about the use of QALYs within their organisation in general.
Figure 17. Respondents knowledge of the use of QALYs within their organisation (N=345)
Figure 18 shows that nearly three quarters of the 345 respondents either indicated that they did not know if QALYs were used as a decision-making tool in their organisation or left the question blank