6.3 Model validation and credibility
6.3.3 Validate components of the model by quantitative techniques (sensitivity analyses)
ANALYSES)
Law discusses several quantitative techniques to validate components. We use two of these techniques. The first technique is used to analyze the fit of a theoretical probability distribution to a set of observed data. We use this to determine the probability distribution functions of the
60 consultation times. The second technique we use, called sensitivity analysis, is to investigate whether a particular factor appears to be important. We discuss these techniques below.
Probability distribution of the consultation times
We use a quantitative technique to determine the consultation times per care provider when we analyze the fit of a theoretical probability distribution to a set of observed data. We use ExpertFit to make a graphical plot of the observed data, followed by a Q-Q plot to test the discrepancies with the proposed distribution. Last, a goodness-of-fit test was performed. With this procedure we found an acceptable probability distribution function for all consultation times per care provider. More information on the consultations times can be found in Section 4.4.2.
Sensitivity analyses
Another quantitative technique we use is sensitivity analysis. We use this technique to investigate whether a particular factor appears to be important. For our model, sensitivity analysis is used to investigate the effect on the one stop shop percentage, percentage of patients with consent on time, waiting times, and access times. We perform sensitivity analysis on the arrival rates, mean consultation time, standard deviation of consultation time, and duration of extra examination by incrementally changing the value of one factor. The effects per analysis are discussed below. Appendix J shows more details and figures of the analyses.
Arrival rates: We investigate the arrival rates to analyze the influence of very busy and quiet periods. As suspected, we find that the average waiting time per care provider is influenced by the number of patients arriving at the PAC. Care providers with high walk-in percentages (nurse and anesthesia assistant at Hengelo) are influenced more than ‘appointment based’ care providers. The average waiting time for the care providers doubles when comparing very busy and very quiet periods Moreover, the waiting time is more influenced during the busy periods than during quiet periods. This is in line with Pollaczek-Khinchine formula which states that mostly equipment loading (utilization) drives the expected work in progress (WIP) and therefore waiting time [58]. In our simulation model we only take into account one standard deviation from the mean. Therefore, extremely busy or quiet days are not included. This may lead to a lower mean and standard deviation of the waiting time in the simulation model than in reality.
Mean consultation time: During the interviews we noticed that employees often noted that the consultation time is highly dependent on who is screening a patient. For example, the consultation time for the same patient would be 12 minutes for nurse A, 15 minutes for nurse B, and 18 minutes for nurse C. Employees suggest that causes for these differences are experience and the capability of some care providers to cut to the chase easily, whereas for others this is more difficult. We investigate the influence of the mean consultation times per care provider and whether these differences influence the waiting time for patients significantly. For example, when we analyze the data in Figure 6.2, we conclude that if a care provider at Hengelo needs 30% more consultation time per patients (factor 1.3), the waiting time can increase with more than 100%. If a care provider needs 30% less consultation time (factor 0.7), the waiting time can decrease with almost 50%. Again, this can be explained with Pollaczek-Khinchine formula [58]. More consultation time per patient leads to higher utilization and more WIP, whereas low consultation time leads to lower utilization and less WIP. According to this formula, a higher utilization has more influence on the WIP and therefore on the waiting time than a low utilization. This suggests that waiting times in the simulation model can be 50% less or 100% more in reality depending on the care provider on duty Moreover, a
61 consequence of increased waiting times is that patients choose to come back another day. This leads to longer access times.
Figure 6.2: Influence of mean consultation time per care provider on the waiting time at Hengelo
Standard deviation of consultation time: We investigate the standard deviation of consultation times to analyze the influence of a high or low variance in consultation of patients. As the formula of Pollaczek and Khinchine [58] already indicates, a higher variance of consultation times leads to more WIP, and therefore to longer waiting times. Moreover, we find that the waiting times are more influenced for individual care providers (anesthesiologist and anesthesia assistant) than for nurses. The reason is that varying the standard deviation has more influence on the utilization of the individual care providers as well. We conclude that reducing the variability between patients leads to a lower standard deviation for the care providers and to less waiting time.
Duration of extra examination: Figure 6.3 shows the outcome of the last sensitivity analysis we performed. This analysis was related to the time till the results of extra examination are in. We research the effect of varying the duration for the extra examination type ‘consult with cardiologist’ from 5 till 35 days (currently 20 days). We find that the duration has more effect on the percentage of patients with consent on time at Hengelo than at Almelo. With almost three times more patients sent to the cardiologist and on average an earlier operation date than at Almelo, this is not surprising. We use a fixed number of days in the simulation model, whereas in reality there is more flexibility for a consult with a cardiologist. However, this analysis gives a good indication on the influence of the cardiologist on the percentage of consent on time.
Figure 6.3: Influence of the duration of a consult with a cardiologist on the percentage of patients with consent on time 0 20 40 60 80 100 0,7 0,8 0,9 1 1,1 1,2 1,3 M in u te s Factor
Waiting time Hengelo
Nurse Anesthesia Assistant Anesthesiologist 0,9200 0,9400 0,9600 0,9800 1,0000 1,0200 5 10 15 20 25 30 35Days before results of extra consult
Percentage of patients with
consent on time
Almelo Hengelo
62 In this section we discussed the effects of several factors on the one stop shop percentage, percentage of patients with consent on time, waiting times, and access times. We conclude that the waiting time in our simulation model may be different than in reality. This is due to the assumptions not to include extremely busy or quiet days and not considering the difference between the mean consultation times per care provider. Quiet days and lower mean consultation times lead to lower waiting times, whereas busy days and higher mean consultation times have the opposite effect. We also found that the access time and consent on time is influenced by the duration of extra examination. In the simulation model this variable is fixed, whereas in reality it is more flexible. Some patients will be able to schedule a consult with a cardiologist earlier and for others the access time may be higher. However, these figures give a good indication on the influence of the variables without the interference of other aspects.