Cost Effectiveness Evaluation
9. Quality Analysis
9.1 On-Time Analysis
Pick-up Definition
On-time pick-ups have three components: “Early” pick-up rates, “Late” pick-up rates and “On-time” pick-up rates. On-time data is computed from the differences between the agency’s promised and actual pick-up times. An arrival is “early” if the manifest time stamp is posted as earlier than fifteen minutes before the promised pick-up time. “On-time” is defined as the interval between fifteen minutes before the promised pick-up and fifteen minutes after the promised pick-up. “Late” is defined as any manifest time stamp more than fifteen minutes after the promised pick-up.
Drop-Off Definition
On-time drop-offs have three components: “Early” drop-off rates, “Late” drop-off rates and “On-time” drop-off rates. On-time data is computed from the differences between the agency’s promised and actual drop-off times. A drop-off is “early” if the manifest time stamp is posted as earlier than fifteen minutes before the promised drop-off time. “On-time” is defined as the interval between fifteen minutes before the promised drop-off and fifteen minutes after the promised drop-off. “Late” is defined as any manifest time stamp more than fifteen minutes after the promised drop-off.
System Definition
System on-time rates are the aggregated on-time rates of both pick-up and drop-off manifest time stamps. System rates are computed in the same fashion as pick-up and drop-off rates; any
manifest time-stamp (pick-up or drop-off) more than fifteen minutes before the promised pick-up or drop-off time is “Early”, any pick-up or drop-off time stamp between fifteen minutes before and fifteen minutes after the promised pick-up or drop-off is “On-Time”, and time stamps later than fifteen minutes after the promised pick-up or drop-off is late. On-time pick-up and drop-off
analysis examined data from individual passenger records as shown on passenger manifests. No distinction was made between trips to or from either home another destination. Data were aggregated to include all pick-up and all drop-off observations in only two categories. “Pick-up”
refers to the portion of the trip either at home or at the return trip were the rider waits for the vehicle to arrive. “Drop-off” refers to the portion of the trip when the rider is being dropped off at either home or another destination.
Table 55 Pre vs. Post Implementation On-Time Performance Test of Proportions
Sample Proportions
(*All pre to post-implementation changes except “Early Pick-up” are statistically significant at α =.05)
Table 14 illustrates on-time ratios Analysis of before and after sample proportions reveal significant change occurring in all but one category of on-time performance measures. On-time pick-ups changed a statistically significant (α =.05) percentage from 72% to 81% in the implementation period. Early pick-up rates remained constant at 12% in both the pre- and post-implementation periods. The ratio of late pick-ups decreased dramatically from 17% in the pre-implementation period to 7% in the post-pre-implementation period. Although the mean late pick-up difference between promised and actual time was less than a minute, mostly due to a peak at the twenty-minute range, from the rider’s perspective, post-implementation vehicles are late far less often.
Drop-off on-time ratios followed the same trend as pick-up proportions – early pick-ups decreased, on-time ratios increased dramatically and late arrivals decreased from 17% to only 9%. Overall system on time ratios also display this same pattern. This same information is also shown in Table 15 in which mean differences from on time in minutes are shown. All
differences except “Early Pick-Up” are significant at the .05 level.
Table 56 Pre vs. Post Implementation On-Time Performance
Mean (Minutes)
Pick-Up Drop-Off System
Pre Post Pre Post Pre Post Early -30.10 -29.67* -31.59 -29.47 -31.44 -29.62 On-Time 0.318 -0.437 -0.106 -0.328 0.061 -0.414 Late 31.79 30.93 31.68 29.97 32.00 30.77 (*All differences except “Early Pick-Up” are significant at α =.05)
Figure 11 displays pre- and post-implementation promised vs. actual pick-up time differences.
The figure shows the most distinct change from the pre and post-implementation periods in the time increments between five minutes early and five minutes late. Pre-implementation data in
this range is just over 30%, while post-implementation data shows an increase to just under 70%
in the same range. The change in on-time arrival deviations is driven almost entirely by a reduction in arrivals between five and fifty minutes after the promised time. The overall percentage of early arrivals (greater than 15 minutes early) has remained almost constant between the pre and post-implementation periods.
Figure 11 Promised vs. Actual Pick-Up Differences
Figure 12 displays promised vs. actual drop-off time differences between the pre- and post-implementation periods. This figure shows similar changes in on-time performance observed in pick-up data. Both early and late drop-offs have been reduced. The change in arrival time rates reflects a fundamental change in how dispatching and scheduling has been modified because of CASD implementation.
Pre-implementation manual scheduling operated with an emphasis on pick-up arrival time;
CASD scheduling places the emphasis on drop-off arrival time. In practice, this means that scheduler/dispatchers who previously arranged times with riders and spoke in terms of arrival time at the pick-up location, must now convert to a specific drop-off time. This basic change in operations has resulted in significantly tighter pick-up and drop-off time windows.
Promised vs. Actual Pick-Up Time Differentials (Difference from promised pick-up time at and aw ay from home)
0%
10%
20%
30%
40%
50%
60%
70%
80%
-60 -50 -40 -30 -20 -10 -5 0 5 10 20 30 40 50 60 > 60
Minutes From Promised Pick-up Percent of Rides per Time Increment
Pre Post
Figure 12 Promised vs. Actual Drop-Off Differences
Figure 13 displays aggregated system wide on-time rates for both ends of a rider’s trip. The figure shows total on-time rates for both pick-up and drop-off. The figure shows a system that has been overall positively impacted by CASD implementation, providing more predictability in both pick-up and drop-off times and simultaneously providing measurably fewer late and early vehicle arrival experiences to riders.
Figure 13 Aggregate On-Time Rates
System On-Time Differentials
(Aggregate Differences from promised/actual pick-up and drop-off time at and aw ay from home)
0%
10%
20%
30%
40%
50%
60%
70%
80%
-60 -50 -40 -30 -20 -10 -5 0 5 10 20 30 40 50 60 More
Minutes From Promised Pick-up or Drop-off Time Percent of Rides per Time Increment
Pre Post Promised vs. Actual Drop-Off Time Differentials
(Difference from promised drop-off time at and aw ay from home)
0%
10%
20%
30%
40%
50%
60%
70%
-60 -50 -40 -30 -20 -10 -5 0 5 10 20 30 40 50 60 More
Minutes From Promised Drop-Off Percent of Rides per Time Increment
Pre Post