Cost-Effectiveness Framework

We now return to the main goal: to assess cost-effectiveness of advanced planning and routing approaches. As can partly be seen from Figure 2.1, cost-effectiveness

can be affected in three ways: through a direct or indirect impact on (1) costs, (2) delay, and/or (3) the relationship between delay and disutility. Next, we combine insights obtained from our interviews and literature review to design a framework of determinants impacting these three factors. Determinants are classified as organi-zational, planning system, operational context, and demand-related factors, and are depicted in Figure 2.2. We assume all costs and resources to be constant, except for those related to the planning system.

Cost-effectiveness

Figure 2.2: Contextual framework for operational cost-effectiveness.

Organizational factors. Program staff tends to reveal its transportation needs at the last moment, although often they could have anticipated them much earlier (Pedraza-Martinez et al., 2011). These request delays create an “artificial demand uncertainty”, which affects cost-effectiveness. A second determinant is the level of earmarked vehicle use (Besiou et al., 2012). This can lead to a dedicated planning system with some disadvantages. Effectiveness is also affected when specific teams must fulfill certain requests as opposed to any team being able to fulfill any request.

For example, ethnic violence made one organization stop sending drivers from certain

tribes to Darfur. Vehicle characteristics, like the age of a vehicle or specific equip-ment, may also induce assignment constraints (Eftekhar and Van Wassenhove, 2016).

Organizational culture is a fourth determinant, as it determines the acceptability and thereby the effectiveness of a planning system. We return to this issue later. Finally, program size determines relative cost-effectiveness of different approaches, as we dis-cuss later.

Planning system factors. Cost-effectiveness is affected by each of the five char-acteristics of planning systems. First, decentralized systems can result in a lack of coordination (cf. Pedraza-Martinez and Van Wassenhove, 2012; Stapleton et al., 2009;

UNHCR, 2006). As a potential consequence, urgency levels are not incorporated ap-propriately in routing and prioritization decisions. Planners in centralized systems may lack (reliable) contextual information. Indeed, IT systems and dispatchers may have relatively little insight into security issues, weather and road conditions, demand mobility and their effects on travel times (Pedraza-Martinez and Van Wassenhove, 2012; UNHCR, 2006).

“There is no single software that knows the roads better than us. We go to the places where nobody else wants to go. You won’t find these roads on Google.”

[Respondent 6]

In case of such information gaps, hybrid systems may be more effective, e.g. by combining priority setting on a centralized level with autonomy in planning and routing on a local level. Second, a periodic review system has the advantage that the pool of destinations to be planned tends to be larger, yielding more efficient routes (cf. Daganzo, 1984). Planning delays induced by such system form a disadvantage.

Third, visiting more than one destination in one trip tends to decrease travel and queuing delays (cf. Daganzo, 1984). Fourth, systems in which vehicles are dedicated can block the assignment of available vehicles to requests, and thereby induce queuing delays. Moreover, pools of destinations will be smaller, which increases travel delays.

Finally, impact-based planning has an obvious effectiveness advantage over proxy metric-based planning, but may bring about substantial additional costs in terms of urgency assessments and implementation and maintenance of solution methods.

This brings us to a general major determinant of cost-effectiveness: the costs of a planning system. Such a system may require an IT solution, possibly including expen-sive vehicle routing software, and a planner or dispatcher. Moreover, implementing (rolling out) such a system will require substantial amounts of training and consume scarce human resources and budgets (cf. Winters et al., 2008). IT systems typically also require expensive support and maintenance services (Koskinen, 2010). Finally, planning may require time-consuming activities such as data gathering, information exchange (planner vs. staff and drivers), and urgency assessments.

Operational context factors. Travel times are determined by the road network, security issues, weather, and humanitarian issues (e.g., disasters might destroy roads and bridges). These issues have an obvious impact on cost-effectiveness as they determine travel and queuing delays and whether or not a destination can be visited at all. Sparsity also has an impact on the relative cost-effectiveness of advanced planning systems compared to simpler ones, as we discuss later. The same holds in general for stochasticity in travel times, as caused by varying operating conditions.

Demand-related factors. Urgency levels of requests constitute a major deter-minant of cost-effectiveness. Higher urgency levels imply that larger amounts of disutility can potentially be averted and hence an improved cost-effectiveness could be attained.

Quality of the urgency assessment and possible communication noises determine to what extent the planning decisions reflect the relative urgency levels of requests, and hence the disutility averted (see, e.g., Frykberg, 2005). Assessment quality may also be affected by misaligned incentives. For example, one interviewee reported that teams sometimes misrepresent needs in order to visit girlfriends or to reach certain targets.

Homogeneity of requests in terms of the required skills, equipment, and/or cul-tural background of staff is another demand-related factor. Heterogeneity can induce vehicle assignment constraints and hence queuing and travel delays. Finally, cost-effectiveness is also affected by demand mobility. For example, in case of conflicts,

demand locations and demand sizes are difficult to predict (Holguín-Veras et al., 2012), which hinders effective resource allocation and prioritization.

Interaction effects. The extent to which a given factor affects cost-effectiveness can be affected by interaction effects. First, the amount of variation in urgency lev-els determines the relevance of prioritization and hence the relative effectiveness of impact-based planning. Second, the amount of operational uncertainty will be highly correlated with the size of information gaps on a central level and hence with the rela-tive effecrela-tiveness of centralized planning systems. Real-time information systems are virtually absent in the humanitarian context and much local information is not be-ing institutionalized (Pedraza-Martinez and Van Wassenhove, 2012; UNHCR, 2006).

Third, the number of possible routes tend to be smaller in sparser networks, rendering the corresponding optimization problems easier and hence the relative effectiveness of advanced planning approaches compared to simpler ones smaller:

“In most operations, on the local level, there is normally only one route, which can either be closed or open due to the security situation. The planning options on using different routes are very few.” [Respondent 2]

Similarly, security issues affect relative effectiveness when they require routes to be unpredictable rather than efficient.

“(...) there are often kidnappings of the UN workers in the area. One has to change and you should not be predictable in order to survive (...).” [Respon-dent 6]

Fourth, program size and planning system factors have a major joint effect on cost-effectiveness. Investments needed to install and run an advanced planning system may, for example, have a relatively smaller impact on cost-effectiveness when program size grows. Fifth, request homogeneity interacts with vehicle, skill, and cultural mix.

These factors jointly determine the space of feasible assignments and hence what queuing and travel delays will be incurred. Finally some planning systems may better fit the culture of a humanitarian organization than others. Systems involving black box optimization, for example, may fail to meet humanitarian standards concerning

transparency (Vinck, 2013). Moreover, having a dispatcher telling field staff what to do and where to go may cause frustrations and discrepancies between what staff perceives as being needed and what is requested. As a result, such systems may fail.

“You cannot expect staff to simply accept [planning] recommendations.” [Respon-dent 1]

“When you tell them “this is not efficient”, they get angry (...).” [Respondent 7]

“Planning is about culture.” [Respondent 4]

More generally, the system affects the amount of autonomy staff has and the amount of bureaucracy they encounter. Studies among social workers show that these two factors have a substantial impact on job satisfaction, the risk of getting a burnout, and staff turnover (Kim and Stoner, 2008; Arches, 1991), each of which may clearly affect effectiveness.