A simulation study is now carried out to investigate the performance of the EBC forecast approximation against both the Miller approximation and the benchmark.
The aims of the simulation study are the following:
1. To identify how the EBC approximation performs compared with the Miller approximation, and the benchmark, in terms of inventory performance metrics that are relevant to practitioners.
2. To enumerate different conditions under which the difference between the two approaches may be greater, or smaller.
3. To examine whether using a heuristic to pool forecast errors together improves inventory performance in the presence of price cuts and promotions.
4. To explore whether there is a possible link between the bias, variance and ac- curacy properties of the forecasts, and the resulting inventory performance.
500 data series are generated from the model in Equation 4.2.1 for each simu- lation run. The quantities we fix in these results are: (i) the lead time between a replenishment order and its arrival, set to 1 period, (ii) the review period, also set to 1 period, and (iii) the baseline sales α, set to 100. Also fixed by implication is the forecast horizon over which the forecasts are assessed, at 2 periods (review + lead time). Varying these conditions did not yield figures that alter the narrative of the results, and so they are omitted for concision.
The quantities that are varied are: (i) the price elasticity of demand, taking values -1,-2, and -4 (ii) the ‘noise’ parameter σ in the sales model, taking values 0.2,0.5 and 0.8 (iii) the proportion of promoted periods, which varies between 1 promotion every 5 periods (0.2), 1 in 10 (0.1), and 1 in 20 (0.05), and (iv) the history length, which
Origin β = −2, σ = 0.5, Promo. prop. = 0.1, Hist. length = 20
Dimension Alternative values
Elasticity -1, -4
Noise 0.2, 0.8
Promo. proportion 0.05, 0.2
History length 10, 30
Table 4.4.1: Showing the four different dimensions across which we vary parameters in the simulation study. The origin represents the parameter combination sitting in the middle of the range in each direction, whilst the other rows show the higher and lower alternatives for each dimension
is varied between 10 periods, 20, and 30. In addition, for each set of parameters we average the results obtained across 3 different target service levels: 90%, 95% and 99%. Table 4.4.1 displays the selected parameter values.
The choices of values of price elasticities, noise, and promotional frequencies are intended to represent the range of these conditions that may be experienced in prac- tice. For example, the promotional frequencies selected range from roughly once a month to roughly twice/three times a year, if the sales frequency is taken to be weekly. The noise parameter values were chosen by visual assessment of the series generated, whilst the elasticities were chosen partly by reviewing literature and real-world data. The history lengths chosen are on the shorter end of what is typically encountered in practice. For longer histories of 50 or more, the difference between the two approaches is expected to narrow considerably, as sufficient data to estimate the parameters more
accurately means that there is much less need for any approximation. Therefore, it is of most interest in this situation to analyse the situations where the most difference may occur. With regard to the service level targets, we choose high values of 90% or more, since higher service targets of this level are generally the most desirable in retailing.
The main inventory metrics utilised to judge performance are:
1. Average on-hand inventory (OHI), defined as:
Av. OHI = 1
T
T
X
t=1
Stock at beginning of period t + Stock at end of period t 2
(4.4.1) where T is the total number of periods in the test set. The assumption here is that demand comes at a constant rate throughout the period, and therefore the average inventory held during that period is the midpoint between the starting and finishing points.
Since average OHI is somewhat dependent on the level of the series and hence the choice of parameters, we introduce the following relative measure, termed the relative average on-hand inventory (RelAvOHI):
RelAvOHIfi = hYT t=1 Av. OHIfi t Av. OHIbt iT1 , (4.4.2)
where fi is a specified forecast method (here, either the Miller or EBC approx-
imation) and Av. OHIbt is the average OHI for the benchmark in period t. We use the geometric mean rather than the arithmetic since it is known to treat ratios of greater and smaller than 1 more symmetrically (Davydenko and Fildes,
2013). Whilst other references, such as the paper just mentioned, apply geo- metric averaging for evaluation purposes, the application to on-hand inventory is a new one to our knowledge.
2. Service level difference, defined as:
SL diff. = CSL (achieved) − CSL (target) , (4.4.3)
Ideally, the service level difference will be close to zero, although it is common that for high service level targets, the achieved service level rarely reaches the target level. The measure is not always symmetric; from an organisation’s per- spective, overperformance may be preferred to underperformance (this is more common than vice-versa). This caveat should be noted whilst, for simplicity, in these results values closer to zero will be indicated as superior.
The forecast performance metrics, to be used later on in the results, are:
1. Forecast bias, expressed as the mean error (ME):
ME = 1 T T X t=1 Et (4.4.4)
2. Forecast accuracy, expressed as the root mean squared error (RMSE):
RMSE = v u u t 1 T T X t=1 E2 t (4.4.5)
3. Forecast variance, the variance of the forecasts themselves:
Var = PT
t=1(Ft− ¯F )2
The errors Et used in these metrics are cumulative with respect to the forecast
horizon (2 periods) mentioned earlier in the section. We note that whilst forecast accuracy and bias are commonly used to evaluate forecast performance, the use of forecast variance is more unusual. In this setting, however, forecast variance possibly feeds into inventory performance via its influence on calculating the order-up-to level S. More variable forecasts may lead to S fluctuating more rapidly, with consequences for stock levels.