5.6 Extensions
6.1.4 Simulation study and a closer look at jump regularization effects
To investigate the performance of the estimation method described in section 5.4, we make a test series consisting of 4 observation scenarios – full and partial state observations, exact and noisy measurements – and two choices of jump regularization weights to investigate their impact on the estimation.
The resulting 8 scenarios are:
(A) exact measurements, full observation, jump weight 1.0 (B) exact measurements, full observation, jump weight 10.0 (C) exact measurements, partial observation, jump weight 1.0 (D) exact measurements, partial observation, jump weight 10.0
(E) noisy measurements, full observation, jump weight 1.0 (F) noisy measurements, full observation, jump weight 10.0 (G) noisy measurements, partial observation, jump weight 1.0 (H) noisy measurements, partial observation, jump weight 10.0
For each setting, 100 realizations(3)of the stochastic FITZHUGH-NAGUMOoscillator eq. (6.4) driven by a WIENER process acting on the first component with diffusion coefficient D = 0.1, and kinetic parameter values given in eq. (6.3) were made, and the parameters therein esti- mated using the method proposed in section 5.4.
(a) stochastic (SDE) interpretation of the FITZHUGH-NAGUMOmodel
(b) deterministic (ODE) interpretation of the same FITZHUGH-NAGUMOmodel
Figure 6.3: Simulation of the FITZHUGH-NAGUMO oscillator with a driving WIENER process effec-
tive on the first component. In the SDE interpretation, the first component (blue) of the FITZHUGH- NAGUMOoscillator is disturbed by a driving standard WIENERprocess, visible as a “noisy” trajectory for
that component. The stability properties of the FITZHUGH-NAGUMOoscillator push both components
towards the steady state, unless the displacement by the WIENERprocess is too high; in that case, the
oscillator traverses its limit cycle and returns towards its steady state again.
The ODE interpretation (which is equivalent to an SDE interpretation with zero diffusion), shows no activity at all, since the oscillator remains at its steady state.
The kinetic parameters area = 0.02, b = 0.7, c = −0.8, z = 0.25, diffusion parameter D = 0.1; initial
values were chosen close to the steady state. Both simulations are generated by an Euler scheme with stepsize10−3.
6.1.4.1 Measurement functions and weights
All settings share the same sample interval of 5 time units, resulting in 201 full or partial state observations. In the case of partial observation, only the first component x1 is measured. In
the “noisy” scenarios, normally distributed noise with zero mean and a standard deviation of 0.1 (about 10% of the steady state value) is added to the measurements.
That is, we have as measurement functions(4):
hi(x(ti)) = x(ti) ∈ IR2 (i = 1, ..., 201) for scenarios (A), (B), (E), (F)
hi(x(ti)) = x1(ti)∈ IR1 (i = 1, ..., 201) for scenarios (C), (D), (G), (H)
with an equidistant measurement grid TM=
{0, 5, 10, 15, ..., 1000}. Note that we have omitted writing the dependence on the parameter vector p.
In the undisturbed settings, the measurement weights are chosen as 1.0, in the “noisy” scenarios, the reciprocal of the above standard deviation is used(5).
6.1.4.2 Shooting grid and node initialization
The time domain [0, 1000] is divided into 50 evenly sized intervals, i.e. the shooting grid is TMS=
{tMS
0 , ..., tMS50} with tMS0 = 0, tMS50= 1000, tMSk = 20k (k = 1, ..., 49)
and the shooting node variables sk (k = 0, ..., 49) are initialized with their temporally most
proximate measurement.
In the partial observation settings (C), (D), (G), (H), the unobserved species x2 is
initialized with its approximate steady state value −0.7506.
6.1.4.3 Initial parameter guess and stopping criterion for the GAUSS-NEWTON solver The initial guess of the parameters is set to 50% of the true values (see table 6.1), ensuring that the local area of contraction of the GAUSS-NEWTONmethod is left, thus globalization takes effect. We remark that also for more distant as well as randomized initial guesses, convergence to the solution is observed.
The optimization is stopped when the maximum norm of the search direction k∆xkk∞
(see section 1.3 on page 14) falls below 10−3. 6.1.4.4 Constraints on optimization variables
The following constraints (beyond reachable values for the FITZHUGH-NAGUMOoscillator) on the state variables at the shooting nodes are set:
x1, x2 ∈ [−4, 4].
The following constraints on the kinetic parameters ensure the right sign of the parameters: a∈ [0.001, 0.1], b ∈ [0.01, 2], c ∈ [−2, −0.01], z ∈ [0.01, 1].
In the test series, no constraints are active in the solutions.
(4) we use a vector-valued measurement function here solely for the sake of convenient notation. Using scalar
measurement functions hi as required in section 2.1.2, we might write hi(x(ti)) =
nx1(ti) if i ∈ 2IN − 1 x2(ti) if i ∈ 2IN with a
multiset TM
= {0, 0, 5, 5, 10, 10, ..., 1000, 1000} holding the (now non-unique) measurement times.
1 104 1 50 100 150 200 250 300 350 400 450 500 nz = 3088 (a) JacobianJ 1 104 1 104 nz = 756 (b) decomposition factorR
Figure 6.4: Sparsity pattern of the combined residual vector’s Jacobian - FITZHUGH-NAGUMO
oscillator with full state measurements.
(a) Jacobian J with dimension 500× 104 (52000 elements), nonzero elements: 3088 (5.9%)
(b) decomposition factorR with dimension 104× 104 (10816 elements), nonzero elements: 756 (7.0%)
See section 5.5.1 for details on the sparsity pattern.
6.1.4.5 Sparsity pattern of the combined residual vector’s Jacobian
Figure 6.4 shows the sparsity pattern of the the combined residual vector’s Jacobian J, i.e. the system matrix of the linearized problem without constraints, as well as the sparsity pattern of its decomposition factor that may be used for solving the linearized problem.
With a total dimension of 500× 104, only 3088 (5.9%) out of 52000 elements are nonzero. The decomposition factor R∈ IR104×104 has roughly the same low occupancy rate of 7.0%.
See section 5.5.1 for details. 6.1.4.6 Results of the test series
The results of the test series are shown in table 6.1. As one would expect, having exact full state measurements (scenarios (A) and (B) gives the best estimation results. For parameters a, b, and z, the mean estimate is very close (0.3–2.6% relative error) to the true parameter values, and the standard deviation of the parameter estimates is satisfying.
As already foreshadowed in section 6.1.2, parameters b and c that describe the steady state value of the non-excited FITZHUGH-NAGUMOoscillator, show a small bias in most of the testing scenarios, because of the enduring excitation by the driving WIENER process and thus moving the steady state value to a (varying) elevated steady state (see figure 6.1b). Parameters a and z may be recovered fairly well in all experimental settings.
For partial noisy observations (scenarios (G) and (H)), stronger jump regularization leads to considerable improvement, as both the relative error of the estimates as well as their variance is reduced (see the lower part of table 6.1; also compare the similar findings in the calcium oscillation example in the next section).
Table 6.1: Estimation test series: FITZHUGH-NAGUMOoscillator. Results of parameter estimation on 100 independent simulations (in each setting) of a FITZHUGH-NAGUMOoscillator, whose first component is affected by a WIENERprocess, eq. (6.4). For a discussion of the bias in parametersb and c, see sec- tion 6.1.2. Initial guess for the kinetic parameters was50% of the true values. Values rounded to 3 digits.
exact observations without measurement error
scenario (A) scenario (B)
true full observation full observation
parameter all jump weights 1.0 all jump weights 10.0
name value estimate± SD (SD%) RelErr% estimate± SD (SD%) RelErr%
a 0.02 0.021± 0.001 (3.8%) 2.6% 0.020± 0.001 (6.6%) 2.0%
b 0.7 0.698± 0.021 (3.0%) 0.3% 0.689± 0.059 (8.5%) 1.6%
c −0.8 −0.728 ± 0.057 (7.8%) 9.0% −0.692 ± 0.085 (12.2%) 13.5%
z 0.25 0.251± 0.009 (3.4%) 0.5% 0.253± 0.027 (10.8%) 1.2%
scenario (C) scenario (D)
true partial observation partial observation
parameter all jump weights 1.0 all jump weights 10.0
name value estimate± SD (SD%) RelErr% estimate± SD (SD%) RelErr%
a 0.02 0.021± 0.001 (4.0%) 6.0% 0.021± 0.001 (3.1%) 3.3%
b 0.7 0.645± 0.068 (10.6%) 7.9% 0.670± 0.039 (5.8%) 4.2%
c −0.8 −0.782 ± 0.092 (11.8%) 2.3% −0.713 ± 0.063 (8.9%) 10.9%
z 0.25 0.240± 0.013 (5.5%) 3.9% 0.236± 0.013 (5.3%) 5.7%
SD%: standard deviation of the estimate · RelErr%: relative deviation of the estimate from the true parameter value
noisy observations with measurement error ∼ N 0, 0.12
scenario (E) scenario (F)
true full observation full observation
parameter all jump weights 1.0 all jump weights 10.0
name value estimate± SD (SD%) RelErr% estimate± SD (SD%) RelErr%
a 0.02 0.020± 0.001 (6.7%) 0.8% 0.021± 0.002 (7.7%) 4.0%
b 0.7 0.686± 0.077 (11.2%) 2.0% 0.682± 0.064 (9.4%) 2.5%
c −0.8 −0.673 ± 0.108 (16.0%) 15.9% −0.770 ± 0.120 (15.6%) 3.7%
z 0.25 0.250± 0.009 (3.7%) 0.1% 0.255± 0.018 (7.1%) 1.9%
scenario (G) scenario (H)
true partial observation partial observation
parameter all jump weights 1.0 all jump weights 10.0
name value estimate± SD (SD%) RelErr% estimate± SD (SD%) RelErr%
a 0.02 0.021± 0.003 (14.1%) 6.4% 0.021± 0.001 (6.8%) 7.3%
b 0.7 0.509± 0.165 (32.4%) 27.3% 0.652± 0.094 (14.4%) 6.9%
c −0.8 −0.887 ± 0.238 (26.8%) 10.9% −0.783 ± 0.136 (17.4%) 2.1%
z 0.25 0.227± 0.020 (8.6%) 9.2% 0.237± 0.019 (7.9%) 5.1%
6.1.4.7 A closer look at a single realization
Figure 6.5 shows the fitted trajectories for a certain realization of a WIENER-driven FITZHUGH-
NAGUMOoscillator for each of the eight testing scenarios discussed in section 6.1.4.
As one can see by comparing figure 6.5a and figure 6.5b, smaller jump weights allow a better reproduction of the influces of the driving WIENER process, while stronger jump regularization leads to a “more steady” trajectory (in both settings, exact measurements are used). Depending on the user’s intention, each of them might be more suitable.
The figures 6.5c to 6.5h show detail enlargements of the approximate interval [280, 620] of the fitted trajectories in the respective scenarios. The interval contains one transit in the limit cycle and a subsequent stay around the steady state.
Especially in the “noisy” scenarios, i.e. with additional measurement noise, the application of higher jump regularization weights delivers a much “smoother” trajectory, which would be beneficial if state estimation is a user goal.
Scenario (A): Exact measurements, full observation, jump weight 1.0
Scenario (B): Exact measurements, full observation, jump weight 10.0
Figure 6.5: FITZHUGH-NAGUMO example fit (continues on the facing page)
Fitted trajectories for scenarios (A) and (B) for a certain realization of the FITZHUGH-NAGUMOoscillator.
While in the low jump weight scenario (A) the trajectory “mimics” the driving WIENERprocess affecting
the first component, the higher jump weights in (B) act as regularization, leading to a “more continuous” trajectory. The detail enlargements for scenarios (C) to (H) on the next page show this more clearly. Fitted trajectory of componentx1as blue line (—), of componentx2as green line (—), shooting nodes as dots in the respective colors (•,•). Measurements ofx1 as small light blue dot (•), ofx2 as small light
Scenario (C): Exact measurements, partial observation, jump weight 1.0
Scenario (D): Exact measurements, partial observation, jump weight 10.0
Scenario (E): Noisy measurements, full observation, jump weight 1.0
Scenario (F): Noisy measurements, full observation, jump weight 10.0
Scenario (G): Noisy measurements, partial observation, jump weight 1.0
Scenario (H): Noisy measurements, partial observation, jump weight 10.0