Modelling dynamic systems with differential equations
3.4 Regressing experimental data by curve fitting
3.4.2 Nonlinear least-squares model identification
whereǫis a small quantity (such as nearly the machineeps)
1. What is the (algebraic) inverse ofA?
2. Obviously if ǫ = 0 we will have trouble inverting since A is singular, but what does the pseudo-inverse,A+, converge to asǫ→ 0?
3.4.2 Nonlinear least-squares model identification
If the model equation is nonlinear in the parameters, then the solution procedure to find the opti-mum parameters requires a nonlinear optimiser rather than the relatively robust explicit relation given by Eqn.3.41or equivalent. Nonlinear optimisers are usually based on iterative schemes additionally often requiring good initial parameter estimates and even then may quite possibly fail to converge to a sensible solution. There are many algorithms for nonlinear optimisation problems including exhaustive search, the simplex method due to Nelder-Mead, and gradient methods.
MATLABprovides a simple unconstrained optimiser,fminsearch, which uses the Nelder-Mead simplex algorithm. A collection of more robust algorithms and algorithms to optimise con-strained systems possibly with integer constraints is the OPTItoolbox5.
A nonlinear curve-fitting example
The following biochemical example illustrates the solution of a nonlinear algebraic optimisation problem. Many biological reactions are of the Michalis-Menton form where the cell number y(t) is given at time t by the relation
y = αt
βt + 1 (3.53)
Suppose we have some experimental data for a particular reaction given as
time, t 0 0.2 0.5 1 1.5 2 2.5
cell count, y 0 1.2 2.1 2.43 2.52 2.58 2.62
where we wish to estimate the parameters α and β in Eqn3.53using nonlinear optimisation tech-niques. Note that the model equation (3.53) is nonlinear in the parameters. While it is impossible to write the equation in the linear form y = f (t)· θ as done in §3.4, it is possible to linearise the equation (Lineweaver-Burke plot) by transforming the equation. However this will also trans-form and bias the errors around the parameters. This is not good practice as it introduces bias in the estimates, but does give good starting estimates for the nonlinear estimator. Transforming Eqn3.53we get
y = α β − y
βt
5The OPTItoolbox is available fromwww.i2c2.aut.ac.nz
Thus plotting y/t against y should result in a straight line with an intercept of α/β and a slope of
−1/β.
Assuming we have the experimental data stored as column vectors[t,y]in MATLAB, we could plot
t = [0, 0.2, 0.5:0.5:2.5]';
y = [0,1.2,2.1,2.43,2.52,2.58,2.62]';
3 %ignore divide-by-zero plot(y./t,y,'o-')
0 1 2 3 4 5 6
1 1.5 2 2.5 3 3.5
y/t
cell count y
which should given an approximately straight line (ignoring the first and possibly the second points). To find the slope and intercept, we use thepolyfitfunction to fit a line and we get a slope =−0.2686 and intercept = 2.98. This corresponds to α = 11.1 and β = 3.723.
Now we can refine these estimates using thefminsearchNelder-Mead nonlinear optimiser.
We must first construct a a small objection function file which evaluates the sum of the squared error for a given set of trial parameters and the experimental data. This is succinctly coded as an anonymous function in Listing3.5following.
Since we require the experimental data variablesyandtin the objective function, but we are not optimising with respect to them, we must pass these variables as additional parameters to the optimiser.
Listing 3.5: Curve fitting using a generic nonlinear optimiser
1 %Compute sum of square errors given trial θ and (t, y) data.
sse= @(theta,t,y) sum((y-theta(1)*t./(theta(2)*t+1)).ˆ2);
optns = optimset('Diagnostics','on','Tolx',1e-5);
theta = fminsearch(sse,[11.1 3.723]',optns,t,y) %Polish the estimates.
6 plot(t,y,'o',t, theta(1)*t./(theta(2)*t+1)) %Check final fit in Fig.3.19.
Listing3.5returns the refined estimates for θ as θˆ=
α β
=
12.05 4.10
and a comparison of both the experimental data• and the model’s predictions is given in Fig.3.19.
Curve fitting using theOPTIoptimisation toolbox
The MATLABOPTIMISATION toolbox contains more sophisticated routines specifically intended for least-squares curve fitting. Rather than write an objective function to compute the sum of squares and then subsequently call a generic optimiser as we did in Listing3.5, we can solve the problem in a much more direct manner. Theopti_lsqcurvefitis the equivalent routine in the OPTItoolbox that solves least-squares regression problems.
Listing 3.6: Curve fitting using the OPTIoptimisation toolbox. (Compare with Listing3.5.)
>> theta0 = [11,3]'; %Initial guess θ0
Figure 3.19: The fitted (dashed) and experi-mental, •, data for a bio-chemical reaction.
The asymptotic final cell count, α/β, is given
by the dashed horizontal line. 0 0.5 1 1.5 2 2.5
0 0.5 1 1.5 2 2.5 3
time
cell count
Exptal data Best fit curve α/β
>> F = @(x,t) x(1)*t./(x(2)*t+1); %f (θ, t) = θ1t/(θ2t + 1)
>> >> theta = opti_lsqcurvefit(F,theta0,t,y)
4 theta = 12.0440
4.1035
Note that in Listing3.6we encapsulate the function in an anonymous function which is then passed to the least-squares curve fit routine,opti_lsqcurvefit.
Higher dimensional model fitting
Searching for parameters where we have made two independent measurements is just like search-ing when we have made one independent measurement, except that to plot the results we must resort to contour or three-dimensional plots. This makes the visualisation a little more difficult, but changes nothing in the general technique.
We will aim to fit a simple four parameter, two variable function to model the compressibility of water. Water, contrary to what we taught in school, is compressible, although this is only noticeable at very high pressures. If we look up the physical properties for compressed water in steam tables,6 we will find something like table3.4. Fig.3.20(a) graphically illustrates the
Table 3.4: Density of compressed water, (ρ× 10−3kg/m3)
Pressure (bar) Temperature,◦C
0.01 100 200 250 300 350 374.15
100 1.0050 0.9634 0.8711 0.8065 0.7158 — —
221.2 1.0111 0.9690 0.8795 0.8183 0.7391 0.6120 — 500 1.0235 0.9804 0.8969 0.8425 0.7770 0.6930 0.6410 1000 1.0460 1.0000 0.9242 0.8772 0.8244 0.7610 0.7299
strong temperature influence on the density compared with pressure. Note how the missing data represented by NaNs in MATLABis ignored in the plot.
Our model relates the density of water as a function of temperature and pressure. The proposed
6Rogers & Mayhew, Thermodynamic and Transport properties of Fluids, 3rd Edition, (1980), p11
0
(a) True density of compressed water as a function of tem-perature and pressure.
Figure 3.20: A 2D model for the density of compressed water. In Fig.3.20(b), the• mark experi-mental data points used to construct the 2-D model given as contour lines.
model structure is where the constants (model parameters θ) need to be determined. We define the parameters as;
θdef=
a b c k ⊤
(3.55) Again the approach is to minimise numerically the sum of the squared errors using an optimiser such asfminsearch. We can check the results of the optimiser using a contour plot with the experimental data from Table3.4superimposed.
The script file in Listing 3.7 calls the minimiser which in turn calls the anonymous function J_rhowat which given the experimental data and proposed parameters returns the sum of squared errors. This particular problem is tricky, since it is difficult to know appropriate starting guesses for θ, and the missing data must eliminated before the optimisation. Before embarking on the full nonlinear minimisation, I first try a linear fitting to obtain good starting estimates for θ. I also scale the parameters so that the optimiser deals with numbers around unity.
Listing 3.7: Fitting water density as a function of temperature and pressure rhowat = @(a,P,T) P.ˆa(1).*exp(a(2) + ...
1e2*a(3)./T + 1e7*a(4)./T.ˆ3) %Assumed model ρ = Pkexp a +Tb +Tc3
J_rhowat = @(a,P,T,rho) ...
4 norm(reshape(rho,[],1)-reshape(rhowat(a,P,T),[],1)); %SSEJ =P(ρ − ˆρ)2 T = [0.01, 100, 200, 250,300,350, 374.15]'; %Temp [deg C]
P = [100 221.2 500 1000]'; %Pressure [Bar]
rhof = 1.0e3*[1.00250, 0.9634, 0.8711, 0.8065, 0.7158, NaN, NaN; ...
9 1.0111, 0.9690, 0.8795, 0.8183, 0.7391, 0.6120, NaN; ...
1.0235, 0.9804, 0.8969, 0.8425, 0.7770, 0.6930, 0.6410; ...
1.0460, 1.0000, 0.9242, 0.8772, 0.8244, 0.7610, 0.7299]; %density kg/m3 [TT,PP] = meshgrid(T+273,P); Tv = TT(:); Pv = PP(:); % vectorise
14 A = [ones(size(Tv)), 1.0e2 ./Tv, 1.0e7 ./Tv.ˆ3, log(Pv)]; %Scaled data matrix idx = isnan(rhof(:)); %find missing data points
rhofv = rhof(:); rhofv(idx) = []; %remove bad points Tv(idx) = []; Pv(idx) = []; A(idx,:) = [];
theta = A\log(rhofv); %first (linear) estimate
19
%Do nonlinear fit
theta_opt = fminsearch(@(theta) J_rhowat(theta,Pv,Tv,rhofv),theta);
[Ti,Pi] = meshgrid([0.01:10:370]+273,[100:100:1000]); %Compare fit with data
24 rho_est = rhowat(theta_opt,Pi,Ti);
Fig.3.20(b)compares contour plots of the density of water as a function of pressure and temper-ature derived from the experimental data and the fitted model. The• shows the location of the experimental data points. The solid contour lines give the predicted density of water compared with the contours derived from the experimental data (dashed lines). The optimum parameters found above are