3.3 Projection
4.1.5 The Frisch scheme
The Frisch scheme, another approach to estimating both model and noise parameters, pro- vides a recursive algorithm strikingly similar to theBCLSapproach so that many of its variants may be interpreted as a special form ofBCLS, operating on similar extended models [9] with
comparable performance results [34]. It is based on the idea that the sample covariance ma- trix ¯D of the true values of observations ¯x yields the zero vector when multiplied by the true parameter values g, i.e.
¯ Dg = 1 N − mX¯ >Xg =¯ 1 N − mX¯ >0 = 0 This implies that in terms of the measured quantities, it holds that
¯
Dg = (D − ˜D)g = 0 (4.11)
where we have used that D = ¯D + ˜D where ˜D = ˜D(σ2y,σ2u) is a(n estimated) covariance matrix corresponding to white noise on both output and input. Similarly to theBCLScase, we have more unknowns than equations in (4.11), namely, m linearly dependent unknowns in g and the two unknownsσ2yandσ2u. However, assuming an estimate ofσ2uis available,σ2ymay be computed such that the difference matrix D − ˜D is singular. First, we have
D = ·
Dy y Dyu Du y Duu
¸
in which Dy y, Dyu, Du yand Duudenote the sample covariance matrices belonging to output-
and input-related entries in x where Du y= D>yu due to symmetry. By construction, we have
¯
Dyu = Dyu since the input and output noises are assumed to be independent, and ¯Dy y = Dy y− ˜Dy y= Dy y− σ2yI and ¯Duu= Duu− ˜Duu= Duu− σ2uI: ¯ D = · Dy y− σ2yI Dyu Du y Duu− σ2uI ¸
What we seek is det ¯D = 0. Expanding the determinant we have
det ¯D = det µ· Dy y− σ2yI Dyu Du y Duu− σ2uI ¸¶ = det¡Duu− σ2uI¢ det ³ Dy y− σ2yI − Dyu¡Duu− σ2uI ¢−1 Du y ´
If the input signal provides sufficient excitation, det¡Duu− σ2uI¢ > 0, which implies
det³Dy y− Dyu¡Duu− σ2uI
¢−1
Du y− σ2yI
´
= 0 (4.12)
Since the only unknown in the above equation is the scalarσ2y, (4.12) is the characteristic equation of the matrix
Dy y− Dyu¡Duu− σ2uI
¢−1 Du y.
As we are interested in the smallest possible noise contribution, it follows that
σ2
y= λmin¡Dy y− Dyu(Duu− σ2uI)−1Du y
¢
where operatorλmin(M) denotes the minimum eigenvalue of the operand matrix M.
In order to determine σ2u, one of the more robust approaches is to compute so-called residuals and compare their statistical properties to what can be predicted from the model [21]. A residual is defined as
ε(t) = A(q−1)y
k− B(q−1)uk.
Introduce the covariance vector r belonging to shift k=1 with components
ri= E (wkwk+i)
and its estimate from finite samples as
ˆ ri= 1 N − i N −i X k=1 wkwk+i
The idea is to compute the sample covariance vector r usingεkwhere εk( ˆθ) = ˆA(q−1)yk− ˆB (q−1)uk
in which ˆA(q−1) and ˆB (q−1) encapsulate current model parameter estimates, and compare it to a theoretical covariance vector based on
εk³ ˆθ, ˆσ2y, ˆσ2u
´
= ˆA(q−1) ˜yk− ˆB (q−1) ˜uk
where ˜yk as well as ˜uk are independent white noise sequences with variance as determined
by the current estimates ˆσ2yand ˆσ2u. The dimension of the covariance vector r (i.e. the maxi- mum shift i ) is a user-supplied parameter. Essentially, we are minimizing
δ = r¡εk( ˆθ)¢ − r
³
εk³ ˆθ, ˆσ2y, ˆσ2u
´´
with some weights attached to the entries in r to reflect their relative importance. A possible choice is W = 2n 2(n − 1) . .. 1 where n is the size of r and the other entries in W are zero.
The entire algorithm runs as follows: 1. Assume an initial value for ˆσ2u.
2. Compute an estimate ˆσ2y using (4.13).
3. Compute model parameters based on (4.11).
4. Determine the residualsεk( ˆθ) using estimated model parameters in ˆA and ˆB as well as
observed output and input sequences y(t ) and u(t ), and compute the related sample covariance vector ˆr .
5. Determine the theoretical reference covariance vector using residuals εk³ ˆθ, ˆσ2y, ˆσ2u
´ generated by estimated process parameters and white noise sequences ˜ykand ˜ukwhere
E¡ ˜yk
¢2
= σˆ2y E( ˜uk)2 = σˆ2u
Compare the sample and the reference covariance vectors by setting V = δ>Wδ where
δ is a difference vector of covariances and W is a weighing matrix.
6. Repeat from step 1 minimizing V .