System Identification

In applied mathematics, and indeed, systems theory, the building of mathematical models from measured experimental data of dynamical systems is generally referred to as system identification. It is an important sub-set of statistics; hence, many identification techniques as well as the tools for analyzing their inherent properties are very much rooted in statistical theory. A review of the literature suggests that research in this area of system theory can be traced back to the mid-1960s – perhaps to the works of Åström and Bohlin (1965), and Ho and Kalman (1965). In the former, the very fundamental principle behind the maximum likelihood methods for parametric input-output models (which later became known as prediction error identification) was presented; while the latter presented the first known solution of state-space realization theory, which subsequently led to stochastic realization and finally to the birth of subspace identification methods. Ever since, system identification has experienced tremendous growth, which can rightly be said to have been spurred by the enormous interest in model-based control strategies as well as the advancements in optimal control theory by Rudolf Kalman and his peers.

Evidently, building mathematical models from first principles material and energy conservations often require considerable expert knowledge, and can be expensive in terms of man-hour requirements. The resulting models are referred to as white-box, and are often characterized by high complexity which makes them very unsuitable for real-time model- based control applications. In other words, physics-based white-box models are able to capture process behavior over a wide range of operation; they are however, not suitable in applications where small computation times are crucial. In contrast, systems identification provides a suitable alternative to these so-called first principle models. It results in simple compact models that can be used in real-time model-based controllers; it also provides an enablement for the construction of process models that are able to reproduce process data and therefore exhibit accurate description of the local behavior of the system. To this end, identified models are often used in control design, in the adjustment of free parameters in first principle models, in fault detection techniques and process monitoring, Larimore (1997). Based on the physical interpretation of the parameters of the identified model, the resulting model may be referred to as black-box (if there is no physical interpretation of parameters), or grey-box (i.e. if there is little meaning of model parameter).

39

System identification of linear time-invariant (LTI) models can be broadly classified as parametric or non-parametric. The parametric approach includes the prediction error methods – which have strong relations to maximum likelihood estimation, the output error methods – where the governing criterion is to minimize the error of the output measurements, and the subspace methods – which intersect system theory, numerical linear algebra and geometry. The idea behind SubID is based on the possibility of retrieving certain subspaces which are related to the system state-space matrices, according to block-Hankel matrices, structured from input output data, Overschee and de Moor (1996) and Verhaegen and Verdult (2007). Non-parametric approach to system identification includes frequency domain identification techniques such as the correlation and spectral analysis methods, Brillinger (1981) and Pintelon and Schouken (2012). By and large, the theory of systems identification LTI systems is considered mature, Ljung (1987, 1999). One reason attributed to this fact is the simplicity of modeling and implementation of black-box linear models in several control applications.

However, it is important to note that the dynamics of most industrial processes, for example, the waterflooding process, are governed by highly nonlinear equations; therefore, the use of data-driven LTI models for model-based control strategy in such nonlinear dynamical processes suffer severe performance limitations. Thus, while LTI models can be sufficient for the purpose of control in some nonlinear dynamical systems, they usually come short in situation where the underlying system is highly nonlinear, or where the dynamics of the system varies a lot for different operating point, Tayamon (2012). To this end, nonlinear system identification is becoming popular, and the identification of nonlinear black-box model has received much attention in the past decade.

The resulting models from nonlinear system identification include the Nonlinear AutoRegressive eXogenous input (NARX) model, Nonlinear AutoRegressive Moving Average eXogenous input (NARMAX) model, Volterra model; which are considered to be the nonlinear extension of the popular AutoRegressive eXogenous input (ARX), AutoRegressive Moving Average eXogenous input (ARMAX) and Finite Impulse Response (FIR) models respectively. A special sub-group of nonlinear models often referred to as block-oriented class include the Hammerstein, Wiener and Hammerstein-Wiener model structures. This family of nonlinear models consist of linear dynamic and nonlinear static blocks connected in series; it is important to note that the nomenclature of the resulting model structure is solely based on the relative position of the linear dynamic block with respect to

40

the nonlinear static block. If the linear dynamic block is preceded by a static input nonlinear block, the model is referred to as a Hammerstein; however, if the linear dynamic block is followed by the static output nonlinear block, the model is referred to as a Wiener. Because both Hammerstein and Wiener models are basically composed of the same components connected in reverse order, one is in every sense the dual of the other. When the linear dynamic block of the model is sandwiched in between the static nonlinear blocks, we obtain a Hammerstein-Wiener model.

Generally, the various system identification techniques often involve the following steps:

1. An appropriate experiment is designed and executed on the system such that those properties that are deemed to be relevant for the model are excited.

2. A set of candidate models or model structure has to be chosen which consists usually

of a dynamic model that connects the excited inputs with the measured outputs and contains unknown parameters or free variables on various locations inside the model.

3. To determine the best model in the set, some criterion function is chosen that measures the distance between model predictions and the process measurements as a function of the free variables. By some mathematical optimization procedure this cost function is minimized to find optimal parameter values.

4. The last step is the model validation step. This aims to assess whether the model is ―good enough‖ for its purpose. Common validation tools are residual analysis and the so-called cross-validation – where the identified model is simulated using new data and the output compared to the measured output.

Note that if the initial model fails to pass the validation tests, some or all of the above steps have to be repeated iteratively, until a model that passes the validation tests is found.