Data Reconciliation

9.3.1. State observers and estimation

Before the 1960’s the contemporary control system designs required that all state variables (variables that contain essential information that describe the behaviour of a system) of a particular system be available for measurement. Control system design for many practical situations were limited since only a few outputs can typically be measured. The theory of state observers was introduced in the 1960’s to address the problem and construct estimates of state variables from limited input and output measurements of a system (Luenberger, 1963: 74).

The Kalman filter is one of the first optimal state estimators, typically used for linear dynamic systems with variables contaminated by normally distributed noise. Research of Kalman filters is abundant in state estimation literature and was introduced in the 1960’s especially for sampled (discrete) dynamic systems (Luenberger, 1963: 74; Mohd Ali, Ha Hoan, Hussain & Dochain, 2015: 28). Constrained, extended and unscented Kalman filters have been developed to extend the application of simple linear Kalman filters to more complex problems. Kalman filters and their

139 extensions are known as Bayesian estimators in that priori process knowledge is used to acquire posteriori information and that essentially maximum likelihood principles are used to minimise the difference between the values of estimated and true states of process variables (Mohd Ali, Ha Hoan, Hussain & Dochain, 2015: 28).

Similar reasoning to the Kalman filter can be used to develop state observers for steady state systems. Steady state observers are useful in chemical and especially metallurgical plants where it is desired to maintain design conditions. This will typically be an optimal steady state for stable operation and maximum product quality. The necessary observer can be derived via the maximum a posteriori method shown by Equation 9.15 (Romagnoli & Sánchez, 2000: 200). The derivation is shown in Romagnali & Sánchez (200: 201).

max

𝑥 𝑃𝑟(𝑋|𝑌) = max𝑋

𝑃_𝑟(𝑌|𝑋)𝑃_𝑟(𝑋)

𝑃𝑟(𝑌) (Eq. 9.15)

Where

• The state vector is 𝑋.

• The measurement vector is 𝑌.

• 𝑃𝑟(𝑌|𝑋) is the likelihood distribution of 𝑌 given 𝑋. 𝑃𝑟(𝑋) and 𝑃𝑟(𝑌) is typically known as the prior and evidence terms (Von der Linden, Dose & Von Toussaint, 2014: 28).

In Equation 9.15 𝑋 represents the plant variables’ estimated states and 𝑌 represents their measurements corrupted by noise. The maximum a posteriori method seeks to maximize the probability of the estimated states given the measurements by using Bayes rule and assuming a suitable probability distribution model for 𝑃𝑟(𝑌|𝑋). When uninformative priors (such as uniform distributions) are used, Equation 9.15 will reduce to an equivalent maximum likelihood estimation problem. Typically, one will obtain a constrained weighted least squares estimation problem (from Equation 9.15) for the steady state system case. This creates the opportunity to not only acquire reliable estimates of measured variables but also to estimate unmeasured variables (given that a minimum estimability constraint is met) (Romagnoli & Sánchez, 2000: 53).

In the 1980s use of data reconciliation gained popularity in the mineral processing industry. Metallurgical plants typically have many unmeasured variables and measurements that are made usually have large errors. Development of software such as the BILMAT algorithm (Hodouin, Kasongo, Kouame & Everell, 1981) allowed reconciliation of mass balances in flotation and comminution circuits (improving the analysis and quality of data obtained during measurement campaigns). Hodouin et al. (1981) showed that metallurgical plant flow rate, composition and PSD data can be successfully reconciled (in a maximum likelihood sense).

Data reconciliation has been implemented in past spiral research that focused on iron ore concentration (Sadeghi, Bazin & Renaud, 2016: 53; Sadeghi, 2015). Sadeghi et al. (2016: 53) used the

140 BILMAT algorithm to reconcile mass flow rates of different mineral species at different particle sizes. The analysis provides the intended precise estimates but also serves as a method to determine the quality of experimental data. Sadeghi et al. (2016: 53) found that the majority of their data points were adjusted by less than 3 % of the measured value(s) and concluded that it validates the quality of the data obtained (thus further analysis could continue with more confidence).

9.3.2. Data processing, reconciliation and rectification

Confident estimates of a plant’s state (including variables such as flow rates, temperatures and concentrations) are necessary before a process can be optimised, evaluated or controlled. Process models and measurements are typically used to evaluate the behaviour of a plant and can be reconciled to provide more precise estimates of plant states (Romagnoli & Sánchez, 2000: 2; Johnston & Kramer, 1998: 591). Mass and energy balances (fundamental models) can be used to formulate process models while stochastic models are used to represent how measurements are inferred and behave (Romagnoli & Sánchez, 2000: 2).

The measurement system is typically expressed as 𝑦 = 𝑞(𝑋) + 𝑒 with 𝑒 ~ 𝑁(0, 𝑉𝑒) and the system of process models as 𝑧(𝑋) = 𝜖 with 𝜖 ~ 𝑁(0, 𝑉𝜖) (Romagnoli & Sánchez, 2000: 13). In symbolic terms a data reconciliation problem can be stated by Equation 9.16 to Equation 9.18 in the form of a weighted least squares problem (Romagnoli & Sánchez, 2000: 13).

min 𝑋 𝐽 = 𝑒 𝑇_𝑉 𝑒−1𝑒 + 𝜖𝑇𝑉𝜖−1𝜖 (Eq. 9.16) With constraints: 𝑧(𝑋̂) = 0 (Eq. 9.17) ℎ(𝑋̂) ≤ 0 (Eq. 9.18)

Here it is tacitly understood that errors of the measurement and model are Gaussian (as previously stated). Equations 9.16 to 9.18 are not formulated for the case of gross error (also called systematic or bias error) detection but is still useful in the case of steady state operation and design of sensor networks (Romagnoli & Sánchez, 2000: 114).

Figure 9.10 shows the probability distribution of a process variable measurement and its process model value and how the variable estimate compares with them. The measurement has the largest standard deviation with the process model providing a more precise distribution around its mean. The estimate value has the smallest standard deviation and, therefore, has a much larger probability density value around the estimate mean (indicating a more likely state).

141

Figure 9.10: State estimation example

Looking toward simple estimation equations one can see why the estimate will be more precise: • Variable 𝑋₁’s estimate is obtained via a transform of the measurement 𝑦:

𝐽 =(𝑦 − 𝑋1) 2_𝜎 𝑒−2 2 + (𝑋1− Ε(𝑋1))2𝜎𝜖−2 2 0 = 𝜕𝐽/𝜕𝑋1= −(𝑦 − 𝑋̂1)𝜎𝑒−2+ (𝑋̂1− Ε(𝑋1))𝜎𝜖−2 𝑦𝜎𝑒−2+ Ε(𝑋1)𝜎𝜖−2= 𝑋̂1(𝜎𝑒−2+ 𝜎𝜖−2) 𝑋̂1= 𝑦𝜎𝑒−2+ Ε(𝑋1)𝜎𝜖−2 (𝜎𝑒−2+ 𝜎𝜖−2)

• Variable 𝑋₁’s variance can be obtained by transforming the measurement variance 𝜎̂2_:

𝑉𝑎𝑟(𝑋̂1) = 𝜎𝑒−2 (𝜎𝑒−2+ 𝜎𝜖−2) 𝑉𝑎𝑟(𝑦) 𝜎𝑒 −2 (𝜎𝑒−2+ 𝜎𝜖−2) , 𝑉𝑎𝑟(𝑦) = 𝜎𝑒2 𝜎̂2 ₌ 𝜎𝑒 −2 (𝜎_𝑒−2_{+ 𝜎} 𝜖−2)2

In the simple case of Figure 9.10 the estimate variance will at least either be equal to the measurement variance or lower than it. This is one of the desirable properties of state estimation allowing us to produce more reliable data on the current state of a system. State estimation alone can produce more

142 trustworthy data but in the context of a chemical or metallurgical plant may still provide dubious results (Romagnoli & Sánchez, 2000: 4).

Reconciliation of plant data involves estimating plant state variables that are consistent with relevant conservation equations of a system (Romagnoli & Sánchez, 2000: 4). Measurements of state variables are typically corrupted by either uncorrelated errors (noise) and can also be affected by systematic or gross errors (Johnston & Kramer, 1998: 591). Thus, data reconciliation provides a means to transform observed state variables to more precise estimates that satisfies the physical constraints of the plant.