Online optimal control of uncertain nonlinear systems is a challenging problem due to the difficulty of solving the Hamilton-Jacobi-Bellman (HJB) equation which does not have a closed-form solution. In addition, controlling a nonaffine nonlinear discrete- time system in general is a major challenge due to the coupled nonlinear relationship between the states and the control input within the unknown nonlinearity. Recently, neural networks (NN) as online approximators have been successfully applied to learn the uncertain nonlinear system dynamics in an online fashion because of their universal function approximation property.
The NN-based optimal control of affine nonlinear systems is now available in the literature either in continuous or discrete-time systems [1]-[4] by using HJB equation in forward-in-time manner via value and policy iterations. While [1] and [1] present offline based schemes, others [3] address optimal control in an online manner for affine nonlinear discrete-time systems.
In [1] and [3] the input gain matrix1 (IGM) of the affine system is considered known while the internal system dynamics are considered unknown. The work in [5] introduces an adaptive dynamic programming (ADP)-based scheme for optimal control of unknown affine systems. The authors in [1] and [6] deal with online optimal control of affine nonlinear system whose input gain matrix (IGM) is considered known. Here in these works [1] and [6], the cost function is estimated through the HJB equation offline,
1
In a general form of discrete time affine systems i.e. , and are considered as internal dynamics and input gain matrix respectively.
1 ( ( )) ( ( ))
k k
whereas the work in [3] estimates the cost function with an online NN based estimator while proving the overall convergence of the NN based controller. In [6], convergence of the heuristic dynamic programming algorithm (HDP) via value and policy iterations is demonstrated and closed-loop stability is not shown. It is found that an insufficient number of iterations in the value and policy iteration-based optimal control schemes [3,6] will not only cause convergence issues but also instability. Therefore the optimal controller in [3] is developed without using value and policy iterations and closed-loop stability analysis is demonstrated. However, all these methods [1-6] assume that the states of the system are measurable. Unfortunately, in many practical applications, such as the proposed control of HCCI engines, states are not available which necessitates an output feedback based optimal control scheme.
Therefore, this paper addresses forward-in-time based optimal control of unknown nonaffine MIMO discrete-time systems by transforming the nonaffine nonlinear discrete-time system into an affine-like equivalent system in the input-output form with higher order terms. The input-output form relaxes the need for state availability. Next, a NN identifier is proposed to learn the unknown IGM matrix online whose estimation is required in the optimal controller design. Next, in order to mitigate the modeling errors due to higher order terms, an auxiliary term is designed via fast dynamic inversion technique. The fast dynamic solver, along with the closed loop system, forms a singularly perturbed system whose stability is shown to be guaranteed. Thus this auxiliary term ensures robustness against modeling errors and reduces the ultimate bounds of the closed-loop system by mitigating the effect of higher order terms.
Subsequently, the forward-in-time approach similar to [3] is introduced to the generic unknown affine-like equivalent system by using output feedback without using value and policy iterations. Here, the value function and the control input are updated once a sampling interval. Using an initial stabilizing control, a NN online approximator (OLA) is tuned to learn the cost function which is subsequently utilized along with the estimated IGM to generate the optimal control input. The nonaffine nature of the system, online identifier and lack of system states complicate the stability analysis whereas the boundedness of all the closed-loop signals and the actual control input to the optimal value are demonstrated. The net result is the output feedback-based robust optimal controller design for nonaffine MIMO nonlinear discrete-time systems.
Finally, the proposed optimal controller is applied to a homogeneous charge compression ignition (HCCI) engine which is a practical example of an unknown nonlinear MIMO discrete-time system with structural uncertainties due to variations in the fuel type or ambient operating conditions [12]. Compared to regular spark ignition (SI) engines, the HCCI engines have the advantage of increased thermal efficiency and low nitrous oxides, NOx, and particulate matter emissions [8]-[10]. The HCCI engines do not have an ignition system, and managing the combustion appears to be a challenging control problem. In other words, achieving and maintaining HCCI mode of operation in diverse operating situations requires an appropriate closed loop control strategy. The control approach should be optimal under a variety of fuel types which in turn imposes a variety of combustion chemical kinetics and that being unknown, necessitates an online learning feature for the controller.
Due to complex engine dynamics [11] and the presence of uncertainties, the HCCI engine dynamics in nonaffine form are transformed into the input-output form for control purposes. Numerical results show that the proposed NN-based robust optimal control scheme can successfully control the engine dynamics and is able to adaptively tune the initial admissible controller to attain an optimal controller online. The paper is organized as follows.
The state space representation of the MIMO system dynamics are given in Section II. Section III introduces a system identification technique while Section IV establishes an overall robust online optimal control approach where the identified information of the system dynamics is used. Here the robust term is utilized first to mitigate the higher order terms that appear as the result of transformation from nonaffine to affine nonlinear discrete-time system. Next it is shown, in this section, that any initial admissible controller can be tuned to an optimal online controller that minimizes a desired performance index. Section V introduces the experimentally validated representation of a HCCI engine and performance of the proposed online optimal controller.
II. STATE SPACE REPRESENTATION OF THE OUTPUT CONTROL