Universal Singular Optimal Control: Affine Systems

: In this study, the problem of finding an optimal controller for nonlinear systems with one input and a reference tracking signal is approached. With the problem's formulation, any desired signal can be tracked instantly with a closed-loop controller without the need for integral terms. Presentation lies at the heart of optimal control. This study, however, does not consider the integral term, allowing tracking and stability to occur naturally. It has a broad scope with a wide range of applications, namely when dealing with affine nonlinear systems, which provide geometric control unification with asymptotic stability in some cases. A common scenario that comes from optimal control involves the minimization of integral cost functionals. Issues like asymptotic stability or even tracking to the desired reference signal have always been the main limitations. In this study, the main theorem allows the solution of optimal control problems with no-integral terms, in other words tracking problems with input/state constraints, providing closed-loop controllers. A DC motor with a pendulum in upright position is an example of an application for which singular optimal control is tested in this study. The results confirm both asymptotic stability and optimal tracking with an accuracy of 95%. The main contributions of this study include an optimal closed-loop controller with no mixed initial/final conditions, input/state constraints, asymptotic stability guarantee, a strong connection with geometric tools and finally the possibility to generalize to systems with multiple inputs. As a conclusion, general nonlinear control systems can be included in the optimal control methodology presented in this study including input/state constraints. Due to the lack of integral terms, the problem can be solved in closed form by using an optimal closed-loop controller .


Introduction
Control systems are a very significant engineering branch. Nowadays, it is not possible to design any new system development without taking into consideration the control side (Ivanov et al., 2018). Different strategies to model and control real systems have been proposed throughout the control history (Kozak, 2014), however, a very common and useful approach is the well-known model-based control (Hamid and Ahmad, 2022).
Once a model is obtained, a very wide range of applications can be written as a set of non-linear Ordinary Differential Equations (ODEs) with a set of free parameters to be determined to steer the system's states to the desired location (Slotine and Li, 1991;Li et al., 2022).
In the field of control, the historical development of control strategies has been divided into two main branches (Iqbal et al., 2017): • Linear systems • Non-linear general systems While for linear systems many methodologies and algorithms can be found with well-studied and verified results, non-linear control systems are an active research area nowadays (Iqbal et al., 2017;Xiao et al., 2022).
For non-linear control systems, several criteria and methods can be applied (Kozak, 2014), all of them with their advantages and disadvantages. Moreover, two prominent general and promising strategies since their introduction are geometric control and optimal control (Isidori, 1995;Pontryagin et al., 1962;György and Galaczi, 2020).
From an optimal control point of view, very general cost functionals can be written with initial and final conditions or even with time optimization (Bertsekas, 1995;Geering, 2007;Peitz and Dellnitz, 2018).

295
However, the necessary condition given by Pontryagin´s principle requires, in the majority of optimal formulations, solving a set of ODEs with mixed-initial and final conditions, providing an open-loop control law. In contrast, Hamilton-Jacobi-Bellman (HJB), which provides a closed-loop control law, aims to solve a Partial Differential Equation (PDE) with viscous solutions (Bertsekas, 1995).
Considering the most general scenario for a control/optimal control formulation, general non-linear ODEs must be written (Chicone, 2006): , However, taking into account that many real systems and in particular, those coming from mechanical modeling, can be written in affine forms (Sarkar et al., 1994;García et al., 2009): These kinds of systems are the ones considered in the well-known geometric control scenario (Isidori, 1995;Nijmeijer and Van der Schaft, 1990).
Besides the complications of nonlinearities, the addition of input and/or state constraints adds an extra obstacle to tackle when solving nonlinear control problems (Geering, 2007).
Constraints on states/inputs are a very significant issue when dealing with real-world problems. A recent example is the recent research on energy harvesting, especially those techniques based on wave ocean energy extraction (Liu et al., 2020;Zhu et al., 2022).
On the other hand, many optimal control problems cannot ensure stability, at least in the sense of Lyapunov (Eberhardt, 1997).
The aforementioned drawbacks to solving optimal control problems can be summarized as follows: On this list, one of the most difficult obstacles is balancing mixed initial and final conditions. Moreover, taking a close look at how the mixed condition behaves, it is clear that the problem surrounds the complementary ODEs (Geering, 2007).
However, some classical optimal control formulations can be solved by avoiding mixed initial/final conditions and, instead, solving a closed-loop ODE with initial conditions. These formulations can be found in the cost function that does not include time minimization, which means the final time is not subject to minimization (free parameter). However, if optimal tracking is desired, let us assume that a time-varying reference signal or even, a nonlinear function of the states needs to be minimized instant to instant. Integral terms in the cost function cannot cover this possibility.
These reasons led to the introduction of a novel optimal control scenario utilizing singular formulations without integral terms (García and Pons, 2017;Monte et al., 2018;Garcia et al., 2020).
In this study a singular optimal control problem is solved for cost functionals that do not include integral terms (singular), allowing the time variable to be explicit. In this case, an input-state constraint can be added and for certain affine systems, asymptotic stability is also proved with a closed-loop control law, avoiding mixed initial-final conditions. This study is organized as follows: Section Preliminary machinery and notation prepare the machinery to be used in the main theorem and corollary's proofs, Section Main theorem: Optimal control in closed-form formalize the definition and proof of the main theorem and its corollary, Section Application example analyzes a complex non-linear example: The inverted pendulum's with the adding of the DC motor's dynamic, Section Materials and Methods presents the simulation machinery, Section Results and Discussion discuss the results obtained along with the contributions on this study, whereas Section Conclusion presents some conclusions, contributions, and future work.

Preliminary Machinery and Notation
In this section, the notations and optimal control results are recalled.

Lie Derivatives
To provide a neat and compact notation, successive derivatives are going to be denoted using Lie derivatives (Sastry, 2013):

Singular Optimal Control
An optimal control problem can be formulated as (Geering, 2007): Possibly with state constraints: However, when the integral term is null and the remaining term is not constant but time-dependent, the problem becomes more involved and known as a singular control problem:

Affine Control System
A general control system can be written to be: However, certain common structures appeared in mechanical systems are modeled as affine structures (Bloch and Brogliato, 2004;Sarkar et al., 1994): The next section formalizes a theorem to obtain the controller in closed form without the hard inconvenience of partial derivatives (Hamilton-Jacobi-Bellman equation) or mixed initial-final conditions (Pontryagin's principle).

Main Theorem: Optimal Control in Closed-form
Taking into consideration a general singular optimal formulation with one of the most complicated scenarios, let's say only one control input: , n x f x u x u = Using Pontryagin's principle, the following main result can be proved.

Theorem 1 (Closed-form Control)
Given a singular optimal control problem: Then, the optimal controller yields.
G(x, u) < 0: where, sign(.) is the well-known sign function. García and Agamennoni, 2008;García et al., 2009 pioneer the idea of changing the time variable, so the given optimal control problem can be reformulated to be: In this way, the problem can be converted to a nonlinear programming problem via Pontryagin's principle (Geering, 2007, pp. 49-51): Notice that, unlike the classical Pontryagin's necessary condition, where the complementary variable λ(T) is only known at time T (final time), in this study, the value of λ(t) is known at every instant of time t.
The classical formulation in optimal control does not consider the possibility for constraints in both: States and inputs, however, the proof in Geering, 2007, pp. 51 makes use of the well-known Karush-Kahn-Tucker conditions, 297 then adding the derivatives of the constraints until the input appears, it is rather straightforward to include the case in this study.
On the other hand, as in Geering, 2007, two main cases must be considered, active and inactive constraints.
G(x, u) < 0. In this case, the minimization runs without taking into account the constraints, so: The problem becomes a linear programming problem if Lg  0, so when this is not the case: G(x, u) = 0. This case is rather simple, providing:

This Completes the Proof
Note: Having formulated the optimal control problem with a cost function not containing integral terms and being time-varying, the time change of variables allows one to obtain the value of the complementary variables (t) (Pontryagin) at every instant of time. This avoids the classical drawback of mixed initial/final conditions. As mentioned in the Introduction, stability is also important in control systems, however, in this optimal control scenario, some cases can also provide asymptotic stability by choosing an appropriate Lyapunov function:

Corollary (Stability)
Given a control system to the form: This is a Lyapunov function, which is radially unbounded and zero at the equilibrium point. On the other hand, to ensure asymptotic stability (according to Lyapunov's theorems, Khalil, 2002), one needs: Then, with the controller in Theorem 1: (2) is satisfied. Finally, if Lg(x) = 0, this conforms an embedded submanifold containing the trajectories. This completes the proof.
It is clear that this corollary is useful in systems without drift terms, for instance, unicycle like-robots (Deepak et al., 2011).

Application Example
A simple nonlinear control problem that could exhibit complexity and control challenges is the well-known inverted pendulum (Boubaker, 2013). However, even when this benchmark is very well-known, a few times the control action, let's say a DC motor is included into the dynamics.
It is worth noticing that, not including the dynamics of the DC motor, would become the problem very simply: To add more complexity and, on the other hand, to take into consideration a more realistic modeling, the DC motor dynamics will add an extra state variable and a richer non-linear model to solve. Figure 1 shows a schematic picture of the inverted control pendulum, exhibiting the angle to control and the voltage-torque as the control input (DC motor).
where,  Figure 3 shows a Simulink model presented with separate components: Sign functions and cost functions. Figure 4 shows the simulation results in Matlab/Simulink, as it is clear, the angular velocity ̇ tends to zero as required, whereas the angle  tends to 180º smoothly, as required.

Materials and Methods
All the simulation were performed in a core i7 laptop with 20GB RAM, with Matlab release 2020b under ODE 113 (Adams) solver with a maximum relative tolerance error of 1e-1.

Results and Discussion
The methodology in this study turns to be universal, in the sense that any affine control system with one input can be controlled in an optimal way according to Theorem 1.
Moreover, the optimal control formulation presented avoids the problem of mixed initial/final conditions providing also a closed-loop control policy.
The asymptotic stability proved in the Corollary for a kind of affine system, becomes very useful and very uncommon in the context of optimal control.
Finally, a connection with geometric control via Lie derivatives, opens the door to more intense research and extensions to this methodology, in order to analyze observability and controllability. Also notice that the complex structure of an inverted pendulum including the DC motor dynamics can be successfully controller with the universal optimal control in this study.

Conclusion
In this study, the important concept of singular optimal control with no integral cost functional terms was formalized. Even with the pioneering work of (García et al., 2009;García and Pons, 2017), a formal and general proof including state/input constraints in this context, was lacking.
One of the main contributions in this direction is about the closed-form control law, that is, a closed-loop avoiding the classical problem o mixed initial-final conditions when applying Pontryagin's principle.
On the other hand, the connection with geometrical control using Lie derivatives opens a new direction in optimal control exploring also the future connections with non-linear controllability and observability.
A final contribution in this study is about Corollary 1, even for simplified versions of affine systems (f(x)=0) the connection with Lyapunov stability unifies the three branches of control: Optimal control, geometric control, and Lyapunov stability. The practical implication of such a methodology is rather straightforward, possessing a closed-loop rather than open-loop strategy (as it comes from pure Pontryaing's principle) the controller becomes very effective and simple to implement in real control systems.
A summary comparison between the classical Pontryagin's principle and the methodology with no integral term (singular) in this study is shown in Table 1.
In future work, further connections with geometrical control and asymptotic stability will be studied.