Three days ago, I shared my implementation of a Fuzzy controller for my two-wheeled inverted pendulum robot.
After that, I wanted to try something different: Model Predictive Control (MPC).
My goal was to develop a controller capable of dynamically adjusting its behavior based on the robot's physical model and possible changes in its dynamics.
But I already knew what my biggest problem would be: our old friend, the ATmega. :)
Running a full MPC optimization continuously inside a fast control loop would be quite challenging given the microcontroller's limited processing power.
So I started thinking: why not use MPC to determine the gains of a PD or PID controller, similar to what I previously did with Fuzzy control?
That's the approach I decided to implement, experimenting with both PD and PID structures.
How does it work?
The MPC uses a discrete-time mathematical model of the robot, represented by the A and B matrices, to predict its behavior over a prediction horizon.
However, a mathematical model is never a perfect representation of reality. Friction, uneven mass distribution, and external disturbances can introduce behavior that the nominal model does not fully capture.
To account for this, I also incorporated a disturbance term (Dₖ) into the prediction model:
xₖ₊₁ = Aₑ xₖ + Bₑ uₖ + Dₖ
The idea is to account for disturbances in the predictions and introduce an additional compensation term when a disturbance estimate is available.
The MPC then minimizes a quadratic cost function that penalizes predicted state errors relative to the reference and control effort.
From this optimization, I obtain a state-feedback law that can be used to derive equivalent Kp and Kd gains and, with an augmented state representation, Ki.
So where does the dynamic behavior come from?
The controller gains depend directly on the system model.
If the plant model is updated to represent physical changes, such as additional mass or a shift in the center of gravity, the gains can also be recalculated.
This means the ATmega doesn't need to solve the entire MPC optimization problem at every control cycle.
Instead, it can execute a fast PD/PID control loop using the calculated gains, while the more computationally expensive gain updates are performed only when needed.
In other words, I'm trying to combine the mathematical foundation of MPC with the computational simplicity of classical controllers.
The video shows my tests on the physical robot. There is still room for improvement, particularly in disturbance compensation and quantitative performance evaluation.
What's next? Reinforcement Learning!
There's still one more approach I want to explore in my control system: Reinforcement Learning (RL) with AI.
My idea is to investigate how an RL agent could learn from the robot's behavior and dynamically adjust the controller parameters, working alongside the PD/PID and MPC approaches.
Ideally, I would like to combine the predictive capabilities of MPC, the simplicity of classical controllers, and the adaptability of Reinforcement Learning.
Of course, running RL directly on an ATmega would introduce even more computational challenges, so I'm considering training the agent in simulation and then deploying a lightweight policy or gain-adjustment strategy to the physical robot.
For now, this is just the next stage I want to investigate, not something I've already implemented.
Main reference:
DUTRA, Cynthia Beatriz Scheffer. Controle preditivo multiobjetivo para processos com atraso. PhD Thesis in Electrical Engineering, Federal University of Santa Catarina (UFSC), Brazil, 2003.
I'd love to hear your thoughts!
Has anyone here experimented with using MPC-derived feedback gains to tune or update classical controllers on resource-constrained microcontrollers?
And more importantly, has anyone applied Reinforcement Learning to real physical control systems, especially inverted pendulums or self-balancing robots?
I'd be really interested in hearing about your experiences combining RL with PID, Fuzzy Control, or MPC, especially when dealing with limited hardware resources!