Uncertainty, if not explicitly accounted for in controller design, can significantly degrade the optimal control performance of greenhouse production systems. Scenario-based stochastic MPC (SMPC) addresses uncertainty by approximating its underlying probability distributions through sampling. However, SMPC rapidly becomes computationally intractable and can suffer from growing uncertainty with longer prediction horizons. Terminal costs and constraints ensure closed-loop performance of SMPC, but designing these for greenhouse systems is challenging since they rely on steady-state targets that often do not exist in greenhouse production systems. To overcome these challenges, this work introduces RL-SMPC, which uses reinforcement learning (RL) to learn a control policy that constructs both terminal region constraints and a terminal cost function. Additionally, this policy serves as a nonlinear feedback policy to attenuate uncertainty growth in the open-loop solution of scenario-based SMPC. RL-SMPC's closed-loop performance is compared against standalone RL, MPC, and scenario-based SMPC on a greenhouse lettuce model under parametric uncertainty. Simulation results showed that RL-SMPC outperformed MPC across all prediction horizons and surpassed SMPC for horizons shorter than five hours. Moreover, the results indicated that at equal online computational cost, RL-SMPC outperformed SMPC.

Achieving closed-loop hydroponics necessitates precise adjustment of individual macro- and micronutrients within the nutrient solution. However, nutrient management in hydroponics remains constrained to electrical conductivity (EC) and pH-based approaches, due to the complexity of steering individual ions and the coupling inherent in multi-element fertilizer formulations. In this study, an optimal control framework, termed OptiDose, is implemented to optimize daily fertigation strategies for hydroponically grown lettuce. The system integrates six fertilizer sources—calcium nitrate, magnesium sulfate, monopotassium phosphate, potassium nitrate, magnesium nitrate, and potassium sulfate—to maintain the concentrations of the macronutrients nitrogen (N), phosphorus (P), potassium (K), calcium (Ca), magnesium (Mg), and sulfur (S) within crop-specific adequacy ranges. Five scenarios are tested in the simulator to evaluate system performance under varying operational constraints. Results indicate that OptiDose maintained suitable nutrient concentrations for plants throughout the growth cycle—without nutrient deficiencies or toxicities—while markedly improving resource-use efficiency. Relative to a single-shot nutrient preparation (baseline), the strategy using properly sized solution tanks with daily recipe adjustment (Scenario 1) increased water-use efficiency sixfold and doubled fertilizer-use efficiency, achieving 32.3 ± 1.4 g/L and 12.3 ± 0.3 g/g, respectively. Additionally, water and fertilizer costs decreased significantly (p < 0.05), by approximately 76% and 51%, respectively. The results underscore the promise of element-specific fertigation and optimization for precision nutrient management in controlled environment agriculture.

Nonlinear model predictive control (MPC) and reinforcement learning (RL) are two powerful control strategies with complementary advantages. This work shows how actor-critic RL techniques can be leveraged to improve the performance of MPC. The RL critic is used as an approximation of the optimal value function, and an actor rollout provides an initial guess for the primal variables of the MPC. A parallel control architecture is proposed where each MPC instance is solved twice for different initial guesses. Besides the actor rollout initialization, a shifted initialization from the previous solution is used. The control actions from the lowest-cost trajectory are applied to the system at each time step. We provide some theoretical justification of the proposed algorithm by establishing that the discounted closed-loop cost is upper-bounded by the discounted closed-loop cost of the original RL actor plus an error term that depends on the (sub)optimality of the RL actor and the accuracy of the critic. These results do not require globally optimal solutions and indicate that larger horizons mitigate the effect of errors in the critic approximation. The proposed algorithm is intended for applications where standard methods to construct terminal costs or constraints for MPC are impractical. The approach is demonstrated in an illustrative toy example and an autonomous driving overtaking scenario.

We propose a frequency-domain state representation to improve the performance and reduce the computation and data requirements of reinforcement learning. This approach is tailored to tracking tasks of periodic trajectories. We apply the proposed methodology to an active knee prosthesis application. Using the high-fidelity simulator MuJoCo, we demonstrate significant performance improvements (in terms of Bellman error) for the proposed frequency-domain state representation relative to the current state-of-the-art time-domain state representation used in these applications.

We establish a collection of closed-loop guarantees and propose a scalable optimization algorithm for distributionally robust model predictive control (DRMPC) applied to linear systems, convex constraints, and quadratic costs. Via standard assumptions for the terminal cost and constraint, we establish distributionally robust long-term and stagewise performance guarantees for the closed-loop system. We further demonstrate that a common choice of the terminal cost, i.e., via the discrete-algebraic Riccati equation, renders the origin input-to-state stable for the closed-loop system. This choice also ensures that the exact long-term performance of the closed-loop system is independent of the choice of ambiguity set for the DRMPC formulation. Thus, we establish conditions under which DRMPC does not provide a long-term performance benefit relative to stochastic MPC. To solve the DRMPC optimization problem, we propose a Newton-type algorithm that empirically achieves superlinear convergence and guarantees the feasibility of each iterate. We demonstrate the implications of the closed-loop guarantees and the scalability of the proposed algorithm via two examples. To facilitate the reproducibility of the results, we also provide open-source code to implement the proposed algorithm and generate the figures.

This paper introduces a cascaded climate control framework in which a primary economic model predictive controller (EMPC) determines climate bounds for a secondary rule-based controller, based on industrial practice. The proposed controller may therefore serve as a blueprint for control design for existing greenhouse climate control systems while retaining the reliability and safety of legacy systems. The framework's performance is evaluated through simulations of a lettuce greenhouse model and compared against a state-of-the-art EMPC that controls all actuators directly. The results show that the proposed approach achieves comparable performance to the ideal state-of-the-art EMPC, demonstrating negligible performance loss from retaining rule-based control in the climate control system.

The efficient operation of greenhouses is essential for enhancing crop yield while minimizing energy costs. This paper investigates a control strategy that integrates Reinforcement Learning (RL) and Model Predictive Control (MPC) to optimize economic benefits in autonomous greenhouses. Previous research has explored the use of RL and MPC for greenhouse control individually, or by using MPC as the function approximator for the RL agent. This study introduces the RL-Guided MPC framework, where a RL policy is trained and then used to construct a terminal cost and terminal region constraint for the MPC optimization problem. This approach leverages the ability to handle uncertainties of RL with MPC's online optimization to improve overall control performance. The RL-Guided MPC framework is compared with both MPC and RL via numerical simulations. Two scenarios are considered: a deterministic environment and an uncertain environment. Simulation results demonstrate that, in both environments, RL-Guided MPC outperforms both RL and MPC with shorter prediction horizons.

Agricultural production of annual crops is often hampered by annual weeds, which compete with planted crops and persist through the collection of dormant seeds in the soil called the weed seed bank. Conventional weed management relies heavily on chemical herbicides, which are not sustainable. A complementary method that reduces the need for herbicides is ‘cultural control’, in which the crop rotation is designed in part to manage the weed population. We propose a methodology that optimizes the crop rotation, here defined as periodic crop planting densities, subject to periodic weed dynamics. We adopt a well-established model of discrete-time annual weed seed bank dynamics with crop planting density inputs, and show that any periodic weed seed bank trajectory corresponding to a periodic crop rotation is globally exponentially stable. This guarantees convergence to the optimal periodic trajectory obtained by solving a nonlinear optimal control problem with periodic constraints, which we formulate as a nonlinear program.

We establish sufficient conditions for the terminal cost and constraint such that economic model predictive control (MPC) is robustly recursively feasible and economically robust to small disturbances without any assumptions of dissipativity. Moreover, we demonstrate that these sufficient conditions can be satisfied with standard design methods. A small example is presented to illustrate the inherent robustness of economic MPC to small disturbances.