Uncertainty, if not explicitly accounted for in controller design, can significantly degrade the optimal control performance of greenhouse production systems. Scenario-based stochastic MPC (SMPC) addresses uncertainty by approximating its underlying probability distributions through sampling. However, SMPC rapidly becomes computationally intractable and can suffer from growing uncertainty with longer prediction horizons. Terminal costs and constraints ensure closed-loop performance of SMPC, but designing these for greenhouse systems is challenging since they rely on steady-state targets that often do not exist in greenhouse production systems. To overcome these challenges, this work introduces RL-SMPC, which uses reinforcement learning (RL) to learn a control policy that constructs both terminal region constraints and a terminal cost function. Additionally, this policy serves as a nonlinear feedback policy to attenuate uncertainty growth in the open-loop solution of scenario-based SMPC. RL-SMPC's closed-loop performance is compared against standalone RL, MPC, and scenario-based SMPC on a greenhouse lettuce model under parametric uncertainty. Simulation results showed that RL-SMPC outperformed MPC across all prediction horizons and surpassed SMPC for horizons shorter than five hours. Moreover, the results indicated that at equal online computational cost, RL-SMPC outperformed SMPC.