This thesis discusses the chemical composition (basicity) control problem of HIsarna, an experimental iron furnace which operates with 30% less CO2 emissions than its traditional blast furnace counterparts. The control challenge is keeping the basicity of the plant in a narrow operating region. A mass balance model of the plant was constructed - as it is common practice in the literature for traditional blast furnaces - which we combined with a parameter search method to find the most optimal model parameters from data. Next a stochastic system model of the plant was derived using the prediction errors of the plant on a new dataset. The novelty of our work is the chosen dynamic programming controller approach that we used for controller synthesis that enables optimal control of the plant with respect to the known model error distribution. A continuous Markov Decision Process based infinite horizon controller was devised by using Value Iteration to find a fixed point in our value function space with respect to the Bellman operator: our static value function. We used multilinear interpolation and a state and inputs grid to define our value function for our continuous state space. The controller map was derived from the static value function and we used multilinear interpolation again in order to obtain a continuous controller. We validate our controller by simulating the controller performance on our stochastic system model and evaluating it versus the recorded operator runs according to our running cost function defined in the Value Iteration. In summary the resulting controller outperforms the operator on average and in the worst case has comparable performance to the operator from 1000 simulation runs. Improving the controller performance further would be possible by using a more accurate system model or using a different grid parameterization than the one we used for computational efficiency reasons.