District heating systems (DHSs) have the potential to play a big part in the energy transition. The efficient operation of DHSs is therefore also an important subject of study. The operation of DHSs where combined heat and power (CHP) plants are used are particularly interesting, because CHPs can operate with high efficiency. In this work, the operational optimization of DHSs with CHP plants is considered. Determining the optimal heat and electricity production for CHPs for multiple time steps into the future is a complex problem. Because of the heat storage capabilities in the network many solutions are feasible, but determining which solutions are infeasible because of constraint violations in the DHS involves computing time delays that depend on complex network dynamics. In this work, the possibility of using an input convex neural network (ICNN) to learn the network dynamics of a DHS is explored. ICNNs have limitations on their learning capabilities, but theoretically allow for easier optimization. Experiments on the learning capabilities of ICNNs reveal that caution should be used when they are used to learn non-convex constraints, as the accuracy of the ICNN highly depends on how non-convex the function is. Experiments on the feasible space of supply temperatures to a small district heating network (DHN) reveal that although the network does not provide the same flexibility as heat storage tanks, still some flexibility in the operation can be found. This is due to the fact that water with a higher supply temperature is consumed by consumers at a slower pace and this increases the time delay between production and consumption. Supply temperatures that follow can then be lowered if the increased time delay causes this water to arrive when the heat demands are lower. In the experiments it was found that this flexibility in operation translates to non-convex areas in the feasible space. When this space would be learned by an ICNN, this space would be made convex. How much of the flexibility would be removed by doing this is yet unknown and could be researched in future work. Other future work can be done on safely learning non-convex constraints with an ICNN.