The growth of renewable energy technologies is leading to energy systems that are more reliant than ever on renewables such as Wind and Photovoltaic (PV) power. Despite their benefits in terms of sustainability, their ubiquity poses challenges in maintaining grid stability given their intermittency, emphasising the prediction of power fluctuations. Physical models and statistical approaches, especially for nowcasting (forecasting for 0-6 hours in the future), have been superseded by Machine Learning (ML) methods in terms of forecast accuracy (below 3% Root Mean Squared Error (RMSE)). Within ML, Artificial Neural Network (ANN) methods seem to perform particularly well for nowcasting. This project focuses on predicting solar and wind meteorology with that level of accuracy, and on how to best use the prediction to minimize the cost of maintaining a balanced energy system, i.e. one where power consumption matches production at any moment. Producing accurate power predictions based on Multi-Modal (MM) data and the extent to which prediction accuracy reduces system cost are challenges to be addressed in this thesis. MM and End-to-End (E2E) training (with the system cost as the task of an ANN based algorithm) are investigated to this end. MM learning involves handling information from multiple types of input (audio and visual, for example) for performing a ML task such as regression or classification. It is of interest for this project because it has been shown to outperform other NN approaches in predicting sudden changes in solar irradiance. E2E learning entails an algorithm design which predicts the end goal of a ML process directly from the inputs. This is pursued because it addresses the true task (cost minimization) of system operators as the focus of the ML algorithm. The proposed method consists of a NN architecture that learns to fuse features from MM data (sky imagery and meteorological sensor data) at intermediate layers of the network in order to predict PV or Wind generation. This prediction is then used as an input to an Optimal Power Flow (OPF) problem (which seeks to minimize generation costs in a power system, considering power balance and transmission network constraints to ensure the twin goals of economic and secure system operation). The proposed model is trained E2E, therefore it is informed by the minimized cost solved by the optimization, rather than the intermediate power prediction (as conventional approaches would involve). In an IEEE 6-bus system with PV generation, a sequential training baseline results in costs 10% higher than a perfect forecast, while our proposed MM4-E2E approach achieves costs only 7% higher, a significant improvement. The intermediate prediction of PV power by MM4-E2E is also improved, with 18% lower RMSE by the proposed model compared to the baseline, explained by the enhancement of one modality by the other through MM learning. In a power system with two renewable sources, costs are reduced through the proposed model compared to a conventional approach (4% excess cost compared to 7%, measured against a perfect forecast), but power prediction accuracy is worse, sue to convergence to local minima.