Pre-training is a process used to enhance the learning of deep reinforcement learning (RL) algorithms through initial guidance from an expert demonstrator. This involves training a neural network to replicate the outputs of the selected expert before allowing the RL agent to specialise and develop its own policy. This paper outlines a study that aims to analyse the impact of pre-training on deep RL algorithms used in ramp metering. Specifically, behaviour cloning is performed for increasing lengths of time (0-10,000 epochs), with ALINEA as the chosen expert algorithm guiding a proposed Proximal Policy Optimisation (PPO)-based system. The results confirm that, with the same length of training, some initial guidance through pre-training can significantly improve the system’s effectiveness in reducing congestion compared to no pre-training. Otherwise, excessive pre-training may lead to overfitting and reduced generalisability. Design issues resulting in weak model convergence, however, limit the algorithm’s overall performance in the chosen scenario.

The primary goal of a strong democracy should be to most accurately represent its electorate, and the way they are divided into electoral districts can drastically affect this. As a result, many methods have been proposed to algorithmically generate fairer boundaries, the majority of which focus on eliminating bias through qualitative measures, however, these often fail to produce truly fair results. This paper, therefore, aims to demonstrate how fairness can and should become a higher priority within our electoral systems through the development, implementation and application of a new reinforcement learning-based method for algorithmic redistricting that directly optimises for fairness. Specifically, the model has been applied to the parliamentary system of the UK, filling a significant gap within the literature, meaning the paper also outlines a new metric for measuring fairness in parliamentary systems that directly rewards proportionality, the seats-votes difference. The algorithm has then been evaluated on the current parliamentary constituency boundaries in the UK and was ultimately found to fulfil all initial goals as the algorithm was able to improve the map’s fairness in all experiments performed. The paper subsequently concludes with some of the limitations of the model and the seats-votes difference and ways the redistricting algorithm could be further expanded in the future.