The current state-of-the-art solutions for playing Chess, are created using deep reinforcement learning. AlphaZero, the current world champion, uses ’policy networks’ and ’value network’ for selecting moves and evaluating positions respectively. However, the training of these networks are done using reinforcement learning from games of selfplay. There are many factors which determine the learning speed of reinforcement learning algorithms, where the size of the search space is a main one. In this research, we have tried to see the effect of the size of the search space on the time it takes the reinforcement learning agent to learn.