Gas Source Localization (GSL) is a challenging field of research within the robotics community. Existing methods vary widely and each has its own strengths and weaknesses. Existing GSL evaluations vary in environment size, wind conditions, and gas simulation fidelity, thereby complicating objective comparison between algorithms. They also lack photo-realistic rendering for the integration of obstacle avoidance. In this paper, we propose GSL-Bench, a benchmarking suite to evaluate the performance of GSL algorithms. GSL-Bench features high-fidelity graphics and gas simulation. Realism is further increased by simulating relevant gas and wind sensors. Scene generation is simplified with the introduction of AutoGDM+, capable of procedural environment generation, CFD and particle-based gas dispersion simulation. To illustrate GSL-Bench's capabilities, three algorithms are compared in six warehouse settings of increasing complexity: E. Coli, dung beetle and a random walker. Our results demonstrate GSL-Bench's ability to provide valuable insights into algorithm performance.