A reliable multi-agent decision-making system is highly demanded for safe and efficient operations of connected and autonomous vehicles (CAVs). In order to represent the mutual effects between vehicles and model the dynamic traffic environments, this research proposes an integrated and open-source framework to realize different Graph Reinforcement Learning (GRL) methods for better decision-making in interactive driving scenarios. Firstly, an interactive driving scenario on the highway with two ramps is constructed. The vehicles in this scenario are modeled by graph representation, and features are extracted via Graph Neural Network (GNN). Secondly, several GRL approaches are implemented and compared in detail. Finally, The simulation in the SUMO platform is carried out to evaluate the performance of different G RL approaches. Results are analyzed from multiple perspectives to compare the performance of different G RL methods in intelligent transportation scenarios. Experiments show that the implementation of GNN can well model the interactions between vehicles, and the proposed framework can improve the overall performance of multi-agent decision-making. The source code of our work can be found at https://github.com/Jacklinkk/TorchGRL.