Optimizing asphalt mix design at the indoor stage is of significant importance for enhancing the rutting resistance of asphalt mixture, which is affected by its structural characteristics. In this work, the coordination number of particle contact force (CNpcf) was proposed as an indicator to represent contact characteristics of skeleton structure aggregates in asphalt mixture. Nine asphalt mixtures with different gradations were designed, and the relationship of CNpcf with the number of aggregate contact zones (CZ) was established by combining rutting tests and digital image processing technique (DIP). The Mann-Whitney U test was implemented to analyze the distribution properties of inter-particle contacts before and after the rutting test. In addition, the resistance to the further expansion of rutting was analyzed. The results revealed a significant positive correlation (PCCs = 0.843, R2 = 0.711) between CNpcf and CZ. The content of coarse aggregates in the dominant structure did not exhibit monotonic related to anti-rutting performance of the asphalt mixture. Therefore, an optimum aggregate content of 57% was utilized. The Mann-Whitney U test revealed that the mesoscale skeleton structure of the asphalt mixes before and after rutting exhibited excellent stability. This study further indicated the applicability of combining CNpcf to adjust the mix design to enhance the rutting resistance of asphalt mixture and to prevent rutting expansion in flexible pavement.@en