Pore structure characteristics of cementitious materials play a critical role in the transport properties of concrete structures. This paper develops a novel framework for modeling chloride penetration in concrete, considering the pore structure-dependent model parameters. In the framework, a multi-scale transport model was derived by linking the chloride diffusivities with pore size distributions (PSDs). Based on the three-dimensional (3D) microstructure generated by “porous growth” and “hard core-soft shell” methods, two sub-models were computationally developed for determining the multi-modal PSDs and pore size-related chloride diffusivities. The predicted results by these series of models were compared with corresponding experimental data. The results indicated that by adopting pore size-related diffusivities, even if the total porosities were the same, the proposed multi-scale chloride transport model could better capture the effect of different PSDs on chloride penetration profiles, while the model without pore structure-depended parameters would ignore the differences. Compared with the reference transport models, which adopt averaged chloride diffusivities, the chloride penetration depths predicted by the proposed multi-scale model are in better agreement with experimental data, with 10%–25% reduced prediction error. This multi-scale transport model is hoped to provide a novel computational approach on 3D microstructure generation and better reveal the underlying mechanism of the chloride penetration process in concrete from a microscopic perspective.