Asphalt mixtures show complex mechanical behavior due to their heterogeneous structure. Traditionally, the mechanical characterization of asphalt mixture is done through laboratory testing or micromechanical modeling. While laboratory tests and micromechanical models provide reliable measurements and physical interpretability, they are often resource-intensive and demand extensive calibration. Recent advances in machine learning address some of the above issues by enabling accurate predictions, though often lacking physical interpretability and stability. Hence, this study aims to present a novel micromechanics-infused neural network (MINN) framework for predicting asphalt mixture stiffness. The framework embeds micromechanical principles derived from the modified Hirsch model into the neural network's loss function, allowing the model to learn from experimental data while adhering to micromechanics-based constraints. In this study, feature selection is performed using BorutaShap, and Bayesian optimization is applied for hyperparameter tuning. Results show that MINN improves prediction accuracy, interpretability, and robustness.