Surface-related multiple elimination remains one of the most robust primary estimation approaches for decades, in which the adaptive subtraction step is a non-trivial task. Due to imperfections in the made assumptions during prediction, the perfect adaptive subtraction is a highly non-linear and non-stationary process, which is suitable for the popular deep learning (DL)-based image processing. Different from the most straightforward DL-based adaptive subtraction (i.e., the full wavefield and the advanced estimated primary training pair), we propose to include both the original full wavefield and the initial globally estimated surface multiples as the two-channel input, and train a DL neural network (U-Net) on synthetic modeled primaries. In this way, the robust physics (i.e., the globally estimated multiples) is utilized, and the ground truth primary labels can be beneficial to the framework. Both synthetic and field examples are provided to demonstrate the current performance of our proposed framework.