Parallel imaging is a widely-used acceleration technique for magnetic resonance imaging (MRI). Conventional linear reconstruction approaches in parallel imaging suffer from noise amplification. Recently, a non-linear method that utilizes subject- specific convolutional neural networks for k-space reconstruction, Robust Artificial-neural-networks for k-space Interpolation (RAKI) was proposed and shown to improve noise resilience over linear methods. However, the linear convolutions still provide a sufficient baseline image quality and interpretability. In this paper, we sought to utilize a residual network architecture to combine the benefits of both the linear and non-linear RAKI reconstructions. This hybrid method, called residual RAKI (rRAKI) offers significant improvement in image quality compared to linear method, and improves upon RAKI in highly- accelerated simultaneous multi-slice imaging. Furthermore, it establishes an interpretable view for the use of CNNs in parallel imaging, as the CNN component in the residual network removes the noise amplification arising from the linear part.