In image registration the optimal transformation parameters of a given transformation model are typically obtained by minimizing a cost function. Stochastic gradient descent (SGD) is an efficient optimization algorithm for image registration. In SGD optimization, stochastic approximations of the cost function derivative are used in each iteration to update the transformation parameters. The stochastic approximation error leads to large variance in the parameters. To enforce convergence nonetheless, SGD methods are typically implemented in combination with a gradually decreasing update step size. However, selecting a proper sequence of step sizes is a major challenge in practice. An alternative strategy in numerical optimization is to use a constant step size and enforce convergence by averaging the parameters obtained by SGD over several iterations. It was proven mathematically that the highest possible rate of convergence is achieved in this way. Inspired by this work, we propose an averaged SGD (Avg-SGD) method for efficient image registration. In the Avg-SGD approach, a constant step size is used, in combination with an exponentially weighted iterate averaging scheme. Experiments on 3D lung CT scans demonstrate the effectiveness of the Avg-SGD method in terms of convergence rate, accuracy and precision.

Image registration is typically formulated as an optimization process, which aims to find the optimal transformation parameters of a given transformation model by minimizing a cost function. Local minima may exist in the optimization landscape, which could hamper the optimization process. To eliminate local minima, smoothing the cost function would be desirable. In this paper, we investigate the use of a randomized smoothing (RS) technique for stochastic gradient descent (SGD) optimization, to effectively smooth the cost function. In this approach, Gaussian noise is added to the transformation parameters prior to computing the cost function gradient in each iteration of the SGD optimizer. The approach is suitable for both rigid and nonrigid registrations. Experiments on synthetic images, cell images, public CT lung data, and public MR brain data demonstrate the effectiveness of the novel RS technique in terms of registration accuracy and robustness.