Convergence of Expectation-Maximization Algorithm with Mixed-Integer Optimization

Convergence of Expectation-Maximization Algorithm with Mixed-Integer Optimization

Joseph, G. (TU Delft Signal Processing Systems)

2024

The convergence of expectation-maximization (EM)-based algorithms typically requires continuity of the likelihood function with respect to all the unknown parameters (optimization variables). The requirement is not met when parameters comprise both discrete and continuous variables, making the convergence analysis nontrivial. This paper introduces a set of conditions that ensure the convergence of a specific class of EM algorithms that estimate a mixture of discrete and continuous parameters. Our results offer a new analysis technique for iterative algorithms that solve mixed-integer non-linear optimization problems. As a concrete example, we prove the convergence of an existing EM-based sparse Bayesian learning algorithm that estimates the state of a linear dynamical system with jointly sparse inputs and bursty missing observations. Our results establish that the algorithm converges to the set of stationary points of the maximum likelihood cost with respect to the continuous optimization variables.

Discrete non-linear optimization
global convergence theorem
sparse Bayesian learning
bursty missing data

http://resolver.tudelft.nl/uuid:f2c11c39-e22c-44d8-9db2-dbd4891fd518

https://doi.org/10.1109/LSP.2024.3393352

2024-10-28

1558-2361

IEEE Signal Processing Letters, 31, 1229-1233

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

journal article

© 2024 G. Joseph