In many fields we are interested in inference for a complex stochastic process given limited observations regarding its state over time. This thesis therefore introduces an expectation propagation approach to backward filtering forward guiding for high-dimensional finite-state space models. The backward filtering forward guiding method is first derived for such models with a specific emphasis on the temporal dynamics, after which factorised guiding terms which exploit the inherent structure of the latent state space are introduced. Performance of the method is assessed by comparing numerical results for statistical inference of a Susceptible-Infected-Recovered example problem. The expectation propagation approach performs comparably with existing methods in a particularly simple setting where state variables are observed individually, and performs very well in a more difficult setting which the familiar methods can not deal with. We conclude that a more advanced treatment of the approximate likelihood filtering phase may be warranted in such complex settings. The research summarised in this work provides a first effort towards the development of more general expectation propagation based backward filtering procedures for other types of high-dimensional sequential data models. The work additionally elucidates connections between backward filtering with backward marginalisation and alternative approximate likelihood filtering procedures, suggesting multiple avenues for future research.