Balancing Stakeholder Needs in Adolescent mHealth Personalization using Multi-Objective Reinforcement Learning

Abstract

To support adolescents' well-being, mobile health (mHealth) applications aimed at promoting habits that improve well-being and prevent mental health problems are being increasingly developed. To increase uptake and sustained usage, advancements in personalization for these applications have been made. However, personalization is not straightforward, as it should consider the diverse and sometimes conflicting needs of stakeholders: adolescents may prioritize enjoyment and desire low time and effort, while clinical experts emphasize adherence and usefulness. Existing reinforcement learning (RL) approaches either only optimize for one of such outcomes, or collapse several objectives into a single scalar number. As a result, these methods may fail to adequately capture and address all stakeholder needs, especially when objectives conflict. This thesis explores the use of multi-objective reinforcement learning (MORL) for personalization in an mHealth application for adolescents that is aimed at promoting four different coping categories. Through literature, we identified relevant needs and concerns from adolescents, clinical experts, parents, and society, and translated these into the design of our personalization algorithm through model components and algorithmic choices. We implemented a model-based MORL algorithm based on decomposition (MORL/D) with policy iteration, using data from 317 participants from an existing study. This produces a set of Pareto optimal policies, each representing a different trade-off between the objectives. We further proposed and evaluated three metapolicies for selecting from this Pareto set: user choice, expert priority, and a combination of both. Simulation-based evaluations revealed that conflicting objectives exist and that no single-objective policy is able to improve all outcomes simultaneously, revealing inherent trade-offs across and within stakeholders. In contrast, the user choice metapolicy that provides users with three options from the Pareto set achieved consistent improvements across all outcomes compared to a non-personalized baseline, while addressing adolescents' desire for autonomy and control by allowing them to select from a set of good options. This approach enhances user experience, engagement, and diversity of the practiced coping strategies and thus may better equip adolescents with a broad set of coping skills. These results support the use of MORL and demonstrate it can be a promising approach for mHealth personalization when balancing diverse and potentially conflicting stakeholder needs.