With the fast integration of Machine Learning (ML) across industries, effective pedagogical strategies are essential for teaching this complex and evolving field. Machine Learning is now widely integrated into various university programs and introduced at earlier educational stages, including high school and secondary school. However, ML pedagogy lacks standardized teaching methods compared to other science-related subjects, which have established norms for topic introduction, teaching tools, and assessment methods. Inspired by other fields, this research explores the use of interactive visualizations in teaching ML topics, more specifically in teaching Gradient Descent (GD) and Principal Component Analysis (PCA). The target population consists of Computer Science and Engineering Bachelor students who have not yet followed any Machine Learning courses but have foundational knowledge in calculus, linear algebra, and statistics. The evaluation measures knowledge gained and student motivation, compared to a static version of the materials. Results show a significant positive effect in knowledge related to PCA with interactive visualizations, but no differences in knowledge gain for GD or in learning motivation for either topic. With these results, we contribute to the body of evidence-based teaching methods in Machine Learning and identify further research needed to generalize the effect of interactive visualizations as a teaching method for teaching ML basic concepts.

Background: Despite several randomised controlled trials (RCTs) on the use of adjuvant treatment with corticosteroids in patients with community-acquired pneumonia (CAP), the effect of this intervention on mortality remains controversial. We aimed to evaluate heterogeneity of treatment effect (HTE) of adjuvant treatment with corticosteroids on 30-day mortality in patients with CAP. Methods: In this individual patient data meta-analysis, we included RCTs published before July 1, 2024, comparing adjuvant treatment with corticosteroids versus placebo in patients hospitalised with CAP. The primary endpoint was 30-day all-cause mortality, collected across all trials, and analyses followed the intention-to-treat principle. We analysed HTE using risk and effect modelling. For risk modelling, patients were classified as having less severe or severe CAP based on the pneumonia severity index (PSI), comparing PSI class I–III versus class IV–V. For effect modelling, we trained a corticosteroid-effect model on six trials and externally validated it using data from two trials, received after model preregistration. This model classified patients into two groups: no predicted benefit and predicted benefit from adjuvant treatment with corticosteroids. The literature search was registered on PROSPERO, CRD42022380746. Findings: We included eight RCTs with 3224 patients. Across all eight trials, 246 (7·6%) patients died within 30 days (106 [6·6%] of 1618 in the corticosteroid group vs 140 [8·7%] of 1606 in the placebo group; odds ratio [OR] 0·72 [95% CI 0·56–0·94], p=0·017). The corticosteroid-effect model, which selected C-reactive protein (CRP), showed significant HTE during external validation in the two most recent trials. In these trials, 154 (11·4%) of 1355 patients died within 30 days (88 [13·1%] of 671 in the placebo group vs 66 [9·6%] of 684 in the corticosteroid group; OR 0·71 [95% CI 0·50–0·99], p=0·044). Among patients predicted to have no benefit (CRP ≤204 mg/L, n=725), no significant effect was observed (OR 0·98 [95% CI 0·63–1·50]), whereas for those with predicted benefit (CRP >204 mg/L, n=630), 39 (13·0%) of 301 patients died in the placebo group compared with 20 (6·1%) of 329 in the corticosteroid group (0·43 [0·25–0·76], p_interaction=0·026). No significant HTE was found between less severe CAP (PSI class I–III, n=229) and severe CAP (PSI class IV–V, n=1126). Corticosteroid therapy significantly increased hyperglycaemia risk (44 [12·8%] of 344 in the placebo group vs 84 [24·8%] of 339 in the corticosteroid group; OR 2·50 [95% CI 1·63–3·83], p<0·0001) and hospital re-admission risk (30 [3·7%] of 814 in the placebo group vs 57 [7·0%] of 819 in the corticosteroid group; 1·95 [1·24–3·07], p=0·0038). Interpretation: Overall, adjuvant therapy with corticosteroids significantly reduces 30-day mortality in patients hospitalised with CAP. The treatment effect varied significantly among subgroups based on CRP concentrations, with a substantial mortality reduction observed only in patients with high baseline CRP. Funding: None.

Objective and continuous monitoring of Parkinson’s disease (PD) tremor in free-living conditions could benefit both individual patient care and clinical trials, by overcoming the snapshot nature of clinical assessments. To enable robust detection of tremor in the context of limited amounts of labeled training data, we propose to use prototypical networks, which can embed domain expertise about the heterogeneous tremor and non-tremor sub-classes. We evaluated our approach using data from the Parkinson@Home Validation study, including 8 PD patients with tremor, 16 PD patients without tremor, and 24 age-matched controls. We used wrist accelerometer data and synchronous expert video annotations for the presence of tremor, captured during unscripted daily life activities in and around the participants’ own homes. Based on leave-one-subject-out cross-validation, we demonstrate the ability of prototypical networks to capture free-living tremor episodes. Specifically, we demonstrate that prototypical networks can be used to enforce robust performance across domain-informed sub-classes, including different tremor phenotypes and daily life activities.

Aims We aimed to compare performances of conventional survival models with machine learning (ML) survival models for incident heart failure (HF) in men and women without prevalent HF, cardiomyopathy (CM) or ischaemic heart disease (IHD), and to identify potential high-risk precursors overlooked by conventional survival models. Methods and results We predicted 10-year risk of incident HF in 266 306 women (2894 events) and 212 061 men (4213 events). We constructed multivariable Cox models, first using ∼ 400 baseline characteristics, and subsequently only those remaining after LASSO stability selection. We also used Random Survival Forest (RSF) and eXtreme Gradient Survival Boosting (XGBoost). Performances were assessed using internal cross validation and hold-out sets, with C-indices, calibration curves and net-benefit analyses. Model performances were comparable during internal validation: XGBoost (C-index ± SE) (men: 0.79 ± 0.0040, women: 0.83 ± 0.0023) showed similar performance to the multivariable Cox model (men: 0.80 ± 0.0031, women: 0.83 ± 0.0022) and Cox models after LASSO stability selection, while RSF showed numerically slightly lower performance (men: 0.78 ± 0.0025, women: 0.81 ± 0.0015). Findings were similar in the hold-out sets. Age, cystatin-C, lifetime treatments/medications, other heart disease, systolic blood pressure, and spirometry measures were identified as high-risk factors in both model types for both sexes. Additionally, sex-specific and model-specific risk factors were identified. Conclusion Machine learning models and Cox proportional hazard models performed well and similarly for 10-year incident HF risk prediction in the general population. However, sex-specific and model-specific risk predictors were found. Spirometry measures, rarely included in existing models, were identified as important risk factors. Our results suggest that ML models for HF prediction in the general population reveal insights that would otherwise remain unnoticed.

Introduction: Dynamic survival analysis has become an effective approach for predicting time-to-event outcomes based on longitudinal data in neurology, cognitive health, and other health-related domains. With advancements in machine learning, several new methods have been introduced, often using a two-stage approach: first extracting features from longitudinal trajectories and then using these to predict survival probabilities.

Methods: This work compares several combinations of longitudinal and survival models, assessing their predictive performance across different training strategies. Using synthetic and real-world cognitive health data from the Alzheimer's Disease Neuroimaging Initiative (ADNI), we explore the strengths and limitations of each model.

Results: Among the considered survival models, the Random Survival Forest consistently delivered strong results across different datasets, longitudinal models, and training strategies. On the ADNI dataset the best performing method was Random Survival Forest with the last visit benchmark and super landmarking with an average tdAUC of 0.96 and brier score of 0.07. Several other methods, including Cox Proportional Hazards and the Recurrent Neural Network, achieve similar scores. While the tested longitudinal models often struggled to outperform simple benchmarks, neural network models showed some improvement in simulated scenarios with sufficiently informative longitudinal trajectories.

Discussion: Our findings underscore the importance of aligning model selection and training strategies with the specific characteristics of the data and the target application, providing valuable insights that can inform future developments in dynamic survival analysis.

A major challenge in estimating treatment effects in observational studies is the reliance on untestable conditions such as the assumption of no unmeasured confounding. In this work, we propose an algorithm that can falsify the assumption of no unmeasured confounding in a setting with observational data from multiple heterogeneous sources, which we refer to as environments. Our proposed falsification strategy leverages a key observation that unmeasured confounding can cause observed causal mechanisms to appear dependent. Building on this observation, we develop a novel two-stage procedure that detects these dependencies with high statistical power while controlling false positives. The algorithm does not require access to randomized data and, in contrast to other falsification approaches, functions even under transportability violations when the environment has a direct effect on the outcome of interest. To showcase the practical relevance of our approach, we show that our method is able to efficiently detect confounding on both simulated and semi-synthetic data.

Background
Switching from controlled to assisted ventilation is crucial in the trajectory of intensive care unit (ICU) stay, but no guidelines exist. We described current practices, analyzed patient characteristics associated with switch success or failure, and explored the feasibility to predict switch failure.

Methods
In this retrospective study, we obtained highly granular longitudinal ICU data sets from three medical centers, covering demographics, severity scores, vital signs, ventilation, and laboratory parameters. The primary endpoint was switch success, considering a switch attempt to be successful if a patient did not return to controlled ventilation for the next 72 h while alive, and to be failed otherwise. We compared the characteristics of patients with successful vs. failed first switch attempts at ICU admission, immediately before, and 3 h after the attempt. We trained LASSO logistic regression models to predict switch failure.

Results
In 4524/6715 (67%) patients attempting a switch, the first attempt failed. The first switch attempt, regardless of success or failure, was generally made at normalized PaCO2 and pH levels, with PEEP < 10 cmH2O and PaO2/FiO2 indicating mild injury. Despite very similar baseline disease severity, switch failure was associated with significantly worse outcomes, including a 28-day mortality of 27% vs. 16% and median ventilator-free days of 16 vs. 22 (p < 0.001). Failed attempts were initiated significantly earlier than successful ones (median 1.8 vs. 1.3 days, p < 0.001). Before the switch, PaO2/FiO2, if measured at PEEP > 10 cmH2O, and respiratory system compliance was lower in patients with switch failure (median 185 vs. 205 mmHg, p < 0.001; 39 vs. 41 mL/cmH2O, P = 0.001), and post-switch, patients with switch failure experienced greater deterioration in gas exchange and minimal improvement in ventilatory parameters post-switch. Contrary to our hypotheses, patient characteristics for failed vs. successful switches were surprisingly similar, resulting in prediction models with limited discriminative performance.

Conclusions
Approximately two-thirds of attempts to switch patients to assisted ventilation fail, which are associated with significantly worse clinical outcomes, despite similar baseline disease severity. Contrary to our hypotheses, patients with successful and failed attempts showed similar characteristics, making switch failure difficult to predict. These findings underscore the importance of preventing switch failures and, given the retrospective nature of this study, highlight the need for prospective studies to better understand the reasons for switch failure and when spontaneous breathing can be safely initiated.

In her pioneering work, Calfee et al. [1] addressed the clinical and biological heterogeneity of acute respiratory distress syndrome (ARDS), a factor likely contributing to the poor track record of randomized trials (RCTs) in this patient population. Using latent class (or profile) analysis (LCA), a method for identifying unobserved subgroups from observed data, they identified two distinct ARDS sub-phenotypes (hypo- and hyperinflammatory), which showed association with clinical outcomes and, crucially, heterogeneity of treatment effect (HTE) [2], demonstrating different responses to higher vs. lower PEEP regimes. [...]

Prediction models are popular in medical research and practice. Many expect that by predicting patient-specific outcomes, these models have the potential to inform treatment decisions, and they are frequently lauded as instruments for personalized, data-driven healthcare. We show, however, that using prediction models for decision-making can lead to harm, even when the predictions exhibit good discrimination after deployment. These models are harmful self-fulfilling prophecies: their deployment harms a group of patients, but the worse outcome of these patients does not diminish the discrimination of the model. Our main result is a formal characterization of a set of such prediction models. Next, we show that models that are well calibrated before and after deployment are useless for decision-making, as they make no change in the data distribution. These results call for a reconsideration of standard practices for validation and deployment of prediction models that are used in medical decisions.

Clinicians increasingly rely on prediction models to guide treatment choices. Most prediction models, however, are developed using observational data that include some patients who have already received the treatment the prediction model is meant to inform. Special attention to the causal role of those earlier treatments is required when interpreting the resulting predictions.

“Causal blind spots” were identified in 3 common approaches to handling treatment when developing a prediction model: including treatment as a predictor, restricting to persons taking a certain treatment, and ignoring treatment. Through several real examples, this article illustrates how the risks obtained from models developed using such approaches may be misinterpreted and can lead to misinformed decision making. The discussion covers issues attributable to confounding, selection, mediation, and changes in treatment protocols over time.

An extension of guidelines for the development, reporting, and evaluation of prediction models is advocated to avoid such misinterpretations. Developers must ensure that the intended target population for the model, and the treatment conditions under which predictions hold, are clearly communicated. When prediction models are intended to inform treatment decisions, they need to provide estimates of risk under the specific treatment (or intervention) options being considered, known as “prediction under interventions.” Next to suitable data, this requires causal reasoning and causal inference techniques during model development and evaluation. Being clear about what a given prediction model can and cannot be used for prevents misinformed treatment decisions and thereby prevents potential harm to patients.

We appreciate the opportunity to further clarify our findings in response to the insightful comments from Shota Yamamoto and colleagues and Luis Felipe Reyes and Ignacio Martin-Loeches regarding our recent community-acquired pneumonia (CAP) study. [...]

Background: Reports regarding the relationship between tacrolimus exposure and the risk of acute kidney allograft rejection are conflicting. This may be explained by the previous use of methodological approaches that disregarded important factors in the analysis of longitudinal measurements and time-to-event data. Therefore, in this study, joint models were used to investigate the relationship between repeated measurements of tacrolimus predose concentrations (C₀) and time to acute biopsy-proven acute rejection (BPAR). Methods: This was a post hoc analysis of a randomized controlled trial in which living-donor kidney transplant recipients (KTR) received either a standard, bodyweight-based or CYP3A5 genotype-based tacrolimus starting dose. Joint modeling was performed by coupling a mixed-effects model for tacrolimus C₀ with a Cox proportional hazards model for the risk of rejection. Only the first episode of rejection was considered. Results: A total of 229 KTRs were included, of whom the incidence of BPAR was 10.5% (n = 24 KTRs) in the first 3 months posttransplant. A total of 3069 tacrolimus measurements were available for the analysis. A joint model adjusted for recipient age and peak panel reactive antibodies demonstrated that tacrolimus C₀ was associated with risk of rejection. A 1-unit increase in the time-normalized area under the curve for logarithmically (log)-transformed C₀ represented a change of 22.65 in the log of the relative hazard (95% credible interval: 25.05 to 20.36, P = 0.022). Conclusions: A negative association between the cumulative effect of tacrolimus C₀ and BPAR was observed using joint modeling. This demonstrated that KTRs with lower tacrolimus exposure were at a higher risk of rejection.

Prediction models are used among others to inform medical decisions on interventions. Typically, individuals with high risks of adverse outcomes are advised to undergo an intervention while those at low risk are advised to refrain from it. Standard prediction models do not always provide risks that are relevant to inform such decisions: for example, an individual may be estimated to be at low risk because similar individuals in the past received an intervention which lowered their risk. Therefore, prediction models supporting decisions should target risks belonging to defined intervention strategies. Previous works on prediction under interventions assumed that the prediction model was used only at one time point to make an intervention decision. In clinical practice, intervention decisions are rarely made only once: they might be repeated, deferred, and reevaluated. This requires estimated risks under interventions that can be reconsidered at several potential decision moments. In the current work, we highlight key considerations for formulating estimands in sequential prediction under interventions that can inform such intervention decisions. We illustrate these considerations by giving examples of estimands for a case study about choosing between vaginal delivery and cesarean section for women giving birth. Our formalization of prediction tasks in a sequential, causal, and estimand context provides guidance for future studies to ensure that the right question is answered and appropriate causal estimation approaches are chosen to develop sequential prediction models that can inform intervention decisions.

Learning curves illustrate how generalization performance of a learner evolves with more training data. While this is a useful tool to characterize learners, not all learning curve behavior is well understood. For instance, it is sometimes assumed that the more training data provided, the better the learner performs. However, counter-examples exist for both classical machine learning algorithms and deep neural networks, where errors do not monotonically decrease with training set size. Loog et al. [12] describe this monotonicity problem, and present several regression examples where simple empirical risk minimizers display unexpected learning curve behaviors. In this paper, we will study two of these proposed problems in detail and explain what caused the odd learning curves. For the first, we use a bias-variance decomposition to show that the monotonic increase in the learning curve is caused by an increase in the variance, which we explain by a mismatch between the model and the data generating process. For the second problem, we explain the recurring increases in the learning curve by showing only two solutions are attainable by the learner. The probability of obtaining a configuration of training objects that leads to the high risk solution typically decreases as the training set size increases. However, for particular training set sizes, additional configurations that produce the high risk solution become possible. We prove that these additional configurations increase the probability of the high risk solution and therefore explain the unusual learning curve. These examples contribute to a more complete understanding of learning curves and the possibilities and reasons behind their various behaviors.

This scoping review focuses on the essential role of models for causal inference in shaping actionable artificial intelligence (AI) designed to aid clinicians in decision-making. The objective was to identify and evaluate the reporting quality of studies introducing models for causal inference in intensive care units (ICUs), and to provide recommendations to improve the future landscape of research practices in this domain. To achieve this, we searched various databases including Embase, MEDLINE ALL, Web of Science Core Collection, Google Scholar, medRxiv, bioRxiv, arXiv, and the ACM Digital Library. Studies involving models for causal inference addressing time-varying treatments in the adult ICU were reviewed. Data extraction encompassed the study settings and methodologies applied. Furthermore, we assessed reporting quality of target trial components (i.e., eligibility criteria, treatment strategies, follow-up period, outcome, and analysis plan) and main causal assumptions (i.e., conditional exchangeability, positivity, and consistency). Among the 2184 titles screened, 79 studies met the inclusion criteria. The methodologies used were G methods (61%) and reinforcement learning methods (39%). Studies considered both static (51%) and dynamic treatment regimes (49%). Only 30 (38%) of the studies reported all five target trial components, and only seven (9%) studies mentioned all three causal assumptions. To achieve actionable AI in the ICU, we advocate careful consideration of the causal question of interest, describing this research question as a target trial emulation, usage of appropriate causal inference methods, and acknowledgement (and examination of potential violations of) the causal assumptions.

Background: Currently available treatment options for Parkinson's disease are symptomatic and do not alter the course of the disease. Recent studies have raised the possibility that cardiovascular risk management may slow the progression of the disease. Objectives: We estimated the effect of baseline cardiovascular risk factors on the progression of Parkinson's disease, using measures for PD-specific motor signs and cognitive functions. Methods: We used data from 424 de novo Parkinson's disease patients and 199 age-matched controls from the observational, multicenter Parkinson's Progression Markers Initiative (PPMI) study, which included follow-up of up to 9 years. The primary outcome was the severity of PD-specific motor signs, assessed with the MDS-UPDRS part III in the “OFF”-state. The secondary outcome was cognitive function, measured with the Montreal Cognitive Assessment, Symbol Digit Modalities Test, and Letter-Number Sequencing task. Exposures of interest were diabetes mellitus, hypertension, body mass index, cardiovascular event history and hypercholesterolemia, and a modified Framingham risk score, measured at baseline. The effect of each of these exposures on disease progression was modeled using linear mixed models, including adjustment for identified confounders. A secondary analysis on the Tracking Parkinson's cohort including 1,841 patients was performed to validate our findings in an independent patient cohort. Results: Mean age was 61.4 years, and the average follow-up was 5.5 years. We found no statistically significant effect of any individual cardiovascular risk factor on the MDS-UPDRS part III progression (all 95% confidence intervals (CIs) included zero), with one exception: in the PD group, the estimated effect of a one-point increase in body mass index was 0.059 points on the MDS-UPDRS part III per year (95% CI: 0.017 to 0.102). We found no evidence for an effect of any of the exposures on the rate of change in cognitive functioning in the PD group. Similar results were observed for the Tracking Parkinson's cohort (all 95% CIs overlapped with PPMI), but the 95% CI of the effect of body mass index on the MDS-UPDRS part III progression included zero. Conclusions: Based on this analysis of two large cohorts of de novo PD patients, we found no evidence to support clinically relevant effects of cardiovascular risk factors on the clinical progression of Parkinson's disease.

Arguably, a desirable feature of a learner is that its performance gets better with an increasing amount of training data, at least in expectation. This issue has received renewed attention in recent years and some curious and surprising findings have been reported on. In essence, these results show that more data does actually not necessarily lead to improved performance—worse even, performance can deteriorate. Clustering, however, has not been subjected to such kind of study up to now. This paper shows that k-means clustering, a ubiquitous technique in machine learning and data mining, suffers from the same lack of so-called monotonicity and can display deterioration in expected performance with increasing training set sizes. Our main, theoretical contributions prove that 1-means clustering is monotonic, while 2-means is not even weakly monotonic, i.e., the occurrence of nonmonotonic behavior persists indefinitely, beyond any training sample size. For larger k, the question remains open.

Artificial intelligence (AI) research in the intensive care unit (ICU) mainly focuses on developing models (from linear regression to deep learning) to predict out-
comes, such as mortality or sepsis [1, 2]. However, there is another important aspect of AI that is typically not framed as AI (although it may be more worthy of the name), which is the prediction of patient outcomes or events that would result from different actions, known as causal inference [3, 4]. This aspect of AI is crucial for decision-making in the ICU. To emphasize the impor- tance of causal inference, we propose to refer to any data- driven model used for causal inference tasks as ‘action- able AI’, as opposed to ‘predictive AI’, and discuss how these models could provide meaningful decision support in the ICU.