Erik van Zwet | TU Delft Repository

Enhancing feedback on performance measures

The difference in outlier detection using a binary versus continuous outcome funnel plot and implications for quality improvement

Journal article (2021) - Laurien Kuhrij, Erik Van Zwet, Renske Van Den Berg-Vos, Paul Nederkoorn, Perla J. Marang-Van De Mheen

Background Hospitals and providers receive feedback information on how their performance compares with others, often using funnel plots to detect outliers. These funnel plots typically use binary outcomes, and continuous variables are dichotomised to fit this format. However, information is lost using a binary measure, which is only sensitive to detect differences in higher values (the tail) rather than the entire distribution. This study therefore aims to investigate whether different outlier hospitals are identified when using a funnel plot for a binary vs a continuous outcome. This is relevant for hospitals with suboptimal performance to decide whether performance can be improved by targeting processes for all patients or a subgroup with higher values. Methods We examined the door-to-needle time (DNT) of all (6080) patients with acute ischaemic stroke treated with intravenous thrombolysis in 65 hospitals in 2017, registered in the Dutch Acute Stroke Audit. We compared outlier hospitals in two funnel plots: the median DNT versus the proportion of patients with substantially delayed DNT (above the 90th percentile (P90)), whether these were the same or different hospitals. Two sensitivity analyses were performed using the proportion above the median and a continuous P90 funnel plot. Results The median DNT was 24 min and P90 was 50 min. In the binary funnel plot for the proportion of patients above P90, 58 hospitals had average performance, whereas in the funnel plot around the median 14 of these hospitals had significantly higher median DNT (24%). These hospitals can likely improve their DNT by focusing on care processes for all patients, not shown by the binary outcome funnel plot. Similar results were shown in sensitivity analyses. Conclusion Using funnel plots for continuous versus binary outcomes identify different outlier hospitals, which may enhance hospital feedback to direct more targeted improvement initiatives. ...

Background Hospitals and providers receive feedback information on how their performance compares with others, often using funnel plots to detect outliers. These funnel plots typically use binary outcomes, and continuous variables are dichotomised to fit this format. However, information is lost using a binary measure, which is only sensitive to detect differences in higher values (the tail) rather than the entire distribution. This study therefore aims to investigate whether different outlier hospitals are identified when using a funnel plot for a binary vs a continuous outcome. This is relevant for hospitals with suboptimal performance to decide whether performance can be improved by targeting processes for all patients or a subgroup with higher values. Methods We examined the door-to-needle time (DNT) of all (6080) patients with acute ischaemic stroke treated with intravenous thrombolysis in 65 hospitals in 2017, registered in the Dutch Acute Stroke Audit. We compared outlier hospitals in two funnel plots: the median DNT versus the proportion of patients with substantially delayed DNT (above the 90th percentile (P90)), whether these were the same or different hospitals. Two sensitivity analyses were performed using the proportion above the median and a continuous P90 funnel plot. Results The median DNT was 24 min and P90 was 50 min. In the binary funnel plot for the proportion of patients above P90, 58 hospitals had average performance, whereas in the funnel plot around the median 14 of these hospitals had significantly higher median DNT (24%). These hospitals can likely improve their DNT by focusing on care processes for all patients, not shown by the binary outcome funnel plot. Similar results were shown in sensitivity analyses. Conclusion Using funnel plots for continuous versus binary outcomes identify different outlier hospitals, which may enhance hospital feedback to direct more targeted improvement initiatives.

A dynamic quality assessment tool for laparoscopic hysterectomy to measure surgical outcomes

Journal article (2017) - Sara R C Driessen, Erik W. Van Zwet, Pascal Haazebroek, Evelien M. Sandberg, Mathijs D. Blikkendaal, Andries R H Twijnstra, Frank Willem Jansen

Cocontraction measured with short-range stiffness was higher in obstetric brachial plexus lesions patients compared to healthy subjects

Journal article (2017) - Galia V. Anguelova, Erwin de Vlugt, Alistair N. Vardy, Erik W. Van Zwet, J. Gert van Dijk, Martijn J.A. Malessy, Jurriaan H. de Groot

We suggest short range stiffness (SRS) at the elbow joint as an alternative diagnostic for EMG to assess cocontraction.Elbow SRS is compared between obstetric brachial plexus lesion (OBPL) patients and healthy subjects (cross-sectional study design). Seven controls (median 28. years) and five patients (median 31. years) isometrically flexed and extended the elbow at rest and three additional torques [2.1,. 4.3,. 6.4. N. m] while a fast stretch stimulus was applied. SRS was estimated in silico using a neuromechanical elbow model simulating the torque response from the imposed elbow angle.SRS was higher in patients (250. ±. 36. N. m/rad) than in controls (150. ±. 21. N. m/rad, p = 0.014), except for the rest condition. Higher elbow SRS suggested greater cocontraction in patients compared to controls. SRS is a promising mechanical alternative to assess cocontraction, which is a frequently encountered clinical problem in OBPL due to axonal misrouting. ...

Hospital versus individual surgeon’s performance in laparoscopic hysterectomy

Journal article (2017) - Sara R C Driessen, Markus Wallwiener, Florin Andrei Taran, Sarah L. Cohen, Bernhard Kraemer, Christian W. Wallwiener, Erik W. Van Zwet, Sara Y. Brucker, Frank Willem Jansen

Purpose: To compare hospital versus individual surgeon’s perioperative outcomes for laparoscopic hysterectomy (LH), and to assess the relationship between surgeon experience and perioperative outcomes. Methods: A retrospective analysis of all prospective collected LHs performed from 2003 to 2010 at one medical center was performed. Perioperative outcomes (operative time, blood loss, complication rate) were assessed on both a hospital level and surgeon level using Cumulative Observed minus Expected performance graphs. Results: A total of 1618 LHs were performed, 16 % total laparoscopic hysterectomies and 84 % laparoscopic supracervical hysterectomies. Overall outcomes included mean (SD±) blood loss 108.9 ± 69.2 mL, mean operative time 95.4 ± 39.7 min and a complication occurred in 76 (4.7 %) of cases. Suboptimal perioperative outcomes of an individual surgeon were not always detected on a hospital level. However, collective suboptimal outcomes were faster detected on a hospital level compared to individual surgeon’s level. Evidence of a learning curve is seen; for the first 100 procedures, a decrease in operative time is observed as individual surgeon experience increases. Similarly, the risk of conversion decreases up to the first 50 procedures. Conclusion: An individual outlier (i.e., surgeon with consistently suboptimal performance) will not always be detected when monitoring outcome measures only on a hospital level. However, monitoring outcome measures on a hospital level will detect suboptimal performance earlier compared to monitoring only on an individual surgeon’s level. To detect performance outliers timely, insight into an individual surgeon’s outcome and skills is recommended. Furthermore, an experienced surgeon is no guarantee for acceptable surgical outcomes. ...

Identification of risk factors in minimally invasive surgery

A prospective multicenter study

Journal article (2017) - Sara R C Driessen, Evelien M. Sandberg, Sharon P. Rodrigues, Erik W. van Zwet, Frank Willem Jansen

Background: Since the introduction of minimally invasive surgery (MIS), concerns for patient safety are more often brought to the attention. Knowledge about and awareness of patient safety risk factors are crucial in order to improve and enhance the surgical team, the environment, and finally surgical performance. The aim of this study was to identify and quantify patient safety risk factors in laparoscopic hysterectomy and to determine their influence on surgical outcomes. Methods: A prospective multicenter study was conducted from April 2014 to January 2016, participating gynecologists registered their performed laparoscopic hysterectomies (LHs). If deemed necessary, gynecologists could fill out a checklist with validated patient safety risk factors. Association between procedures with and without an occurred risk factor(s) and the surgical outcomes (blood loss, operative time, and complications) were assessed, using multivariate logistic regression and generalized estimation equations. Results: Eighty-five gynecologists participated in the study, registering a total of 2237 LHs. For 627(28 %) procedures, the checklist was entered (in total 920 items). The most reported risk factors were related to the surgeon (19.6 %), the surgical team (14.4 %), technology (16.6 %), and the patient (26.8 %). The procedures where a risk factor was registered had significantly less favorable outcomes, higher complication rate (10.5 vs. 4.8 % (p = 0.002), longer operative time [114 vs. 95 min (p < 0.001)], and more blood loss [110 vs. 168 mL (p = 0.047)], which was mainly due to the technological and patient-related risk factors. Conclusion: Technological incidents are the most important and clinically relevant risk factors affecting surgical outcomes of LH. Future improvements of MIS need to focus on this. As awareness of safety risk factors in MIS is important, embedding of a safety risk factor checklist in registration systems will help surgeons to evaluate and improve their individual performance. This will inherently improve the surgical outcomes and thus patient safety. ...

Background: Since the introduction of minimally invasive surgery (MIS), concerns for patient safety are more often brought to the attention. Knowledge about and awareness of patient safety risk factors are crucial in order to improve and enhance the surgical team, the environment, and finally surgical performance. The aim of this study was to identify and quantify patient safety risk factors in laparoscopic hysterectomy and to determine their influence on surgical outcomes. Methods: A prospective multicenter study was conducted from April 2014 to January 2016, participating gynecologists registered their performed laparoscopic hysterectomies (LHs). If deemed necessary, gynecologists could fill out a checklist with validated patient safety risk factors. Association between procedures with and without an occurred risk factor(s) and the surgical outcomes (blood loss, operative time, and complications) were assessed, using multivariate logistic regression and generalized estimation equations. Results: Eighty-five gynecologists participated in the study, registering a total of 2237 LHs. For 627(28 %) procedures, the checklist was entered (in total 920 items). The most reported risk factors were related to the surgeon (19.6 %), the surgical team (14.4 %), technology (16.6 %), and the patient (26.8 %). The procedures where a risk factor was registered had significantly less favorable outcomes, higher complication rate (10.5 vs. 4.8 % (p = 0.002), longer operative time [114 vs. 95 min (p < 0.001)], and more blood loss [110 vs. 168 mL (p = 0.047)], which was mainly due to the technological and patient-related risk factors. Conclusion: Technological incidents are the most important and clinically relevant risk factors affecting surgical outcomes of LH. Future improvements of MIS need to focus on this. As awareness of safety risk factors in MIS is important, embedding of a safety risk factor checklist in registration systems will help surgeons to evaluate and improve their individual performance. This will inherently improve the surgical outcomes and thus patient safety.