An analysis of impact evaluations of water, sanitation, and hygiene (wash) interventions in rural sub-saharan Africa

Abstract

This report presents the findings of a comprehensive analysis of the literature on impact evaluations of foreign aid funded interventions in the water sector of rural Sub-Saharan Africa (SSA). The study was conducted using a dataset extracted from the International Initiative for Impact Evaluation (3ie) Development Evidence Portal (DEP), which is one of the largest repositories for evidence-based research in Low and Middle Income Countries (LMICs). The extracted publications underwent systematic screening based on predetermined criteria. Studies that pertained to interventions including infrastructure development and water resource management were included, while papers focusing solely on education, behavioral change, awareness campaigns, or the exclusive provision of soap were excluded. After classification, it was found that interventions related to Water, Sanitation and Hygiene (WASH) dominated the dataset, so studies related to agricultural productivity were excluded. The final dataset consisted of 30 papers. These publications were classified and analyzed using bibliographic and network analyses to explore the structure of the research community. The content of the papers was then analyzed and situated within external literature to validate conclusions, contextualize findings, and identify gaps and areas for further research.

The dataset consisted solely of impact evaluations of randomized controlled trials (RCTs) related to WASH interventions. The trials were spread across nine countries, with the majority conducted in Kenya, and their duration varied from eight weeks to 29 months. Most interventions focused on low-cost household water treatment technologies, with chlorination being the most common. Regarding outcomes, most trials investigated health-related outcomes for children under five years, with diarrhea prevalence being the most frequently studied outcome. The results showed significant heterogeneity among trial findings, especially for diarrhea prevalence, suggesting that intervention effectiveness may depend on contextual factors that are not yet fully understood.

The research community within the dataset showed disparities in the distribution of trials across SSA, with most studies conducted by organizations from the Global North and published in journals targeting a Global North audience. The studies used similar approaches and focused on highly recurrent outcomes, suggesting a potential lack of diversity in the knowledge base. The dataset emphasized low-cost interventions suitable for large-scale implementation and showed a preference for RCTs due to their ability to establish causal relationships, minimize bias, and produce generalizable findings. However, the RCTs included design limitations that compromised these strengths, including limited blinding and reliance on subjective outcome indicators. Furthermore, reporting on field challenges and logistical information was insufficient, limiting generalizability.

The study highlights the need for greater diversity in approaches and knowledge, stronger local representation, and increased innovation in research on WASH interventions in SSA. To address these issues, several recommendations are made. First, greater representation of organizations from the Global South and increased collaboration with the Global North are needed, as such partnerships were associated with high-impact research. Second, standardized reporting guidelines and data-sharing protocols should be developed to improve transparency and reproducibility of impact evaluations. Third, greater diversity in targeted outcomes and intervention approaches is recommended, along with overall improvements in research quality.

Overall, this study provides insights into the state of impact evaluations in LMICs, highlighting both progress and remaining challenges. By identifying gaps and offering recommendations, it contributes to improving the quality and impact of future research in development contexts.