Holy grail or convenient excuse? Stakeholder perspectives on the role of health system strengthening evaluation in global health resource allocation

Abstract

Background: The role of evaluation evidence in guiding health systems strengthening (HSS) investments at the global-level remains contested. A lack of rigorous impact evaluations is viewed by some as an obstacle to scaling resources. However, others suggest that power dynamics and knowledge hierarchies continue to shape perceptions of rigor and acceptability in HSS evaluations. This debate has had major implications on HSS resource allocation in global-level funding decisions. Yet, few studies have examined the relationship between HSS evaluation evidence and prioritization of HSS. In this paper, we explore the perspectives of key global health stakeholders, specifically around the nature of evidence sought regarding HSS and its potential impact on prioritization, the challenges in securing such evidence, and the drivers of intra- and inter-organizational divergences. We conducted a stakeholder analysis, drawing on 25 interviews with senior representatives of major global health organizations, and utilized inductive approaches to data analysis to develop themes. Results: Our analysis suggests an intractable challenge at the heart of the relationship between HSS evaluations and prioritization. A lack of evidence was used as a reason for limited investments by some respondents, citing their belief that HSS was an unproven and potentially risky investment which is driven by the philosophy of HSS advocates rather than evidence. The same respondents also noted that the ‘holy grail’ of evaluation evidence that they sought would be rigorous studies that assess the impact of investments on health outcomes and financial accountability, and believed that methodological innovations to deliver this have not occurred. Conversely, others held HSS as a cross-cutting principle across global health investment decisions, and felt that the type of evidence sought by some funders is unachievable and not necessary – an ‘elusive quest’ – given methodological challenges in establishing causality and attribution. In their view, evidence would not change perspectives in favor of HSS investments, and evidence gaps were used as a ‘convenient excuse’. Respondents raised additional concerns regarding the design, dissemination and translation of HSS evaluation evidence. Conclusions: Ongoing debates about the need for stronger evidence on HSS are often conducted at cross-purposes. Acknowledging and navigating these differing perspectives on HSS evaluation may help break the gridlock and find a more productive way forward.