Assessing the scalability of health sector interventions: An economic perspective

Professor Ajay Mahal queries the economics of health sector interventions at various scales.

Suppose that an experimental evaluation or a small pilot project demonstrates that a health intervention – say, the training of community health workers in roles that task shift from nurses in rural settings – is highly cost-effective. Can it be assumed that if this intervention were to be replicated ad infinitum in a province or a country, it would yield close to proportionate, or better still, more than proportionate increases in outcomes relative to costs? In other words, is the intervention ‘scalable’?

There are at least three types of arguments suggesting that transforming an intervention from a study to a broader geography or population will not lead to a straightforward retention of its experimental (or pilot) outcomes. On the supply side, a program that is readily manageable at the study site may present serious challenges related to coordination and monitoring when extended beyond it, including potential absenteesim among workers, or inconsistent quality of services delivered. Expanding intervention implementation may therefore call for changes in the organization of service delivery, including measures such as decentralized management and payment incentives, which were likely not part of the initial experimental assessment. Relatedly, unit costs may even decline if some inputs remain fixed, such as training facilities for community health workers, even as their numbers increase; or if large scale of program delivery allows for specialization of tasks, such as for procurement and data management.

Second, a key channel through which expanded implementation of interventions is likely to influence costs and input use is by influencing worker salaries. Rapid expansion in the demand for community health workers and their span of duties (owing to task shifting initiatives) may drive up the market wages needed to attract them and hence the per unit cost of the intervention. A concern for rising wages need not be an issue in economies with high levels of baseline unemployment among the potential pool of community health workers. However, the experimental or pilot research on which the estimates of intervention effectiveness are based often involves high levels of program inputs and supervision and motivated health workers, and the latter will be limited in supply. The alternative is to sacrifice worker quality in the interests of conserving wage costs as intervention coverage is increased, but then the expanded program may experience lower levels of cost-effectiveness, or not at all.

Third, the outcome and cost implications of any increases in intervention coverage must account for the interaction with political leaders and bureaucracy who are often central to the delivery of health sector programmes at scale. Small-scale experimental and pilot exercises funded by external donors that are carried out independently cannot capture the outcomes that result when the responsibilities for scale-up become the realm of government action. In a regime of budget constraints, political factors can drive the process of scale up, or they may lead to the neglect of a programme that is attractive on grounds of cost-benefit analysis. Politics may be especially salient in settings where implementing officials have limited bandwidth in terms of attention span, with competing initiatives crowding their plate. On the flip side, pilot and experimental intervention effectiveness estimates can be misleading even with close government partnership, if the context is which they arise differ substantially from the broader population and geographical landscape, e.g., when the pilot project is set up in richer areas or in areas represented by the political leadership.

How then is one to decide on the scalability of an intervention? Is it only possible to assess scalability once an intervention is undergoing a process of scaling up (or failing in the process)? This last option sounds unattractive, and potentially highly inefficient. Perhaps the right approach is not to focus solely on learning from small-scale intervention studies and then ‘hope for the best’ when scaling up. It may also be useful to introduce learning from previous efforts at large scale implementation of interventions, when deciding on the intervention itself. Finally, engaging early with parties that will implement the intervention at scale, both in designing interventions and assessing their feasibility at scale-up, may be desirable.