Two years into a pioneering three-year development impact bond (DIB), Educate Girls—which tackles gender inequality in India’s educational system, is showing promising results. With a DIB, a service provider receives upfront working capital under a pay-for-success contract through a third party investor. This insulates the service provider—Educate Girls in this case—from the financial risk. The DIB goals here were increased enrollment of out-of-school girls in primary school, and progress of both boys and girls in English, Hindi, and mathematics. Educate Girls has achieved nearly 88 percent of the enrollment target and 50 percent of the 3-year learning target in a remote, rural district in Rajasthan, India. The DIB spurred Educate Girls to fundamentally overhaul its curriculum, which they’re now rolling out across 5,000 non-DIB schools this year. We talked with Maharshi Vaishnav and Alison Bukhari from Educate Girls about how their education intervention goals fared with a tight timeline, what they’ve learned by implementing this new funding approach, and what funders could learn as well.
Zach Slobig: Yours is the first DIB in education in the world. What are you learning as you pioneer the approach?
Maharshi and Alison: Because there was no template to follow, we started off with a clean slate and have learned things the hard way, and we’ve learned a lot. Be prepared to do your homework to assess if a DIB is a good fit for your organization. Conduct a thorough baseline without relying on secondary data. Seek as much evidence as possible to support the valuation of your selected outcomes. Conduct a thorough risk assessment looking particularly at potential externalities. Be prepared to support your staff through a culture shift. Keep it simple: work towards a less complex results-based contract.
Zach: This DIB funds your work in a single district in rural Rajasthan. How was the district selected?
Maharshi and Alison: The government has a list of what they call ‘Educationally Backward Blocks’—a cluster of villages which score low on both educational outcomes as well as gender parity. We work with every village in a block rather than a scatter shot approach, and we’re now working in over 100 blocks across 13 districts in Rajasthan and bordering Madhya Pradesh, two central states of India. Bhilwara was selected because of low percentage of girls’ enrollment, poor learning outcome, and high percentage of marginalized population.
Zach: The targets you set with UBS Optimus Foundation and Children’s Investment Fund Foundation focus not just on enrollment for girls but academic outcomes. How did you choose which key metrics to tie to the DIB?
Maharshi and Alison: Choosing metrics was difficult. We’ve always seen ourselves as focused primarily on establishing gender equity via community mobilization and systemic reform, but these aren’t easily quantifiable outcomes. We’ve always believed that girl enrollment is the key to reduce the gender gap in education. We fought hard to keep enrollment as one of the main outcomes, and we wouldn’t have joined in the DIB without it. We settled for 80-20 weightages allocation for learning outcomes and enrollment because a large percentage of the population had learning lags and a smaller percentage of out-of-school girls needed enrollment support.
Zach: It looks like at the current pace, you are on track to exceed your enrollment target. Progress towards the learning targets appear to be moving just a bit more slowly. Why?
Maharshi and Alison: We are indeed on track with our enrollment target and could exceed expectations. We have a long track record with our behavior change communications and community level engagement, and the DIB has improved our ability to tailor our approach with more data to inform our strategy.
In terms of learning, we are at the early stages of refining our approach. In the first year we did not get the planned 24 weeks in the classroom—only 14 weeks. Certain patterns in learning gaps became clear and our curriculum was not designed to fill those gaps. Halfway through that first year we totally revamped the curriculum in response.
Given that we were almost back to a pilot phase in year two, we are actually happy with the results. The children who faired best were those who had been consistent in the classroom over two years. This indicates to us that longevity in the program is essential and there is an expected lag in terms of seeing impact of the new curriculum. We work with first generation school goers who have missed a lot of their primary education and don’t have the benefit of literate parents at home. We have to remember that educational interventions are never quick wins, and even a three-year horizon to achieve a change in pace of learning gains is ambitious.
Zach: What is not working with this new approach? What did you not foresee?
Maharshi and Bukhari: Overall, this has been transformational for our organization, but we’ve struggled at the planning stage, and that can be expected for a first of-its-kind contract. Our baseline relied on flawed government data, so we conducted our own baseline at the beginning of the project which delayed our contract. The valuation of the outcomes was based on a somewhat outdated RCT, and so that undervalued the outcome price. We also underestimated the cost of senior management required for this proof of concept.
One of the greatest risks with a pay-for-performance contract is the exclusion of externalities. We didn’t foresee that the government would close or merge schools for instance, and that in certain areas there are no upper primary schools. There will always be things you can’t foresee.
Zach: How has the laser focus on outcomes spurred innovation in programs and curriculum?
Maharshi and Alison: It depends on how you define innovation and some of our examples we might prefer to call adaptive management!
We integrated a high-end performance management system to slice and dice data to analyze reasons for older girls not enrolling in schools, and we revamped our curriculum to sharpen micro-competencies in children. We pulled resources from one geography and deployed them in another geography with greater need. The funder trusted us to adapt in order to achieve results.
At the community level, the field workers are given the flexibility to test different approaches and tailor messages to a girl’s unique challenges. The flexibility inherent in the DIB design certainly has created an enabling environment.
Zach: What is the lesson here for funders? As pay-for-performance funding grows as a practice, what risks should we be mitigating?
Maharshi and Alison: Even in more traditional grant agreements funders can adopt more of an outcomes focus. What our DIB has shown us is that the flexibility to innovate, redeploy, and react to on-the-ground conditions enables us to be more accountable to the students, not just accountable to funders.
Pay-for-success is only suitable if the organization has the ability to deliver outcomes, their M&E and performance management capacity is in place, and their organizational culture is ready to adapt. Funders must do their due diligence and select the right service provider and ensure it has the capacity to deliver. A majority of NGOs may require intense training before they could manage such contracts and the costs would have to be factored in. Even though our M&E was strong, we found we had to adapt.
One must be careful about where the power and decision making lies—outcomes should be aligned with the needs of the communities and universally agreed upon. Outcomes should not be experimental, and they should be measurable. The service provider must be deeply involved in setting the outcomes and metrics to not create incentives that could be detrimental to the community. Funders should take a hands-off approach—after all, flexibility spurs innovation.
