Sector Results and Learning:
Education

Random assignment can be a fair and transparent mechanism to distribute finite resources and allow for a rigorous impact evaluation. In El Salvador, random assignment was seen as a fair way to distribute scholarships when qualified demand exceeded supply. Scholarships were offered for multiple years and the random selection was only conducted in the year where there were more qualified applicants who met the minimum criteria than scholarships available. By randomly selecting scholarship recipients, the applicants who were not offered scholarships were similar to those offered scholarships and thus created a strong control group to measure impact. The random assignment method (a computerized lottery conducted publicly) was transparent and it was clear that each eligible applicant had an equal chance to receive the scholarship.

Improvements in enrollment and progression drove improved levels of learning, but without efforts to address weak overall performance of schools, the rate of learning may not improve. As education access (availability, distance) and girls’ participation were constrained in rural Burkina Faso, the project’s approach of bundled interventions to address these two constraints seem to have driven improvements on these dimensions. Teacher housing and performance awards seems to have helped BRIGHT schools attract more female teachers and allow schools to reduce teacher/student ratio. However, despite improving girls’ engagement and smaller classroom sizes, BRIGHT schools did not seem to have a steeper learning profile than non-BRIGHT schools (i.e. how long it takes a given student to arrive at a skill/competency such as recognizing letters or reading a sentence). In contexts where education quality lags act as a more binding constraint than enrollment, improvements to the academic environment – including quality instruction, better educational management and leadership, and academic support for struggling students – might help steepen learning profiles.

New systems or service offerings should include well-designed data systems. The evaluation reported inconsistencies in the administrative data collected within and across the RSRCs. For example, the final report describes the visitor counter at the Oshana RSRC, where patrons who use the restroom or retrieve items from a locker must exit and be counted again upon reentry, resulting in systematically overcounting library visits at that location. The weak data systems at the RSRCs limited the evaluation but more importantly, they also undermine performance management efforts taking place at the individual RSRC and central levels. Future investments in entirely new systems or service offerings like the RSRCs should develop systemwide data systems that are both appropriate given the infrastructure itself (e.g., layouts that position visit counters between the restrooms and the rest of the facility are going to skew the visit counts) and facilitate the types of planning and decision-making that needs to happen at various levels within the system. MCC recognizes the importance of data systems to performance management and nearly all of the agency’s ongoing education interventions include investments in data systems.

The ILEI evaluation’s use of pairwise randomization implemented by construction phase helped mitigate risks to the evaluation due to uncertainties around the number of treatment schools and construction timelines. MCC should build flexibility into evaluation design whenever possible.

Key policy decisions that affect sustainability should be taken before MCC compacts end. To the extent possible, policy and regulatory hurdles should be identified before Entry into Force, with consultant support as needed, to ensure that partner governments can reap the benefits of intensive technical assistance to operationalize new government functions and increase the likelihood of sustainability. Important policy decisions were still to be determined at the end of the Namibia compact. For example, the process Industry Skills Committees would use to identify priority skill areas and what constitutes vocational training were both unclear. This lack of clarity could easily impede the Namibia Training Authority’s ability to identify the skills that are truly in demand and prevent employers from being reimbursed for sponsored trainings. It will be important to resolve these issues soon so that stakeholders maintain support for the system.