Final Evaluation Brief
Improving Vocational Training Capacity in Namibia
Scholarships increased training completion but did not increase employment
MCC’s $295.7 million Namibia Compact (2009-2014) funded the $145 million Education Project, which included the $8.2 million Vocational Training Grant Fund (VTGF) Sub-Activity. The sub-activity was designed to address the skills gap in Namibia’s labor force by providing grants to expand domestic training capacity and fund scholarships in high-priority skills areas, thereby allowing more trainees to complete training and experience increased employment and earnings.
MCC commissioned Mathematica Policy Research to conduct an independent impact evaluation of the VTGF Sub-Activity under the Namibia Education Project. Full report results and learning: https://data.mcc.gov/evaluations/index.php/catalog/90.
- The VTGF grants were largely implemented as planned, but faced challenges, especially in determining the market demand for specific vocational skills.
- The VTGF Sub-Activity trained nearly 92 percent of its compact target of 1,638 trainees. This target was reduced from 5,838 trainees due to implementation limitations.
- Components were added to the VTGF training grants during implementation to build provider capacity and reduce dropouts among VTGF-funded trainees.
Training Enrollment and Completion
- The offer of VTGF scholarships resulted in a 34 percentage point increase in enrollment and a 29 percentage point increase in self-reported training completion rates compared to the control group.
- Impacts on both enrollment and completion were almost 50 percent larger for female applicants compared to males.
Employment and Income
- The offer of VTGF scholarships had no statistically significant impact on employment compared to the control group, but did lead to some participants re-enrolling in training in lieu of immediately pursuing employment.
- VTGF also had no impact on applicants’ earnings or on the distribution of total individual or household income compared to the control group.
This impact evaluation was conducted over two distinct rounds of data collection and analysis. The first round was designed to answer a set of questions focused on program implementation and beneficiaries’ perceptions of the program. The second round focused on the VTGF’s impact on training applicants’ employment and income relative to a control group following completion of vocational training courses. Key evaluation questions include:
Was the VTGF Sub-Activity implemented as planned?
To what extent did training completion, employment, earnings, and income improve for VTGF-funded trainees relative to outcomes for non-funded qualified applicants?
Did the effects of the VTGF funding vary by trainee characteristics?
The VTGF Sub-Activity was largely implemented as planned, but faced multiple challenges, including the small market for vocational training in Namibia, which made it difficult to identify sufficient training providers. Additionally, there was no clear process for determining market demand for training in specific skills. While the industry skills committees were expected to fulfill this role, they were not functioning in all industries and did not produce rigorous guidelines for skill prioritization.
At the start of the compact, the Namibian accountable entity (MCA-N) set a target to train 5,838 individuals through VTGF training grants, but this target was revised down to 1,638 due to the limited number of qualified training providers, their ability to accommodate additional trainees, and limited course offerings. Approximately 1,500 participants were trained through VTGF training grants, roughly 92 percent of the revised compact target.
In addition to planned activities, capacity-building grants were provided to increase both the physical capacity and the quality of training providers. These grants incentivized participation in the VTGF by facilitating long-term changes to providers’ operations and by helping them meet formal requirements for registration and accreditation. A board and lodging stipend was also established for applicants who received VTGF scholarships, increasing trainee attendance and reducing dropouts due to financial hardship.
Training Enrollment and Completion
At follow-up, about 59 percent of the treatment group had enrolled in vocational training—a 34 percentage point increase in enrollment compared to a control group that did not receive training scholarships. In addition, 46 percent of VTGF treatment group members reported that they had completed their vocational training, compared to 17 percent of the control group. The treatment group was also more likely to enroll in further vocational training, with 14 percent of treatment group members continuing their training compared to 9 percent of the control group, largely driven by women choosing further training over employment.
The VTGF Sub-Activity sought to diversify and achieve greater gender balance in the workforce. Despite concerns that the random selection of participants for VTGF training and traditional views on gender in vocational fields might result in training dominated by men, 61 percent of the treatment group was female. In addition, the impact on enrollment in training for women was 13 percentage points (48 percent) higher than men and the impact on training completion was about 11 percentage points (46 percent) higher, both statistically significant at the 5 percent level.
Employment and Income
While there were significant impacts on training participation from VTGF grants, this did not translate into positive impacts on employment. At follow-up, 44 percent of VTGF treatment group members held any type of paid employment, compared to 50 percent of the control group. However, treatment group members were 5 percentage points more likely to re-enroll in vocational training compared to the control groups, an effect driven mostly by female applicants.
The offer of VTGF scholarships did not have any impact on applicants’ individual or household earnings or income, with a mean monthly income for the VTGF treatment group of N$1,328 compared to the control group mean of N$1,400. This difference between the treatment and control groups’ incomes was not statistically significant. The evaluation found that earnings were generally low: more than half the treatment group had no earnings in the month prior to data collection, and only about one quarter earned more than N$2,000. The estimated impacts on the distribution of total individual income (including non-earnings components) and household income in the month before the follow-up survey were also not statistically significant.
Improving the quality and relevance of vocational training programming requires a package of interventions which are rooted in the key institutional problems identified in the given context. MCC would benefit from conducting a more thorough assessment of the institutions and stakeholders involved at early stages of compact development.
Vocational training interventions need a credible, demand-based approach for identifying skills gaps in the labor market. Vocational training efforts aiming to improve employment outcomes should better calibrate training programs to the needs of the labor market.
If scholarships do not cover all costs associated with training, they may not be enough to ensure people enroll in and complete training. To the extent that funding is a concern, training providers may want to assess how scholarships meet trainees’ total costs, and identify ways to address funding gaps in order to increase take-up and completion of vocational training courses.
The VTGF evaluation used a mixed-methods approach, which included a rigorous impact evaluation, complemented by a qualitative performance evaluation. The impact evaluation used a random assignment design to estimate impact on the key outcomes of interest. For each training where the number of applicants exceeded the number of training slots, applicants were randomly assigned to either receive VTGF funding (treatment group) or receive no VTGF funding (control group). Because the offer of funding was randomly assigned, the treatment and control group for each training were expected to be similar on average in all respects other than funding.
The performance evaluation primarily drew upon interviews with key stakeholders and document analysis. Data collection was conducted between October and November 2014. The exposure period was between one and two months.
The impact evaluation involved two rounds of quantitative data collection that were administered to 1,250 applicants. The baseline survey was administered between December 2011 and July 2014 and the follow-up survey was administered between March 2014 and April 2016. Although the plan was for the follow-up survey to occur roughly one year after the scheduled end of each training, in practice the timing varied considerably (between six and 28 months). However, the median was close to one year after the end of training (13 months). The exposure period for the follow-up survey was between six and 28 months.
The follow-up analysis sample included applicants from 26 VTGF trainings, conducted by 10 training providers. These trainings cover about one-third of all VTGF trainings and about one-half of all VTGF-funded trainees, but are not representative of the full set of trainings or trainees.