After independence, Georgia inherited a run-down Soviet TVET system, with policy geared to a centralized and top-down bureaucracy. TVET institutions were in a state of disrepair, staff skills were neglected, and many graduates were not well-positioned to obtain employment in a changing labor market. The GoG had been prioritizing TVET sector reform, but there remained a mismatch between the types of skills training being offered by TVET providers and the skills demanded by industry. Moreover, pursuing technical or vocational education was regarded by most potential students and their parents as the option of last resort, training one would seek only if unable to successfully pursue higher education.
In existing TVET programs, substantial gender disparities were observed in STEM fields. The wages of female TVET graduates were significantly lower than wages of male TVET graduates, attributed to the fact that many women TVET students pursued less lucrative non-STEM related fields of study. In 2010-2012, female enrollment in engineering and agriculture TVET tracks was approximately 14 percent. A tracer study of TVET graduates conducted by the National Center for Educational Quality Enhancement with support from MCC and the World Bank in 2012-2013 found that women earned less than men, with 22 percentage points more women making less than 500 Georgian lari per month. 1
The Industry-Led Skills and Workforce Development (ISWD) Project aimed to address the gaps between labor market demand for skilled workers and the supply of Georgians with technical skills relevant to the labor market through investments in TVET. The project supported industry-relevant training and education programs and increased the capacity of education providers to deliver programs in accordance with international best practices. It placed an emphasis on supporting skills development in STEM occupations, as well as in agriculture and tourism, growth industries where there were also significant gaps in the labor market. By providing the technical skills demanded by employers, the project aimed to further build the pipeline of Georgians ready to enter the workforce and build Georgia’s economy. The project consisted of two activities:
- Competitive Program Improvement Grants Activity
- Strengthening Sector Policy and Provider Practice Activity
By using a holistic approach to reforming the TVET sector—through public relations and communications to improve the perception of technical and vocational education, improved curricula and programs, technical assistance to the GoG’s reform efforts, increased involvement of the private sector, and improved pedagogy—the project contributed to improved perceptions of and increased demand for TVET in Georgia. The project, and particularly the Strengthening Sector Policy and Provider Practice Activity, was designed and implemented in close cooperation with the EUVEGE—the flagship project of the European Union for TVET and employment policy development in Georgia—and with the German Development Agency, Swiss Development Corporation, and UN Development Programme projects. These projects all coordinated their efforts to improve the quality of TVET provision through stronger partnerships with employers.
Competitive Program Improvement Grants Activity
The objective of the Competitive Program Improvement Grants (PICG) Activity was to create a competitive grants program awarding TVET programs that introduced or expanded innovative approaches to building skills for employment. MCA-Georgia ran a $12 million competitive grant mechanism to solicit and fund innovative, industry-driven proposals. Applicants were TVET providers that partnered with employers and/or international organizations. The competitive grant facility encouraged innovation and the development of new programs that could not be procured through the existing market. Grantees developed, expanded, or improved TVET programs to meet identified skills demanded by the labor market. Each grant required matching funds of at least 10-15 percent from the grantee and its partners in order to stimulate a private-public partnership model for TVET program delivery.
Under the activity, Georgian public and private TVET providers received grants to scale up or introduce newly authorized, higher-level programs in partnership with national and international companies, universities, and industry representatives. These included globally recognized firms and institutions such as CISCO, British Petroleum, Pearson, the University of Finland, the Technical University of Munich, Wageningen University and Research, Deutsche Bahn, and multiple others that contributed financial or material resources or provided technical assistance on top of MCC’s investment.
The activity awarded $12 million in grants to more than 50 new or expanded TVET degree and certificate programs at 10 public and private education institutions. All programs received accreditation from the Georgian board of certification. Shorter, more targeted certificate programs (as opposed to the 18-24-month traditional diploma programs) were developed and drew many applicants, both from private companies wanting to improve their employees’ skills and knowledge, and from individual students seeking employment. In total, these programs graduated 727 students during the compact period with skills demanded by the private sector in growth sectors including tourism, agriculture, infrastructure, and STEM fields. It generally takes less time to earn a technical qualification than to get a university degree, resulting in several outcomes: these new programs primarily enrolled Georgians who could not afford higher education given the opportunity cost of studying without pay for multiple years, or increased participation from Georgians who were already in the workforce and returning to the classroom to obtain additional technical qualifications in hopes of higher salaries/compensation.
At the project level, grantees and their private sector partners exceeded MCC’s cost-sharing requirement, contributing $5.96 million in total. This collectively represents nearly three times more than the required amount, and almost half the amount that MCC invested into the program. According to reporting from the Millennium Foundation in April 2020, the programs developed under the compact continue to enroll new students, and additional degree programs have been developed and accredited since compact closure.
This activity also aimed to significantly contribute to the goal of increasing career opportunities and pathways for girls and women in the Georgian economy. In partnership with UN Women, MCA-Georgia provided tailored gender and social inclusion training for administrators from the TVET providers that received grants, instilling more gender-aware and inclusive TVET teaching. While total enrollment in the PICG courses exceeded the compact target, only 14 percent of the participants were female. Although recruitment of women into traditionally male-dominated TVET fields proved challenging for grantees, partnerships with the U.S. Embassy in Georgia and the private sector exposed TVET providers to targeted recruitment tools in an effort to create employment opportunities specifically targeted at increasing female enrollment. These opportunities included a three-week tour of U.S. community colleges with a successful track record of recruiting and graduating women.
Strengthening Sector Policy and Provider Practice Activity
At the start of the compact, Georgia’s TVET policies were in the process of being elaborated, in collaboration with the European Union and other donors. No system was in place to track TVET best practices or disseminate them, nor was there a viable pathway to engage industries in TVET trainings or employ graduates. Engaging industries allows for TVET programs to align with labor market demands. To strengthen TVET policies and practices, technical assistance was provided to the Ministry of Education and Science’s TVET Department. The topics covered by the technical assistance were annually agreed upon with the Ministry of Education, Science, Culture, and Sport. Assistance included extensive support on industry engagement, quality assurance, and career guidance policy development.
To promote good practices and pilot meaningful engagement with industry, MCA-Georgia launched a second, smaller competitive grant facility for TVET providers to strengthen, document, and disseminate existing or recognized good practices in vocational education. MCA-Georgia awarded about $400,000 in “best practices” grants to 27 TVET providers. Though a match was not required, small grant recipients collectively contributed 18 percent of the grant totals in co-financing.
MCA-Georgia also held annual TVET conferences in 2016, 2017, and 2018 to showcase and promote TVET best practices and recognize industry leaders, with significant co-funding from private sector in the case of the second and third conferences. The conference served as a forum for dialogue and information sharing among TVET stakeholders, and the dissemination of best practices. The conferences were seen by stakeholders to contribute to improving perceptions of TVET in Georgia, and Millennium Foundation held the annual TVET conference in July 2020. Continuing these conferences in the future will depend on funding and organizational support.
To further the impact and sustainability of the project, the compact envisioned the development of partnerships between education providers and employers, in order to promote private investment in Georgian technical and vocational education. During implementation, partnerships remained a key priority for both MCC and the GoG. Private sector partners brought significant co-funding and in-kind contributions to each TVET institution that received MCC assistance. These partnerships improved the quality of TVET education in Georgia by bridging the gap between labor supply and industry demand.
Technical assistance provided to the Ministry of Education, Science, Culture, and Sport aimed to increase the project’s sustainability. The GoG developed a four-pillar reform plan for the education center in 2019. Aspects of the plan focused on tertiary education and industry engagement in TVET. The GoG also passed a TVET Law in 2018, committing to the continued improvement and development of the sector. These policy reforms will allow the public-private partnership model for TVET developed by MCC, MCA-Georgia, and the GoG to continue for years to come.
Environmental sustainability was built into the project from the start. Grant criteria were developed in advance to ensure compliance with MCC’s Environmental Guidelines. Grant recipients prepared environmental, social, health, and safety plans for all activities and identified and developed opportunities to incorporate environmental sustainability into training program curricula.
At the end of the compact, the Ministry of Education, Science, Culture, and Sports planned to continue hosting annual TVET conferences, and Millennium Foundation held a TVET conference in July 2020 with more limited participation due to the pandemic. Other international donors, such as USAID, the European Union, German Development Agency and United Kingdom Good Governance Fund, noted the success of this project and have developed their own interventions that will continue the technical assistance, TVET reform, and industry-led TVET programs. The European Union developed a new “Skills Development and Matching for Labor Market Needs” program, worth about €48 million, that builds on their EUVEGE project and MCC’s experience in TVET in Georgia and incorporates the technical assistance developed by the compact.
Original Compact Project Amount: $16 million
Total Disbursed: $15.7 million
Estimated benefits at compact closure correspond to $15.7 million of MCC project funds, $5.97 million of country contribution project funds, as well as administrative and M&E costs, which are all included in the related cost-benefit analysis.
|Project||Activity||Estimated Economic Rate of Return over 20 years 2||Estimated beneficiaries over 20 years||Estimated net benefits over 20 years|
|Industry-Led Skills and Workforce Development Project||At the time of investment decision||21%||26,000||$22.9 million|
|Updated at entry into force||13%||25,000||$3.6 million|
|Updated after compact closure||19.1%||81,769||$36.4 million|
|Competitive Program Improvement Grants Activity||At the time of investment decision||22%||26,000||$24.5 million|
|Updated at entry into force||14%||25,000||$5.3 million|
|Updated after compact closure||20.9%||81,769||$39.1 million|
The original ERRs were estimated in April 2013. Among others, this estimate included the assumption that a nominal 23.8 percent increase in income would accrue to graduates of technical vocational programs receiving grants. The costs of the Strengthening Sector Policy and Provider Practice Activity were included in the project-level ERR, but no benefits were assigned because of the lack of project definition and quantitative data to estimate potential outcomes. The ERR estimates were revised shortly after the compact entered into force. The reduction came from a slight increase in estimated costs, a decrease in expected beneficiaries, and accounting for variation in several key parameters including proposal amount, private sector investment contribution, and increases in wages and employment. The grantees under the Competitive Program Improvement Grants Activity were not selected until after implementation began, so the EIF ERR was calculated based on assumptions about costs and benefit streams. TVET center ERRs were later estimated and used as part of the grantee selection process.
The economic logic underpinning the closeout CBA model used to calculate the ISWD Project and PICG Activity ERRs remains the same. The PICG Activity still accounts for all benefit streams in the project-level ERR. The Strengthening Sector Policy and Provider Practice Activity appears to have been implemented well and achieved the intended outputs, thereby further supporting the short-, medium- and long-term outcomes of the ISWD investment. However, the Activity does not appear to have any specific or additional benefit streams or beneficiaries to incorporate. The ISWD Project ERR includes all costs from both activities. 3
The main benefit stream of the PICG Activity comes from improved employment outcomes and higher wages for graduates of MCC-supported training programs. Graduates are expected to be more likely to enter the labor force and be employed, particularly in their field of study and in positions that provide them with more job-related benefits. Individuals expected to obtain those benefits are considered beneficiaries. These outcomes indicate that firms would have the potential to become more productive, reduce labor costs incurred from hiring foreign workers, and potentially grow.
The CBA model structure changed substantially given a shift from hypothetical to the actual 10 TVET centers selected as grantees in 2016. Earlier ERRs were based on similar assumptions and parameter estimates across all training programs, as details were unknown at the time, while closeout ERRs consider key TVET center characteristics (e.g., international certification, level of training, economic sector’s growth potential, collaboration with employers) to inform labor market outcomes. For each center, the model follows cohorts that enroll and graduate across an assumed 20-year life of the investment. The analysis also considers a 10-year lifetime of the investment and still obtains an ERR well above 10 percent, but with a clear reduction in beneficiaries.
The ISWD Project is viewed as having been successfully implemented across various measures, with itemized data on the inputs to outputs outlined in the program logic, and inclusion of clear progress on several short-term outcomes. Although, several training programs experienced lower enrollment numbers and higher dropout rates than expected, leading to fewer beneficiaries, preliminary evidence on short-term and medium-term outcomes remains positive. Future analysis will inform key assumptions (e.g., continuation of training programs, updated enrollment numbers and dropout rates, linkages to employers, labor market outcomes) to calculate the evaluation-based ERRs, as several remain risks to long-term project success.
The Industry-Led Skills and Workforce Development Project aimed to strengthen the linkage between market-demanded skills and the supply of Georgians with technical skills relevant to the local economy. The project’s mixed-methods performance evaluation will assess project implementation, project sustainability, policy reform, and changes to labor market outcomes of graduates from project-supported courses. Specifically, the evaluation will seek to answer the following questions:
- How did the implemented PICG courses compare with the original grant proposals, and what were the reasons for any deviations?
- Did trainees enroll in PICG courses and graduate from them at targeted levels?
- What were the labor market outcomes (employment and wages) for graduates from PICG courses?
- What are TVET providers’ perceptions of the best practices identified and disseminated by the program, to what extent have they adopted them, and what are the main barriers to doing so?
- To what extent have the Ministry of Education and Science (MES) and its agencies adopted the policy reforms supported by the program, and what have been the main challenges in doing so?
- How and to what extent has the annual TVET conference influenced providers, employers, the MES, and other TVET sector stakeholders?
Key Findings from the Interim Evaluation Report’s Evaluation Brief include:
- New Courses Successfully Established
- Through the Competitive Program Improvement Grants Activity, the Industry-Led Skills and Workforce Development Project established 51 new or improved TVET courses in Georgia.
- These courses were established through close cooperation between the TVET provider and private sector partners, who provided valuable knowledge and material support.
- Course Participation and Early Perceptions
- Total enrollment in supported courses exceeded the compact’s target.
- Trainees and teachers had positive first impressions of the new courses and were optimistic about trainees’ labor market prospects.
- Other Project Activities
- The small grants supported dissemination of best practices in ways that could be replicated by others.
- The technical assistance component delivered support for a wide range of policy reforms, but more work will have to be done after the compact.
- The Annual TVET Conference, which aimed to strengthen ties between TVET providers and private sector firms, was well-attended and well-received by stakeholders.
|Baseline||Data collected May 2017 to June 2018. Because courses were staggered in their start dates, baseline data collection was also staggered. The baseline data analysis is in the Interim Report.|
|Interim||Data collection was completed in December 2018. The Interim Report was published in August 2019.|
|Endline||Data collection will be completed in mid-2021. The Endline Report is expected in late 2021.|
Lessons from the evaluation:
- The grant application process was rigorous and well-managed, resulting in high quality grant recipients. MCC should consider the importance of the grant application process and grant management capacity when designing and implementing grant facility projects in the future.
- Although the program sought to increase female participation in STEM fields, only 14 percent of all trainees in the PICG-supported courses were female. In order to recruit more women into traditionally male-dominated fields, future programming should seek to better understand the underlying causes of low female participation and target specific interventions to address those causes.
- Overall, initial impressions of the PICG-supported courses were positive; however, both instructors and trainees noted a lack of instructional materials in Georgia in technical subjects. Future TVET projects should strive to provide resources for written materials in the local language if not already available.
- The sustainability of MCC’s investments under the ISWD project is still uncertain. MCC should plan from the start of project design for funding and support post-compact.
Key Output and Outcome Indicators
Outcomes: Increased availability of higher quality TVET courses and better alignment with industry needs. Graduates of the new and improved TVET courses are expected to have increased employment opportunities and earn higher salaries.
|Key Performance Indicators||Baseline||End of Compact Target||Q1-Q20 Actuals (As of July 2019)||Percent Target Satisfied|
|Students participating in MCC-supported education activities||0||1,500||1,935||129%|
|Graduates from MCC-supported education activities||0||No Target 4||727||N/A|
|Industry co-investment in TVET provision||0||$1,800,000||$5,967,379||331.52%|
Explanation of Results
The Industry-Led Skills and Workforce Development Project proved to be one of the most successful compact projects in terms of meeting output targets. The project far exceeded its target for industry co-investment, suggesting the eagerness of industry to engage on TVET.