Georgia II Compact

In 2013, MCC signed a $140 million compact with the Republic of Georgia to increase the earning potential of Georgians through holistic improvements in education. This was the first MCC compact to focus exclusively on education. To address the identified constraint of a lack of an adequately trained workforce to meet labor demands, MCC and Georgia designed a compact that targeted: the quality of general education through rehabilitation of deteriorating schools, training for educators and school directors, and support for education assessments; technical and vocational training programs to provide the skills that businesses in the country demand; and the development of a higher education partnership to help modernize science, technology, engineering, and math education in three of the country’s top universities.

  • Signed: July 26, 2013
  • Entry into Force: July 1, 2014
  • Compact End Date: July 1, 2019

Executive Summary

MCC signed a $140 million compact with the Government of Georgia (GoG) in 2013. The compact was designed to increase the earning potential of Georgians through holistic improvements in the quality of education in science, technology, engineering, and math (STEM) fields, and included strategic investments in general education, technical training, and advanced degree programs. These investments sought to build upon the success of MCC’s first compact with Georgia, which was signed in 2005 and worked to address the lack of reliable infrastructure and the slow development of businesses, focusing particularly on agribusiness.

This was the first MCC compact to focus exclusively on education. The focus on education was based on the GoG’s analysis of constraints to economic growth in the spring of 2011, which identified a lack of an adequately trained workforce to meet labor demands. To address this constraint, MCC and the GoG designed a compact that targeted: the quality of general education through rehabilitation of deteriorating schools, training for educators and school directors, and support for education assessments; technical and vocational training programs to provide the skills that businesses in the country demand; and the development of a higher education partnership to help modernize STEM education in three of the country’s top universities.

Under MCC’s country ownership model, governments receiving MCC assistance are responsible for implementing the MCC-funded programs. Partner governments establish units known as accountable entities, referred to as MCAs, to manage implementation of compact projects. The Georgia II Compact was largely implemented by MCA-Georgia, but the GoG assigned several implementation responsibilities—such as utility connections, education assessments and teacher training—to other GoG agencies to help ensure long-term sustainability. This implementation strategy was important for country ownership. However, it presented several obstacles and delays. Early in the compact, some entities were beset by staffing and capacity challenges, as well as technical differences between MCC and GoG policies and practices. Ultimately, MCC and the GoG overcame those challenges and successfully executed the compact.

The GoG contributed $32.96 million of its own funds towards the compact and proved itself to be a strong partner throughout implementation of both compacts by meeting key conditions required to release compact funds and passing legislation to support the sustainability of compact benefits.

The compact entered into force on July 1, 2014 and concluded on July 1, 2019. Approximately $138.6 million (99 percent) of the compact was disbursed. MCC’s support for improvements to education and workforce training in the country is expected to benefit 1.2 million people over 20 years. 1

The Star Report for the Georgia II Compact provides a summary of the compact’s results, documents key changes in compact activities and the reasons behind them, details information on performance against targets in the monitoring plan, and summarizes the results of independent evaluations that have been completed. It also outlines relevant partnerships and learning as a result of compact investments. This report will be updated to include data from evaluations that are not yet complete.

The Georgia II Compact consisted of three projects:

The Improving General Education Quality Project

The objective of the Improving General Education Quality Project was to improve general education quality in Georgia through infrastructure enhancements to the physical learning environment in schools, training for educators and school managers, and support to national and international education assessments. The project sought to rehabilitate dilapidated school facilities, upgrade utilities such as heating, electrical, water supply, and sanitation systems, and provide science laboratories and basic equipment to 91 Georgian public schools. It also established a framework for public school system-wide operations and maintenance. Additionally, the project included activities to provide training to all public secondary school STEM and English teachers, all public school principals, and school-based professional development facilitators. Beyond training, this project also financed Georgia’s participation in three international assessments and the implementation of five national assessments focused on math and science. Interim evaluation findings showed progress towards outcomes in both school infrastructure and teacher training results. One year after the treatment schools were rehabilitated, students and teachers reported substantial improvements in the school infrastructure, including heating systems, science labs, and classroom appearance, as compared to the schools in the control group. There was no significant change in student learning outcomes, but the program logic posited that it would take longer to see an effect on those outcomes. Interim results from the teacher training activity showed that teachers felt more confident in their knowledge of the new pedagogy, but that teachers had not yet begun to use those pedagogical practices in the classroom. Preliminary results from the final data collection round showed teachers started to utilize the new pedagogical practices at a higher rate during years one and two  after completing the training, especially younger certified teachers.

The Industry-Led Skills and Workforce Development Project

The objective of the Industry-Led Skills and Workforce Development Project was to strengthen the linkage between market-demanded skills and the supply of Georgians with technical skills relevant to the local economy. The project aimed to increase the number of Georgians with in-demand technical skills that would boost their employability. The project provided an initial investment in competitive grants to technical and vocational education and training (TVET) programs that develop, test, and disseminate innovative and effective approaches to employment-oriented skills, in partnership with public and private sector employers. Additionally, the project worked to provide technical assistance to develop policy that supports industry engagement and matches private sector demand with labor supply; invests in small-scale competitive grants programs that identify, document, and disseminate best practices by TVET providers; and fosters international best-practice exchanges between industry and government leaders. Interim evaluation findings for this project revealed good progress in the areas of developing new or improved TVET courses in Georgia. Enrollment in the new courses generally met or exceeded expectations and participants reported high rates of satisfaction and that they expected increases in income after completing the courses.

The STEM Higher Education Project

The objective of the STEM Higher Education Project was to support delivery of high-quality STEM degree programs in Georgia. The project sought to modernize science, technology, engineering, and math higher education through a partnership that introduced high-quality, U.S.-accredited STEM bachelor’s degrees in Georgia. Following the GoG’s competitive selection process, San Diego State University and three Georgian public partner universities were awarded contracts to administer bachelor’s degree programs targeting enhanced employment opportunities for Georgian students. The project also funded rehabilitation and construction of modern lab and classroom facilities, essential equipment upgrades, curriculum development, and institutional support towards obtaining international accreditation. Interim evaluation findings for this project have shown mixed results, specifically: students were very satisfied with new facilities and equipment but were less satisfied with their experience with the faculty and the courses offered.  This may have been due to student expectations and their understanding of the American bachelor’s degree, which includes liberal arts course requirements that many students did not want to take, as well as online course delivery.

Country Context

Georgia has been a strategic partner of the United States since the country’s independence from the Soviet Union in 1991. Georgia’s growth since independence has been characterized by sharp disparities, with poverty persisting notably in rural areas. Situated in a region of the world fraught with conflict since the breakup of the Soviet Union, Georgia has made remarkable progress, having enacted numerous reforms while benefitting from almost $545 million in MCC investments over 15 years of partnership.

Shortly after the Rose Revolution in May of 2004, MCC’s Board of Director’s selected Georgia for a $395 million compact focused on infrastructure and business development. Upon compact completion in April of 2011, the compact had delivered targeted interventions for the poor through increasing access to jobs, basic services, and capital for business development.

The first Georgia compact rehabilitated 220 kilometers of highway, connecting the capital of Tbilisi to key agriculture corridors in the southwest, Armenia, and Turkey. Improved highways drove international trade and investment, and increased transportation access, and speed to both the capital city of Tbilisi and local markets decreased. The improved roads significantly increased industrial investment in nearby communities. 2   The creation of a business development program, which was designed to accelerate the shift from subsistence to commercial agriculture, contributed to an increase in firms’ revenue by over $3.8 million and supported the creation of over 3,000 jobs. 3 Critical repairs were made to the country’s main gas pipeline, to provide better access to reliable energy. The compact also worked to increase the reliability of water supply in five cities and reduce costs for more than 67,000 residents and businesses.

Georgia achieved wide-scale economic growth during the span of the first compact. Georgia moved from 100th to 11th place in the World Bank’s “Doing Business Indicators” between 2006 and 2010. The country implemented sweeping reforms that strengthened public finances, deregulated business, improved the business environment (e.g., the process of registering a new business was dramatically simplified), and enhanced social protection. 4 As a result of these reforms, Georgia enjoyed rapid economic growth, averaging 9 percent annual GDP growth from 2004-2007.

In 2008, Georgia’s foreign-investment-driven economy faltered with the double shock of the global economic downturn and conflict with Russia. Although economic growth somewhat rebounded in the subsequent years, poverty rates remained high, increasing from 22.7 percent to 24.7 percent after the 2008 conflict with Russia. The poverty gap between urban and rural areas also widened during this period. Though less than half of Georgia’s population lived in rural areas, 64 percent of Georgia’s poor were rural residents. While the country had made progress, it still lacked the inclusive economic growth necessary for long-term poverty reduction, which is, in part, why it was selected for a second compact.

As part of the compact development process, the GoG conducted an analysis of constraints to economic growth in the spring of 2011 and identified the lack of an adequately trained workforce to meet labor demand as one of two binding constraints. 5 This finding aligned with the 2011 World Economic Forum’s Global Competitiveness Report, which identified an “inadequately educated workforce” as the second most problematic factor for doing business in Georgia in 2011-2012.  

MCC and the GoG agreed to target Georgia’s education system to improve the skills of the Georgian workforce. Education was lacking in STEM fields in particular. In 2007, Georgia ranked 33 out of 48 countries in an international math assessment of grade 8 students, and 37 out of 48 in science. 6 In 2009, Georgia’s scores in an international student assessment revealed that two-thirds of 15-year-old students performed below the baseline proficiency in reading, math, and science. 7 The low quality of education continued at higher rungs of education. There were not enough quality STEM courses in Georgian universities and technical colleges. This deficiency was reflected in Georgia’s ranking 120 out of 142 countries in the availability of scientists and engineers. 8

The constraints analysis also found that gender disparities were prominent in STEM careers in Georgia. Data showed girls outperformed boys in school, yet schools and communities steered girls into less profitable sectors of the economy, resulting in lower earnings on average for women than for men. In December 2011, the GoG submitted a proposal to MCC outlining objectives to improve education quality through investments in general education, TVET, and STEM higher education. MCC approved the proposed project areas, and the GoG subsequently held meetings with over 50 different international higher education institutions in Georgia and the U.S. in order to obtain market feedback to help define the parameters of an MCC compact-funded partnership for improved higher education in Georgia. Additionally, to understand the gap between labor supply and demand, MCC officials met with more than 70 private sector representatives throughout Georgia.

MCC’s Board of Directors approved the compact with Georgia in June of 2013, and it was signed in July of 2013. The objective of the compact was to strengthen the quality of general education (particularly STEM education), TVET, and higher education through these interrelated projects:

  1. Improving General Education Quality Project;
  2. Industry-Led Skills and Workforce Development Project; and
  3. STEM Higher Education Project.

The compact took a lifecycle approach to education, investing in general education, technical training, and tertiary education. The aim was that Georgians would enter the workforce better trained, earn higher wages, and be better equipped to meet industry demands. The project originally intended to benefit 1.7 million Georgians, nearly half the population in the long-term, and to reduce poverty and spur economic stability and growth.

To address the gender disparities identified in the constraints to growth analysis, the compact aimed to increase women’s participation in the workforce and in STEM fields, in particular. MCC partnered with the GoG, the private sector, the U.S. Embassy, and other donors to amplify the impact of the compact, including on Georgian women and girls. Gender and social inclusion best practices were used to better assist Georgia’s most vulnerable citizens, namely the poor and ethnic minority students attending neglected schools outside of Tbilisi, and high school graduates entering TVET as a path to prosperity. These targeted investments aimed to ensure that the compact’s economic growth benefits also reduced poverty.

The GoG remained a strong and supportive partner of MCC throughout implementation and beyond. Georgia contributed $32.96 million to the compact and in 2019, after compact closure, the GoG announced that it would continue the work of the MCC compact through investing in education, partnering with donors such as the World Bank to build on MCC’s efforts to enhance education quality. The GoG contributed an additional $10.5 million to continue supporting the STEM Higher Education Project after compact closure. At the end of the compact, the GoG established and funded the Millennium Foundation, an entity in charge of sustaining the program. The GoG’s commitment to continue and replicate the compact’s student-centered investments, from the start of a child’s schooling to his or her placement in the job market, bodes well for sustainability of the compact’s results.

At a Glance

  • Original Amount at Compact Signing: $140 million
  • Amount spent: $138.6 million
  • Total country contribution: $32.96 million (end of compact)
  • Signed: April 7, 2011
  • Entry Into Force: September 20, 2013
  • Closed: September 20, 2018


Economic Analysis of the Compact

All numbers are at compact closure

Compact Project Estimated Economic Rate of Return over 20 years 9 Estimated Beneficiaries over 20 years 10 Estimated Net Benefits over 20 year
Georgia II Compact Not available 1,165,821 $106.3 million (2013 USD)
Improving General Education Quality Project 15.5% 1,067,817 $78.9 million
Industry-Led Skills and Workforce Development Project 19.1% 81,769 $36.4 million
STEM Higher Education Project 9.0% 16,235 -$9 million

Improving General Education Project

Project Summary

The 2011 constraints analysis identified insufficient qualifications of potential employees as a barrier to employment and wages. It was determined that a major barrier to target was weaknesses in the education system, particularly in STEM fields. Thus, the Improving General Education Quality Project began by approaching the problem at its starting point: at the general education level. There, the broadest and youngest group of Georgians would benefit from investment in improved school infrastructure, teachers and principals trained in student-centered instruction, and a curriculum policy feedback loop generated by regular assessment and evaluation of learning outcomes. By improving learning outcomes in the general education system, the project would contribute to economic growth through workforce development in Georgia.

International best practice recognizes that education quality is achieved when the following components are present in an education system: knowledgeable and motivated teachers, strong school management, a well-designed curriculum with good teaching materials, student testing and performance evaluation, and a safe and conducive physical learning environment. At the time of compact signing, the Georgian public education system was underperforming in each of these areas. Teachers were underprepared with respect to both subject and pedagogical knowledge, and school directors lacked adequate professional training. Schools lacked teaching materials required for hands-on, student-centered learning. School facilities were in severe disrepair and lacked adequate heating or protection from the elements. Student performance was rarely assessed nationally. As such, the objective of the Improving General Education Quality Project was to improve education quality by targeting the physical learning environment, secondary school teacher subject knowledge and pedagogical skills, school management, and education assessments, with an emphasis on the STEM subjects.

Problems of quality were most pronounced in two specific areas: schools in rural areas and areas with larger proportions of ethnic minorities. Poor and ethnic minority children were more likely to start secondary education late and drop out early. Additionally, students in rural schools and poorer children also scored lower than students in urban schools on international assessments. Teachers in these targeted areas also struggled. Based on poor teacher certification rates, it was also noted that teachers in poor and ethnic minority schools were less qualified. For example, of the teachers that attempted the exams, 80 percent of teachers at schools in Tbilisi, the capital, were certified in their subject matter, compared to only 41 percent of teachers in the regions from more rural or remote schools. To address this, the general education investments targeted regions outside of Tbilisi, where poverty and ethnic minorities were concentrated.

Another glaring disparity was the issue of gender. The transition rates for girls from secondary to tertiary programs in STEM areas were low compared to boys, despite the fact that girls outperformed boys in these subject areas. Through specific interventions designed to benefit female students, the project aimed to increase girls’ participation in these fields.

The Improving General Education Quality Project was comprised of three activities: the Improved Learning Environment Infrastructure Activity, the Training Educators for Excellence Activity, and the Education Assessment Support Activity. Three Georgian government implementing entities were in charge of project management and oversight for the General Education Quality Project: the Educational and Scientific Infrastructure Development Agency (ESIDA) for the Improved Learning Environment Infrastructure Activity; the Teachers’ Professional Development Center (TPDC) for the Training Educators for Excellence Activity; and the National Assessment and Examination Center (NAEC) for the Education Assessment Support Activity. These three entities are under the purview of the Ministry of Education, Science, Culture, and Sport, and each played a key role in school infrastructure management, educator professional development, and education assessment activities, respectively. Although this approach introduced some delays and operational challenges for implementation, close collaboration with these organizations was an explicit strategy to build long-term organizational and management capacity in Georgia and to sustain project investments after the end of the compact.

Improved Learning Environment Infrastructure Activity

Most Georgian public schools were built during the Soviet era and were not properly operated or maintained. Aside from emergency repairs and partial refurbishments, much-needed large-scale repairs and new school construction had also been neglected and under-funded, especially since Georgia regained independence in 1991. In general, heating systems did not work and were poorly vented wood- or coal-burning stoves that generated smoke and poor air quality but not warmth in most classrooms. Leaky roofs contributed to the active decay of the buildings and systems, including collapsing ceilings, rotting floors, unsafe wiring, and decaying concrete and plaster. Lighting was limited and hallways and classrooms were poorly lit. Schools also generally lacked quality teaching equipment and learning materials. In sum, prolonged periods of insufficient maintenance and neglect resulted in school facilities that were in poor physical condition, adversely affecting student attendance, learning, and educational outcomes.

Although there is not much international literature on the impacts school rehabilitation can have on student learning compared to new construction, U.S. literature indicates that physical infrastructure has an important impact on learning outcomes in general education. The characteristics that have the greatest impact are classroom temperature, air quality, lighting, and science labs/equipment. 11 Better maintained and more conducive learning environments facilitate student learning and attract as well as retain qualified, motivated teachers. The Activity provided an opportunity to learn the extent of the impacts of school rehabilitation on student learning. In fact, the interim evaluation report showed significant improvements in the quality of classrooms and general school infrastructure, and that students and teachers were more satisfied with their learning environments. The final evaluation will examine the changes in student learning as a result of the improved infrastructure.

The Improved Learning Environment Infrastructure Activity aimed to rectify key aspects of the learning environment, including inadequate heating systems, poor indoor air quality, and inadequate lighting. At the school level, interventions undertaken through the compact to address these issues included: full interior and exterior rehabilitation of classroom and support buildings; new or significant upgrades to utilities such as electricity, water, and wastewater; and new laboratory classrooms. Designs to increase student health and safety in rehabilitated schools complied with domestic regulations and reflected international good practice.

A root cause of the poor quality of school infrastructure was the absence of any significant, systematic operations and maintenance programs in Georgia. Thus, the Improved Learning Environment Infrastructure Activity included a school Operations and Maintenance (O&M) component. This established a viable public-school O&M program at a national level. This program was intended to help address this root cause of the poor physical condition of the learning environment and promote the sustainability of Improved Learning Environment Infrastructure Activity investments in school rehabilitation.

Under the activity, the GoG, with and through MCA-Georgia, developed a national school O&M framework plan, established a dedicated national budget line for school O&M, conducted comprehensive inspections of most school buildings, developed software to plan and manage school O&M and minor repairs, designed and executed urgent repairs in two municipalities, and trained key personnel on school O&M good practices. These practices served to operate and maintain new assets installed at rehabilitated schools, such as wastewater bio-treatment plants at schools lacking connections to municipal wastewater systems. MCC supported these and other efforts through a dedicated school O&M Incentive Fund within the compact of up to $2,500,000. Through this incentive fund, the GoG committed funds for O&M, and subsequently MCC “matched” the GoG’s contribution through a commitment of compact funds. The O&M sub-activity evolved substantially over time, as described under the “Compact Changes” section of this report.

The Improved Learning Environment Infrastructure Activity:

  • Rehabilitated 91 schools across the country;
  • Introduced science laboratory classrooms and equipment in 91 schools; and
  • Provided data, information, and planning to improve school operations and maintenance practices for all public schools in Georgia.

The GoG and MCC identified schools to be targeted for rehabilitation using a formula that prioritized schools outside of the capital Tbilisi according to their physical condition (dilapidated physical infrastructure), social vulnerability (higher proportion of socially vulnerable students), number of students enrolled, and utilization rate. This approach prioritized variables including school size and enrollment-to-capacity ratio (in combination, key to the ERR) and the proportion of socially vulnerable students (key to equity).

ESIDA, the government agency designated by the GoG as the implementing entity for this activity, created an 11-person project management unit (PMU) within the agency, focused exclusively on managing the MCC school infrastructure investment. The GoG funded all of the PMU’s operating costs as part of its country contribution to the compact. Although MCC was aware of capacity challenges within the agency, the decision to work through ESIDA despite these challenges was an explicit strategy aimed to build long-term capacity to sustain project investments after the compact. Unfortunately, staffing and capacity challenges and technical differences of opinion early on in implementation led to delays in the beginning of rehabilitation works. In response, MCC and the GoG decided to shift most of the responsibilities for building rehabilitation to MCA-Georgia, while tasking ESIDA with management and oversight of utility connections and ancillary works outside of the school building. MCC worked to keep ESIDA engaged in the school rehabilitation process from start to finish, with an aim to equip ESIDA with the knowledge and resources to properly maintain MCC-funded school sites post-compact term.

MCC and MCA-Georgia education and infrastructure specialists collaborated to improve the effectiveness, safety, and sustainability of science laboratories funded through the compact. The provision of science laboratory facilities, durable equipment and consumable supplies triggered significant curriculum revisions related to chemistry and physics experiments, modification of technical design criteria and installed systems for science classroom ventilation, the substitution of lower hazard chemicals for experiments, the refinement of supplies and experimental equipment to improve laboratory safety, and a comprehensive laboratory safety training program translated from English into local languages of instruction, including Georgian and Armenian.

ESIDA was also the implementing entity engaged in school O&M institutional reforms and executing Georgia’s country contribution to the school O&M Incentive Fund. During the final two years of the compact, the GoG started to implement decentralization reforms which resulted in ambiguity of institutional roles and responsibilities for school O&M. MCC focused more attention and resources on the O&M of compact-funded rehabilitated schools during the final two years of the compact, partly in reaction to this decentralization reform but also due to technical challenges obtaining data and information about more than 2,200 public school buildings that fell under the purview of the national O&M framework plan and system.

Training Educators for Excellence Activity

Some evidence shows that students have better learning outcomes when they have better trained teachers. 12 In the TEE Evaluation Design Report, the evaluators cite Hanushek and Chetty’s papers from 2010 which demonstrate that variation in teacher quality is causally linked to student learning outcomes. 13 However, the evidence on effect size is mixed. At the time of compact development, the GoG had established a framework for continuous professional development of teachers, including associated salary increases, in an attempt to improve teacher quality. The government entity responsible for teacher professional development, the Teachers’ Professional Development Center (TPDC), was charged with implementation of this framework, and needed support to transition from exclusively providing certification-oriented short-term, subject matter-focused trainings to facilitating continuous professional development. At the school level, it was acknowledged that school directors needed support, as well as strong educational leadership support for improved learning outcomes.

The objectives of the Training Educators for Excellence Activity were to: (1) improve math, science, information and communication technology, and English teaching in order to improve learning in grades 7-12; and (2) improve school management. To accomplish these objectives, the activity offered training to all 2085 public school principals in Georgia, at least one school professional development facilitator per school, and all 18,750 secondary school STEM, geography and English teachers in the country. The activity aimed to have at least 74 percent of these educators complete the full training course, taking into consideration the length of the trainings and turnover of teachers. The full training program was delivered over a multi-year period and included several cycles of in-person training modules for teachers and principals, complemented with tasks and activities between the modules. The trainings covered leadership skills, student-centered pedagogy approaches, innovative and interactive teaching methods, subject matter expertise, science lab health and safety, and gender bias.

Another aspect of the activity was the provision of trainings in minority languages. The Georgian population is ethnically diverse and in many instances members of minority groups have limited command of the Georgian language. Limited language comprehension affects student’s access and ability to actively participate in a mainstream education. In addition to Georgian language schools, the GoG funds public schools where the primary languages of instruction are Armenian, Azerbaijani, and Russian. Prior to the compact, ethnic minority population teachers had fewer opportunities than their ethnic majority counterparts to participate in professional development programs due to language barriers. Teachers and principals at these schools did not have access to the same training materials or trainers as Georgian-speaking teachers and principals, which furthered their exclusion from the education quality gains that the government was supporting. Notably, for the first time in Georgia’s history, through the TEE Activity, ethnic minority teachers and principals received trainings in minority languages (Armenian, Russian, and Azerbaijani) that were identical in content and format to the trainings that Georgian-speaking educators received, delivered by minority language-speaking trainers. TPDC targeted 2,177 minority language teachers, 213 school professional development facilitators and 213 school directors.

The Training Educators for Excellence Activity was expected to improve student learning through improved teaching and management of schools. A dedicated unit (PMU) within TPDC was responsible for overall implementation of the activity. Delays in forming the PMU, as well as early management challenges, caused the initial roll-out of the trainings to take place later than originally planned. Once on board however, the TPDC PMU staff had strong technical and managerial skills, and the activity was quickly back on track, as the PMU successfully managed and oversaw numerous, complex trainings and follow-on activities.

The structure of the trainings required a significant time commitment from school principals and teachers. School principals and professional development facilitators were trained through a three-part “Leadership Academy” training program. TPDC delivered this program regionally, which included up to 160 hours of intensive, face-to-face engagement. In order to foster exchange and collaboration among principals, those who attended the Leadership Academies were also required to attend quarterly meetings. At these meetings, principals from the same regions were able to discuss common questions and concerns, reinforce the techniques and approaches acquired during trainings, and hold one another accountable for completing assignments from the trainings. TPDC also organized two annual principals’ conferences which brought together school principals from across Georgia to share knowledge and best practices.

Teacher training consisted of intensive “core” and “subject matter” modules. This included 60 hours of face-to-face training on student-centered, participatory approaches to learning, complemented by additional non-contact hours. TPDC also organized regular study group meetings for teachers to foster “communities of practice” for STEM teachers at the school level, while allowing teachers to reflect on the material presented during the trainings. TPDC developed 16 hours of lab safety and experiment methods training modules with the support and guidance of the US-based Lab Safety Institute. These trainings were delivered to all STEM teachers in schools where new science labs were installed through the Improved Learning Environment Infrastructure Activity.

Integrated into the teacher and principal trainings were sessions on gender and social inclusion theory and practice, which TPDC developed through a partnership with UN Women. As teachers and principals play a vital role in the early empowerment of girls by creating an equal, inclusive environment for STEM learning, this additional training component aimed to have a strong multiplier effect. Teachers and principals were challenged to rethink cultural norms about the preference of boys and young men in the STEM fields, which then translated into more inclusive teaching styles. These investments aimed to increase the number of better educated girls able to pursue vocational and STEM higher studies.

After two years of training module development and activity design, implementation began in spring 2016 with trainings of trainers. Although attendance was optional, participation rates in both teacher and principal trainings were high. TPDC trained and certified more than 300 principals and teachers to serve as trainers to deliver the modules nationwide. By the end of the compact, approximately 18,000 teachers participated in compact trainings, nearly 12,000 of whom completed the full program and received a certificate. Over 2,000 principals and another 2,100 school-based professional development facilitators completed at least one course in the training program.

The interim evaluation report showed that by 2018, 93 percent of school directors attended all five training modules, and 82 percent of teachers from the first cohort completed the training. The training modules developed through this activity were subsequently integrated into the GoG’s continuous teacher professional development system, which provides salary and career incentives to encourage teachers to pursue professional development. The nationwide reach of this activity is the driver of the compact’s beneficiary number of just over 1 million students over twenty years. 14 Teachers who received the training are considered participants of an MCC project, and the beneficiary number represents the students who would benefit from better trained teachers for years to come, as a result of improved learning outcomes and therefore higher long-term earnings, as outlined in the description of the ERR for the TEE Activity.

The interim evaluation found that almost all school directors and teachers liked the trainings and saw value in participating. School directors reported that the training improved their capacity to provide instructional leadership through curriculum guidance, classroom observation, and supporting teachers’ professional development. Teachers also became more confident in their ability to teach higher-order thinking skills, promote cooperation through group work, and use lesson plans that include formative assessments and differentiated instruction for students with different abilities. However, there was minimal evidence of immediate changes in teachers’ classroom instruction practices, and in focus groups some teachers voiced concerns about the amount of time and effort needed to implement these practices consistently.

Education Assessment Support Activity

In support of improved education outcomes, the GoG committed to fostering enhanced, modern, and results-oriented schools. Assessment of student learning provides critical information on the operation of schools regarding the knowledge, attitudes, and skills obtained by learners as a result of their exposure to schooling. However, little work had been done to evaluate the performance of students across the nation. Education assessment results were mostly used to evaluate teachers, rather than to help educators better address the needs of their students. The National Assessment and Examination Center (NAEC) was also in need of support to ensure Georgia’s participation in international assessments.

Through this activity, the compact provided financing and technical support to NAEC to carry out national and international assessments. The compact provided funding to support national assessments of secondary school student achievement in math, biology, chemistry, physics, and Georgian as a second language. The compact also funded Georgia’s participation in two rounds of international assessments including the Teaching and Learning International Survey, the Trends in International Mathematics and Science Study, and the Program for International Student Assessment. Through its country contribution, the GoG also funded the country’s participation in the Electronic Progress in International Reading Literacy Study. The compact also funded additional training and salaries for key NAEC staff, as well as tools for school evaluations.

Data generated by national and international assessments enabled policymakers to observe trends in student achievement, both nationwide and as compared to other countries. Based on assessment outcomes, the Ministry of Education and Science would then be better positioned to plan, adjust, and implement policy decisions to support improvement of the teaching quality. In the final year of the compact, when there was large turnover within NAEC, particularly in the assessment division, the compact supported the training of new staff. By the end of the compact term, NAEC had completed two rounds of the three international assessments and six national assessments.

In addition, reports on assessment results were published and disseminated, and a national conference was held in 2017 to discuss the results of the international assessments. This conference was an important first step for the government, policymakers, and other stakeholders to demonstrate their willingness to define policy priorities based on assessment results. Early signs indicate that this activity is informing policy changes and shaping foreign assistance in the education sector. For example, the most pronounced finding of the assessments was a growing performance gap between students in urban and rural areas, with students in urban areas doing far better academically. As a result, during the unveiling of its education sector strategy in 2018-19, the GoG announced it would prioritize support to improve learning outcomes for students in rural areas and other socioeconomically disadvantaged students. In addition, in 2019 the GoG announced that, as a result of this finding, “school leaving” exams were determined to disproportionately impact ethnic minority students and were therefore abolished. Assessment results are also being widely used by other donors, such as USAID and the World Bank, to inform their new education assistance programs.

Project Sustainability

Sustainability was incorporated into the Improving General Education Quality Project from its inception. The GoG’s budgetary contributions laid the foundation for sustaining MCC’s investments in the future. Close partnership with implementing entities (ESIDA, TPDC, and NAEC), an approach that was also successful in the first Georgia compact’s Energy Rehabilitation Activity, was intended to help develop long-term institutional capacity in Georgia. From the start, this capacity building was an important way to increase the sustainability of the compact.

Improved Learning Environment Infrastructure Activity

The Improved Learning Environment Infrastructure Activity depended on collaboration with public schools and the Ministry of Education, particularly ESIDA. ESIDA was charged with implementing a nationwide O&M plan for all public schools, including those rehabilitated by MCC. During the compact term, ESIDA also assumed responsibilities for funding, designing, and connecting utilities, including water, electricity, and natural gas supply, as well as wastewater disposal, fencing, gates, and asphalt at school sites. Though it was at times challenging to align ESIDA’s roles and responsibilities for school construction, rehabilitation and operations and maintenance with MCA-Georgia school rehabilitation works timelines, ESIDA generally fulfilled responsibilities in these areas, and as a result, made a meaningful contribution to school rehabilitation. Following completion of MCC-funded rehabilitation works and prior to school re-openings, ESIDA-funded site perimeter fencing and paving work was generally completed and ESIDA-financed utility connections to power and water were finished. Schools were fully operational

During the last two years of the compact, government educational reforms focused on decentralizing school management. This decentralization introduced ambiguity as to which entity would be responsible for, among other things, school O&M funding, prioritization, and execution of O&M responsibilities. The revised National Whole School O&M System Framework Plan, developed and updated under the auspices of MCA-Georgia in January 2019, reassigned some roles and responsibilities from ESIDA to municipalities. Although MoES verbally expressed support and commitment to school O&M throughout the compact term, the GoG had difficulty committing and obligating funds identified for matching by the O&M Incentive Fund defined by the compact and undertake other actions required to meet the two school O&M conditions precedent in a timely manner. A high degree of turnover at MoES during compact years 4 and 5 compounded the lack of ownership and limited capacity to produce results related to systematically improving school O&M nationwide. In hindsight, establishing a viable, national-scale school O&M system over a five-year timeframe may have been overly ambitious and more difficult than anticipated for the GoG to sustain after the end of the compact. The role of ESIDA continues to evolve, as the GoG institutionalizes changes to roles and responsibilities within the general education sector.

The Millennium Foundation, the successor entity to MCA-Georgia and funded by the GoG, is supporting the sustainability of compact investments in school infrastructure. This entity is tasked with monitoring the quality of compact-funded school rehabilitation works and addressing defects that arise, as well as contributing to the implementation of the school O&M plan. The Millennium Foundation continues to raise the problem of insufficient nationwide school O&M financing, to ensure that a new funding structure for Georgian public schools will allocate specific funding for O&M purposes. ESIDA’s O&M budget helps guarantee that the public schools rehabilitated with compact funds are relatively well-maintained, however, the overall policy of school funding needs revision.

Training Educators for Excellence Activity

Sustainability was built into the Training Educators for Excellence Activity from the start by integrating all training activities into the GoG’s own teacher professional development system and building the capacity of TPDC staff to manage the system after the compact.  

After the compact, TPDC maintained responsibility for continuing to manage teachers’ professional development. In order to be financially sustainable, the compact also funded the development of online modules that would allow TPDC to deliver the same training content online at a significant cost savings. TPDC had already tested those modules and built capacity through delivering the online trainings to other non-STEM teachers before the compact end date. As future advances in the STEM fields are made, TPDC plans to continuously update the content of subject matter modules. In addition, in November 2018, TPDC began scaling up delivery of the core pedagogy modules developed through the compact, registering over 9,000 non-STEM teachers using its own resources. To ensure the sustainability of the teacher study groups formed during the trainings post-compact, TPDC provided grants to chemistry, biology, and math teachers’ associations. All of these follow-on activities were designed to ensure that compact investments are sustained and have a system-wide impact on future generations of Georgians.

Other donors will also continue to support educator professional development in Georgia. The Training Educators for Excellence Activity was designed in close coordination with USAID’s Georgia Primary Education Project, which ended in 2018. This program provided student-centered training to primary education teachers. MCC participated as a member of the working group to prepare a document appraising USAID’s new general education project Achieving Student-Centered Education for a New Tomorrow (ASCENT), which was launched in January 2020. ASCENT is focused on the primary grades and teachers’ and school administrators’ in-service and pre-service education and will therefore both complement and help sustain MCC’s TEE investments. The Peace Corps plans to use the English teacher subject-matter modules developed through the compact as well.

Education Assessment Support Activity

The compact aimed to ensure sustainability for the Education Assessment Support Activity by integrating activities into NAEC. Activities were led by NAEC personnel, with technical assistance provided as needed to build additional capacity within NAEC to administer testing, evaluate results, and share findings. Over the course of the compact, NAEC staff executed a number of national and international assessments, which provided them with the skills and experience necessary to continue them without MCC support. The GoG has expressed its intention to continue carrying out periodic national assessments and to participate in international assessments post-compact. One of the key objectives of the USAID ASCENT program will be to support the GoG in using data and evidence, including the results of education assessments funded through the compact, to inform policy decisions.

Economic Analysis

  • Original Compact Project Amount: $76.5 million
  • Total Disbursed: $70.7 million

Estimated benefits at the time of investment decision and EIF corresponded to $76.5 million of project funds, where cost-benefit analysis was conducted.

Estimated benefits at compact closure corresponded to $70.7 million of MCC project funds, $17.0 million of country contribution project funds as well as administrative and M&E costs, which are all included in the cost-benefit analysis.

Project Activity   Estimated Economic Rate of Return over 20 years 15 Estimated beneficiaries over 20 years 16 Estimated net benefits over 20 years
Improving General Education Quality Project   At the time of investment decision 17 13% 1,700,000 $28.6 million
Updated at entry into force 11% 1,700,000 $7 million
Updated after compact closure 15.5% 1,067,817 $78.9 million
  Improved Learning Environment Infrastructure Activity At the time investment decision 8% 18 423,000 $8 million
Updated at entry into force 10% 348,000 $23,000
Updated after compact closure 8.1% 66,266 19 -$13.2 million
  Training Educators for Excellence Activity At the time of investment decision 27% 1,700,000 $60.9 million
Updated at entry into force 18% 1,700,000 $19.1 million
Updated after compact closure 26.5% 1,067,817 $104.1 million

MCC presented the original economic rate of return (ERR) estimates for an investment decision in November 2012 and April 2013. This data was later revised after the compact entered into force (EIF). At that time, the estimated ERR for the Improved Learning Environment Infrastructure (ILEI) Activity increased from 8 percent to 10 percent. This change was due to the use of more recent data from the Georgian Integrated Household Survey indicating that the employment rate, calculated as the share of the working age population engaged in either formal or informal employment, was higher than originally estimated. The higher estimated employment rate increased the projected future earnings for students studying in schools rehabilitated under the compact.

The economic logic underpinning the investment decision and EIF cost-benefit analysis (CBA) models used to calculate the ILEI Activity ERR remains largely the same in the compact closure CBA model. However, updated data based on CBA model peer review, evaluation reports, and new information on GoG O&M commitments resulted in a reduction of the closeout ERR to 8.1%.

The main benefit stream from the ILEI Activity is an increase in educational attainment that results in increased probabilities of employment and increased long-term earnings. Increased time on task and reduced student absenteeism, as a result of an improved learning environment, are the channels through which student learning results are expected to improve. 20 These results are measured using completion rates (for 12th grade, reflecting graduation from upper secondary school) and transition rates (from upper secondary to TVET and from upper secondary to university) for secondary school students in Georgia and income, labor force participation, and employment data by school completion level. Due to weak evidence on the effects of school rehabilitation on education quality and educational attainment, 21 the closeout CBA model assumed more conservative increases in completion and transition rates for students in project-rehabilitated schools. 22

Improved learning outcomes and increased educational attainment leads to increased income through three pathways: (i) students who have higher educational attainment are more likely to be employed; (ii) students who have higher educational attainment earn more income when employed; and (iii) students who experience an improved learning environment in project-rehabilitated schools are assumed to earn higher incomes relative to without-project school students with the same level of education. The closeout ERR decreased from earlier estimates in part because of updated data on baseline graduation and transition rates, which were higher than initially thought. Updated data shows nearly universal completion from lower secondary school, which leaves little room for improvement among students who would not have completed 9th grade without the project.

The largest change in assumptions in the ILEI closeout CBA model reflects updated estimates on GoG commitments to post-compact O&M. This resulted in an assumption of a reduced project life, where benefits in the CBA model extend to only 10 cohorts of students instead of 20. While this reduced the number of beneficiaries by half, it did not negatively affect the ERR since the reduction in the benefits are offset by the reductions in O&M costs.

The ERR for the Training Educators for Excellence (TEE) Activity decreased from 27 percent at the time of the investment decision to 18 percent just after entry into force. This change reflected the evolving project design and additional research that suggested that the degree to which the training would affect students’ grades was less than originally anticipated. Although fewer teachers were expected to be trained, updates in class sizes meant that the same number of students were expected to be reached. This did not change the number of Georgians that benefited from the program.

The main changes to the closeout CBA model, and the cause of the more than 8 percentage point increase in the ERR, are from using updated data from project completion reports, interim independent impact evaluation, the latest national labor force and household data, and two comprehensive literature reviews. The main benefit stream for the TEE Activity is a relative increase in wages as a result of improved student learning. The interim evaluation demonstrated that the TEE Activity was implemented well, and its design was consistent with many of the most effective practices identified in MCC’s updated literature review. This supported the use of a slightly larger increase in test scores for the closeout CBA.

The next step in estimating the impact of these efforts are to consider how improved learning outcomes will result in increased future earnings. The closeout CBA updated estimates of long-term earnings based on more recent literature on the subject. The positive change in these two key variables (i.e., student learning and incremental wage increases) contribute to the large improvement in the closeout ERR for the TEE Activity.

There was no ERR estimated for the Education Assessment Support Activity. The costs of this activity are included in the project-level ERR calculations, but no specific benefit streams were included in the CBA models due to the lack of rigorous literature available to support its inclusion. While not included in the economic analysis, MCC and the GoG agreed on the importance of the activity, which aligns closely with MCC’s commitment to data collection and the use of evidence for decision making.

Evaluation Findings

The Improving General Education Quality Project aimed to improve the quality of public STEM education in grades 7-12. The project invested in rehabilitating education infrastructure and constructing science laboratories in targeted schools. A one-year sequence of training activities was provided to STEM educators and school directors on a nationwide basis. While the effects of the project are assessed by one evaluation, because the Improved Learning Environment Infrastructure and Training Educators for Excellence Activities were implemented separately and on different schedules, the evaluation uses two evaluation methodologies implemented on different timelines, one for each of the activities. The Education Assessment Activity has been closely monitored, but because the assessment portion was not integrated in the logic of the Training Educators for Excellence Activity, as originally planned, it was not included in the evaluation design.

The Improved Learning Environment Infrastructure Activity’s portion of the evaluation aims to measure the effect of improved infrastructure on attendance, enrollment, retention rates, and test scores. Specifically, the evaluation seeks to answer the following evaluation questions:

  1. Did school rehabilitation deliver improved facilities?
  2. What are the impacts of rehabilitation on the school environment, including temperature, lighting, equipment, and infrastructure maintenance?
  3. What are the perceptions of students, parents, teachers, and school directors about the effects of rehabilitation on safety, comfort, and the extent to which time in school is used effectively for learning?

The Training Educators for Excellence Activity’s portion of the evaluation will assess the impacts of teacher training in pedagogy and STEM subjects. Specifically, the evaluation seeks to understand to what extent the management skills of school directors and instructional skills of teachers have improved because of the activity. The evaluation will seek to answer the following questions:

  1. Did training initiatives for teachers and school directors succeed in delivering training on a nationwide basis?
  2. To what extent do school directors perceive that their instructional leadership and school management skills have changed as a result of the new training intervention?
  3. To what extent do teachers perceive that their pedagogical and classroom management practices have changed as a result of the new training intervention?
  4. Did teacher training modules improve teachers’ knowledge of and willingness to use practices related to student-centered instruction, formative assessments, and improved classroom management?
Bar charts of school rehabilitation evaluation findings

Key Findings from the Interim Evaluation Report’s Evaluation Brief include:

School Rehabilitation

  • In the first phase of school rehabilitation (29 schools), students experienced large improvements compared to baseline in heating, lighting, sanitation, building quality, and access to science laboratories and recreation facilities.
  • Students and teachers agreed that these improvements addressed barriers to using classroom time effectively on instruction.
  • The final report will estimate impacts for all rehabilitated schools and measure whether infrastructure upgrades improved learning outcomes.

Educator Training

  • The teacher training component was successfully delivered on a nationwide scale, with high completion rates for school directors and teachers.
  • Infographic of educator training evaluation findings
  • One month after the one-year training sequence concluded, teachers reported that they had improved confidence using student-centered teaching practices, and school directors reported that they had increased delivery of instructional leadership. However, there was little evidence of immediate changes in teachers’ classroom instruction practices.
  • Those involved in the design and implementation of the teacher training component anticipate that further changes in instructional practices could develop over time. The final report will examine trends in teaching practices several years after the training sequence ended.
Status of the evaluation: Improving General Education Quality Project
Component Status
Baseline Learning Infrastructure: Data collected from 2015 to 2017, in tranches as schools were rehabilitated. The baseline report was released in December 2017.

Training Educators: Data was collected in October 2017.

Midline Learning Infrastructure: Data collection was completed in March 2019.

Training Educators: Data collection was completed in October 2018. The Interim Report was published in August 2019.

Endline Learning Infrastructure: Data collection will be completed in 2022.

Training Educators: Data collection was completed in September 2019. The Final Evaluation Report is expected in 2023.

Lessons from the evaluation:

  • Construction timelines for the Improved Learning Environment Activity were significantly longer than originally anticipated. For future school infrastructure investments MCC should stress realistic work planning from the start of the compact.
  • The success of the TEE Activity in reaching its participation targets across Georgia was due to a great deal of collaboration between the MCA, Program Management Unit (PMU) within the Teacher Professional Development Centre (TPDC), and Program Management Consultant, IREX.
  • Stakeholder input is critical to survey module design and made the TEE data more useful to both MCC and the Ministry of Education and Science. Future evaluations should plan time for a survey design workshop in country.

Key Output and Outcome Indicators

Key Performance Indicators Baseline End of Compact Target Q1-Q20 Actuals (as of July 2019) Percent Target Satisfied
Educational facilities constructed or rehabilitated  0 130 91 70%
Science labs installed and equipped 0 130 91 70%
Students benefitting from rehabilitated school buildings 0 37,450 39,830 106%
School-based professional development facilitators who complete Leadership Academy 3 0 1,528 1,409 92%
School principals who complete Leadership Academy 1 0 1,668 1,823 109%
Teachers who have completed Core Module 3 0 14,578 14,859 102%
Teachers who completed the full course and received a certificate 0 13,666 11,829 87%
National assessments developed and implemented with MCC funding 0 10 6 60%
International assessments developed and implemented with MCC funding 0 5 6 120%

Explanation of Results

Improved Learning Environment Infrastructure Activity

The target for students benefitting from rehabilitated school buildings was met. However, the original target of rehabilitated education facilities was not met. Based on preliminary cost estimates, 130 educational facilities were envisioned to be constructed or rehabilitated under this activity. MCC and MCA-Georgia reassessed eligible schools in 2014, and in an effort to make the best use of funds available, weighted schools with more students per square meter more heavily. MCC and MCA-Georgia did not officially change the target for schools or the target for students benefitting from those schools because cost estimates and target numbers for each grouping of schools were set with each procurement. However, 100 control and 100 treatment schools were selected prior to the start of rehabilitation, due to MCC’s reservations about reaching the 130-school target and the implications for statistical power. The 2014 retargeting of schools with larger student populations helped to increase the number of beneficiaries for the activity.

Training Educators for Excellence Activity

In general, participation rates for trainings were high, especially for principals. Participation rates for teachers were somewhat lower, in part due to the number of trainings required to complete a full course, as well as the timing of the trainings (not all the teachers could participate in all of them since they were held on weekends and over holidays, even when make-up sessions were offered later in the year). In addition, geographic isolation of some of the teachers in rural schools posed challenges to participation in the trainings (due to road and accessibility constraints, the trainings could be offered only during specific times of the year).

Education Assessment Support Activity

The national assessments under the activity included two rounds of testing 9th graders in math, chemistry, physics, biology, and Georgian as a second language for 7th graders. The first round of testing was carried out, as well as the second round of the math assessment; the remaining four had to be postponed due to a change of personnel at NAEC during the final year of the compact, which disrupted the programming and administration of those national assessments. The World Bank’s Innovation, Inclusion and Quality project supports the GoG in continuing to carry out periodic national assessments post-compact.

Industry-Led Skills and Workforce Development Project

Project Summary

After independence, Georgia inherited a run-down Soviet TVET system, with policy geared to a centralized and top-down bureaucracy. TVET institutions were in a state of disrepair, staff skills were neglected, and many graduates were not well-positioned to obtain employment in a changing labor market. The GoG had been prioritizing TVET sector reform, but there remained a mismatch between the types of skills training being offered by TVET providers and the skills demanded by industry. Moreover, pursuing technical or vocational education was regarded by most potential students and their parents as the option of last resort, training one would seek only if unable to successfully pursue higher education.

In existing TVET programs, substantial gender disparities were observed in STEM fields. The wages of female TVET graduates were significantly lower than wages of male TVET graduates, attributed to the fact that many women TVET students pursued less lucrative non-STEM related fields of study. In 2010-2012, female enrollment in engineering and agriculture TVET tracks was approximately 14 percent. A tracer study of TVET graduates conducted by the National Center for Educational Quality Enhancement with support from MCC and the World Bank in 2012-2013 found that women earned less than men, with 22 percentage points more women making less than 500 Georgian lari per month. 23

The Industry-Led Skills and Workforce Development (ISWD) Project aimed to address the gaps between labor market demand for skilled workers and the supply of Georgians with technical skills relevant to the labor market through investments in TVET. The project supported industry-relevant training and education programs and increased the capacity of education providers to deliver programs in accordance with international best practices. It placed an emphasis on supporting skills development in STEM occupations, as well as in agriculture and tourism, growth industries where there were also significant gaps in the labor market. By providing the technical skills demanded by employers, the project aimed to further build the pipeline of Georgians ready to enter the workforce and build Georgia’s economy. The project consisted of two activities:

  1. Competitive Program Improvement Grants Activity
  2. Strengthening Sector Policy and Provider Practice Activity

By using a holistic approach to reforming the TVET sector—through public relations and communications to improve the perception of technical and vocational education, improved curricula and programs, technical assistance to the GoG’s reform efforts, increased involvement of the private sector, and improved pedagogy—the project contributed to improved perceptions of and increased demand for TVET in Georgia. The project, and particularly the Strengthening Sector Policy and Provider Practice Activity, was designed and implemented in close cooperation with the EUVEGE—the flagship project of the European Union for TVET and employment policy development in Georgia—and with the German Development Agency, Swiss Development Corporation, and UN Development Programme projects. These projects all coordinated their efforts to improve the quality of TVET provision through stronger partnerships with employers.

Competitive Program Improvement Grants Activity

The objective of the Competitive Program Improvement Grants (PICG) Activity was to create a competitive grants program awarding TVET programs that introduced or expanded innovative approaches to building skills for employment. MCA-Georgia ran a $12 million competitive grant mechanism to solicit and fund innovative, industry-driven proposals. Applicants were TVET providers that partnered with employers and/or international organizations. The competitive grant facility encouraged innovation and the development of new programs that could not be procured through the existing market. Grantees developed, expanded, or improved TVET programs to meet identified skills demanded by the labor market. Each grant required matching funds of at least 10-15 percent from the grantee and its partners in order to stimulate a private-public partnership model for TVET program delivery.

Under the activity, Georgian public and private TVET providers received grants to scale up or introduce newly authorized, higher-level programs in partnership with national and international companies, universities, and industry representatives. These included globally recognized firms and institutions such as CISCO, British Petroleum, Pearson, the University of Finland, the Technical University of Munich, Wageningen University and Research, Deutsche Bahn, and multiple others that contributed financial or material resources or provided technical assistance on top of MCC’s investment.

The activity awarded $12 million in grants to more than 50 new or expanded TVET degree and certificate programs at 10 public and private education institutions. All programs received accreditation from the Georgian board of certification. Shorter, more targeted certificate programs (as opposed to the 18-24-month traditional diploma programs) were developed and drew many applicants, both from private companies wanting to improve their employees’ skills and knowledge, and from individual students seeking employment. In total, these programs graduated 727 students during the compact period with skills demanded by the private sector in growth sectors including tourism, agriculture, infrastructure, and STEM fields. It generally takes less time to earn a technical qualification than to get a university degree, resulting in several outcomes: these new programs primarily enrolled Georgians who could not afford higher education given the opportunity cost of studying without pay for multiple years, or increased participation from Georgians who were already in the workforce and returning to the classroom to obtain additional technical qualifications in hopes of higher salaries/compensation.

At the project level, grantees and their private sector partners exceeded MCC’s cost-sharing requirement, contributing $5.96 million in total. This collectively represents nearly three times more than the required amount, and almost half the amount that MCC invested into the program. According to reporting from the Millennium Foundation in April 2020, the programs developed under the compact continue to enroll new students, and additional degree programs have been developed and accredited since compact closure.

This activity also aimed to significantly contribute to the goal of increasing career opportunities and pathways for girls and women in the Georgian economy. In partnership with UN Women, MCA-Georgia provided tailored gender and social inclusion training for administrators from the TVET providers that received grants, instilling more gender-aware and inclusive TVET teaching. While total enrollment in the PICG courses exceeded the compact target, only 14 percent of the participants were female. Although recruitment of women into traditionally male-dominated TVET fields proved challenging for grantees, partnerships with the U.S. Embassy in Georgia and the private sector exposed TVET providers to targeted recruitment tools in an effort to create employment opportunities specifically targeted at increasing female enrollment. These opportunities included a three-week tour of U.S. community colleges with a successful track record of recruiting and graduating women.

Strengthening Sector Policy and Provider Practice Activity

At the start of the compact, Georgia’s TVET policies were in the process of being elaborated, in collaboration with the European Union and other donors. No system was in place to track TVET best practices or disseminate them, nor was there a viable pathway to engage industries in TVET trainings or employ graduates. Engaging industries allows for TVET programs to align with labor market demands. To strengthen TVET policies and practices, technical assistance was provided to the Ministry of Education and Science’s TVET Department. The topics covered by the technical assistance were annually agreed upon with the Ministry of Education, Science, Culture, and Sport. Assistance included extensive support on industry engagement, quality assurance, and career guidance policy development.

To promote good practices and pilot meaningful engagement with industry, MCA-Georgia launched a second, smaller competitive grant facility for TVET providers to strengthen, document, and disseminate existing or recognized good practices in vocational education. MCA-Georgia awarded about $400,000 in “best practices” grants to 27 TVET providers. Though a match was not required, small grant recipients collectively contributed 18 percent of the grant totals in co-financing.

MCA-Georgia also held annual TVET conferences in 2016, 2017, and 2018 to showcase and promote TVET best practices and recognize industry leaders, with significant co-funding from private sector in the case of the second and third conferences. The conference served as a forum for dialogue and information sharing among TVET stakeholders, and the dissemination of best practices. The conferences were seen by stakeholders to contribute to improving perceptions of TVET in Georgia, and Millennium Foundation held the annual TVET conference in July 2020. Continuing these conferences in the future will depend on funding and organizational support.

Project Sustainability

To further the impact and sustainability of the project, the compact envisioned the development of partnerships between education providers and employers, in order to promote private investment in Georgian technical and vocational education. During implementation, partnerships remained a key priority for both MCC and the GoG. Private sector partners brought significant co-funding and in-kind contributions to each TVET institution that received MCC assistance. These partnerships improved the quality of TVET education in Georgia by bridging the gap between labor supply and industry demand.

Technical assistance provided to the Ministry of Education, Science, Culture, and Sport aimed to increase the project’s sustainability. The GoG developed a four-pillar reform plan for the education center in 2019. Aspects of the plan focused on tertiary education and industry engagement in TVET. The GoG also passed a TVET Law in 2018, committing to the continued improvement and development of the sector. These policy reforms will allow the public-private partnership model for TVET developed by MCC, MCA-Georgia, and the GoG to continue for years to come.

Environmental sustainability was built into the project from the start. Grant criteria were developed in advance to ensure compliance with MCC’s Environmental Guidelines. Grant recipients prepared environmental, social, health, and safety plans for all activities and identified and developed opportunities to incorporate environmental sustainability into training program curricula.

At the end of the compact, the Ministry of Education, Science, Culture, and Sports planned to continue hosting annual TVET conferences, and Millennium Foundation held a TVET conference in July 2020 with more limited participation due to the pandemic. Other international donors, such as USAID, the European Union, German Development Agency and United Kingdom Good Governance Fund, noted the success of this project and have developed their own interventions that will continue the technical assistance, TVET reform, and industry-led TVET programs. The European Union developed a new “Skills Development and Matching for Labor Market Needs” program, worth about €48 million, that builds on their EUVEGE project and MCC’s experience in TVET in Georgia and incorporates the technical assistance developed by the compact.

Economic Analysis

Original Compact Project Amount: $16 million
Total Disbursed: $15.7 million
Estimated benefits at compact closure correspond to $15.7 million of MCC project funds, $5.97 million of country contribution project funds, as well as administrative and M&E costs, which are all included in the related cost-benefit analysis.

Project Activity Estimated Economic Rate of Return over 20 years 24 Estimated beneficiaries over 20 years Estimated net benefits over 20 years
Industry-Led Skills and Workforce Development Project At the time of investment decision 21% 26,000 $22.9 million
Updated at entry into force 13% 25,000 $3.6 million
Updated after compact closure 19.1% 81,769 $36.4 million
  Competitive Program Improvement Grants Activity At the time of investment decision 22% 26,000 $24.5 million
Updated at entry into force 14% 25,000 $5.3 million
Updated after compact closure 20.9% 81,769 $39.1 million

The original ERRs were estimated in April 2013. Among others, this estimate included the assumption that a nominal 23.8 percent increase in income would accrue to graduates of technical vocational programs receiving grants. The costs of the Strengthening Sector Policy and Provider Practice Activity were included in the project-level ERR, but no benefits were assigned because of the lack of project definition and quantitative data to estimate potential outcomes. The ERR estimates were revised shortly after the compact entered into force. The reduction came from a slight increase in estimated costs, a decrease in expected beneficiaries, and accounting for variation in several key parameters including proposal amount, private sector investment contribution, and increases in wages and employment. The grantees under the Competitive Program Improvement Grants Activity were not selected until after implementation began, so the EIF ERR was calculated based on assumptions about costs and benefit streams. TVET center ERRs were later estimated and used as part of the grantee selection process.

The economic logic underpinning the closeout CBA model used to calculate the ISWD Project and PICG Activity ERRs remains the same. The PICG Activity still accounts for all benefit streams in the project-level ERR. The Strengthening Sector Policy and Provider Practice Activity appears to have been implemented well and achieved the intended outputs, thereby further supporting the short-, medium- and long-term outcomes of the ISWD investment. However, the Activity does not appear to have any specific or additional benefit streams or beneficiaries to incorporate. The ISWD Project ERR includes all costs from both activities. 25

The main benefit stream of the PICG Activity comes from improved employment outcomes and higher wages for graduates of MCC-supported training programs. Graduates are expected to be more likely to enter the labor force and be employed, particularly in their field of study and in positions that provide them with more job-related benefits. Individuals expected to obtain those benefits are considered beneficiaries. These outcomes indicate that firms would have the potential to become more productive, reduce labor costs incurred from hiring foreign workers, and potentially grow.

The CBA model structure changed substantially given a shift from hypothetical to the actual 10 TVET centers selected as grantees in 2016. Earlier ERRs were based on similar assumptions and parameter estimates across all training programs, as details were unknown at the time, while closeout ERRs consider key TVET center characteristics (e.g., international certification, level of training, economic sector’s growth potential, collaboration with employers) to inform labor market outcomes. For each center, the model follows cohorts that enroll and graduate across an assumed 20-year life of the investment. The analysis also considers a 10-year lifetime of the investment and still obtains an ERR well above 10 percent, but with a clear reduction in beneficiaries.

The ISWD Project is viewed as having been successfully implemented across various measures, with itemized data on the inputs to outputs outlined in the program logic, and inclusion of clear progress on several short-term outcomes. Although, several training programs experienced lower enrollment numbers and higher dropout rates than expected, leading to fewer beneficiaries, preliminary evidence on short-term and medium-term outcomes remains positive. Future analysis will inform key assumptions (e.g., continuation of training programs, updated enrollment numbers and dropout rates, linkages to employers, labor market outcomes) to calculate the evaluation-based ERRs, as several remain risks to long-term project success.

Evaluation Findings

The Industry-Led Skills and Workforce Development Project aimed to strengthen the linkage between market-demanded skills and the supply of Georgians with technical skills relevant to the local economy. The project’s mixed-methods performance evaluation will assess project implementation, project sustainability, policy reform, and changes to labor market outcomes of graduates from project-supported courses. Specifically, the evaluation will seek to answer the following questions:

  1. How did the implemented PICG courses compare with the original grant proposals, and what were the reasons for any deviations?
  2. Did trainees enroll in PICG courses and graduate from them at targeted levels?
  3. What were the labor market outcomes (employment and wages) for graduates from PICG courses?
  4. What are TVET providers’ perceptions of the best practices identified and disseminated by the program, to what extent have they adopted them, and what are the main barriers to doing so?
  5. To what extent have the Ministry of Education and Science (MES) and its agencies adopted the policy reforms supported by the program, and what have been the main challenges in doing so?
  6. How and to what extent has the annual TVET conference influenced providers, employers, the MES, and other TVET sector stakeholders?

Key Findings from the Interim Evaluation Report’s Evaluation Brief include:

  • New Courses Successfully Established
    • Through the Competitive Program Improvement Grants Activity, the Industry-Led Skills and Workforce Development Project established 51 new or improved TVET courses in Georgia.
    • These courses were established through close cooperation between the TVET provider and private sector partners, who provided valuable knowledge and material support.
  • Course Participation and Early Perceptions
    • Total enrollment in supported courses exceeded the compact’s target.
    • Trainees and teachers had positive first impressions of the new courses and were optimistic about trainees’ labor market prospects.
  • Other Project Activities
    • The small grants supported dissemination of best practices in ways that could be replicated by others.
    • The technical assistance component delivered support for a wide range of policy reforms, but more work will have to be done after the compact.
    • The Annual TVET Conference, which aimed to strengthen ties between TVET providers and private sector firms, was well-attended and well-received by stakeholders.
Component Status
Baseline Data collected May 2017 to June 2018. Because courses were staggered in their start dates, baseline data collection was also staggered. The baseline data analysis is in the Interim Report.
Interim Data collection was completed in December 2018. The Interim Report was published in August 2019.
Endline Data collection will be completed in mid-2021. The Endline Report is expected in late 2021.

Lessons from the evaluation:

  • The grant application process was rigorous and well-managed, resulting in high quality grant recipients. MCC should consider the importance of the grant application process and grant management capacity when designing and implementing grant facility projects in the future.
  • Although the program sought to increase female participation in STEM fields, only 14 percent of all trainees in the PICG-supported courses were female. In order to recruit more women into traditionally male-dominated fields, future programming should seek to better understand the underlying causes of low female participation and target specific interventions to address those causes.
  • Overall, initial impressions of the PICG-supported courses were positive; however, both instructors and trainees noted a lack of instructional materials in Georgia in technical subjects. Future TVET projects should strive to provide resources for written materials in the local language if not already available.
  • The sustainability of MCC’s investments under the ISWD project is still uncertain. MCC should plan from the start of project design for funding and support post-compact.

Key Output and Outcome Indicators

Outcomes: Increased availability of higher quality TVET courses and better alignment with industry needs. Graduates of the new and improved TVET courses are expected to have increased employment opportunities and earn higher salaries.

Key Performance Indicators Baseline End of Compact Target Q1-Q20 Actuals (As of July 2019) Percent Target Satisfied
Students participating in MCC-supported education activities 0 1,500 1,935 129%
Graduates from MCC-supported education activities 0 No Target 26 727 N/A
Industry co-investment in TVET provision 0 $1,800,000 $5,967,379 331.52%

Explanation of Results

The Industry-Led Skills and Workforce Development Project proved to be one of the most successful compact projects in terms of meeting output targets. The project far exceeded its target for industry co-investment, suggesting the eagerness of industry to engage on TVET.

STEM Higher Education Project

Project Summary

Feedback from more than 70 Georgian employers during compact development indicated high interest from the private sector in improved quality of higher education in Georgia. Highly trained professionals in the fields of computer science and civil, electrical, and mechanical engineering were in high demand and short supply in Georgia. This shortage deprived businesses of the ability to employ updated technology and business practices needed to innovate and increase their productivity, forcing them to hire foreigners to fill these jobs in some cases. MCC and the GoG conducted a wage survey of over 50 Georgian businesses, and found that employers would pay 44 percent more to a U.S.-educated engineer than to the “best” Georgian-educated engineer.

While access to higher education was widespread in Georgia, institutions with STEM programs were not of sufficient quality to provide the skilled graduates needed by industry. In particular, there were two factors preventing the establishment of quality STEM programs in Georgia: 1) the outdated knowledge and approach of faculty educated largely under the Soviet system; and 2) poor quality laboratory facilities and equipment, due to the substantial cost necessary to establish a modern STEM program. Women’s self-selection into non-STEM fields, with women comprising only 27 percent of engineering students in 2009, posed a major gender equity challenge in higher education.

To address this human capital binding constraint to growth, the STEM Higher Education Project was designed to support the delivery of high-quality STEM bachelor’s degree programs in Georgia by: 1) bringing a U.S. university to Georgia to partner with Georgian public universities to offer U.S. bachelor’s degree programs in the STEM academic disciplines; and 2) providing capacity enhancement for Georgian public universities with the goal of reaching international standards and acquiring international program accreditation.

San Diego State University (SDSU) was selected through a competitive process to offer U.S. accredited degrees in Tbilisi, Georgia. In the fall semester of 2015, SDSU began offering bachelor’s degrees in STEM fields, in partnership with three public Georgian universities: Tbilisi State University; Georgian Technical University; and Ilia State University.

In the long run, the STEM Higher Education Project was expected to improve the productivity of firms, reduce reliance on foreign graduates, and reduce the need for Georgian students to study abroad to obtain an adequate education. Students benefiting from the project were expected to have better employment opportunities and higher incomes. The graduates of SDSU in Georgia and accredited partner university programs would gain the skills demanded by industry and go on to earn higher wages, contributing to Georgia’s economic growth by building the pipeline of skilled higher education graduates.

In 2014, the GoG and SDSU signed a twenty-year memorandum of understanding. During the compact term, SDSU offered six U.S. Bachelor of Science programs in chemistry/biochemistry, computer science, electrical engineering, computer engineering, civil engineering, and construction engineering in Georgia. More than 600 students enrolled in these programs over the life of the compact, and the first 55 graduates received their degrees in June 2019. Students receive both an SDSU and a partner university degree upon graduation. With their U.S. degrees in hand, graduates of SDSU in Georgia are expected to have better employment opportunities and higher incomes.

The project invested in facility upgrades at the public partner universities, including:

  • Rehabilitation of nearly 5,000 square meters of science and engineering laboratories and classrooms. These laboratories are very similar to those at SDSU’s home campus in San Diego, are the only labs of their kind in the country and are providing the students enrolled at SDSU in Georgia with opportunities to advance their skills in a high-demand technical field.
  • Construction of a new four-story building on the campus of Ilia State University to house additional engineering laboratories and serve as a central hub for the three partner universities.
  • Installation of world-class equipment at all three partner universities.

The classrooms, laboratories, and equipment provide the infrastructure required for partner universities to apply for and obtain international accreditation for several degree programs.

The project also invested significantly in Georgian partner university faculty development, contributing to the objective of acquiring international accreditation or certification for partner university programs. Eighty-six Georgian faculty received training in the U.S., emphasizing how to use and deliver SDSU’s curricula with compact-provided state-of-the-art laboratory equipment. Faculty training is expected to continue through July 2023.

The STEM Higher Education Project was successful at recruiting female and socially vulnerable students into the SDSU-G STEM majors. Targeted recruitment for this program resulted in a student body that was 36 percent female, an impressive achievement even by American STEM higher education standards. In 2018, only 26 percent of students enrolled in STEM degree programs at SDSU’s home campus in California were female.

SDSU in Georgia worked to empower female students in various ways. They created a Women’s Empowerment Club, which provides peer mentoring and support to female STEM students through mentorship by female faculty. Through the compact, SDSU in Georgia also created a career development center and held internship and career fairs to introduce students to employers in STEM fields. These events prepared female students in particular for their eventual job search process while challenging stereotypes by introducing potential employers to qualified female STEM students. The career development center also provided access to internship and other professional development opportunities, which was particularly valuable for students from the regions outside of Tbilisi who may not have had access to this information otherwise.

The STEM Higher Education Project was also successful in recruiting those students who would not otherwise have been able to afford a U.S. university degree. Generally, students interested in obtaining an American education and with the ability to pay still went to U.S.-based universities rather than enroll at SDSU in Georgia. As a result, the highly qualified students who were interested in enrolling at SDSU in Georgia largely came from families that lacked the means to afford the $7,500 annual tuition. Eighty-five percent of all Georgian students enrolled at SDSU-Georgia during the compact term came from families whose mean income was less than 40,000 Georgian lari, or under $17,000, and 30 percent of students came from outside Tbilisi. 27 The project was only able to successfully enroll these students because of significant financial aid packages made available through the GoG and private sector contributions, provided through SDSU in Georgia’s public-private partnership fund. Given this student body makeup, the upward mobility impacts of the project on graduates and their families are potentially quite large.

During compact development, MCC, the GoG, and SDSU envisioned that this partnership would continue well beyond the life of the compact, with SDSU continuing to offer U.S. degrees to Georgian students for at least 20 years from the start of program delivery. That vision, however, was premised on certain financial assumptions that did not prove accurate. In particular, the number of students enrolling and the ability of these students and their families to pay full tuition was significantly lower than estimated, and budget gaps ensued due to lower revenues. SDSU, MCC, and the GoG worked collaboratively over the life of the compact to successfully cover these budget gaps, including through: 1) GoG co-financing for student scholarships, student support services, and research grants; 2) compact reallocations (described in the Compact Changes section of this report); and 3) the mobilization of more than $3.5 million in private sector funds for student scholarships through a public private partnership fund.

The project envisioned that SDSU would initially deliver U.S. degrees, but that they would later transition to a support role as Georgian universities acquired and maintained international accreditation and certification, allowing them to eventually deliver internationally recognized degrees on their own. While this transition was originally planned to occur after twenty years, priorities of both SDSU and the GoG shifted once the shortcomings of the financial assumptions came to light. Given the higher-than-expected financial risk of the partnership, SDSU and the GoG agreed to accelerate the transition process, with SDSU delivering degrees directly to five cohorts of students (through 2023), after which partner university programs are expected to have achieved accreditation and to assume responsibility for direct degree delivery.

Georgian universities are delivering the following U.S. degree program: 28

  • Tbilisi State University: Chemistry (Biochemistry Focus), Computer Engineering, Electrical Engineering, and Computer Science
  • Ilia State University: Computer Engineering and Electrical Engineering
  • Georgian Technical University: Computer Engineering, Electrical Engineering, Civil Engineering, and Construction Engineering

Project Sustainability

International accreditation of partner university degree programs is critical to project sustainability, and SDSU considered accreditation equivalent to having a U.S. degree in the global labor market. As part of the project, SDSU assisted the three partner universities in meeting the requirements for the Accreditation Board for Engineering and Technology (ABET) and American Chemical Society (ACS), and MCA-Georgia contracted the ABET Foundation to deliver additional technical assistance to partner university accreditation efforts. Two programs at TSU received ABET accreditation retroactive to 2017 and 2018. Accreditation offers benefits similar to those guaranteed by a U.S. degree by providing external validation of program quality and ensuring programs incorporate industry needs, thereby enhancing employability of graduates and driving quality enhancement across the university. The project has placed significant emphasis on the partner universities developing their own ABET/ACS accredited programs. Nonetheless, it will take sustained government support and coordination with the partner universities to continue the program.

With internationally accredited programs, the project aimed to allow the partner universities to continue to produce a pipeline of world-class engineers and scientists and grow the Georgian economy long after the compact ended. Each of the project’s investments—from building state-of-the-art infrastructure, training faculty, and introducing continuous improvement in program delivery—supports this sustainability objective. These internationally accredited programs, which are expected to require students to pay somewhat higher tuition fees than the average Georgian university tuition, may also be an important revenue stream for sustaining program quality.

During the compact term, SDSU’s three partner universities made progress towards receiving international accreditation, with the first two-degree programs from Tbilisi State University (Computer Science and Electrical Engineering) receiving ABET accreditation in October 2020. These programs are the first STEM programs in Georgia and the larger region to receive this global stamp of quality. Ilia State University and Georgian Technical University are on track to obtain accreditation for several degree programs in 2022 and 2023. Tbilisi State University’s chemistry/biochemistry program is on track to receive ACS certification by 2021.

To ensure the sustainability of the project’s infrastructure and equipment investments, SDSU signed partnership agreements with each of the three partner universities, which include plans for facilities-sharing and operations and maintenance through 2023.

Since the compact ended, SDSU has maintained a presence in Georgia and continues to assist all three partner universities in obtaining internationally recognized accreditation and certification. The successor entity to MCA-Georgia will also continue to assist SDSU and partner universities with the international accreditation process. SDSU’s final cohort of U.S. degree students will graduate in spring 2023. MCC, SDSU, and the GoG are exploring opportunities for a longer-term SDSU engagement in Georgia, including potentially through summer programs, executive education, continued accreditation support, and/or other collaborative opportunities.

Economic Analysis

Original Compact Project Amount: $30 million
Total Disbursed: $36.1 million

Estimated benefits at compact closure corresponds to $36.1 million of MCC project funds, $8.4 million of country contribution project funds, and administrative and M&E costs, as included in the related cost-benefit analysis.

Estimated Economic Rate of Return over 20 years Estimated beneficiaries over 20 years 29 Estimated net benefits over 20 years
At the time of investment decision 10% 31,000 $3.5 million
Updated at entry into force 11% 47,000 $19.8 million
Updated at compact closure 9.0% 16,235 $-9 million

The original ERRs were estimated for an investment decision by MCC on April 3, 2013, and revised after the compact entered into force. The updated ERR for the STEM Higher Education Project increased from 10 to 11 percent to reflect several assumptions including a combined annual student intake of 495 in the first year of the program, rising to 610 by year five, and an average annual operating cost intended to be covered by tuition of $7,500 per student. The economic model also assumed a 44 percent wage differential attributed to a Georgian student who graduates with a U.S. degree in a STEM field, compared to a Georgian student who graduates with a non-U.S. degree from a program in the same STEM field that is not internationally accredited (for students who earn an ABET-accredited degree from a Georgian partner university, the assumed wage differential is 22 percent). This wage differential is the key benefit of the CBA model and was based on a wage survey of more than 50 Georgian companies in fall 2012.

The economic logic underlying the STEM Higher Education Project CBA model remained consistent in calculating the project-level closeout ERR. The major change in the closeout CBA model—a reduction of the ERR from 11 percent at EIF to 9 percent—is due to lower-than-expected enrollment, particularly for U.S. degree programs, that drastically increased per-student operating costs and reduced the number of project beneficiaries. Whereas the EIF CBA model assumed an initial cohort size of 495 students, which would increase to 610 by year five, total first-year enrollment in U.S. degree programs (for five cohorts) was only 780 students (from a minimum of 86 students enrolled across all programs in the first cohort to a maximum of 230 students in the fourth cohort in the final year of the compact). However, the decline in the closeout ERR due to low U.S. degree program enrollment is mitigated by an accelerated transition of SDSU to a support role for Georgian partner universities. The lower ERR at compact closure also incorporates the probability of employment in the CBA model and beneficiary count, which reflects a change from previous analysis.

A secondary benefit stream of the STEM Higher Education Project is savings to the Georgian economy from reduced imports of highly educated and more expensive expatriate STEM professionals. Due to skills shortages in Georgia, some high-skill positions are currently occupied by expatriate workers. Due to low enrollment in U.S. degree programs, however, these benefits contributed only 4 percent to the closeout ERR.

While small due to low enrollment in U.S. degree programs, additional benefit streams include (i) increased “productivity spillovers,” where having more well-educated STEM professionals in the workplace increases earnings and wages of other workers; and (ii) cost savings for students who, without access to the U.S. degree programs offered in Georgia, would have pursued undergraduate STEM degrees at U.S. or European universities. Because of the large increase in per-student operating costs for U.S. degree programs, the “cost savings” benefit stream negatively affects the compact closure ERR.

Evaluation Findings

The independent evaluation utilizes a mixed-methods approach that involves a quantitative analysis of surveys along with document and literature reviews and qualitative analyses of interviews, focus groups and case studies. Such mixed-methods evaluation will allow us to examine a variety of aspects of the STEM Higher Education Project including partnerships, design and implementation, outcomes, barriers to and facilitators of high-quality implementation, cost, and sustainability.

The study utilizes student and alumni surveys as a quantitative approach to examine such project outcomes as student perceptions of the SDSU-Georgia degree programs, employment during university studies and after graduation, and wages after their degree is completed. The evaluation will examine how the outcomes of interest compare between SDSU-Georgia students and other students who attended and completed degrees in similar disciplines at different university programs.

Evaluation Questions:

  1. How was the partnership established and carried out? How did it change over time?
  2. Were the activities implemented through the project aligned with the program design, as documented in the logic model?
  3. How do SDSU and comparison group students view their programs?
    To what extent are the project activities sustainable?

Key Findings from the interim report’s Evaluation Brief include:

  • University Partnerships
    • San Diego State University (SDSU) led the development of partnerships with three Georgian public universities: Georgian Technical University, Ilia State, and Tbilisi State.
  • Project Implementation
    • Georgians found tuition too expensive but MCA-Georgia, MCC, and GoG collaborated to generate scholarship funds to increase demand for the program.
  • Student Perceptions
    • Surveys show that students in the program were very satisfied with new facilities and equipment but less so with their experience with faculty and courses offered.
  • Sustainability
    • Partners have taken steps to sustain STEM programs and relationships after the compact ended in July 2019 through program accreditation and other activities.
    • Readiness for international program accreditation and certification varies across the partner universities. Once accredited or certified, the tracks at Georgian Technical University, Ilia State University, and Tbilisi State University will be the first STEM programs in Georgia and the larger region to receive this global stamp of quality.
Component Status
Interim Data collection completed in 2019. Interim Report published in January 2021.
End-line Data collection to be completed mid-2023. Report expected late 2023.

Lessons from the evaluation:

  • MCC should work to obtain a better understanding of students’ ability and willingness to pay for higher education. MCC should have a clear picture of what student expectations are for higher education, and, if a partner government is proposing something new/innovative that neither they nor MCC have undertaken in the past, make sure there is a market for it.
  • Sharing potential student enrollment numbers from MCC’s initial Cost-Benefit Analysis as part of the initial Request for Proposals gave bidders the incentive to overestimate their ability to recruit and retain students.
  • New education programs require ramp up periods for outreach and to build their image or “brand.” MCC should plan for an initial subsidy period, with mechanisms in place (for example, marketing of success stories and job placements) that reduce the risk that the subsidy phase-out period is insufficient to generate demand at cost. In addition, MCC should have in place proper mechanisms that allows the program to improve based on feedback from students.
  • Creating a partnership between a U.S.-based university and public Georgian universities was more complicated than any of the parties anticipated. New partnerships should be structured to be flexible to allow for adaptation by all stakeholders, with certain “non-negotiables” clearly identified from the outset, e.g., there was a push in the third year of the compact to recruit international students that took away valuable time and human resources away from other program activities, as the GoG sought to make the SDSU program in Georgia part of a Georgian “hub” to attract international students.
  • MCC should agree on site identification with partner governments prior to committing compact resources to infrastructure rehabilitation. The partner universities identified about 20 locations across Tbilisi for science labs and classrooms, but many were located far apart from each other and it was hard for students to travel between sites and arrive on time for class. The dispersed locations of renovated spaces across three partner universities also made MCC’s infrastructure and equipment investments less visible to the public.
  • Getting responses from comparison group students at Georgian partner universities was initially very difficult. Survey planning should consider the lack of incentives for respondents to answer surveys when they are not benefiting from the compact-funded program. Especially when surveying groups that were not targeted for the program. It is increasingly important to plan adequate time and effort for building relationships and troubleshooting problematic respondent groups.

Key Output and Outcome Indicators

Outcome: Increased quality of tertiary education in STEM fields in Georgia. Improved employment opportunities and higher income for graduates of the improved programs.

Key Performance Indicators Baseline End of Compact Target Q1-Q20 Actuals (as of July 2019) Percent Target Satisfied
Students participating in MCC-supported education activities 0 2,008 642 32%
ABET Committee Formed N/A No Target April 27, 2016 N/A
Retention Rate of Students in MCC-funded Bachelor’s programs 0 80% 77.2% 96.5%

Explanation of Results

The STEM Higher Education Project did not meet its original target for student participation. MCC, SDSU, and MCA-Georgia determined early on in implementation that the willingness or ability of Georgian students and their families to pay full tuition at SDSU in Georgia was significantly lower than originally estimated by all parties involved, resulting in a need for additional scholarship financing and lower than expected student enrollment. While enrollment in SDSU-Georgia increased from cohorts one through four, the amount of family tuition pay has been lower than expected, and in the first two cohorts more than 75 percent of tuition was paid through scholarships provided by MCC/MCA-Georgia or the GoG. This gap led to an annual budget deficit that the GoG was able to cover but had to be re-visited each year, including a contribution of $11.2 million to sustain the program through at least five cohorts (through June 2023). This deficit ultimately led to a cap on student enrollment for cohorts four and five.

SDSU has projected eight percent student attrition annually through graduation of the final class of U.S. degree students in spring 2023. Out of the seniors that graduated in May 2019 after four years of SDSU education in Georgia, 20 percent graduated summa cum laude. After the unexpectedly low enrollment of the first cohort, SDSU and MCA intensified recruitment efforts, and at the same time expanded the pool of candidates by offering a “bridge” or preparatory program, whereby students could enroll in the SDSU degree program, take required additional math and English courses in the fall semester, and then enroll in the full program in the freshman spring semester. Although this increased the number of enrolled students, it also increased the level of attrition, as many students who had been accepted into the “bridge” program were not able to keep up in subsequent years. Others did not, or will not, complete the program in four years, partially due to the Georgian regulations which allow students to take a year or two off without losing their student status, and then return to their same Georgian university to complete their degree. Besides attrition, this also affects the overall graduation rate.

Compact Changes

Over the course of the compact term, no major modifications to the Georgia II Compact were introduced. The compact is expected to largely achieve the original program and project objectives, as measured by MCC-funded independent evaluations. Changes in the scope of several activities described below were agreed between MCC and the GoG.

Number of Schools Rehabilitated: Under the Improved Learning Environment Infrastructure Activity, the final number of schools MCA-Georgia rehabilitated was 91. As the compact noted that “up to 130 schools” would be rehabilitated, there was an expectation from the GoG that MCC would be able to fund rehabilitation of 130 schools, which turned out not to be the case. In order to maximize the number of schools and beneficiaries of the activity, the GoG financed all “ancillary works” outside of the school building, including paving, fencing, utility connections and upgrades, and water discharge routes. When schools were closed for rehabilitation, the government funded transportation for students to an alternative schooling location until their school reopened. This, as well as the rebalancing of the school selection formula in 2014, allowed MCC and MCA-Georgia to maintain the target beneficiary number for this activity.

In addition, in 2017 MCA-Georgia reallocated $600,000 from the Monitoring & Evaluation Budget to the activity’s budget to ensure that MCA-Georgia had sufficient funds to rehabilitate schools randomly assigned to the treatment group. The M&E team agreed to this reallocation to help preserve the fidelity of the evaluation design. It was critical to rehabilitate as many of the treatment schools as possible to be able to detect an effect on the outcomes of interest.

Combination of Capacity-Building and Degree Delivery Partnership in the STEM Higher Education Project: At the time of compact signature, a partner higher education institution for the STEM Higher Education Project had not yet been identified. Both MCC and the GoG envisioned that the project’s two objectives of degree delivery and capacity building for accreditation could be treated separately. However, once the compact officially began and the SDSU partnership was in place, SDSU proposed to deliver both objectives. Ultimately, given financial constraints, SDSU increased its focus on capacity building for partner universities in the latter years of the compact as the transition timeline was accelerated.

Reallocations to STEM Higher Education Project: In March 2018, $3.2 million in compact funding was reallocated to the STEM Higher Education Project in order to finance SDSU-Georgia in fall 2018. The compact funds were reallocated from the Training Educators for Excellence Activity, Education Assessment Support Activity, Industry-Led Skills and Workforce Development Project, Monitoring & Evaluation budget, and MCA-Georgia’s program administration budget due to cost savings. This reallocation was prompted by several developments in the compact. The willingness or ability of Georgian students and their families to pay full tuition at SDSU in Georgia was significantly lower than originally estimated by all parties involved, resulting in financial constraints. Both MCC and the GoG agreed that admitting additional cohorts of students was critical to sustaining the education quality gains made during the compact term under the STEM Higher Education Project, in part because additional U.S. degree cohorts would increase the likelihood of partner university international accreditation and certification.

On top of the compact reallocation, the GoG committed additional funding in a revision of the twenty-year memorandum of understanding with SDSU in March 2018, and by February 2019 had transferred the full amount upfront, as requested by SDSU as a condition to the enrollment of a fifth cohort in fall 2019. The GoG’s financial contribution to the STEM Higher Education Project, came primarily in the form of student scholarships and infrastructure support, exceeded $8 million during the compact plus an additional $10.5 million for the post-compact period.

Change in Scope of Education Assessment Support Activity: In September 2018, 80 percent of the National Assessment and Examination Center’s technical research division resigned following the dismissal of the center’s leadership. The skillset required for these positions was difficult to find in Georgia and needed to be developed over a period of intensive capacity building. As a result, the Center could complete the second round of only one national assessment during the compact period. Although the compact originally envisioned that the National Assessment and Examination Center would complete two rounds each of five different national assessments during the compact term, the GoG agreed to continue investing in national assessments after the closure of the compact. MCC agreed to re-program $165,000 of compact funds that were originally intended for the administration of these national assessments to support capacity building for newly hired staff.

Operations and Maintenance: The original, stated purpose of the school operations and maintenance (O&M) sub-activity was to support the establishment of an O&M program for all Georgian public schools to ensure the sustainability of MCC’s investment in rehabilitated schools in particular and all public schools in general. A distinctive element of the school O&M sub-activity was a School O&M Incentive Fund of up to $2.5 million to match each dollar of the GoG’s contribution to school O&M activities.

Beginning in January 2018, MCC and MCA-Georgia increasingly focused the school O&M sub-activity on the 91 schools rehabilitated with compact funding. This was done because decentralization reforms led to a lack of clarity as to who was responsible for each aspect of school O&M. In order to ensure that MCC-funded school infrastructure was properly maintained, MCC worked with the GoG to ensure that O&M financing was secured to operate and maintain MCC-funded schools.

Coordination and Partnerships

The effects of the compact were amplified through successful partnerships between MCC and the GoG, and also the U.S. Embassy, Georgian private sector, and a number of domestic and international organizations. As important context for the U.S.-Georgia partnership specifically, at the time of compact signing, MCC’s $140 million Georgia II Compact was the single largest bilateral economic growth program in Georgia and the largest donor investment ever made in Georgia’s education sector. Its scale and nationwide reach have helped to further strengthen the relationship of the United States with Georgia and advance the shared goal of contributing to greater economic prosperity and regional stability.

Over the course of the compact, MCA-Georgia and MCC partnered with the U.S. Embassy in Tbilisi, the GoG, and private sector partners to run the Millennium Innovation Awards, an annual national competition to support innovation in key STEM fields among youth ages 13 to 18. The Millennium Innovation Awards continues to grow year-after-year. Since MCA-Georgia launched this competition in 2014, 900 students from nine regions of Georgia have participated. The U.S Embassy in Georgia continues supporting this partnership after the compact.

Throughout the Georgia II Compact, MCC worked with the U.S. Embassy. In 2016, to mark the 25th anniversary of U.S.-Georgia relations, the U.S. Embassy funded five full tuition scholarships (valued at $30,000 each) for ethnic minority students accepted to SDSU-Georgia. The Embassy joined MCC in bringing the public-private partnership “Women in Science (WiSci)” to Georgia. In 2018, the Embassy joined MCC, alongside Google, Intel, the State Department, and the UN Foundation’s Girl Up, in co-sponsoring a two-week international girls STEM camp. Also, in 2019, the Embassy organized and funded a three-week U.S. community college tour for 12 Competitive Program Improvement Grants Activity recipients aimed at increasing female enrollment in newly introduced TVET programs. In 2019, the Embassy joined MCC and the Tiger Woods Foundation to send three Georgian STEM teachers to the U.S. to participate in a weeklong “STEM Studio.”

Finally, over each of the five years of the compact, the Embassy, through the State Department’s International Visitor Leadership Program, partnered with MCC to send multiple compact stakeholders and beneficiaries on relevant professional study tours to help maximize compact results and ensure the sustainability of MCC investments. For example, through the program, MCC and MCA coordinated to have four Georgian participants from MCA and the Educational and Scientific Infrastructure Development Agency visit various organizations and schools in the U.S. over a three-week period. Five Georgian participants were also sent on a second three-week International Visitors Leadership Program on financially sustainable higher education management. These visits to the U.S. provided the participants with tools to support education management and reform efforts already underway in Georgia.

Private sector partnerships aimed to improve the employability of graduates of SDSU’s programs in Georgia. SDSU and MCA-Georgia engaged 17 of Georgia’s largest public and private sector employers through an industry advisory board, held regular internship and job fairs, and expanded outreach to industry through a Career Development Center launched in 2018. In November 2016, a public-private partnership fund was launched under the STEMHigher Education Project to attract public and private sector support for Georgian student scholarships. Over $3.5 million in private sector funds were leveraged from approximately 15 different institutions and individual donors. In fall of 2020, a student loan facility that is funding Georgian students accessing vocational and higher education programs was launched by the Millennium Foundation. Through these efforts, the Millennium Foundation is enabling students to gain access to these educational opportunities that they otherwise would not be able to access.

MCA-Georgia also entered into a wealth of partnerships to ensure that compact investments benefited girls and women. The partnership with UN Women allowed MCA-Georgia to provide world-class gender sensitivity trainings to high school teachers and principals, as well as TVET administrators. MCA-Georgia signed a memorandum of understanding with the Federation of Business Women, “Women for Tomorrow,” which will continue with the “Business for Gender Equality” competition and award process designed to celebrate Georgian businesses for women’s empowerment efforts. These efforts include meaningfully integrating women into the workplace, mentoring, and networking. These partnerships are integral to ensuring the sustainability of the compact’s gender and social inclusion achievements.

Policy and Institutional Reforms

Reforms to improve the GoG’s framework for school operations and maintenance (O&M) were built into the compact design and included both conditions precedent tied to disbursements and an incentive fund dedicated to school O&M. The compact included several prerequisites to school rehabilitation funding that required the GoG to develop and implement a school operations and maintenance plan. After MCC clarified its expectations of what “substantial compliance” with the plan looked like, the GoG met all operations and maintenance-related requirements on time. Although a combination of conditional disbursements and financial incentives focused energy and helped ensure near-term results during the compact term, it is not yet evident if institutional reforms related to school O&M will remain in place.

The compact also planned policy reforms to better link TVET providers and industry, primarily through providing the GoG with technical assistance via the Industry-Led Skills and Workforce Development Project. During compact development, MCC worked with the Ministry of Education and Science to identify how to best provide technical assistance that would help cultivate policies that improved engagement with industry and provided economic returns. As part of the Strengthening Sector Policy and Provider Practice Activity, the GoG, MCC, and MCA-Georgia ensured that the technical assistance provided through MCC built on that provided by other donors. Close collaboration with other donors, including the European Union in particular, ensured smooth coordination on assistance with TVET policy reform.

Compact Component(s) Major CP or Policy Reform Rating: Met on Time/Deferred/Not Met Narrative
Operations and Maintenance Sub-Activity Prior to the initial Disbursement of Program Funding for the initial group of schools to be rehabilitated under the Improved Learning Environment Infrastructure Activity: ESIDA must have hired or designated at least one permanent, dedicated and technically qualified staff member, satisfactory to MCC, to develop and implement the forthcoming School Operations and Maintenance (O&M) Plan Met on time In August 2016 ESIDA hired a staff member dedicated to development and implementation of the school O&M plan.
Operations and Maintenance Sub-Activity Prior to the initial Disbursement of Program Funding for the initial group of schools to be rehabilitated under the Improved Learning Environment Infrastructure Activity: MCA-Georgia must have hired a consultant to coordinate development of the eventual School O&M Plan Met on time In August 2016 MCA-Georgia hired a consultant to coordinate the development of the O&M plan.
Operations and Maintenance Sub-Activity Prior to the initial Disbursement of Program Funding for the second group of schools to be rehabilitated under the Improved Learning Environment Infrastructure Activity, and the initial Disbursement of Program Funding for the O&M incentive fund: MCA-Georgia must have developed the School O&M Plan satisfactory to MCC, that is ready for adoption by the Government Met on time MCC approved the School O&M Plan in March 2017.
Operations and Maintenance Sub-Activity Prior to the initial Disbursement of Program Funding for the second group of schools to be rehabilitated under the Improved Learning Environment Infrastructure Activity, and the initial Disbursement of Program Funding for the O&M incentive fund: ESIDA must have maintained a staff member dedicated to development and implementation of the School O&M Plan, and will have identified the staff team responsible for implementation of the School O&M Plan Met on time In August 2016 and January 2017 ESIDA hired two staff members dedicated to development and implementation of the school O&M plan.
Operations and Maintenance Sub-Activity Prior to the initial Disbursement of Program Funding for the second group of schools to be rehabilitated under the Improved Learning Environment Infrastructure Activity, and the initial Disbursement of Program Funding for the O&M incentive fund: the Government must be in substantial compliance with an Implementation Letter to be issued by MCC regarding the framework for the O&M incentive fund (the “O&M Implementation Letter”) Met on time MCC issued an implementation letter in October 2016. The Government satisfied the requirements of the letter in March 2017.
Operations and Maintenance Sub-Activity Prior to the initial Disbursement of Program Funding for the third and any subsequent tranches of schools to be rehabilitated under the Improved Learning Environment Infrastructure Activity, and for any subsequent Disbursements of Program Funding for the O&M incentive fund: the Government must be in substantial compliance with the School O&M Plan Met on time, after implementation letters were issued to clarify the meaning of “substantial compliance” The Government satisfied the requirements of the letter in March 2018.
Operations and Maintenance Sub-Activity Prior to the initial Disbursement of Program Funding for the third and any subsequent groups of schools to be rehabilitated under the Improved Learning Environment Infrastructure Activity, and for any subsequent Disbursements of Program Funding for the O&M incentive fund: ESIDA must have maintained adequate staffing levels to implement the School O&M Plan Met on time The Government satisfied this requirement in March 2018.
Operations and Maintenance Sub-Activity Prior to the initial Disbursement of Program Funding for the third and any subsequent groups of schools to be rehabilitated under the Improved Learning Environment Infrastructure Activity, and for any subsequent Disbursements of Program Funding for the O&M incentive fund: the Government must be in substantial compliance with the O&M Implementation Letter Met on time The Government satisfied this requirement in March 2018.

Beyond the Compact

The Georgia II Compact led to many additional programs beyond the original scope of investments. Significantly, the compact raised the profile and importance of developing the workforce throughout the country. Like the compact itself, the GoG’s strategy aims to grow the economy by building up Georgia’s workforce, starting from the beginning of schooling to placement in the job market. During the final years of the compact term, the GoG announced plans to more than triple education spending as a percentage of GDP, though Georgia, like many countries, was forced to modify spending due to the impacts of the covid pandemic on economic growth.

The U.S. Embassy in Georgia has amplified the reach of the compact as well. The Embassy funded travel to the U.S. through the International Visitor Leadership Programs for Ministry of Education and Science representatives and TVET Center Directors to learn about TVET marketing, financing, and management. The Embassy also funded the travel of Competitive Program Improvement Grant Activity grantees to attend a study tour of community colleges that successfully recruit women to TVET programs in similar fields in the United States. In both 2016 and 2017, the Embassy funded study tours and professional exchanges in the United States through the State Department International Visitor Leadership Program for members of the MCA and implementing entities focused on school operations and maintenance. Finally, to increase compact sustainability, in April 2019, the U.S. Embassy organized a visit for SDSU Georgian partner university and Ministry representatives responsible for post-compact management of higher education program administration through the STEM Higher Education Project to learn about financing higher education.

In August 2018, MCC partnered with the United Nations Foundation’s Girl Up program, the State Department’s Office of Global Partnerships, Intel, Google, Microsoft, and the American Society for Microbiology to bring a Women in Science “WiSci” Girls’ Camp to Georgia. The camp is a public-private partnership that was designed to encourage adolescent girls to pursue further education and potential careers in STEM. WiSci Georgia expanded the reach of MCC in Georgia. Fifty-five Georgian girls, ages 15-18, and seven female SDSU Georgian students who served as camp counselors benefited from MCC’s participation in the camp. 30 This highly successful partnership was institutionalized through a memorandum of understanding between MCC and the State Department to work together on future camps involving MCC partner countries. In 2019, MCC expanded this partnership, holding a WiSci camp in Kosovo, and is planning WiSci camp in Morocco in 2021. The United Nations Foundation’s Girl Up also held another WiSci camp in Estonia, which Georgian campers attended with support from the U.S. Embassy in Georgia, expanding the number of female students benefiting from this partnership in Georgia post-compact.

During the launch of the United States Women’s Global Development and Prosperity Initiative, Nino Zambakhidze, a beneficiary of MCC’s first Georgia compact and a partner of the second, shared her story of how MCC’s investments in Georgia transformed her life and opened doors for other Georgian women like her. Under the Industry-Led Skills and Workforce Development Project, Nino’s organization, the Georgian Farmers’ Association, partnered with the Agricultural University to pilot applied coursework in agribusiness, including how to deliver targeted support for women leaders in the field. Nino’s story was highlighted by the White House during the launch of the Women’s Global Development and Prosperity Initiative as a prime example of how U.S. Government investments in economically empowering women can transform lives and countries.

Lessons from the Compact

The lessons below come from MCC’s experience developing, implementing, and closing the Georgia Education Compact. Several lessons from the Compact overlap and complement lessons derived from the independent evaluations of each project.

Overarching Lessons

Partnerships outside of the compact can be structured creatively to enhance and contribute to compact goals. The compact’s impact on the education sector and the popularity of STEM education was amplified by strong partnerships. The WiSci Camp in 2018 and Millennium Innovation Awards initiated in 2014 have raised the profile of STEM fields and professions, providing many students typically underrepresented in STEM fields in Georgia (girls, women, socially vulnerable populations in the regions outside of Tbilisi) with a platform to obtain and receive public, widespread positive recognition for STEM achievements. These two activities, running alongside the compact, have amplified the compact’s impact on women’s participation in STEM fields, thus supporting women’s economic empowerment.

When implementing through a government entity, be realistic about what institutional change is achievable during the compact term and adapt the scope and support to existing capacity. Implementing entities within partner governments have varying levels of capacity. MCC’s approach to cooperation with implementing entities should take into account both what is feasible given the level of capacity and what is necessary in order to make the project sustainable. In the case of school rehabilitation, MCC agreed early in the compact to assign substantial and critical responsibility for utility infrastructure as well as operations and maintenance to the Educational and Scientific Infrastructure Development Agency (ESIDA) despite ESIDA’s limited prior experience in this area. While this decision introduced delays in school re-openings and increased the risk that utility connections would not be completed before the end of the compact, it was important that MCC worked through ESIDA in order to ensure the country ownership and capacity building necessary for sustainability of rehabilitated schools. ESIDA was able to successfully complete O&M-related tasks, though a new GoG decentralization plan announced at the end of the compact increased ambiguity and introduced other institutions into school O&M management.

Likewise, the decision was made early on to implement the teacher training component through the Teachers’ Professional Development Center and the Education Assessment Support Activity through the National Assessment and Examination Center. Both had implementation challenges at different times in the compact due to insufficient staffing. The Teachers’ Professional Development Center struggled at the beginning, until MCA-Georgia and MCC asked them to change key personnel. After that, the program ran successfully and exceeded all expectations in the final two-and-a-half years of the compact. Conversely, the Education Assessment Support Activity was run well until the final year of the compact, when changes at the minister level resulted in changes in top management at the National Assessment and Examination Center, which in turn resulted in changes at lower levels. As a result, the National Assessment and Examination Center was unable to complete all planned assessments in the final year and had to re-focus its efforts and re-adjust its budget to acquire technical assistance and train new staff.

Implementation of smaller dollar value compacts still requires significant resources comparable to that of larger programs. MCC assumed that the relatively smaller dollar amount of the compact implied that a smaller MCA could be hired. However, this assumption proved incorrect. Based on original program administration budget allocations, the MCA-Georgia infrastructure team was envisioned to include only two staff to oversee the implementation of the Improved Learning Environment Infrastructure Activity. By the end of the compact, the heavy workload led MCC and MCA-Georgia increase the size of the infrastructure team to six people. Adequately staffing the team earlier may have avoided implementation delays. MCC’s requirements during a tight implementation timeline are demanding and effectively the same for both small and large compacts and projects.

Project-Level Lessons

New tertiary education programs like SDSU-Georgia and degree and certificate offerings require significant ramp-up periods for outreach and to build brand recognition. As demonstrated by lower-than-expected initial enrollment numbers at SDSU and in certain TVET programs, MCA-Georgia, MCC, and SDSU were overly optimistic about student enrollment in year one of operations in first-of-its-kind programs. However, once the “brand” became better known, year-on-year enrollment increased at SDSU. MCC should more conservatively estimate and re-visit assumptions on enrollment numbers that factor into financial commitments and cost-benefit analyses.

Institutionalizing operations and maintenance through building government capacity and reforming policy requires significant resources, local champions, government commitment, and adequate implementing entity capacity. To implement a nation-wide school operations and maintenance reform, MCC should include more upfront technical assistance earlier in order to cultivate greater ownership and an environment ready for a reform activities. In the case of Georgia, a $2.5 million school O&M Incentive Fund matched by $2.5 million in Georgian government funds, was an insufficient financial resource to catalyze institutional changes necessary for a nation-wide reform.

Identifying sectors with a significant demand for specific skill sets is a critical prerequisite for the success of a TVET grant program. In Georgia, this was accomplished through publishing a “Call for Ideas”. The “Call for Ideas” process included strategic engagement with both the GoG and the private sector. This partnership resulted in greater impact and results that should be replicated in other partner countries where grants are used to introduce new and expanded TVET programs.

Requiring industry engagement can create real collaboration between educators and employers when implementing grants. MCC requirements on partnering with industry catalyzed symbiotic relationships between industry and TVET providers as part of the TVET grant selection process. These strong ties between TVET providers and the private sector have directly expanded industry engagement beyond compact-funded activities. This will help sustain the objective of better aligning provision of TVET courses with labor market needs. 31

Footnotes
  • 1. The population of Georgia is approximately 4.9 million according to the CIA World Factbook’s July 2021 estimate.
  • 2. Samtskhe-Javakheti Roads Evaluation, 2007, https://www.mcc.gov/resources/doc/evalbrief-011013-geo-sj-roads
  • 3. Agribusiness Development Evaluation, 2008, https://www.mcc.gov/resources/doc/evalbrief-010914-geo-agriculture
  • 4. World Bank, “Fighting Corruption in Public Services: Chronicling Georgia’s Reforms,” 2012, (retrieved on March 4, 2020)
  • 5. Georgia II Constraints Analysis, July 2011.
  • 6. TIMSS 2007.
  • 7. PISA 2009.
  • 8. 2011 Global Competitiveness Report.
  • 9. The compact closure ERRs are near final but may change slightly during reviews required as part of the publication process.
  • 10. The compact closure beneficiary estimates are preliminary. Updated, final estimates will be published in the CBA model spreadsheet when peer and country team reviews are completed.
  • 11. (1) Earthman, Glen I. (2002), School facility conditions and student academic achievement, Los Angeles CA: UCLA’s Institute for Democracy, Education and Access. (2) Earthman, Glen I. (2004), Prioritization of 31 criteria for school building adequacy, ACLU Maryland
  • 12. Chetty, Raj, John N. Friedman, and Johnah E. Rockoff. “The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood.” National Bureau of Economic Research Working Paper Series, working paper no. 17699. Cambridge, MA: National Bureau of Economic Research, December 2011.
  • 13. Hanushek, Eric. “The Economic Value of Higher Teacher Quality.” National Center for the Analysis of Longitudinal Data in Education Research Working Paper 56. Washington, DC: Urban Institute, December 2010.
  • 14. The closeout estimate of the beneficiaries is preliminary at the time of this report.
  • 15. For education projects MCC’s economic analysis typically considers 20 cohorts of students that are followed for at least 20 years, in order to capture the long-term benefits that are realized during their working years. This is consistent with the economic analysis for other sectors as the 20-year time horizon begins when the investment is completed, and the first cohort enters the education program and includes the cohorts that enter during the 20-year time horizon. Since the benefits are delayed and obtained during their working lifetime, the economic analysis includes at least another 20 years for each cohort to ensure those benefits are captured.
  • 16. The total number of beneficiaries of the Improving General Education Quality Project is less than the sum of the beneficiaries of the project’s three activities because it is adjusted for double counting. The beneficiaries of the Improved Learning Environment Infrastructure Activity are a subset of the Training Educators for Excellence Activity.
  • 17. Economic analysis data (estimated ERRs, beneficiary counts, and net benefits) presented for investment decision and entry into force cost-benefit analysis (CBA) models may not be comparable to compact closure data, as earlier CBA models for the Georgia II Compact were not extensively peer reviewed and only the EIF CBA model for the Industry-Led Skills and Workforce Development Project was published on MCC’s website.
  • 18. As shown in the table, the project-level ERR was above MCC’s 10% hurdle rate at the time of the initial investment decision (13%). For this Activity, MCC wanted to design school selection to ensure a balance between school size (a key variable for the ERR) and equity (as measured by the proportion of socially vulnerable students). Given the 13% ERR for the project, an activity-level ERR of 8% was approved to meet the desired targeting strategy.
  • 19. This beneficiary count applies to only 10 cohorts of students due to reduced operations and maintenance (O&M) commitments that reduced the sustainability of the investment. All other beneficiary counts presented for the ILEI Activity and Improving General Education Project apply to 20 cohorts of students. For economic analysis, direct beneficiaries are students who graduate from with project schools and become employed, resulting in increased incomes. Total beneficiaries, which are reported in the table, is calculated by multiplying the direct beneficiaries by the average household size in Georgia (3.4).
  • 20. While there are potential health benefits associated with rehabilitated schools, these were not quantified in any CBA models for the Activity; in this sense, the reported ERRs are conservative.
  • 21. The 2019 Evaluation Interim Report for the Georgia II Improving General Education Quality Project’s School Rehabilitation and Training Activities by Mathematica (Interim Evaluation Report) summarizes the existing literature on school rehabilitation.
  • 22. Notes on previous economic logic indicate an initial assumption of 20% increases in transition rates. The Interim Evaluation Report indicates the assumption on transition rates had been reduced to 10% but included a transition from lower secondary to upper secondary. In the closeout CBA model, the transition from lower to upper secondary education is not modeled due to nearly universal transition out of 9th grade and into upper secondary school in the data; instead, the model assumes a 10% increase in graduation from upper secondary school and applies higher education transition rates to the set of graduating students.
  • 23. When the study was carried out, 500 Georgian lari was equivalent to approximately $292 and the average monthly salary in Georgia was about 670 Georgian lari per month.
  • 24. For TVET education projects MCC’s economic analysis typically considers all students that enroll during the 20-year lifetime of the investment and are then followed for 20 to 30 years, in order to capture the long-term benefits that are realized during their working years.
  • 25. The ERRs and beneficiary estimates remain preliminary at the time of this report.
  • 26. When the M&E Plan was developed, MCC and MCA-Georgia did not know how many programs, or what type of programs, the Competitive Program Improvement Grants Activity would fund. Thus, they were not able to estimate the number of TVET graduates. The activity let the private sector guide what programs MCA-Georgia would fund, and MCA-Georgia did not mandate that the grantees have a certain number of enrollees.
  • 27. San Diego State University Strategic Growth Plan, September 2017.
  • 28. U.S. degree program enrollment in Computer Science and Construction Engineering concluded in September 2018. U.S. degree program enrollment in the remaining four disciplines concluded in September 2019.
  • 29. For economic analysis, direct beneficiaries are students who graduate from with project schools and become employed, resulting in increased incomes. Total beneficiaries, which are reported in the table, are calculated by multiplying the direct beneficiaries by the average household size in Georgia (3.4).
  • 30. MCC’s participation in WiSci camps was not funded by the compact. MCC’s Office of Strategic Partnership provided a grant to partially fund WiSci Georgia.
  • 31. For additional MCC learning from TVET programming, see MCC’s Principles into Practice Report “Training Service Delivery for Jobs & Productivity: MCC Lessons Learned in Technical and Vocational Education and Training” at https://assets.mcc.gov/content/uploads/paper-2020001233801-p-into-p-tvet.pdf.