Ukraine Threshold Program Monitoring & Evaluation
How MCC Tracks Progress
MCC emphasizes both achieving and measuring results in its threshold programs. MCC worked closely with Ukrainian counterparts in the MCA Working Group and with USAID, MCC’s on-the-ground representative, to design the program. During implementation, MCC and USAID worked closely with the threshold program’s executive board that consisted of 30 representatives of Ukraine’s executive, legislative and judicial branches. Both USAID and the Government of Ukraine contributed to establishing program milestones and refining program indicators for the Ukraine threshold program.
Ukraine’s threshold program indicators were tracked quarterly by MCC in a Results Reporting Table that reported progress over each quarter compared to the baseline and program targets.
Final Implementation Reports
USAID provided MCC a final report based on information provided by implementers and USAID program managers. A brief overview of some highlighted results is also available in the Final Status Report.
Results Reporting Table for Ukraine Threshold Program
| Expected Result |
Performance Indicator Name | Target | Baseline, as of Jan 2007 |
Year & Quarter | FY2007 | FY2008 | FY2009 | FY2010 | Notes | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Q3 | Q4 | Q1 | Q2 | Q3 | Q4 | Q1 | Q2 | Q3 | Q4 | Q1 | ||||||
| C1 Increased public knowledge of corruption reforms and issues | # of citizens expressing awareness of surveys and/or implemented project, millions | 3.73 | – | Target | 1.10 | 1.25 | 1.30 | 1.35 | 2.00 | 2.50 | 3.15 | 3.73 | 3.73 | 3.73 | 3.73 | |
| – | 0 | Actual | 3.69* | 1.83 | 4.10 | 4.10 | 4.10 | 3.61 | 3.61 | 4.10 | 2.72 | 6.11 | 3.84 | |||
| C1 Advocacy campaigns lead to anti-corruption reforms | % (#) of ACTION-supported advocacy campaigns result in government reform (verifiable change in behavior, guidelines, or legislation) | 30%(24) | – | Target | – | 3.75% | 7.50% | 11.25% | 15.00% | 18.75% | 22.50% | 26.25% | 30.00% | 30.00% | 30.00% | |
| – | 0 | Actual | 0 | 4.76% | 7.62% | 11.17% | 15.80% | 21.99% | 24.49% | 27.88% | 30.28% | 35.05% | 37.42% | |||
| C2.1 More transparent case assignment | % (#) of pilot court cases randomly assigned by new software | 100% (161,118) | – | Target | – | Identify 7 pilot courts | Install the system | 40.00% | 60.00% | 80.00% | 100.00% | 100.00% | 100.00% | 100.00% | 100.00% | The USG stopped providing assistance in Q4 FY2009. The target was not reached because one of the pilot courts did not randomly assign cases involving small administrative violations (i.e. traffic violations). |
| – | 0 | Actual | – | Completed | Work in process | 0 | 0 | 0 | 35.20% | 41.45% | 99% | 94% | 96% | |||
| C2.2 More transparent and substantive selection of judges | cumulative # of candidate judges undergoing standardized testing | 300 | – | Target | – | Form a work group | Select 5 pilot test sites | Install the system | 100 | Revise tests | 200 | Revise tests | 50 | 50 | n/a | The USG stopped providing assistance in Q4 FY2009. Due to the longer then expected process of developing and refining standardized tests, there was no time to reach this target during the implementation period of the TCP, which became clear by the end of quarter 2 of FY2009 (March 31, 2009). In the Work Plan for Chemonics cost extension for the period of May 2009 – September 2009 this target was decreased to 50, with a plan to conduct a second pilot test of candidate judges in August-September 2009. This target was not reached because standardized testing was still being refined, as discussed in the final report. |
| – | 0 | Actual | – | Completed | Completed | Work in process | 23 | Work in process | n/a | |||||||
| C2.2 More transparent and objective discipline practice | # of standardized documents/guidelines developed for judicial discipline bodies | 7 | – | Target | – | 3 | 3 | 3` | 4 | 5 | 6 | 7 | 3 | 3 | n/a | The USG stopped providing assistance in Q4 2009. Targets were reset in Q3 2009 to reflect the fact that multiple standardized documents prepared under the TCP were then merged into one document. |
| – | 0 | Actual | – | 0 | 3 | 4 | 2 | 2 | 2 | 2 | 3 | 4 | n/a | |||
| C2.3 Improved transparency in enforcement procedures | % (#) of enforcement procedures available in the Unified State Registry** | 100% (o/a 2.5M) | – | Target | 42.00% | 45.00% | 50.00% | 55.00% | 60.00% | 75.00% | 85.00% | 100.00% | 100.00% | 100.00% | 100.00% | |
| – | 40% (1,015,444) | Actual | 42.00% | 63.00% | 65.00% | 85.00% | 87.00% | 87.00% | 90.00% | 100.00% | 100.00% | 100.00% | 100.00% | |||
| C2.3 Improved enforcement of judgments | % of judgments executed | 28% | – | Target | 22,0% | 22,5% | 23,0% | 24,0 | 25,0% | 26,0% | 27,0% | 28,0% | This indicator was removed from the Results Reporting Table following Q2 2008. The final result for the indicator exceeds the final target, at 28.4%. | |||
| – | 22%; 558,494 judgments | Actual | 22,0% | 26,9% | 32,5% | 15,0 | 25,7% | 28,4% | ||||||||
| C2.5 Opportunity for corrupt licensing practice reduced. | % (#) of newly licensed notaries selected via standardized test | 100% (500) | – | Target | – | – | Draft the tests | Complete the tests | Identify the test sites | Install the software | 50.00% | 100.00% | 100.00% | 100.00% | 100.00% | The test is currently being automated and a pilot was held in December 2009; notary candidates continue to take the paper version of the test. |
| – | 0 | Actual | – | – | Completed | Completed | Completed | Work in process | 100.00% | 100.00% | 100.00% | 100.00% | 100.00% | |||
| C2.5 Improved monitoring of professional standards | % (#) of notaries inspected | 10% | – | Target | 9.00% | 9.00% | Form a work group | Draft the regulations | MOJ reviews regulations | MOJ adopts regulations | Notary SRO is registered | 10.00% | 10.00% | 10.00% | 10.00% | Notary inspections will be governed by the amended Notary Law, which have not yet been passed by the Parliament. 574 inspections were carried out in Q1 2010. |
| – | 9% (1,600) | Actual | 4.50% | 6.75% | Completed | Completed | Work in process | |||||||||
| C 2.5 Increase in sanctions and prosecutions for notary violations | % of violation findings that resulted in sanctions or prosecutions | 9.1% | – | Target | – | – | – | – | 7.5% | 8.0% | 8.5% | 9.1% | 9.1% | 9.1% | 9.1% | As the new notary law has not yet been passed, the Ministry of Justice continues to carry out inspections; this will be taken over by the SRO after passage of the law. |
| – | 7.0% | Actual | – | – | – | 7.0% | 8.0% | 5.0% | 3.5% | 6.4% | 9.4% | 9.0% | 8.5% | |||
| C3.1.1 Implementation of Ethics and Conflict of Interest Code | Major stages in drafting, promoting and adopting legislation | 10 lgsl stages*** | – | Target | – | 1 | 2 | 3-4 | 5-6 | 7-8 | 9-10 | 9-10 | 9-10 | 9-10 | 9-10 | Component 3 drafts on ethics and conflicts of interest were introduced in the Rada on May 15, but have not yet been considered in their first reading. |
| – | 0 | Actual | – | 1 | 2 | 5 | 5 | 5 | 5 | 5 | 6 | 6 | 6 | |||
| C3.1.2 Systematic monitoring of performance of functions in Ministry/Agency (SBGS) with Pilot IIU. | # of investigations (including referrals) conducted by Pilot IIU. | 5/year | – | Target | – | Select pilot IIU site | Draft IIU legislation | Submit IIU bill to CMU | Regulation is in force | 2 | 2 | 2 | 2 | 2 | CMU Regulations are adopted | |
| – | 0 | Actual | – | Completed | Completed | Completed | Regulation is in force | 2 | 2 | 2 | 2 | 218 | 191 | |||
| C3.2 New public official financial declaration regime introduced | Major stages in drafting, promoting and adopting legislation | 10 lgsl stages*** | – | Target | – | – | 1 | 2-4 | 5-6 | 7-8 | 9-10 | 9-10 | 9-10 | 9-10 | 9-10 | The Component 3 draft law on financial declarations was introduced in the Rada on May 15, but it has not yet been considered in its first reading. |
| – | 0 | Actual | – | – | 1 | 5 | 5 | 5 | 5 | 5 | 6 | 6 | 6 | |||
| C4.1.1 Improved customs and transportation regulatory system | # of normative/legislative acts approved | 35 | – | Target | – | 1 | 3 | 7 | 11 | 19 | 20 | 21 | 22 | 23 | n/a | The USG stopped providing assistance in Q3 2009. |
| – | 0 | Actual | – | 0 | 2 | 8 | 10 | 19 | 19 | 27 | 29 | 29 | n/a | |||
| C4.1.2 Participation in the EU New Computerized Transit System (NCTS) | # of system operational manuals and successful tests conducted | 8/35 | – | Target | – | 1/0 | 2/0 | 3/0 | 4/0 | 6/2 | 8/7 | 8/12 | 8/12 | 15-Aug | n/a | The USG stopped providing assistance in Q3 2009. The initial testing period was limited to seven attempted tests, which showed that there was a problem with the security environment. This bug has now been fixed and one successful test has occurred, and further tests were carried out in July. |
| – | 0 | Actual | – | 1/0 | 2/0 | 3/0 | 4/0 | 6/0 | 7/0 | 7/0 | 8/1 | 25-Aug | n/a | |||
| C4.2 Streamlined regulatory processes | # of One-Stop-Shops (OSS) with unified permitting procedures | 25 | – | Target | – | 0 | 5 | 10 | 15 | 25 | 25 | 25 | 25 | n/a | n/a | The USG stopped providing assistance in Q2 2009. |
| – | 0 | Actual | – | 0 | 0 | 0 | 0 | 3 | 13 | 61 | 61 | n/a | n/a | |||
| C5 Tests securely administered | # of tests securely administered, thousands**** | 1,500 | – | Target | – | Form a work group | Review laws, regulations & practices | Develop security rules | 800 | Draft laws are reviewed by the Parliament | Tests quality is improved | Revised security rules are adopted | 1,500 | 2010 admission rules are drafted | 2010 admission rules are announced | |
| – | 247 | Actual | 247 | Completed | Completed | Completed | 1,010 | Work in process | Work in process | Completed | 1,227 | Completed | Completed | |||
SOAG results are in bold
*Number of people is estimated. It is based on the reported proportion of regular omnibus survey participants who knew about corruption assessment surveys sponsored by the implementing partner and estimated total population aged over 18 (37.3 millions).
**USREP is a unified computerized system that posts status of enforcement cases on the internet, thereby increasing access to information on status of enforcement procedures and monitoring bailiff activity.
***(1) WG First Draft, 2)-4) Public Roundtables; 5)WG Final Draft, 6) Review in Rada Committee, 7)-9) 3 readings in Rada, and 10) President signs into Law
****Test is considered to be securely administrated if tests were administered if there are no security violations at printing, packing, delivering and completing stages.
NOTE: One indicator was removed over the course of the TCP: “% of execution cases brought to the ECJ by parties.” This indicator was removed early in the implementation period of the TCP as it was not an adequate measurement for the outputs of the TCP. In recent years, the number of cases brought to the European Court of Human Rights (the initial reference to the European Court of Justice in the indicators was incorrect) by Ukrainians has been increasing, as people become better informed about this option and have more opportunities to appeal Ukrainian court decisions; these court decisions could have been taken even 3-5 years prior to the appeal. Thus, the indicator was removed following an MCC monitoring visit to Ukraine, as it was agreed that the indicator would not be a useful measure of the effects of the TCP.
| Reduction of corruption in areas addressed by TCP activities | Performance Indicator Name | Target | Baseline, as of May 2007 |
Year & Quarter | FY2007 | FY2008 | FY2009 | FY2010 | Notes | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Q3 | Q4 | Q1 | Q2 | Q3 | Q4 | Q1 | Q2 | Q3 | Q4 | Q1 | ||||||
| Judicial system | % of polled citizens that perceived corruption | 44.1% | – | Target | – | – | – | – | 46.6% | – | – | 44.1% | – | – | – | |
| – | 49.0% | Actual | 49.0% | – | – | – | 54.0% | – | – | 59.3% | – | – | – | |||
| % of polled citizens that experienced corruption (solicited payments) | 25.4% | – | Target | – | – | – | – | 28.6% | – | – | 25.4% | – | – | – | ||
| – | 31.8% | Actual | 31.8% | – | – | – | 28.0% | – | – | 30.2% | – | – | – | |||
| % of polled citizens that experienced corruption (unsolicited payments) | 11.0% | – | Target | – | – | – | – | 12.3% | – | – | 11.0% | – | – | – | ||
| – | 13.7% | Actual | 13.7% | – | – | – | 15.0% | – | – | 8.6% | – | – | – | |||
| Customs | % of polled citizens that perceived corruption | 38.5% | – | Target | – | – | – | – | 40.7% | – | – | 38.5% | – | – | – | |
| – | 42.8% | Actual | 42.8% | – | – | – | 46.0% | – | – | 38.6% | – | – | – | |||
| % of polled citizens that experienced corruption (solicited payments) | 29.2% | – | Target | – | – | – | – | 32.9% | – | – | 29.2% | – | – | – | ||
| – | 36.5% | Actual | 36.5% | – | – | – | 42.0% | – | – | 29.5% | – | – | – | |||
| % of polled citizens that experienced corruption (unsolicited payments) | 14.5% | – | Target | – | – | – | – | 16.3% | – | – | 14.5% | – | – | – | ||
| – | 18.1% | Actual | 18.1% | – | – | – | 19.0% | – | – | 16.6% | – | – | – | |||
| Land ownership and use | % of polled citizens that perceived corruption | 33.8% | – | Target | – | – | – | – | 35.7% | – | – | 33.8% | – | – | – | |
| – | 37.6% | Actual | 37.6% | – | – | – | 44.0% | – | – | 47.1% | – | – | – | |||
| % of polled citizens that experienced corruption (solicited payments) | 19.9% | – | Target | – | – | – | 22.4% | – | – | 19.9% | – | – | – | |||
| – | 24.9% | Actual | 24.9% | – | – | – | 26.0% | – | – | 25.3% | – | – | – | |||
| % of polled citizens that experienced corruption (unsolicited payments) | 5.6% | – | Target | – | – | – | – | 6.3% | – | – | 5.6% | – | – | – | ||
| – | 7.0% | Actual | 7.0% | – | – | 10.0% | – | – | 7.1% | – | – | – | ||||
| Education/Universities | % of polled citizens that perceived corruption | 41.7% | – | Target | – | – | – | – | 44.0% | – | – | 41.7% | – | – | – | |
| – | 46.3% | Actual | 46.3% | – | – | – | 51.0% | – | – | 49.0% | – | – | – | |||
| % of polled citizens that experienced corruption (solicited payments) | 37.8% | – | Target | – | – | – | 42.6% | – | – | 37.8% | – | – | – | |||
| – | 47.3% | Actual | 47.3% | – | – | 44.0% | – | – | 43.8% | – | – | – | ||||
| % of polled citizens that experienced corruption (unsolicited payments) | 23.2% | – | Target | – | – | – | – | 26.1% | – | – | 23.2% | – | – | – | ||
| – | 29.0% | Actual | 29.0% | – | – | – | 31.0% | – | – | 27.3% | – | – | – | |||