Agriculture can be a volatile source of income, yet it is the main livelihood for some of the most vulnerable populations in the world. Weather, pests, and price fluctuations can decimate or inflate income for any farmer, but this is especially worrisome when negative shocks affect the poor who are living at subsistence levels. Projects in the agriculture sector make up 15 percent of MCC’s portfolio, accounting for $1.7 billion in investments, and are the subject of dozens of MCC’s independent evaluations. MCC is continually seeking new and better ways to accurately assess the effects of external shocks on both farmer income and the ability of the evaluation to accurately measure impacts.
To mitigate these evaluation challenges, MCC’s Monitoring and Evaluation (M&E) team is researching innovative techniques to better measure key outputs and outcomes of our agriculture and irrigation investments. Traditionally, MCC has measured results in this sector through large household surveys that take place before the project begins to understand the starting point, shortly after the project ends to measure immediate results, and several years after the project ends to measure long-term outcomes. However, the nuances of agriculture investments, such as the year-to-year natural variation in yields and prices, often make reliable measurement of key outcomes—such as improved livelihoods—difficult with our current approaches.
True to our learning and transparency-focused model, in September 2019 MCC hosted an Agriculture and Irrigation Evaluation Day of Learning to explore the latest technologies in unbiased and precise measurement of agriculture. At this all-day, public event, we discussed preliminary findings and lessons learned from our Moldova, Senegal and Burkina Faso midline evaluations. We then used these evaluations to inform discussions with leading experts about improving evaluation methods going forward by incorporating things like satellite imagery and remote sensing to expand our results-measuring toolkit.
Driven by recommendations from leading experts, MCC used this workshop to explore actionable methods to improve our measurement techniques. These include:
- Better ways of collecting data on costs and quantities of inputs such as farm labor and the value of production through rapid surveys;
- Understanding if remote sensing or other approaches could improve crop identification and production data or provide signals that could trigger other data collection efforts; and
- Data collection approaches that might be helpful for non-agricultural income sources and should be considered to capture direct and indirect program effects to ensure valid counterfactual comparisons.
Through these discussions, we realized that these new ways to collect data have the potential to give us greater precision in our measurements. New research has documented that asking farmers about labor used for farm production at the end of the season can skew measurements by as much as 200 percent. We can mitigate that by conducting frequent, rapid surveys. We can also achieve unbiased estimates of key outcomes through remote sensing, which may improve frequency and consistency of estimation of crop yields, cropping intensity, and crop mix. Moreover, remote sensing and rapid surveys can each provide signals of behavior change, such as expected yield increases or unexpected reductions in yield due to reduced cropping intensity.
MCC is now working to implement these ideas to determine if they can augment our measurement toolkit. These new tools are an exciting way to track progress and feed into course-correction during project implementation, and could even be used by beneficiaries to improve their farming practices. Now comes the hard and exciting work of putting these ideas into practice at MCC and with our partners!