Types of Research
This report reviews approaches to results measurement used by multilateral and bilateral donor organizations and highlights trends and gaps in how donors measure and report on their performance. Our review consists of assessing donor organizations in terms of their institutional design and levels of evaluation for results measurement, their organizational processes for measuring types of results including coordination and alignment with recipients, outputs and implementation, outcomes and impacts, and costs and effectiveness, and their processes for reporting and using results information. We collect evidence on 12 bilateral organizations and 10 multilateral organizations. The evidence review includes multi-country reviews of aid effectiveness, peer reviews by other donor organizations, donor evaluation plans and frameworks, and donor results and reporting documents. The report is based on an accompanying spreadsheet that contains the coded information from the 22 donor organizations. We find that donors report several types of results, but that there are challenges to measuring certain results at the aggregate donor level, due to challenges with funding and coordination for results measurement at the project, country, portfolio, and donor levels. Approaches to results measurement vary across donor organizations. We identify some trends and differences among groups of donors, notably between bilateral and multilateral donors, but overall there are no clear delineations in how donors approach results measurement.
Aid results information is often not comparable, since monitoring and evaluation frameworks, information gathering processes, and definitions of “results” differ across donors and governments. This report reviews approaches to results monitoring and evaluation used by governments in developing countries, and highlights trends and gaps in national monitoring and evaluation (M&E) systems. We collect evidence on 42 separate government M&E systems in 23 developing countries, including 17 general national M&E systems and 25 sector-specific national M&E systems, with 14 focused on HIV/AIDS, 8 on health, and 3 on agriculture. The evidence review includes external case studies and evaluations of M&E systems, government M&E assessments, M&E plans, strategic plans with an M&E component, and multi-country reviews of M&E, accountability, and aid effectiveness. We evaluate harmonization of government and development partner M&E systems, coordination and institutionalization of government M&E, challenges in data collection and monitoring, and analysis and use of results information. We also report on key characteristics of M&E systems in different sectors.
Aid donors are interested in the arguments for allocating aid via bilateral versus multilateral channels, and specifically in understanding which channel is more “effective” at supporting positive development and social outcomes. We contribute to the literature on this subject by summarizing recent OECD data on aid flows and reviewing the theoretical arguments from the aid literature on the different characteristics supporting effectiveness of bilateral versus multilateral aid. We then review the empirical literature, analyzing 40 papers that study the effectiveness of different aid channels on various outcomes. Many studies do not directly compare the effectiveness of aid channels, and the studies vary in how aid channels are defined, measured, and evaluated. Further, these studies do not directly test the hypothesized advantages of one channel of aid versus another; rather they test bilateral versus multilateral aid flows associations with development outcomes, assuming some causal mechanism is at work to explain differences in impact. We evaluate studies reporting the impacts of aid on GDP growth, governance, government investment spending, health, the HDI, poverty, and private investment, and find no consistent evidence that either bilateral or multilateral aid is more effective. The lack of conclusive evidence supporting either aid channel is likely related to differences in the methodologies of the studies included in this review, but may also be due to differences in how the theoretical arguments for the effectiveness of either channel apply in different circumstances.
How development organizations, NGOs, and governments can best allocate scarce resources to those in need has long been debated. As opposed to universal allocation of resources, a more targeted approach attempts to minimize program costs while maximizing benefits among those with the greatest need or market opportunity. Drawing on literature from several sectors,this brief presents two categories of beneficiary targeting in the development context: administrative targeting and self-targeting. The paper includes a brief overview of targeting and segmentation in development, a summary of reasons for targeting, theoretical and practical critiques of targeting, and a discussion of targeting methods in research and practice, including examples from the literature. Implementation examples cited in this body of research include food aid program targeting by self-reported household income in Egypt; fertilizer use in low-potential zones of Uganda; and seven strategic initiatives to improve drought and disease resistance in crops in Asia and Sub-Saharan Africa. We find that beneficiary segmentation has several theoretical advantages. Improved targeting may increase the efficiency and equity of organizational and program efforts and help better match interventions to recipient preferences, increasing the likelihood of adoption and participation. Development organizations may improve the focus of both their strategic priorities and budgets through customized targeting methods. However, concerns exist regarding the accuracy, reliability, cost, and time-constraints of targeting methodologies. Creating valid and reliable target groups with implementation potential remains a significant challenge.
How development organizations, NGOs, and governments can best allocate scarce resources to those in need has long been debated. As opposed to universal allocation of resources, a more targeted approach attempts to minimize program costs while maximizing benefits among those with the greatest need or market opportunity. Many international development organizations strategically target clients based on geographic location (e.g., community, region, country) or socio-economic indicators, such as the World Bank’s “$1 a day” poverty line. Drawing on literature from several sectors, this brief presents additional methods of beneficiary targeting that international development organizations might consider. We find that beneficiary targeting/segmentation has the potential to make organizational and program efforts more equitable and efficient. With limited resources, smaller organizations have tended to use single robust indicators or simple heuristics, whereas agribusinesses and private sector firms have used more data-intensive marketing tools to position their products. Technological innovation and better access to data have made targeting more prevalent and potentially more affordable in agricultural development. However, creating valid and reliable target segments remains the most significant challenge.
In recent years, product supply chains for agricultural goods have become increasingly globalized. As a result, greater numbers of smallholder farmers in South Asia (SA) and Sub-Saharan Africa (SSA) participate in global supply chains, many of them through contract farming (CF). CF is an arrangement between a farmer and a processing or marketing firm for the production and supply of agricultural products, often at predetermined prices. This literature review finds empirical evidence that demonstrates that the economic and social benefits of CF for smallholder farmers are mixed. A number of studies suggest that CF may improve farmer productivity, reduce production risk and transaction costs, and increase farmer incomes. However, critics caution that CF may undermine farmers’ relative bargaining power and increase health, environmental, and financial risk through exposure to monopsonistic markets, weak contract environments, and unfamiliar agricultural technologies. There is consensus across the literature that CF has the best outcomes for farmers when farmers have more bargaining power to negotiate the terms of the contract. In reviewing the literature on CF, we find a number of challenges to comparing studies and evaluating outcomes across contracts. This literature review summarizes empirical findings and analyses regarding contract models and best practices to increase farmers’ bargaining power and decrease contract default.
Introducing technology that is designed to be physically appropriate and valuable to women farmers can increase yields and raise income. But gender issues for agricultural technology projects in Sub-Saharan Africa (SSA) are extremely complex. The EPAR series on Gender and Cropping in SSA offers examples of how these issues can affect crop production and adoption of agricultural technologies at each point in the crop cycle for eight crops (cassava, cotton, maize, millet, rice, sorghum, wheat, and yam). This executive summary highlights innovative opportunities for interventions that consider these dimensions of gender. We encourage readers to consult the crop specific briefs for more details. We find that involving both men and women in the development, testing, and dissemination of agricultural technology has been shown to be successful in helping both benefit. Nevertheless, a consistent finding throughout the Gender and Cropping in SSA series is that maximum benefits from technological innovations cannot be realized when upstream factors like education, power, and land tenure heavily influence outcomes. Addressing these more basic upstream causes of gender inequality may be even more important in helping households increase productivity and maximize the benefits of technological interventions.
A widely quoted estimate is that women produce 70 to 80 percent of Sub-Saharan Africa’s (SSA) food. Increasing farmer productivity in SSA therefore requires understanding how these women make planting, harvesting, and other decisions that affect the production, consumption, and marketing of their crops. This brief provides an overview of the gender cropping series highlighting similar themes from the various crops studied, presenting an overarching summary of the findings and conclusion of the individual literature reviews. The studies reviewed suggest that differential preferences and access to assets by men and women can affect adoption levels and the benefits that accrue to men and women. Findings show that women have less secure access to credit, land, inputs, extension, and markets. Similarly, women’s multi-faceted role in household management gives rise to preferences that may very well be different from those of men. Participatory Breeding and Participatory Varietal Selection are two methods shown to be successful in developing technology that is more appropriate and more likely to avoid unintended consequences. Regularly collecting gender-disaggregated statistics can also result in a greater understanding of how technology has affected both men and women. Agricultural technology has the potential to enhance both men’s and women’s welfare and productivity, but unless gender is sufficiently integrated into every step of the development and dissemination process, efforts will only achieve a fraction of their total possible benefit.
Estimates suggest that women grow 70-80 percent of Africa’s food crops, which may constrain their involvement in cash crop production, if food crop production places additional demands their time, resources and labor. There is little evidence regarding women’s motivations or decisions to grow cash versus food crops. Similarly, the policy literature on cotton production and markets in Sub-Saharan Africa (SSA) does not explicitly address the issue of gender, further limiting the information available on the impact of cotton production on women. This brief provides an overview of the role of women in cotton production, and provides a framework for analyzing barriers to women and technology’s impact on women throughout the cropping cycle. We find that women are typically not the primary cultivators of cotton, and that cotton production is a household cultivation strategy, especially in West and Central Africa. Cotton cultivation often provides access to fertilizers, pesticides and extension services that are otherwise unavailable to households. Women have benefitted from household cotton income when they have input in intra-household resource allocation decisions or when they are able to grow cotton on personal plots and have control over the income it generates. Women also benefit from cotton when it offers them the opportunity to engage in paid labor. The data suggests, however, that cotton cultivation can negatively impact women when it increases their unpaid agricultural labor burden or exposes them to harmful chemicals.