Postconflict development in Congo
In early 2012 we completed a major study of the political and economic impacts of development aid in Eastern DR Congo. In addition to using public lotteries to generate a control group we used randomized intervention to examine the impacts of gender quotas for decision making. The big questions here are: is development aid effective in improving local governance? Do provisions to promote women’s representation in decision-making help? See our executive summary below.
- See discussion at The Financial Times, the World Bank, and DFID, and video of presentation given at the University of Toronto.
- See Final Report (June 2012)
- The analysis for this research was specified in advance using “mock report” approach. See the mock report and instruments and a forthcoming paper on design registration.
- Replication Data
This report provides the results of an assessment of the impacts of Tuungane, a major UK government funded Community Driven Reconstruction (CDR) Program implemented by the IRC and CARE International in Eastern Democratic Republic of Congo (DRC).
Project goals and activities: The Tuungane program has been working since 2007 in 1,250 war-affected villages with a targeted beneficiary population of approximately 1,780,000 people. Over this period Tuungane organized the election of village committees in all of these villages, as well as training in leadership, good governance, and social inclusion. The elected committees then worked with populations to select development projects and oversee the implementation of these projects. In the first (village level) stage of Tuungane I, which is the focus of our analysis, the project supported the construction or rehabilitation of 1,700 classrooms and 150 clinics as well as projects to support infrastructure and livelihood development. The theory behind the Tuungane intervention is that training, coupled with exposure to and practice in accountable governance in the context of these projects, can produce learning-by-doing and bring about change in local accountability and social cohesion as well as improve the welfare of communities.
Assessing Impact: This research project, mounted in partnership with IRC, sought to measure whether these objectives were met. In order to measure the causal effects of Tuungane, we employ the method of randomized intervention. The Tuungane communities were randomly selected through public lotteries from a larger pool of potential participating communities. This feature allows us to observe a set of control communities that are similar (in expectation) to the Tuungane communities in every respect except for the presence of the program. Also, among a sub-sample of those selected, a randomly selected set of communities implemented a variation of the program in which community development committees were not required to have gender parity.
Innovations in Measurement: The key innovation of this research is the introduction of a novel collection of measures to assess behavioral changes. The core behavioral measures are generated through the introduction of an entirely new and distinct unconditional cash transfer scheme (RAPID) in which a randomly selected set of 560 villages in treatment and control areas (with populations between 200 and 2,000) were selected to receive block grants of $1,000 which they could manage as they saw fit with minimal oversight and guidance. The key impact question then becomes: did areas that took part in Tuungane engage differently with RAPID relative to those that did not? Monitoring and auditing allowed us to gather measures on participation (Who made the decisions? How were decisions made?), accountability (To what extent did citizens try to oversee the actions of leaders or sanction them for misuse of funds?), efficiency (How effective are communities at managing funds or sharing information), transparency (Does budgetary information made available to leaders reach populations?), and capture (Who gets what?).
Findings on implementation: While the quality of implementation was not the focus of this research, our data confirms that Tuungane was successful in implementing a large number of projects in the target areas, that the projects were in line with the preferences of the populations, and that the populations reported very high levels both of exposure to project activities and of satisfaction with the outcomes of the project.
Findings on Impacts: On most measures we fail to find evidence that these positive experiences with the Tuungane intervention led to behavioral changes. In general, we found that for many local governance measures the outcomes were relatively strong in both control and treatment areas. For example, close to half of all committees were selected through electoral procedures; yet the likelihood of using elections was nearly as great (and statistically no different) in non-Tuungane as in Tuungane areas. Levels of transparency were also similar in both groups. As part of RAPID, villagers were told that at least $900 would be made available in funding; in fact, $1,000 was provided to project leaders. On our return we found, however, that 40% of the general population knew that the final figure was $1,000, not $900; this figure was again almost identical in Tuungane and non-Tuungane areas. Of the $1,000 allocated to communities, an average of $150 was not traceable by our audit teams; again this rate was nearly identical in treatment and control areas. There is evidence that in Tuungane areas, communities included more women on committees to manage RAPID funds, but the substantive effect is weak. There is also evidence that citizens in Tuungane areas are more likely to complain when funds are misused by leaders, suggesting some fostering of bottom up accountability. And there is some weak positive evidence for improvements in a survey based measure of trust (particularly trust in ex combatants, which is generally low), but no effects are seen for other measures of within or between village cohesion. There is little evidence of positive economic effects and some (generally scattered) evidence suggesting adverse effects.
Findings on gender provisions: We provide a small set of results that make use of a variation in design introduced in Tuungane to assess the importance of requiring gender parity as part of the formation of community committees. We find first that even without a requirement of gender parity, women comprise approximately 30% of the committee members. This may reflect pre-existing attitudes towards the role of women or it may reflect features of the general emphasis on gender inclusion as part of Tuungane (our examination of such choices outside of the Tuungane process, suggested considerably lower shares of women selected to these positions). This suggests that within the context of the program the gender parity requirement is not needed to ensure that there is some representation (as might be required to generate a demonstration effect); rather it serves to increase the numerical strength of women on committees. There is some evidence that the inclusion requirement results in fewer projects focused on water and sanitation (watsan) relative to education. This difference does not reflect differences between the stated preferences of men and women in the general population, but is (weakly) reflected in differences in stated preferences between male and female members of Village Development Committees (VDCs). We find, however, no evidence of positive changes in attitudes towards the roles and responsibilities of women as a result of the gender parity requirement. Taken together this evidence does not suggest that the imposition of gender parity requirements is an effective way to strengthen the position of women in this context.
Limitations: A number of features of the research design are worth noting when interpreting these results. First, the results properly address only the first (village level) phase of Tuungane. Although this phase covered the major governance interventions, it involved relatively small projects and accounts for only about 20% ($3.7m out of $18m) of the funding allocated to projects overall. Second, the research took place relatively shortly after the completion of these smaller projects; in cases such as the construction of schoolrooms, potential economic returns could kick in much later. For these reasons we believe that the most significant findings relate to the social effects and not the economic effects of the program. Third, absence of evidence for a positive effect is not the same as evidence of absence — it is possible that in some cases there are small positive effects and we lack the statistical power to identify them. This research is large in scale and power is reasonably good, but assessments of effects, positive and negative, should still focus on effect sizes and not simply the significance of various findings.
Conclusions: Overall, the results on the impacts of the program on the economic well-being and the socio-political attitudes and behaviors of the populations are surprisingly negative. While there is little evidence of adverse effects, the evidence for positive effects is scattered and generally weak. These null findings are broadly in line with results from other studies that have also failed to find evidence for the strong claims made on behalf of the CDR model. We can only speculate on the reasons for these weak effects. For the economic outcomes the low per capita investments and the short timeline to measurement may plausibly account for the results. Although the temporal and spatial coverage is very great, the per capita investments both in terms of finances and training are small. The simplest explanation for the weak effects on governance outcomes is that existing structures are resilient and that while behavior may change temporarily to meet the conditions of development actors, more fundamental change is not being achieved. Another possibility is that the scale of the project was too small to generate governance effects: the numbers directly engaged in training and project management were small and the size of the grants may have been too small to ensure broad engagement. A third possibility is that the program is pitched at the wrong level to effect change in governance structures and social cohesion; Tuungane has focused on the most local levels which may not display the same problems of cohesion and weak governance that are so visible in Congo at the macro level. Coupled with negative evidence from related studies our findings present a challenge to the CDR model and its ability to produce the social and economic impacts that advocates attribute to it.