N2Africa has gathered and monitored a lot of information about the impact of its work. However, the diversity of N2Africa’s interventions, their dynamism, and the widely different contexts where these have been implemented, make it tricky to derive strong inferences about the project’s impacts from suvey-based impact evaluations.
Therefore, N2Africa also uses institutional information and community-level data collected using qualitative methods. Together, this quantitative and qualitative information is expected to provide evidence for N2Africa’s claim of contribution, that is, that the programme’s activities have contributed to the impacts indicated in the project’s theory of change.
N2Africa’s objective is to use these pieces of information to create a coherent contribution narrative, so that those involved in the project can develop more refined theories that may be used to scale and replicate activities in the future.
|
To assist N2Africa with this effort, a team of external evaluators from the Institute of Development Studies (IDS, Brighton, UK) gathered and critically analysed evidence underlying N2Africa’s claims about its impact. The IDS researchers used an approach known as contribution analysis. This evaluation approach is informed by the methodology of process tracing, which is a structured way of critically reviewing a process and sequence of change. The IDS team reviewed available project documentation and other literature, and gathered data through stakeholder interviews that focused on four generic questions, which help to assess the strength and importance of the contribution made by N2Africa. |
 Giel Ton at Contribution Analysis brainstorm session at Wageningen University in February 2019 |
The evaluators applied this approach in two case studies, on Ghana and Ethiopia respectively. The two chosen cases relate to technologies or support interventions that were considered by N2Africa stakeholders to be the most promising for replication or upscaling. The expected impact pathways in each of these cases embody a set of critical assumptions about the immediate and intermediate outcomes of N2Africa’s work, which would need to exist in order to have a fair chance of generating the desired impact at scale.
In Ghana, Giel Ton reviewed the evidence-base behind the contribution claim: “N2Africa has been a relevant contributory factor in the expansion of soyabean production in Northern Ghana.” He differentiated four assumptions that underpin the contribution claim, verified whether the assumed changes had indeed taken place, and assessed what had been the importance of N2Africa’s contribution has been on each of these outcomes.
Did the soyabean expansion take place? Yes. Though data is unreliable, most interviewees believe that an expansion of soyabean production in Northern Ghana has indeed taken place. This is considered as a positive development.
Was N2Africa support used in the process? Yes. In particular, seed multiplication is likely to have had positive effects on the soyabean yields of a large number of smallholder farmers. These farmers use certified seeds and improved soyabean varieties.
Was N2Africa necessary to speed-up or scale faster? Yes. N2Africa-supported seed multiplication work has had a positive, systemic effect for the availability of quality soyabean seed for smallholder farmers. The impact of N2Africa’s work on inoculants and fertilisers is still fairly limited. Uptake seems largely restricted to groups of farmers that are self-organised or who benefit from NGO support, where the demand and supply of inoculants and fertiliser are monitored and communicated to the agro-input shops, and uptake is facilitated by pre-harvest loans.
However, there are some outstanding challenges. Legume fertilisers are increasingly available but still in a ‘pilot distribution mode,’ not part of the regular distribution system. The quantities demanded are too small to make legume fertilisers a regular item in the inventory of agro-input shops. An input-subsidy programme is driving demand for legume fertiliser. N2Africa played a facilitating role in opening up this programme to legumes, but this somewhat artificial market may collapse when the subsidies are discontinued. Inoculants are not part of the subsidy programme. Due to their relatively short shelf-life, they are not yet available to farmers in remote areas.
Was N2Africa a necessary causal factor for the soyabean expansion to take place? No. N2Africa cannot claim to be the trigger that started the soyabean expansion. The main trigger is the demand generated by processing industries that began to be located in the area. At most, N2Africa played a facilitating role in a wide number of partnerships, alongside the efforts of smallholders themselves to benefit from the increasing demand for soyabeans in Ghana. Without N2Africa, this growth of soyabean acreage would have started anyhow, and will likely continue, but with a (even) lower adoption of fertiliser and inoculants.
In Ethiopia, Dominic Glover verified the claim, “N2Africa has contributed to the increase in production, distribution, uptake and expansion of market demand for legume inoculants.” He verified three critical assumptions in the theory of change. The conclusions were as follows:
Did the expansion of inoculants production and use take place? Yes. The volumes of production, distribution, sale and use of inoculants increased significantly, albeit from a very low base. Demand for inoculant products has increased, but the inoculant market remains far below its estimated potential.
Was N2Africa support used in the process? Yes. N2Africa’s support was crucial, especially in two key respects. First, N2Africa played a direct role in securing funds to upgrade and expand inoculant production by a commercial manufacturer. Second, the public—private partnership model brought stakeholders into contact with one another to work in ways that had not existed before the project.
Was N2Africa a necessary causal factor for the expansion of inoculant supply and use to take place? Likely yes. While actors in the system already aspired to develop and expand a market for legume inoculants and improve legume production before N2Africa commenced, the project played a catalytic, facilitating role to strengthen and accelerate these processes. The expansion of inoculant production and use might have occurred in the absence of N2Africa, however, these developments might have been delayed indefinitely or prevented entirely without N2Africa’s intervention. N2Africa’s strategic and targeted approach made a substantial difference.
In summary, there is convincing evidence that N2Africa contributed substantially to a process of technological upgrading of soyabean production in Northern Ghana, which, however, is still only weakly influencing the overall increase in soyabean production in that region. Smallholders cultivate soyabeans in a mixed cropping system, where cash expenses for external inputs are prioritised for other crops (maize, rice), not soyabeans. In Ethiopia, the contribution claims focused on the supply and use of inoculants, where it was clear that N2Africa had made a decisive contribution.
Note: The full report has appeared as an IDS Practice Paper. (Reference: Ton, G. & Glover, D. (2019). Improving Knowledge, Inputs and Markets for Legume Expansion: A Contribution Analysis of N2Africa in Ghana and Ethiopia. In IDS Practice Paper 10 Brighton: Institute of Development Studies. 39 pp.)
Giel Ton, Dominic Glover, Institute of Development Studies, United Kingdom