The impacts of rhizobium inoculation on growth and yield of chickpea have mainly been tested in experiments conducted in greenhouses or on research stations. We report the response of the crop to inoculation (I) and phosphorus fertilizer (P) application across a large number of smallholder’s farms over four regions of Ethiopia, covering diverse soil fertility and agro-ecological conditions. Increased grain yields due to the soil fertility treatments was evident for 99% target farmers. On average, I and P increased grain yield by 21% and 25% respectively, while the combined application of I and P resulted in a 38% increase. However, observed grain yields on control plots and responses to the treatments on individual farms varied greatly, and relative yield responses (%; yield of P and/I minus control yield, divided by control yield) ranged from 3% to 138%. With the exception of a few extremely poorly yielding locations, average responses to P and I were high across a wide range of control yields, indicating the possibility of boosting chickpea productivity for smallholders with P fertilizer and inoculant technology. Variation in response to rhizobium inoculation was mostly independent of agro-ecology and soil type although it was found to be low on a number of farms with extremely high N contents (%). Assuming that a relative yield increase of 10% due to treatment effects is required to be visible, 71%, 73% and 92% of the farmers observed a yield benefit by applying P, I, and P + I, respectively. The results are discussed with respect to the additive benefits of P fertilizers and rhizobial inoculation and their implications for wide scale promotion of inoculant technology to smallholders.