Adoption of improved agricultural technology and its impact on household income: a propensity score matching estimation in eastern Ethiopia

Abstract

Background

Adoption of improved agricultural technologies remains to be a promising strategy to achieve food security and poverty reduction in many developing countries. However, there are limited rigorous impact evaluations on the contributions of such technologies on household welfare. This paper investigates the impact of improved agricultural technology use on farm household income in eastern Ethiopia.

Methods

Primary data for the study was obtained from a random sample of 248 rural households, 119 of which are improved technology users and the rest are non-users. The research employed the Propensity Score Matching (PSM) procedure to establish the causal relationship between adoption of improved crop and livestock technologies and changes in farm income.

Results

Results from the econometric analysis show that households using improved agricultural technologies had, on average, 23,031.28 Birr (Birr is the official currency of Ethiopia. The exchange rate according to the National Bank of Ethiopia (NBE) was 1 USD = 27.6017 Birr on 04 October 2018.) higher annual farm income compared to those households not using such technologies. Our findings highlight the importance of promoting multiple and complementary agricultural technologies among rural smallholders.

Conclusions

We suggest that rural technology generation, dissemination and adoption interventions be strengthened. Moreover, the linkage among research, extension, universities and farmers needs to be enhanced through facilitating a multistakeholders innovation platforms.

Background

Sub-Saharan Africa (SSA) is one of the regions in the world that is mainly characterized by smallholder farm households whose livelihood depends primarily on rainfed agriculture. It is also the region with millions of people living under extreme poverty [1]. It has been widely acknowledged that smallholder mixed crop-livestock agriculture plays a great role in feeding rural and urban populations in Ethiopia. It is considered as a center-stage of economic development and a platform to win the battle to food security and poverty reduction in the country [2]. The agricultural sector contributes 43% of Gross Domestic Product (GDP), 90% of export earnings, and 96% of rural employment [3]. It also provides raw materials for industries in the country.

Realizing the importance of the agricultural sector to the country’s economy, the government of Ethiopia has given a lot of attention to the sector and institutions that support it – research and extension in particular. The government’s emphasis to improve agricultural production and productivity, enhance food security, expedite commercialization and market integration, and improve rural livelihoods of smallholders was deeply reflected in the Agricultural Development Led Industrialization (ADLI) economic growth strategy since 1992, and the 5-year Growth and Transformation Plans (GTP-I&II) since 2010. However, the sector is not well developed especially in terms of technology use and commercialization. Despite its contribution, the performance of the sector has remained largely unsatisfactory [4].

Agricultural production in the country, and especially in the eastern part, is constrained by numerous factors, including small land size that is often not adequate to be economically viable; climate-change induced unreliable and irregular rainfall that frequently results in periodic drought; poor or declining soil fertility; limited input/output market integration; and very low level of use of improved agricultural technologies [5]. Although the country’s agricultural extension system is hailed as one of the strongest in Africa, disseminating and popularizing productivity-enhancing improved agricultural technologies and best practices remains to be of paramount importance in order to foster economic growth and alleviate food insecurity and vulnerability to poverty and its correlates.

In order to accelerate diffusion and adoption of agricultural technologies in the country, the Ethiopian Institute of Agricultural Research (EIAR), Regional Agricultural Research Institutes (RARIs), and universities have been experimenting and releasing several improved agricultural technologies in crops, livestock, and natural resource management. In addition, Agricultural Technical and Vocational Education and Training (ATVET) colleges in the country have been training frontline extension professionals who will station at the Farmers’ Training Centers (FTCs) established at the lowest level of administrative units throughout the country. These are some of the efforts made to increase the adoption of improved agricultural technologies by smallholders in the country.

There are recent studies that investigated the contribution of improved crop, livestock and natural resource management technologies on farm household income across Africa. Regarding crop technologies, for instance, using improved and/or drought-/disease-tolerant maize varieties in Kenya [6], in Zambia [7], in Zimbabwe [8], and in Benin [9] was found to increase farm income. A similar result was also obtained for smallholder maize growers in South Africa who participated in homestead food garden programs [10]. Adopting improved groundnut varieties in Uganda [11] and tissue culture banana technology in Kenya [12] were also found to result in increased household income. In Ethiopia, available literature documents the positive impact of improved wheat technology [13] and high-yielding sorghum varieties [14]. Concerning livestock production and management technologies, [15] in Tanzania and [16] in Rwanda showed the positive causal effects of dairy and sericulture technologies on household income, respectively. Finally, soil fertility management interventions (i.e., mulching) in Nigeria [17] and improved fallow techniques in Zambia [18] were associated with improved farm income of adopters. The abovementioned are some recent examples illustrating the causal relationship between adopting a single agricultural technology and household income.

However, few studies assessed the impact of simultaneous adoption of multiple or a combination of agricultural technologies on household welfare. Such studies can highlight complementarities among technologies and can show how one technology can have a multiplier effect by reinforcing the economic effect of the other technology. An earlier study in Mozambique documented an improved household income as a result of adopting improved seeds and tractors [19]. A similar impact on farm income was found for farmers adopting fertilizer micro-dosing and tied-ridge technologies in Tanzania [20]. Likewise, adoption of Sustainable Agricultural Practices [21] and improved seeds and fertilizer [22] in Ethiopia resulted in better household income for adopters.

Finally, adoption of multiple and/or complementary agricultural technologies–improved seeds, chemical fertilizers, pesticides and soil and water conservation practices–in the country is shown to enhance consumption expenditure and improve poverty status [3]. However, there is limited empirical study in eastern Ethiopia regarding the causal effect of multiple agricultural technology adoption on household welfare. Scarcity of such empirical investigations has created a knowledge gap on the performance and impact of such agricultural technologies and best practices in the region. In addition, there is a scanty empirical evidence on the impact and performance of agricultural technologies developed, disseminated and/or scaled-up by agricultural universities and their roles in improving food security and livelihood outcomes for farm households. Therefore, the current study systematically investigated the impact of improved agricultural technologies on farm household income in eastern Ethiopia.

Methods

Research design and study area

This study relied on an ex-post data collection from a sample of improved agricultural technology users and non-users. Empirical data for the study comes from households residing in six districts of eastern Ethiopia. These are: Kombolcha, Haramaya, Babile, Meta, Girawa, and Sofi. These districts were selected primarily because they are in the mandate area of the Haramaya University’s agricultural technology dissemination and community service activities. On top of this, representativeness to the major agro-ecological zones to represent diversity of livelihood activities, prevalence of food insecurity, and ease of accessibility were taken into account in the selection process. In Table 1, we present a brief description of the selected districts.

Table 1 Description of the study districts

Sampling procedure

A multi-stage sampling procedure was employed in this study. In the first stage, six districts were purposively selected as described above. This study made reference to the following improved crop and livestock technologies developed, disseminated, popularized and/or scaled-up by Haramaya University:

• a. improved crop varieties–Sorghum (Muyira-1, Muyira-2, Melkamash-79, Awash-1050, Dedesa-1050, and Alemaya–70); Maize (Alemaya composite, Rare-1, Bukuri); Wheat (Kulkulu); Groundnut (Oldhale and Roba); Potato (Gudane, Gabisa, Bubu, Chiro, Badhasa, and Harchasa); Sweet Potato; and,

• b. improved livestock technologies – portable poultry houses, poultry birds, cattle breeds, animal feed/forage, and apiculture technologies.

These technologies were chosen following a field scoping survey and a desk review conducted prior to the main survey to map the status and use of improved agricultural technologies that were developed, disseminated, and/or scaled up by Haramaya University.

During the second stage, a list of improved agricultural technology user farmers was generated across the selected districts in consultation with Development Agents (DAs), community/local leaders, administrators of FTCs, and representatives of district Bureau of Agriculture and Natural Resources. The generated list contained 1,785 improved agricultural technology users in the six districts–Girawa (250), Kombolcha (320), Sofi (197), Meta (284), Haramaya (416), and Babile (318). In this study, an improved agricultural technology user is defined as a farm household who has been using one or more of the aforementioned improved agricultural technologies consistently for at least 2 years. From the total households who are currently using improved agricultural technologies, a total of 119 users were randomly selected from the prepared list. Following Probability Proportional to Size (PPS) sampling procedure, this resulted in the random selection of 17 households from Girawa, 21 from Kombolcha, 13 from Sofi, 19 from Meta, 28 from Haramaya, and 21 from Babile districts. Likewise, in order to serve as a comparison group for the purpose of impact evaluation, random samples were drawn from a list of non-users of improved agricultural technologies. This list contained a total of 1,935 households – Girawa (259), Kombolcha (344), Sofi (240), Meta (297), Haramaya (458), and Babile (337). Consequently, a total of 129 households were randomly chosen and included in the study as control groups. Following PPS sampling procedure, this resulted in the random selection of 17 households from Girawa, 23 from Kombolcha, 16 from Sofi, 20 from Meta, 31 from Haramaya, and 22 from Babile. Hence, the overall sample size for this study is 248.

Data collection

This step in the research process started with the selection and training of research assistants (i.e., data collectors/enumerators) as well as translation of the questionnaire to local language (Afan Oromo). Based on information from key-informants, six research assistants who were trained (at agricultural colleges) in crop production, livestock production and management, or natural resource management were selected. Furthermore, three supervisors from Haramaya University were employed in order to closely supervise the process of data collection and provide real-time feedback whenever necessary. Both the enumerators and supervisors were given orientation training on the overall process of data collection.

The questionnaire, developed by the research team, was pre-tested on a randomly sampled 30 non-sample households. Based on the feedback obtained from the pre-testing exercise, additional orientations were given to the enumerators and supervisors. Both qualitative and quantitative data were collected–through face-to-face interviews using paper and pen as well as Focused Group Discussions (FGDs)–from primary and secondary sources. Socio-economic, demographic and institutional data of the study participants (i.e., household heads) were gathered. The participants were also asked about source and time of information on the agricultural technologies/innovations, technology selection criteria, major attributes of the innovations/technologies, crop and livestock technologies adopted and household consumption of food groups, to mention some.

Empirical strategy to data analysis

This study employed descriptive and inferential statistics, and an econometric model to analyze data. Descriptive statistics, such as mean and standard deviation, were used to present summary statistics of quantitative data pertaining to socio-demographic, economic, and institutional characteristics of sample households. Inferential statistics, such as t-test and Chi-Square (${\chi }^{2})$ test, were used to assess the existence of statistically significant differences in observations between improved agricultural technology user and non-user groups of respondents. In this study, farm income, the outcome variable, refers to the annual agricultural income (in Birr) obtained from crop and livestock less associated production costs during the last production season preceding the survey.

The theoretical framework used in this study makes a reference to the process of evaluating the impact of a program or an intervention on an outcome indicator. It requires conceptualizing and answering the tough question: ‘what would have happened to participants of a program/an intervention had they not participated in it?’ Referred to as ‘the fundamental problem of causal inference’ or ‘fundamental evaluation problem’ [23], this is a serious issue since an individual can only be in a state of either participating or not participating in the program at a given time [9; 39]. The ideal way to deal with the problem of counterfactuals is to employ Randomized Control Trials (RCTs) following the potential outcome approach or Roy–Rubin model [24, 25]. However, RCTs were not viable in the present study setting due to non-random allocation of farm households to treatment and control groups (i.e., placement/targeting bias) and selection bias.

The alternative to the experimental approach is the use of quasi-experimental approaches, which seek to create, using empirical methods, a comparable control group that can serve as a reasonable counterfactual [19, 26]. In the present study, among the available non-experimental approaches, the Propensity Score Matching (PSM) procedure is implemented due to the nature of data available for analysis.

Matching methods in evaluating program/treatment effects

The fundamental notion behind matching is to construct a comparable group of individuals–who are similar to the treatment individuals/groups in all relevant pre-treatment characteristics X–from a sample of untreated ones. In practice, a model (Probit or Logit for binary treatment) is estimated in which participation in a treatment/program is explained by several pre-treatment characteristics and then predictions of this estimation are used to create the propensity score that ranges from 0 to 1.

There are different approaches of implementing PSM, including the Nearest Neighbor (NN) matching, Caliper or Radius matching, Stratification or Interval matching, and Kernel and Local Linear matching [27]. In the present investigation, the Nearest Neighbor Matching (with 5-Neighbors and One-to-One matching) is implemented.

There are two assumptions surrounding the implementation of the PSM. The first one is referred to as unconfoundedness [28], selection on observables [29], or Conditional Independence Assumption (CIA) [30]. According to this assumption, the treatment needs to fulfill the criterion of being exogenous, implying that any systematic difference in outcomes between the treatment and comparison groups with the same values for characteristics X can be attributed to the treatment. The second assumption, called common support or overlap, ensures that individuals/groups with the same values for characteristics X have a positive probability of being both participants and non-participants of a program/treatment [23]. The overlap condition enables to compare comparable units. Nevertheless, in order to deal with the ‘curse of dimensionality’ problem, Rosenbaum and Rubin [28] show that if the potential outcomes of treated (${Y}_{1}$) and control (${Y}_{0}$) are independent of treatment allocation conditional on covariates X, then they are also independent of treatment conditional on the propensity score as shown in Eq. 1.

$${\text{P(D}}\;{\text{ = 1|X)}} ={\text{P(X)}}.$$

(1)

Generalizing the above issues, assuming that the unconfoundedness assumption holds and there is sufficient overlap between the treatment and comparison groups, the PSM estimator for the Average Treatment Effect on the Treated (ATT) conditional on the propensity score can be written as.

$$ {\text{ATT }} = \{ {\text{E}}[|{\text{D}} = {\text{1}},{\text{ P}}\left( {\text{X}} \right)\left] {{\text{ }}{-}{\text{ E}}} \right[|{\text{D}} = 0,{\text{ P}}\left( {\text{X}} \right)]\} $$

(2)

This means, the PSM estimator is simply the mean difference in outcomes over the common support region, appropriately weighted by the propensity score distribution of treated participants [31].

A number of techniques are available to check covariate balancing during matching process. In terms of mean comparisons, a two-sample t-test (before and after matching) can be used to check the existence or lack of significant differences in covariate means between the treated and comparison groups [28]. As a rule-of-thumb, there should not be any significant difference in means after matching. Regarding standardized bias, Rosenbaum and Rubin [28] define the absolute standardized bias (for each covariate X) as the absolute difference in sample means between the matched treatment and comparison samples as a percentage of the square root of the average sample variance in the two groups.

The standardized bias before matching can be written as

$${\text{Standardizedbias}}_{{{\text{before}} = 100*\frac{{\mathop {\text{X}}\limits^{ - } 1{\text{ - }}\;\mathop {\text{X}}\limits^{ - } 0}}{{\sqrt {0.5.({\text{V}}1\left( {\text{X}} \right) + {\text{V}}0\left( {\text{X}} \right))} }}}}$$

(3)

The standardized bias after matching can be written as

$${\text{Standardizedbias}}_{{{\text{after}} = 100*\frac{{\mathop {\text{X}}\limits^{ - } 1{\text{M}}\; - \;\mathop {\text{X}}\limits^{ - } 0{\text{M}}}}{{\sqrt {0.5.({\text{V}}1{\text{M}}\left( {\text{X}} \right) + {\text{V}}0{\text{M}}\left( {\text{X}} \right))} }}}}$$

(4)

where,

$\stackrel{-}{\mathrm{X}}1$ (V1) is the mean (variance) in the treatment group before matching.

$\stackrel{-}{\mathrm{X}}0$ (V0) the corresponding values for the comparison group.

$\stackrel{-}{\mathrm{X}}1$ M (V1M) and $\stackrel{-}{\mathrm{X}}0$ M (V0M) are the mean (variance) values for the matched samples.

Sianesi [32] suggests the comparison of Pseudo-R² before and after matching as a method to check balancing. The Pseudo-R² indicates how well the covariates X explain the probability of participating in the treatment. The Pseudo-R² has to be very low after matching to indicate success of the matching process. Moreover, the Likelihood Ratio (LR) test on the joint significance of all covariates in the (Logit) model should not be rejected before matching, but should be rejected afterwards [31].

Results and discussion

Descriptive statistics results

Results related to socio-demographic characteristics are presented in the top section of Table 2. From the results, we note that there is a statistically significant difference in age, gender, school years, and farming experiences between users and non-users of improved agricultural technologies, while the mean value for family size of respondents was found to be not significantly different between the two groups. These observations imply that the farm households who used improved agricultural technologies were majorly male farmers and relatively older than those who did not use the technologies. It is widely acknowledged that male farmers are more likely to adopt agricultural technologies than their female counterparts [33,34,35]. The reasons could be that female farmers have difficulty accessing inputs [36] and other norms and beliefs prevailing in the society [37]. Experience in terms of age is found to positively affect adoption of a System of Rice Intensification (SRI) in India [38]. However, age of farmers did not significantly affect the adoption of soil-improving practices in Ghana [21].

Table 2 Descriptive results. Mean values; standard deviations in parenthesis

Our findings suggest that the farmers who have used technologies from Haramaya University have stayed in schools and on farming activities for more number of years compared to those who did not use the technologies. Our result is consistent with that of Makate et al. [28] who documented a positive effect of education on adoption of multiple climate smart agricultural innovations in Southern Africa. However, our finding contrasts with that of Nata, Mjelde and Boadu [21] for Ghana. In general, however, investment in education is essential for development and would encourage farmers to adopt appropriate technologies and practices [39]. In this study, male household heads appear to have been the preferred targets of improved agricultural dissemination process. Such bias against young, less educated and less experienced, and female household heads is a long-standing bottleneck in adoption and diffusion of agricultural technologies in the country.

The information obtained from several Focused Group Discussions (FGDs) conducted with female headed households specifically revealed that there was a frequent targeting bias against, among others, female farmers and this supports the finding related to gender-based differences among users of the technology [33,34,35, 40,41,42,43]. An FGD participant in Babile revealed that “since most of the extension agents are males, and due to cultural and religious expectation, they do not interact adequately with female headed households as well as female members of the community. This, in my opinion, has created a barrier in obtaining and using relevant information on improved agricultural technologies and practices.” Besides, the qualitative study has also shown that the interventions of Haramaya University were less successful in recognizing gender sensitive preference criteria for the technologies and female targets were found to be more prone to such phenomena compared to the male counterparts.

The second category of explanatory variables are economic factors. Several variables, vis-à-vis livestock ownership, land size, non-farm income, irrigation access, use of conservation practices, and asset value were included in this category. According to the results (depicted in Table 2), a statistically significant difference was observed between the users and non-users for three out of six economic variables. Livestock owned, land size, and SWC practices were the significant variables, while non-/off-farm income, access to irrigation, and asset value were found not significantly varying between the two groups of respondents.

These results indicate that farm households who owned more livestock, operated a relatively large plot of land, and participated in SWC practices had a better chance of improved agricultural technology use due to the fact that such households are better-off in taking risks associated with new technologies and practices, or these households received preferential treatment by the promoters of improved agricultural technologies. The result pertaining to the effect of plot area is related to that of Martey, Kuwornu and Adjebeng-Danquah [44]. Similar findings were documented in Rwanda where asset endowments and participation in farmer organizations, among others, condition adoption of rainwater harvesting technologies to improve agricultural productivity and income [45]. Availability/endowment of farm assets is also found to positively influence the decision to adopt SRI in India [38]. Size of land owned by a farmer is found to have positive effect on adoption of multiple CSA innovations in Southern Africa [28]. Our result contrasts that of Varma [38], who found that small and marginal farmers are more likely to adopt SRI as compared to large farmers. Based on an inventory of Haramaya University’s technological interventions conducted prior to this study, most of the technologies generated and disseminated by the University are on-farm based and hence required an adequate size of plot for the farmers to participate in and draw benefits from the interventions. The fact that the non-user group possesses lesser farm size would tend to hinder them from utilizing such interventions. The finding pertaining to SWC is consistent with the results obtained from FGDs, which indicated that the target farmers have a long history working with the University on various SWC technologies among which cultivation of leguminous crops and tree species, as well as constructing SWC structures are the major ones.

Among the seven institutional variables considered in this study, only two were found to have significantly different distribution between the users and non-users of improved agricultural technologies. These are: participation in FTCs and access to market information. We find that improved agricultural technology users participated more in FTCs than the non-users. In an earlier study conducted in the study area, it was found that participation in FTC-based agricultural extension services significantly improved household income [37]. This confirms that FTCs play a great role in fostering economic development and that farm households should be encouraged to interact continuously with such knowledge and technology generation and transfer institutions in rural areas. We also observe that farm households with greater market information access had a significantly higher rate of improved agricultural technology use in the study area. Access to information is found to be positively associated with adoption of multiple CSA innovations in Southern Africa [28]. Market access has been identified as one of the important elements in enhancing agricultural technology adoption and improved household income. Access to market information is found to significantly influence adoption of improved pigeon peas in semi-arid south-east Kenya [46].

Regarding the outcome variable, i.e., farm income, we find that, on average, non-users of improved agricultural technology obtained 14,923.26 Birr/year while the users obtained 33,082.35 Birr/year. Users tend to earn more income per annum than the non-users, and the mean difference of it is statistically highly significant as shown in the top part of Table 2.

Econometric model estimation results

The causal effect of improved agricultural technology use on farm income is estimated using the Propensity Score Matching (PSM) procedure. The analysis employed Nearest Neighbor Matching (with 5-Neighbors and One-to-One matching algorithms) using psmatch2 command implemented on STATA 15.1 platform. In what follows, the results pertaining to estimation of propensity scores, Average Treatment Effect on the Treated (ATT), and post-matching quality analyses are presented.

Estimation of propensity score

The conditional probability of households’ participation in improved agricultural technology use is estimated using a Logistic Regression model. The model considered all observable covariates that affect participation and farm income and for which observational data were available. The results are given in Table 3. Overall, the model is statistically significant as shown in the lower part of Table 3. Based on the findings, we note the existence of a statistically significant difference between treated (n = 119) and control (n = 129) households regarding the distributions of age, gender, family size, land size, SWC practices, and distance from main market (Column (3), Table 3). As depicted in Table 3, these factors were responsible for households’ differential participation in improved agricultural technology use. Since we are interested in computing the propensity scores, which will be used in the matching process later on, we will not go into the details of why and how each of the covariates affected households’ participation in the intervention. Nevertheless, we indicate that as we proceed with our analysis, these before-matching differences are no longer significant in the aftermath of matching (Column (1), Table 5), which is an indication that the PSM was successful to experimentally create a comparable group of control individuals whose outcomes can be compared to that of the treated ones.

Table 3 Propensity score estimation

Our finding pertaining to the effect of age of on agricultural technology adoption is related to that of Martey, Kuwornu and Adjebeng-Danquah [44] in Ghana and Varma [38] in India. However, age of farmers was found not to significantly affect adoption of soil-improving practices in Ghana [21].Similar to our results, family size and size of cultivated land were found to positively affect agricultural technology use in Ghana [44].Likewise, availability/endowment of farm assets positively influence the decision to adopt Sustainable Rice Intensification (SRI) in India [38] and rainwater harvesting technologies in Rwanda [45]. In a recent study in Southern Africa, size of land owned by a farmer is also found to have a positive effect on adoption of multiple Climate Smart Agricultural (CSA) innovations [28]. Use of fallow, among SWC practices, is found to enhance technology adoption [44]. Finally, distance from main market, as an indicator of access to market information, is shown to have a positive effect on adoption in south-east Kenya [46] and Southern Africa [28].

Estimation of average treatment effect on the treated (ATT)

The estimation of Average Treatment Effect on the Treated (ATT) is performed using the Nearest Neighbour Matching (with 5-Nearest Neighbours and One-to-One matching algorithms). All the control households (i.e., 129) and 117 of the 119 treated households are used in the matching process, since these were found on the common support region (see Fig. 1). The results are presented in Table 4. In addition to the mean values of the outcome variable, Table 4 contains mean differences between treated and control groups (Column 3) and bootstrap standard errors (with 50 replications) on the mean difference (Column 6). Overall, we found convergence of results between the two matching algorithms (Column 7). However, the discussion in this section is based on the results obtained using the One-to-One matching algorithm as this resulted in a higher level of statistical significance.

Fig. 1

Propensity score graph all (129) untreated and 117 out of 119 treated observations are on common support region

Table 4 Nearest Neighbour Matching Results of Average Treatment Effect on the Treated (ATT)

Accordingly, the results show a statistically significant gain in household farm income as a result of using improved agricultural technologies in the study area. More specifically, we found that households using improved agricultural technologies obtained, on average, 23,031.28 Birr higher annual farm income compared to those households not using such technologies. This is a significant result implying that adoption of improved agricultural technologies and practices resulted in improved welfare in the study area. Our result is consistent with previous empirical results of Cunguara and Darnhofer [19] in Mozambique, who showed an improved household income as a result of adopting improved seeds and tractor; Habtemariam et al. [20] in Tanzania, who indicated the positive income effect of adopting fertilizer micro-dosing and tied-ridge technologies; and, Teklewold et al. [21] and Hailu, Abrha and Weldegiorgis [22] in Ethiopia, who documented a positive income effect of adopting Sustainable Agricultural Practices, and improved seeds and fertilizer, respectively.

Matching quality analyses

The matching quality analyses were performed using t-tests and Standardized Percentage Bias (Table 5, Columns (1) and (2), respectively) and other measures of covariate imbalance (Table 6).

Table 5 Matching quality analysis: t-test and standardized percentage bias

Table 6 Other matching quality tests

Looking at the t-test results after matching (Column 1, Table 5), we found that the statistically significant difference between treated and control groups that were observed for some covariates in the unmatched sample were fully removed. This implies that the matching process was effective in balancing the distributions of the covariates in the matched sample. Likewise, the Standardized Percentage Bias (Column 2, Table 5) appears to be in the acceptable range, complementing the post-estimation t-test results and implying further that the PSM performed well in yielding unbiased estimates of ATT.

In addition to the post-estimation t-test and standardized percentage bias results, other measures of covariate imbalance (Table 6) also indicate that the matching process is effective in balancing the pre-treatment characteristics.

Finally, the propensity score graph (psgraph) in Fig. 1 presents treated and untreated households that are found on the common support region (i.e., 117 and 129, respectively) and the two treated observations that are off the support region.

Conclusion and recommendations

In this study, we evaluated the causal effect of using improved crop and livestock technologies on farm household income using 248 randomly selected households in six districts of eastern Ethiopia. The study employed One-to-One and Nearest Neighbor matching algorithms using the PSM estimation procedure. Through collecting data specifically for the purpose of impact evaluation and implementing rigorous evaluation methods, the key findings of the study showed that smallholder farm households using improved agricultural technologies developed, disseminated and/or scaled-up by Haramaya University had a statistically significant household income compared to those not using these technologies. More specifically, we found that improved agricultural technology use resulted in, on average, 23,031.28 Birr higher annual farm income per household compared to non-use of such technologies. This estimate of farm income is robust as confirmed by the convergence of results obtained using the two matching algorithms and the results of matching quality analyses (i.e., post-matching t-test, standardized percentage bias, and other measures of covariate imbalance).

Based on the results of this study, the following recommendations are suggested to improve agricultural technology dissemination and adoption in the study area. To start with, there should be a reinvigorated awareness creation campaign by the university extension wing, district Bureau of Agriculture and Natural Resources, Non-Governmental Organizations (NGOs) and model farmers regarding the importance of improved crop and livestock technologies that can transform the livelihoods of smallholder farmers. In particular, young farmers need to be encouraged to partake in trying new technologies and best practices. Since the default household heads are males, there should be a provision to target young and women farmers (although they are not household heads). One operational suggestion in this regard requires a general shift of focus from household heads to members of the household. Furthermore, a minimum experience in farming should be sufficient to qualify young farm entrepreneurs to involve in improved agricultural technology adoption. Promoting a multi-stakeholder engagement in the design, dissemination, and scaling-up of proven agricultural technologies should also be the model of the university’s research, extension and community service provision.

Although the evaluation technique employed in this paper was based on a rigorous statistical procedure, it used a cross-sectional data and, hence, there are potentials for improvement through further investigation. The first aspect in this regard requires the collection of longitudinal data from more households across the different agro-ecological zones of the region. Such data can contain baseline information that can enable the use of, for example, randomized control trials (RCTs), which are regarded as “gold” standards in impact evaluation. Moreover, the availability of such a rich dataset can enable combining PSM with Difference-in-Difference (DID), which is superior to using each technique single-handedly. Second, more outcome indicators, such as productivity, food and nutrition security, poverty, consumption expenditure, and asset accumulation can be included to examine if improved agricultural technology adoption results in other added advantages in the study area. Another issue for future consideration can be a move away from individual level evaluations to aggregate levels (such as village, district, etc.) or a combination of both of them. Whereas the former can enable a researcher to better capture spillover effects of a program or an intervention as well as any unobserved heterogeneity between treatment and comparison groups, the latter can help in examining the complementarity of individual and group-based approaches to impact evaluation. Finally, we suggest that future research should evaluate the impact of each improved crop/livestock technology independently in order to provide a more detailed and technology-specific policy implication.