UD examines challenges of recruiting farmers for economic field experiments to inform agricultural policy

Researchers in the University of Delaware Center for Experimental and Applied Economics (CEAE) use laboratory-controlled experiments to replicate economic behavior. CEAE uses its findings to inform policy related to water quality, agriculture and provision of public goods.

But, to make these improvements, the team must show research conclusions from the laboratory also hold in the field, where experiments are conducted with real stakeholders, like farmers or homeowners. As with all experiments, conclusions are statistically valid only when a sufficient number of participants take part in a study. In agricultural policy research, recruiting enough farmers is a major challenge — historically more difficult compared to areas like health sciences and education.

The bipartisan Foundations of Evidence-Based Policymaking Act of 2018 — more commonly referred to as simply the “Evidence Act” — requires that federal agencies like the U.S. Department of Agriculture (USDA) develop statistical evidence to inform policy. So Kent Messer, S. Hallock du Pont Professor and Director of CEAE, and postdoctoral research fellow Laura Paul conducted research on the challenges of recruiting farmers, recently published in Applied Economic Perspectives and Policy.

Like most Americans who are asked to take part in a research study, some farmers cite data privacy concerns, but, according to Paul, other factors are at work.

“Farmers might be busier than the average person, especially on a seasonal basis,” explained Paul. “Farmers have extremely time sensitive demands on them; that shortens the window to take part in this research.”

The unyielding demand of farm work makes research recruitment a challenge. Many experimental economics studies take place online and by phone — both a hard sell. Other experiments take place in-person, which Paul says are also tough because of the time and distance from most farms to universities.

“It makes intuitive sense that agricultural policy would be slower to adopt experiments,” said Paul. “Randomized experiments are a very important tool for informing policy. They have been used in medical studies and in the classroom for a long time. In agriculture, however, researchers face hurdles to using the experimental toolkit for policy.”

In this new research, Messer, Paul and collaborators through the Center for Behavioral and Experimental Agri-Environmental Research experimentally evaluated the ten most promising recruitment strategies for farmers. Through two experiments with more than 20,000 combined participants, the authors showed that monetary incentives and reminders are effective, but costly. Costless strategies, such as prominently citing a well-known institution as the sponsor, had positive, albeit small effects on recruitment.

The clearest illustration of the difference in recruiting farmers versus other groups was in email responses. Testing email versus mail participation, zero of the 4,742 farmers surveyed responded to the email while about 2.4 percent (or around 500 of the 20,874) responded to the mailing. Recruitment rates of non-farming adults or students typically ranges from 10 to 30 percent.

“That’s evidence that recruiting farmers is seriously different than recruiting other American adults,” emphasized Paul.

Other findings make more intuitive sense. Paying a participant $100 recruited more people compared to $25; when researchers dropped the difference down to $75 versus $50, there was no difference in recruiting participants. Researchers also found that pre-survey testing on college students can provide an informative, cost-effective result.

The ability to more efficiently and cost-effectively recruit more farmers into studies puts economics researchers in a stronger position to inform agricultural policies like the conservation reserve program—a program with a $2.3 billion annual budget that protects close to 25 million acres of environmentally sensitive cropland and grassland. Ultimately, Paul sees no cheap and easy way to recruit without the close cooperation of the administrators of these programs. Thus, strategic foresight, persistence and partnerships between academia and government (and perhaps industry as well) are musts for investigators.

“Research projects are on a limited budget. Instead of wasting money on an ineffective strategy, you can use the money on strategies that work,” said Paul. “Sure, there’s been testing of these strategies with average consumers, but not specifically on farmers. We are happy to provide new insights on how to conduct recruitment for research in the most cost-effective manner possible.”