Editor's note: This is the final of three parts looking at setting up and running an on-farm trial for 2022. We look at the basics; the finer points of building the trial; and finally, analyzing the trial.

Part 1: 7 tips for setting up an on-farm crop trial

Part 2: Maximizing your on-farm crop trial

The final step in making a successful on-farm trial is to look at the data in more depth than just the averages. Averages are used a lot in product advertising and as a "easy-button" approach to answering a question. "Statistics" is often a term farmers prefer not to use, but they can be vital in understanding the results of a trial. Though statistics and math sometimes seem daunting, there is plenty of assistance to be found, and the results will improve confidence in the conclusions you reach.

Comparing treatments from a field, the measured yields may very well be different, even when the strips are adjacent to each other. Treatment differences may be caused by several factors other than the treatment, including soil variation; soil fertility; moisture availability; insect, weed or, disease pressure; differences in planting or harvesting techniques; or field history.

When at least three replications of a trial are conducted, statistics can be used to determine if variations are attributed to the treatment or to factors not controllable by the on-farm trial. There is quite a bit of difference in looking at data for averages versus statistical significance.

Statistics provide a rigorous process for comparing treatments, with p-values being the key matrix used to determine if differences can be attributed to treatments or to some other factor. The lower the p-value, the more likely it is that the treatments are affecting the independent variable (often yield).

Common benchmarks for p-values determined from on-farm trials are 0.10 and 0.05. If an experiment has a p-value of 0.10, it is said to be 90% confident that differences observed are in response to the treatments. Likewise, if an experiment has a p-value of 0.05, it can be said with 95% confidence that the differences observed were in response to the treatments. Averages are the center or middle of a set of numbers.

Averages are commonly used in agriculture as a simple method to determine if one treatment is different from another. They are the basic middle or center value from a group of like-meaning numbers, such as yields. However, averages can easily be manipulated and not tell you the whole story of a field.

Getting past averages

In the example on-farm trials below, we have two fields of corn that both received treatments of 200 pounds of nitrogen per acre and 125 pounds of nitrogen per acre. The formulation, type and timing of nitrogen application was consistent across both fields, with randomization and replication also utilized to help reduce variation.

Courtesy of Iowa State University Extension

The results taken from this example tell a story of the need for statistical analysis. Alone, on averages, both fields appear to show that the 200 pound-per-acre nitrogen treatment outyielded the 125 pounds acre by 21.5 bushels per acre. This would make for an easy management decision based solely on averages.

In Field 1, you can observe each replication of the 200 pounds per acre outyielded the 125 pounds per acre, which is consistent with the averages assessment. Field 2, however, we see the same yield averages but a vastly different story in the individual replications. In Field 2, we see that in Rep 1 and 2, the 125 pounds per acre outyielded the 200 pounds per acre, while reps 3 and 4 displayed the opposite at a wide margin.

Which replications are too be believed? This is where statistical analysis over averages tells you the story of how confident you can be in your decision-making process with the data collected in the farm trial. For Field 1, there is a p-value of 0.03, which means there is 97% confidence in the treatment differences, as there was consistency across replications. Field 2 only had a 75% confidence, with a p-value of 0.26. If a farmer finds being accurate 75% of the time works, then both results are valid for their operation.

Statistics can be complicated and are often underused in on-farm trials. Raw number comparisons and averages can show differences, but these often mask the real answer to the question. Statistical analysis is often the reason why on-farm trials and their data can be either highly valuable — or not. There are many freely available programs to run statistics or partnering, with organizations such as Iowa State University or other entities, to gain an understanding the true results of on-farm trial data. They will increase your confidence in your tested products.

Treatments that are not statistically different still have value. It is important to remember the data are still extremely useful because they tell us the treatments did not produce a difference in yield (or other factors measured). Simply knowing this can help us learn more about the conditions and performance of a given field, as well as aid future management decisions. Zero difference is just as valuable as a measurable difference in yield results.

Sharing the data

The final tip is that knowledge is the greatest power anyone can have. The more farmers share the data they collect, the more excited others will be to conduct a similar on-farm trial or a different on-farm trial. It is contagious.

More locations and more trial types result in more local information and knowledge to use in making management decisions. Citizen science for farmers has been, and will always be, a key component in increasing farmer knowledge. All products and practices will never work on all acres.

Therefore, farmers must always strive and look for new and innovative ways to improve their operations. On-farm trials are the best way to get results of products and practices in your local area or on your farm to answer the questions that matter to your operation to improve your profitability.

Witt is an Iowa State University Extension agronomist based in Guthrie Center.