Mark Lawson calls it avoiding doing an autopsy on results – just analyzing the numbers. You need to know what was happening to the plot during the season. Lawson, Danville, is a Syngenta agronomist.
Here's an example that makes the point. It is from the Indiana Prairie Farmer and Purdue University Throckmorton farm/Tippecanoe County Extension Service plot this year. The goal was to test downforce settings in soybean planting at two different depths.
Instead of including depth as a factor, four downforce settings were randomized and replicated within each depth, says Jeff Phillips, Tippecanoe County Extension educator. The whole block of one depth and the whole block of the second depth were treated as two different tests, planted the same day into very dry soil in early June.
Looking within each block, the heaviest downforce produced the highest yield, although results were not significant at 0.1 least significant difference. In the 2-inch depth, the heaviest and lightest downforce were likely significantly different at 0.2, a level that allows for more experimental error, Phillips notes.
The striking difference is that the one-inch depth block averaged 53.6, and the two-inch depth averaged 46.8 bushels per acre. However, this is where knowing what went on during the season helps explain the results and prevents one from drawing an erroneous conclusion - deeper planting made a huge difference in yield.
"The two parts of the field used for the different block had been handled differently in the past- this is a research farm," Phillips says. "From the very start, weed pressure was much more intense on the two-inch depth plot. It was largely from lambsquarters. Weed control was better all season in the one-inch block, while they struggled to control weeds in the two-inch block. If any of it was due to differences in emergence, it was likely very slight."
Phillips concludes that weed control, not any treatments within the experiment, likely accounted for differences within yield between blocks. Can he prove it? No. Does it cause him not to conclude that planting depth made a difference? Yes!