Most people don’t enjoy statistics. The “On-Farm Trial Guide” discusses converting trial data into actionable knowledge, based around the use of the ANOVA tool in the “Data Analysis Toolbox”. It’s easy.

Simply comparing the averages from each treatment only gives you a small part of the story. Statistics are just a tool to help you to make sense of trial results.

1             Re-check that data is complete and accurate.

No data is better than wrong data. Make sure that you keep the rubbish out by checking over your data sheets before you start playing statistician.

The first thing to check for is missing data. If you use the pre-made data sheets, missing data should show up fairly readily as gaps in your results.

If you can, it’s best to fill the gaps made by missing data. There are several approaches that you can use.

• The best approach is to try to find the missing data. Was it written somewhere else?
• Second best is to see whether you can reconstruct the missing data using other measurements that you’ve made. For example, if plot area is missing it can be calculated from recorded measurements of plot length and width.
• As a very last resort, you can consider filling the gap with the average value obtained from the other replicates of that treatment. Do not under any circumstances do this for more than 1 out of 4 replicates. Having 25% of your data ‘fake’ is bad enough!

2             Enter data.

First you have to decide which measurements need to be statistically analysed and which don’t. Second is typing the numbers in. You should enter and statistically analyse all relevant qualitative (number) data. You should evaluate and analyse the implications of qualitative (observations or impressions) data.

Whether or not the data requires statistical analysis, you should analyse it. For observations, just think about what the data means and record your thoughts.

Quantitative or numerical data such as yield, biomass, test weight, moisture content, bulk density and lodging should be analysed statistically using the ANOVA tool in the “Data Analysis Toolbox”. You can do statistical analysis on these because they are numbers. You should do analysis on them because it will help you to get the right answer from your trial.

Qualitative data can’t be statistically analysed using standard techniques, because they aren’t numbers. Scores such as ‘poor’, ‘good’ and ‘very good’ are qualitative data that you can’t enter and analyse statistically.

3             Statistically analyse the data

The “On-Farm Trial Guide” ANOVA tool does all of the number crunching and interpretation of statistics for you. It will help you to work out:

• The difference in ‘size’ (eg. yield, biomass) between your two treatments
• Whether or not this difference was caused by a real treatment effect or simply by chance
• The extent to which you can trust the results as the basis for decisions about your farm system

The ANOVA tool does this by giving you 3 pieces of information:

The P-value is the probability that the trial results occurred due to chance, rather than as a result of the treatments that were applied. A P-value of 0.05 means that there’s a 5% chance that the observed difference between treatment A and B was just luck. Looking at it from another viewpoint – there’s a 95% chance that the observed difference between your treatments was caused by the treatments.

The LSD (least significant difference) lets you see whether two numbers really are significantly different from one another. If the difference between your treatment averages is greater than the LSD, you can be confident your treatments really are different.

Thirdly, the ANOVA Tool gives an interpretation of the P value and LSD based on the above.

While essential for understanding your trial, the results from the ANOVA tool have a number of limitations:

• They can only look into the past, not the future. They tell you about what has happened in your trial. They cannot tell you what would happen if you did the trial again.
• They apply only to the conditions in which the trial was conducted.
• They do not give information about why results occurred; only what results occurred.

 Summary:

The ANOVA tool is an essential part of your trial data analysis, because it will help to show how much faith you can have in the results. Differences between treatments can’t be relied upon without P-values and LSDs to support them. If you can’t be at least 95% certain that treatment differences are real (with P-value less than or equal to 0.05), then you should take the view that the treatments may have had no effect, no matter how different they appear to be.

Where statistical analysis tends to be cut-and-dried, analysing the implications of your data relies on judgement. Your statistical analysis will provide you with 3 basic outcomes:

• The results indicate that there was a significant difference between the treatments. You can be confident that treatment A really was different from treatment B.
• The results indicate that some indecision is in order. The treatments weren’t different enough for you to be really confident (eg. 80%) about a significant treatment difference, but nor are they so similar that you’d treat them as alike.
• The results indicate that there was no reliable treatment difference (P-value 0.30 or more).

Each alternative suggests a different set of possible action plans.

If you found significant differences between your treatments:

You can be confident that your treatment differences were real.

You now need to decide what the implications of this result are for your farm operation. Just because one treatment is ‘more’ than another doesn’t mean that it’s ‘better’.

For example, if you did a population trial and found that yield was significantly increased by a higher population, you might want to consider a wide range of other factors before adopting the practice on part or all of your farm.

• What was the effect on profit? You’ll need to do some sums to be certain that higher yield equals higher profit.
• How much consideration should be given to other variables? Did lodging increase? Was this an acceptable increase or is it likely to enlarge your exposure to risk? What happened to quality?
• What is the seasonal context of the results? For instance, if lodging did go up, but just a little, was it because the season wasn’t wet and windy? Would lodging become a problem in those circumstances?
• Scale. If it looks like the new treatment increases both profit and risk, should it be adopted on all or part of the farm, or not at all?

If you found that some indecision is in order

You cannot be confident that your treatment differences were real.

• You needn’t reject a new treatment just because you can’t be confident that it is statistically different from an alternative. As outlined above, the new treatment may have other advantages that make it attractive, despite the lack of significant yield or other benefits.

It’s possible that two treatments had a big numerical difference but only a small or no statistical difference.

• Again you needn’t reject the new treatment, but you might want to proceed with caution.

Large but inconclusive (not significant) differences between treatments usually mean that the data is variable. That is, one or both of the treatments is a bit hit-and-miss.

• When this occurs it is worth examining why the treatments were variable. This will give you some insights into whether the treatments are inherently risky, or whether they varied in response to difficulties with the site, season or trial execution. If you can understand the factors that led to variability you may be able to reduce or avoid them and increase the certainty of obtaining a favourable outcome.
• If you found that there was no reliable difference between treatments
• This is usually pretty conclusive. If there’s no reliable statistical difference your preference between treatments will have to be based factors other than those for which you’ve done statistical analysis.
• For instance, if you find that treatments A and B do not have a statistically reliable difference in yield, then preference will come down to factors such as cost, risk, convenience and ‘gut feel’.
• A ‘no difference’ result may be exactly what you’re after. This will occur particularly where you’re looking at treatments designed to cut costs, such as reduced rates of pesticide or fertiliser application. If you can reduce input costs with no significant effect on yield, you stand to increase profit – in the short term at least.

End of section critical decision point

Have you learned something new about how your crop ticks? Ultimately, this is more valuable than the specific results from your trial. While an individual trial is, by necessity, narrow in focus, the benefits of conducting trials are usually wide-ranging. Hopefully, the knowledge that you’ve gained will help you to improve some part of your cropping system.

Get hold of (and read!) the On-Farm Trial Guide Booklets and use the templates and ANOVA tool.