Good trial design is the foundation stone of reliable trial results. A good trial design is as simple as following the 4 Rs – relevance, replication, randomisation and arrangement.

Relevance suggests that you consider two things when selecting trial sites. First, that you undertake trials in conditions that suit the treatments that you’re examining. Second, that you select sites that represent most of your property.

Replication is a way of increasing the certainty of any treatment comparisons. Four replicates gives 4 head-to-head comparisons of each treatment, and provides an ideal balance between reliability and effort.

Randomisation is just like shuffling a deck of cards. In trials, the aim is to make sure that each treatment has an equal chance of being placed in good or not-so-good conditions.

Arrangement is the art of matching the trial design with the trial site. Trial sites are always variable because soil isn’t uniform and because paddock characteristics are influenced by hillsides, headlands shelter belts, gateways, fence lines, drains and other sundries. Many on-farm trials fail to make valid treatment comparisons because this variability isn’t adequately taken into account when laying down the trial.

Pay special attention to plot width when applying fertiliser treatments. Spreaders aren’t always uniform across their width and edges can easily be under- or over-fertilised depending on the degree of overlap of spreader runs. Ideally plots would be wide enough to allow measurement from the more uniform centres of spreader runs.

MORE

Relevance

Finding this combination of features requires some planning, so select your trial site well before the rush of the cropping season begins. This will enable you to avoid some common pitfalls…

More often than not, on-farm trials are located in funny little paddocks that aren’t good enough for ‘real’ crops. If a site isn’t good enough to farm profitably, why use it to make decisions about farm profit?

Ideally your trial site would be representative of your property as a whole. That way the results will be most relevant to the majority of your business.

Consider giving your trial site priority over your commercial crops. A commercial crop pays you only once, whereas the results from a reliable trial could pay dividends for years.

Failure to recognise and minimise variability in trial paddocks is a common problem. Variable sites can ruin trials by making it impossible to distinguish effects of treatment from effects of variability caused by other factors. Have a look at your commercial crops and avoid putting trials in areas that have had variable establishment, growth or yield (unless you’re wanting to investigate the sources of this variability).

You can minimise variability by avoiding corners of paddocks, which often receive multiple applications of fertiliser or herbicide and are less likely to represent the property than other parts of a paddock.

Stock camping areas, gateways, water troughs, headlands, areas where water gathers, steep slopes, trees, weedy patches, old fence lines or tracks, sheds or other irregularities are best given a wide berth. Hopefully, they’re not like most of your property, so you’re unlikely to get relevant trial results near them.

Replication

The outcome of any given sports match isn’t certain, partly because combinations of opponents and grounds affect the outcome. Some teams play better against a particular style of play and some teams have a marked home ground advantage.

In trials, replication aims to increase the certainty of betting. By having more than one head-to-head comparison, it enables each treatment to play each opponent more than once. In addition, by having treatments repeated at different times or parts of the paddock, it includes at least one ‘home’ game and one ‘away’ game.

Replication refers simply to the number of times that you repeat a treatment using separate plots. Two replicates is where you have two separate plots for each treatment. Three replicates is three separate plots for each treatment, and so on.

Four replicates usually provides the best balance between precision (the ability to separate effects of treatment from other effects, such as site variability) and the amount of work involved. Three replicates might give 25% less work, but it gives about 35% less precision. Five replicates, on the other hand gives 25% more work but only about 15% more precision than four replicates. So four replicates is a good compromise.

Trials without adequate replication are highly unreliable and could harm your business. Four replicates provides a good balance between effort and reliability.

You can achieve replication using space or time, or a combination of the two. Each method works by giving head-to-head comparisons under slightly different conditions.

An example of replication over time is where two treatments are compared head-to-head in the same paddock for four years.

An example of replication over space is where two treatments are compared head-to-head in four different paddocks in one year.

The best method of replication depends partly on the type of business that you run.

Randomisation

Randomisation serves two basic functions. First, it enables you to repeat a head-to-head comparison under a variety of slightly different conditions. Second, it ensures that no treatment has an unfair advantage by always receiving the best spot(s) in the paddock.

You can see that the thoughtful use of replication in blocks and randomisation of treatments within blocks has vastly improved the pretty lousy trial design that we started with in Fig. 1.

The improvements that we’ve made didn’t involve much work. All we had to do was (1) think a little about the characteristics of the trial site, (2) put blocks along the trial ‘path’ and (3) randomise the treatments within the blocks.

This little bit of ‘paper’ effort has paid big dividends. Previously, the trial was effectively ‘rigged’ to show that treatment B was best. Now, we’re in a position to more reliably assess whether treatment A or B is best.

aRrangement

The arrangement of treatments, plots and blocks on the site is as important as the site itself.

The clever use of replication and randomisation has massively reduced the bias that would have occurred had we used the design outlined in Fig. 1. By the time we’d reached the latest design (Fig. 3), we’d blocked the treatments to provide 4 head-to-head comparisons instead of just 1, and we’d randomised the treatments within the blocks to reduce the unfair advantage that consistent allocation to better soil had given to treatment B.

But our trial design (Fig. 3) isn’t perfect yet. The arrangement of the treatment plots within each block still means that one treatment in each pair is assigned (by chance) to better soil conditions than the other. Even though random favouritism of this kind is better than the inherent bias, it’s still not ideal.

A simple trick for banishing this ‘random’ bias is to re-align the plots to better match the trial site. As a first step, we’ll return to the image of the trial as a path. When variation can’t be avoided, blocks should be arranged so that they form a trial path that progresses from block 1 to 4 along the variation in the paddock rather than across it. By this means, each block is most likely to contain a uniform set of properties. Figures 4 and 4 show how this works.

The blocking used in Fig. 4 is correct because it makes a path that goes along the variation of the site, from poorer to better soil. Block 1 contains mostly poorer soil, block 2 is mostly medium-poor, block 3 is mostly medium-good and block 4 is mainly good soil. This arrangement allows each head-to-head comparison (within a block) to occur in reasonably uniform conditions.

This is the same form of blocking that we used in our latest trial design (Fig. 3). But, as we mentioned earlier, that design wasn’t perfect because it didn’t eliminate all of the bias in soil type. The last problem in our trial design can be eliminated by simply rotating the plots within each block. When this is done, the plots run across the trend in the paddock (at right angles to the blocks). This is shown in Fig. 5, below.

Our final trial design is an improvement on the Fig. 3 model. Each treatment within a head-to-head block not only has an equal chance of being assigned to poor or good soil, but each treatment within a block actually receives the same soil type. We can now be fully confident that any bias in our trial design has been minimised.

Arrange plots to form a path that runs across the trends in the trial site. This maximises the head-to-head value of comparisons within blocks.

Changes in soil type or quality aren’t the only things that will introduce trends or random variability in a trial site. The principles outlined above should also be applied to trials established on slopes, near shelter belts or any factor that could result in a trend in the trial site. Of course, if you can, it’s always lovely to pick a site without any obvious trends.

Even when you can’t identify trends in the trial site, it’s essential that you assume that the site is variable. This means that you must block the trial and randomise treatments within each block.

Plot orientation within each block becomes less crucial when a site doesn’t have any obvious trends in it. When this happens you can orient plots within each block so that the trial can be established, monitored and harvested most simply.

Summary:

Arrange treatments to minimise the effects of site variability. Place blocks along the variation and then arrange plots across the variation.

Is bigger better?

Most growers (and many scientists) believe that, for on-farm trials, bigger plots are better than smaller plots. This is simply not true.

Statistically speaking, there is no reason to believe that big plots give better answers than small plots. In fact, the reverse is often the case.

Bigger plots are more likely to contain more variation in soil and other conditions. Unless you can readily define what that variability is, bigger plots may just be creating more uncertainty about the results of the trial. Smaller plots, simply because they occur in a smaller area, tend to be less variable within themselves and, if blocked correctly, create less variability within a block. In addition, being smaller, it’s easier to characterise the variability within or between plots, should any exist.

Just because it’s convenient to sow plots that are full paddock length doesn’t mean that you need to make measurements on the full paddock length.

There’s no doubt that it’s much easier to sow a strip the full length of a paddock than to stop and start all over the place. It’s also easier to find a full paddock length strip than a plot buried in the middle of a heavy crop. The simple compromise between simplicity and reliability is to sow as long a plot as you find convenient, but to make measurements on only those parts of the plots where there aren’t variability problems.

Additional problems related to the ends of paddocks include:

• Gateways mean traffic, which can introduce compaction and different weed and pest populations
• Troughs, stock feeding and camping areas
• Drains provide different drainage characteristics, either because of the drains themselves or the subsoil dumped to the side of them
• Headlands often have different amounts of herbicide, pesticide, fertiliser, seed, cultivation and traffic on them

Headlands are a common problem because, depending on the patterns of cultivation and sowing, they can cover irregular parts of the paddock. A lot of paddocks are cultivated using ever-decreasing circles, giving the characteristic envelope pattern shown in Fig. 7.

When this is added to by one of the common headland sowing patterns (also shown in Fig. 6), the area ‘typical’ of the paddock as a whole is broken up into two big and separate chunks. To ensure that your trial results are representative of the paddock, it’s a good idea to confine your measurements to areas found within ‘typical’ chunks of land. The precise location of these areas will depend on the methods of cultivation and sowing that have been used, as well as the general characteristics of the paddock.

Summary:

• Replication is much more important than plot size in determining the reliability of trial results, so more replication of smaller plots is far better than less replication of larger plots.
• Harvest widths should be smaller than plot widths, to avoid or minimise edge effects.

Plot size, in itself, isn’t usually that important. You end up with the ideal plot size after you’ve ensured that (in order of importance): 1) you have 4 replicate blocks of each treatment, 2) the area within each block is uniform, 3) you can minimise edge effects and, 4) you can minimise measurement errors.

End of section critical decision pointDo you have a map of your trial design in your hand? Does it incorporate the best elements of replication (4 replicates), randomisation (treatments fully randomised within each block) and arrangement (blocks along the site variability and plots across the site variability)? Is the site where you want to enact this plan representative of your farm operation? Is it relevant to the treatments being applied? It is important that you can satisfy these questions if you are to get the best from the next section – Getting the trial site right