|Return Tillage Handbook|
Conservation Tillage Handbook Series No. 9
LSD -- A common statistical tool for on-farm tests is a LSD "Least Significant Difference," which is calculated through standard Analysis of Variance (ANOVA) statistical analysis software or can be calculated by hand. The LSD is a calculation based on the variability of treatment results within the trial and is used to help separate the effects of natural field variability from the treatment effects. If the difference between treatment means in a trial are equal to or larger than the LSD, the difference is statistically significant and believed to be due to the treatment effect and not natural field variability. On the other hand, if the difference between treatment means is smaller than the LSD, the differences are more likely due to natural field variability. The LSD is only used if the ANOVA analysis determines that treatment differences are significant. In the example in Fig. 2, the LSD represents yield in bu/acre, but it can be calculated for any treatment effect you might be interested in comparing.
Probability Level - The subscript number following the LSD refers to the "probability level" at which the LSD was calculated. In Fig. 2, the probability level is 0.05, or 5%. For example, if the difference between treatment means is equal to or larger than the LSD at 0.05, then you are 95% sure that the difference is due to treatment effects and not to natural variability. The smaller the probability level, the greater confidence you can have that differences between treatment means are due to treatment effects. A typical statistical probability level is 0.05, but it can feasiblely range from 0.001 to 0.2 (0.1 - 20%), depending on the economic or environmental implication of choices between management options being tested, interactions with other managements practices, the grower's experience and other factors.
Replication, meaning repetition of each treatment area, provides a critical tool to overcome the fact that any two plots under the same management will not have exactly the same yield, stand count, weed population, fertility, or any other factor because of natural variability. In other words, replication helps you to determine if differences between plots are due to treatments or due to natural field variation. In statistical jargon, this normal variation is called "experimental error." Replication is based on the theory that if one practice is superior to another, it will become evident if you make repeated comparisons.
As part of the STEEP II OFT project in Washington, Oregon and Idaho, field uniformity trials with eight side-by-side combine strips in 14 wheat and barley fields illustrated considerable yield variability even within selected "uniform" areas of fields. We found that yields of combine strips side-by-side (full header cuts with a space between strips) commonly varied from 1 to 10 or more bushels per acre. Fig. 1 show yield variances as great as 9 bu/acre in side-by-side combine strips and 15 bu/acre across the 8 strips in a uniformity trial near Moscow, ID. The further apart the combine strips, the greater the yield differences commonly were in all these trials. This level of field variability emphasized the need for replication as a key step to a successful OFTs.
Fig. 1. Winter wheat yields in eight, 500-foot combine strips in a "uniform" area of a field near Moscow, Idaho in 1992. Full header-width cuts 20 feet wide were harvested in each side-by-side 25 ft X 500 ft plots. Yields of adjacent strips varied as much as 9 bu/A.
Our Northwest research has shown that four replications of the side-by-side comparisons usually give the best chance of success for the amount of effort. In Fig. 2, the LSD required to have significant differences between mean treatment yields on OFT with plots 250 ft long decreased from about 37 bu/acre with 2 replication to 10 bu/acre with 3 replications and 7 bu/acre with 4 replications. Five or six replications can give a slight gain in statistical power in separating treatment differences, but may not be worth the extra effort in some trials. Three replications are less precise in determining treatment differences than four replications, but may be adequate for some management practice comparisons. The danger of starting with three replications is that if you loose data from one plot, you no longer have an effective trial.
Generally, the longer the plots are, the better the results are likely to be, but that depends on the field landscapes, soil variability and yield levels. Figure 2 illustrates how the ability to detect significant difference between treatment yield results increases with increasing plot length. For example, increasing plot length from 250 ft to 1000 ft decreased the LSD from about 10 bu/acre to 5 bu/acre with 3 replications and from about 7 bu/acre to 3 bu/acre with 4 replications. Longer plot lengths typically allow the use of a lower LSD to determine whether or not treatment results are significantly different. The importance of plot length is also influenced by yield. For example, when cereal yields are under 60 bu/acre, a 750 ft or longer plot length would be the best choice. At higher yields, you can achieve accurate comparisons with shorter plot lengths of around 300-500 ft because you have larger volumes of grain from each plot. If you have the space, longer plots will generally give you less trial variability and greater confidence in your results.
Fig. 2. Effects of replication number and plot length on the Least Significant Difference (LSD) values at the 5% statistical probability level based on data from 10 field uniformity trials in Idaho, Oregon and Washington with less than 5 bu/acre yield variance. (Adapted from Wuest et al, 1994).
Selection of treatment locations in a field comparison must be fair or "unbiased." This might seem obvious, but there are many ways to consciously or unconsciously give an advantage to one of the practices being compared. One easy way to choose which of two practices or treatments go in which plot is to flip a coin! This is called "randomization", and the theory is straightforward. Once you have chosen plot areas, that as far as you can tell should perform the same, the logical way to convince yourself and others that you did not consciously or unconsciously favor one of the treatments is to assign them at random.
Remember that there are often gradients of soil and topographic factors across fields that can create gradients in yield and other factors of interest, some you can see and some you can not. For example, on a field slope, yields typically increase downslope with increased soil depth, organic matter content, soil moisture and so on. If the same treatment is always placed on the upper or lower sides of side-by-side comparisons down the slope (or other field gradient factor), it could create an unfair yield bias in the trial and the results would be misleading. Figure. 3 provides an example of the importance of randomizing the order of treatments in OFTs. Canola yields progressively increased downslope in a dry spring that limited overall yields and reduced the potential for a canola response to boron fertilizer. Having the same order of treatments in each replication would have resulted in entirely different conclusions. The randomization of treatments in each replication prevented drawing misleading conclusions from this fertilizer trial because of the yield gradient in the trial.
|Boron Solution #1 560 lb/acre|
|Boron Solution #2 630|
|Untreated Check 726 Rep 1|
|Boron Solution #2 775|
|Untreated Check 822|
|Boron Solution #1 921 Rep 2|
|Untreated Check 992|
|Boron Solution #1 1017|
|Boron Solution #2 1020 Rep 3|
Fig. 3. Yield of Canola with three different boron fertilizer options in an OFT with three replications near Craigmont, Idaho in 1992. Dry spring conditions limited yields and largely overshadowed any crop response to boron fertilizer treatments. Because of the progressive increase in yield down slope (~7% slope), using the same order of treatments in each replication would have biased the results. Fortunately, treatments were randomized in each replication.
Randomization helps to remove any treatment bias due to gradients in field characteristics that can affect yield or other factors being measured. If you are aware of a potential gradient in field condition that could affect the results of an OFT, arrange plots so all the treatments cross the field variability as equally as possibly.
The statistical design generally most appropriate for replicated, randomized OFTs with field-scale equipment is called "Randomized Complete Block" design (see Fig. 4 for two examples). As explained earlier, the "randomized" part of the complete block design means that the order of each treatment in each block or replication is chosen randomly to ensure no bias in assigning treatments to plots. The "complete block" means that each of the two or more treatments are included in side-by-side comparisons in each of the trial "replications or blocks." Because field variability generally increases with distance across the field, data variability is decreased through establishing the replications of side-by-side comparisons. Variability in results due to natural variability between replications can then be separated in the statistical analysis of the results.
The first step in designing an OFT is to decide clearly what you want to learn. This can take more thought than it might appear. Determine what you really need to know before you can make a decision whether or not to adopt the new practice or product. Design your test to provide that information.
Briefly, the steps to laying out a scientific OFT commonly include: 1) choose an area in a field where long, side-by-side plots can be placed with the expectation that the yield (or weed pressure, or other factors to be measured) should be nearly equal; 2) assign the treatments to the plots randomly, such as with a coin toss; and 3) repeat the above process so there are at least four replications. The four replications could be next to each other, or in different areas of the field, or even in different fields (Fig. 4). The best results occur when each replication is positioned so that variations in the field (high and low areas, soil variations, field borders, fertilizer overlaps, etc.) will be encountered equally by each strip in the replication.
Make a map of the field and plot locations, and keep notes on what you observe throughout the trial year. After all the field operations (spraying post emergence herbicides, etc.) mark your plots with stakes tall enough to be easily found at harvest time.
When data measurements are taken, such as stand counts or yield, record them separately for each strip. At harvest, cut a full header width out of the center of each plot (plots wider than the header), and weight the grain from each plot separately. Also measure the length of the harvested strip and header width to calculate the area of each plot. Portable truck scales or weigh wagons typically provide 5 to 10 pound accuracy per load cell in weight measurements. Consequently, your proportional error due to weighing equipment accuracy will be greater with smaller plots, particularly if yields are low. You may want to collect grain samples for moisture content, test weight and quality measurements.
Fig. 4. Potential field layout options for an OFT with two treatments (C = check; N = new practice) and four replications in a randomized complete block design. The bottom layout illustrates a common side-by-side arrangement for all four replications. The top layout illustrated an end-to-end arrangement that can potentially permit faster establishment and harvest of some trials.
With new combine yield monitors and distance measuring devices, OFT may become faster and simpler for growers, but there is still considerable debate whether they are accurate enough for yield comparisons in replicated trials. An article in PrecisionAg Illustrated (Dunn, 1997) provides an overview of some of the advantages and disadvantages of combine yield monitors in field comparisons of management options. Basic OFT principles still need to be followed regardless of harvesting method.
If comparisons of erosion control or soil water storage are important questions in an OFT on sloping cropland, the trial design typically involves adjacent wider blocks instead of long narrow strips. Each plot needs to include much of field slope, starting at the top of the "watershed area" of the field. This means that plots cannot be placed one above another on a slope to prevent the lower plot from receiving runoff from a plot above it with different practice. Each plot must run from the top of the slope down far enough to see how it handles water absorption and runoff. At the same time, if the practices are usually conducted on the slope contour, it should also be done that way in the plots.
Use of basic experimental methods and designs discussed so far are critical to achieving accurate results from OFTs. Once you have collected the data you are interested in, then statistical analysis of the results can separate the effects of natural field variability from the treatment effects for making correct conclusion and management choices. Assistance in analyzing the trial data is often available through your county Cooperative Extension Agent. Even without statistics, a lot can be learned from observation of different treatments to see if one is consistently better than the other in each of the replications.
An easy-to-use statistics computer program called AGSTATS (for IBM compatible computers) is available from Oregon State University for analyzing the results of simple field trials. To request a copy, send a blank 3.5 inch diskette and return postage mailer, or send a $5 check payable to the Ag Research Foundation, addressed to Russ Karow, Extension Agronomist, Crop Science Building 107, Oregon State University, Corvallis, OR 97331-3002. You can also request a free AGSTATS program file sent by e-mail from Russ Karow at (email@example.com).
The data can be statistically analyzed using a hand calculator and step-by-step formulas. "On-Farm Research Guidebook" is a good reference on OFT design and statistical analysis of the data. To request a copy, contact Dan Anderson, Department of Agricultural Economics, 305 Mumford Hall, 1301 W. Gregory Drive, University of Illinois, Urbana, IL 61801.
There are a number of publications and an Internet site that can provide additional information and ideas on on-farm testing. If you have additional questions, contact one of the authors (phone numbers and e-mail addresses on the cover page).
Internet home page on Pacific Northwest On-Farm Testing - part of the Pacific Northwest Conservation Tillage Systems Information Source home page (http://pnwsteep.wsu.edu).
Pacific Northwest On Farm Test Results publications (annual reports from 1991 through 1995) - limited supply of these annual reports are available that describe nearly 250 OFTs. Some are also on the home page above. Request them from WSU Crop and Soil Sciences Dept. Extension Office (509) 335-2915 (e-mail firstname.lastname@example.org or email@example.com)
Idaho On-farm Test Results (annual progress reports of no-farm tests in northern Idaho beginning in 1996). Limited supplies are available. Contact Stephen Guy (208-885-6744; e-mail firstname.lastname@example.org).
On-Farm Testing: A Grower's Guide -- Washington State University Cooperative Extension bulletin EB1706. 1992. A guide to designing and conducting OFTs. Includes forms for record keeping. Order from WSU Cooperative Extension Bulletin Office (509) 335-2999 (e-mail email@example.com; home page http://caheinfo.wsu.edu).
On-Farm Test Record Form -- Pacific Northwest Extension bulletin PNW487. 1995. A convenient form to simplify planning and record keeping for on-farm tests. Order from WSU Cooperative Extension Bulletin Office (509) 335-2999 (e-mail firstname.lastname@example.org; home page http://caheinfo.wsu.edu).
Using an On-Farm Test for Variety Selection -- Pacific Northwest Extension bulletin PNW486. 1995. Specific instructions and considerations for performing on-farm variety comparisons. Order from WSU Cooperative Extension Bulletin Office (509) 335-2999 (e-mail email@example.com; home page http://caheinfo.wsu.edu).
Dunn, Richard F. Jr. 1997. You Can Do Homegrown Research. In March-April 1997 PrecisionAg Illustrated.
Hicks, D.R., R.M. Vanden Heuvel, and Z.Q. Fore. 1997. Analysis and Practical Use of Information from On-Farm Strip Trials. Better Crops with Plant Food. Vol. 81, No. 3. Potash and Phosphate Inst. ISSN-0006-0089.
Wuest, S.B., B.C. Miller, J.R. Alldredge, S.O. Guy, R.S. Karow, R.J. Veseth and D.J. Wysocki. 1994. Plot Length Influences on Experimental Error of On-Farm Tests. Jour. Production Agric. Vol. 7. No. 2.
Pacific Northwest Conservation Tillage Handbook Series publications are jointly produced by University of Idaho Cooperative Extension System, Oregon State University Extension Service and Washington State University Cooperative Extension. Similar crops, climate, and topography create a natural geographic unit that crosses state lines in this region. Joint writing, editing, and production prevent duplication of effort, broaden the availability of faculty, and substantially reduce costs for the participating states.
The Pacific Northwest Conservation Tillage Handbook is a large, three-ring binder handbook that is updated with new and revised Handbook Series publications. It was initiated in 1989 as a PNW Extension publication in Idaho, Oregon and Washington. Updates to the Handbook are provided when the updating card is returned. By 1999, 47 new PNW Conservation Tillage Handbook Series have been added to the original 98. Copies of the complete Handbook are available for $20 through county extension offices in the Northwest or ordered directly by calling state extension publication offices: Idaho -- (208) 885-7982; Oregon -- (541)-737-2513; Washington -- (509) 335-2999 (some shipping and handling charges and sales tax may apply). It's now accessible on the Internet! All of the PNW Conservation Tillage Handbook and Handbook Series are being put on the Internet home page (http://pnwsteep.wsu.edu) Pacific Northwest STEEP III Conservation Tillage Systems Information Source. The home page also contains recent issues of the PNW STEEP III Extension Conservation Tillage Update, listings of other conservation tillage information resources, coming events and much more. For more information on the Handbook or updates to the Handbook, contact Roger Veseth, WSU/UI Conservation Tillage Specialist, Plant Soil and Entomological Sciences Department, University of Idaho, Moscow, ID 83844-2339, phone 208-885-6386, FAX 208-885-7760, e-mail (firstname.lastname@example.org).
Cooperative Extension programs and policies comply with federal and state laws and regulations on nondiscrimination regarding race, color, gender, national origin, religion, age, disability, and sexual orientation. The University of Idaho Cooperative Extension System, Oregon State University Extension Service and Washington State University Cooperative Extension are Equal Opportunity Employers.
us: Hans Kok, (208)885-5971
Accessibility | Copyright
| Policies | WebStats | STEEP Acknowledgement
Hans Kok, WSU/UI Extension Conservation Tillage Specialist, UI Ag Science 231, PO Box 442339, Moscow, ID 83844 USA Redesigned by Leila Styer, CAHE Computer Resource Unit; Maintained by Debbie Marsh, Dept. of Crop & Soil Sciences, WSU