What Does Probability Level Mean?
If we declare two averages are “significantly different” at 5% probability level or P = 0.05, we are saying that we are willing to make a mistake one out of 20 times if in fact they are truly equal. The 5% probability level is the standard used for most field trials. However, 5% may be too conservative or overly cautious for some farmer-researchers. In some on-farm research trials, it may be decided that a wrong decision may not be very costly. This could be the case where treatment costs are essentially the same, e.g., seed costs in variety comparisons. It may be decided to use a probability level of 10% if one is willing to make a mistake one out of 10 times, or 20% for a risk of one out of five. Picking the probability level is a “decision rule.” Increasing the sample size or replicates reduces the chances of making an incorrect decision when the same decision rule is applied. In on-farm research trials, experience has shown that five to six replicates are usually needed to detect m
