What is the smallest significance level, when applied to the entire family of comparisons, at which this particular comparison will be deemed statistically significant?
The idea is pretty simple. There is nothing special about significance levels of 0.05 or 0.01… You can set the significance level to any probability you want. The adjusted P value is the smallest probability at which a particular comparison will be declared statistically significant as part of the multiple comparison testing. Say that a particular comparison is statistically significant at the 5% level but not at the 1% level. The adjusted P value will be between 0.01 and 0.05, say 0.0323. Each comparison will have a unique adjusted P value. But these P values are computed from all the comparisons, and really can’t be interpreted for just one comparison. Don’t confuse multiplicity adjusted individual P values with invidual P values that are not corrected for multiple comparisons at all (the unprotected Fisher’s LSD test).
