How many people need to be included in the job analysis?
This is a tricky question that I typically answer by telling my students to keep interviewing incumbents until you don’t hear anything new. Anecdotally, this seems to be after the third or fourth incumbent for a particular job. However, because my anecdotes don’t carry any weight with my wife, neighbors, or peers – my guess is that they won’t with the readers of this column either. So, let’s see what previous research can tell us. Perhaps the first way to answer this question is to look at the interrater reliability typically found in job analyses. That is, if this reliability is high, incumbents tend to agree with one another; thus using ratings from 30 incumbents would not be any more meaningful than ratings from five incumbents. A look at current job analysis instruments suggests that interrater reliability is probably about .75 For example, interrater reliabilities for the Position Analysis Questionnaire range across studies from .45 to .96 with a mean of about .72. The interrater
