Can Machine Grading Replace Human Grading of Answers to Constructed Response Tests in Large-Scale Assessment Programs?
Computerized scoring technology offers advantages, such as increased speed and reduced costs, over human grading of essay responses, but some researchers are concerned about the level of agreement between human and machine scoring. Kathy Comfort, Principal Investigator and Director of the Partnership for the Assessment of Standards-based Science at WestEd, with colleagues from the RAND Corporation and Research Solutions Group, will present two studies addressing these concerns. In the first study, two machine-scoring companies developed algorithms for scoring 669 9th graders answers on one performance task and two constructed response tasks. The results showed that there can be a high degree of agreement (in means, standard deviations, and correlation) between hand and machine scores, but that level of agreement may differ by task and by machine scoring tool. The second study repeated the first one, using a new sample and applying one of the scoring algorithms from the first study. The
Related Questions
- Have I understood correctly that tutorial answers can be uploaded for grading, but that their grades will not be counted in the final assessment?
- Can Machine Grading Replace Human Grading of Answers to Constructed Response Tests in Large-Scale Assessment Programs?
- Can a machine translation replace a human translator?
