How does the IntelliMetric® artificial intelligence scoring engine work?
IntelliMetric® emulates the process carried out by human scorers. The system must be “trained” with a set of previously scored responses containing “known score” marker papers for each score point. These papers are used as a basis for the system to infer the rubric and the pooled judgments of the human scorers. The IntelliMetric® system “internalizes” the characteristics of the responses associated with each score point and applies this intelligence in subsequent scoring. The approach is consistent with the procedure underlying holistic scoring. IntelliMetric® creates a unique solution for each stimulus or prompt. This is conceptually similar to prompt-specific training for human scorers. For this reason, IntelliMetric® is able to achieve both high correlations with the scores of human readers and matching percentages with scores awarded by humans.
