How do we know that IntelliMetric® works?
In order to evaluate whether IntelliMetric® is able to accurately score essays, we put IntelliMetric® to the same tests that we would an expert human rater. After a human rater is trained, he/she is asked to score a set of essays that have previously been scored by experts. The agreement rate between the new scorer and the “known” scores are compared. If the human rater meets the criteria for acceptable agreement, the human is allowed to score new essays. Similarly, after IntelliMetric® is trained, it is asked to score a set of essays that were previously scored by experts. Just as in the human scoring process, we look at the agreement between IntelliMetric® and the expert scores. If IntelliMetric® meets or exceeds the standards for a human rater, IntelliMetric® is able to be put into use to score new essays. In short, IntelliMetric® is treated much like any expert scorer on the team when evaluating for consistency and accuracy. It must meet or exceed the same high benchmarks of qualit
