How does automatic scoring of spoken language work?
During field testing about half a million responses generated by more than 10 000 native and non-native speakers with 126 different home languages have been gathered. These responses are scored by a pool of human raters on a number of traits (content, vocabulary, fluency, pronunciation). The human scores on a large subset of the responses serve to train the automatic scoring system. The system builds acoustic models and language models for the responses and compares actual responses with the human scores these receive. Based on this large data set the automatic scoring system ‘learns’ what the correspondence is between characteristics of the responses and the human scores that are awarded to these responses. Once the machine has been trained the machine independently scores responses that were not used in the training. These independent machine-generated scores are then compared with the human scores awarded to these responses. If they correlate high enough (typically > 0.90) the autom
