How good is the solver?
Depending on exactly how you define solve you should see the tool find the correct answer for the vast majority of typical cryptic clues. As you add checked letters, the solving percentage rises until it reaches almost 100% The factors that influence the percentage are the quality/easiness of the clue; the number of checked letters; whether by ‘solve’ you mean listing the answer first, in the top five, top with a perfect explanation (etc.) As an example, one benchmark we use is a secret (i.e. not examined by any programmer to ensure the performance is the same as with new clues) list of cryptic clues from the Daily Telegraph (mid-ranking difficulty). The tool will currently produce the correct answer and list it right at the top of the list (i.e. a very strong definition of ‘solve’) 55% of the time with these clues. This rises to 68% if you count times when the answer is listed in the top five solutions. As you use the correct answers to fill in the grid and re-try clues as crossing le
