Best of nine open-source language models reached only 43.2% accuracy predicting next coaching action
Researchers report that when tested on predicting the next coaching action given a question and dialogue context, the best of nine open-source language models reached only 43.2% accuracy.
In cases where effective metacognitive tutoring requires silence, which occurred in 41.7% of turns, the language models predicted no intervention only 6.6% of the time while greatly over-predicting high-intervention moves.
Sources behind this report
Report history
No earlier revision dates are retained here. This does not establish that the report has never changed; read any dated updates and corrections.
This brief was made by AI and checked against its cited public sources. How the newsroom works