Detecting Students Problem Solving Strategies Using Sankey Diagrams
- Tao Gong, Educational Testing Service, Princenton, New Jersey, United States
- Christopher Agard, Educational Testing Service, Princeton, New Jersey, United States
- Gary Feng, Educational Testing Service, Princeton, New Jersey, United States
- Gabrielle Cayton-Hodges, Educational Testing Service, Princeton, New Jersey, United States
- Luis Saldivia, Educational Testing Service, Princeton, New Jersey, United States
AbstractProcess data (e.g., logs of actions, keystrokes, times, or eye tracks) recording students’ interactions with digital assessments are available in many digital educational assessments. They have become the primary focus of cognitive scientists to detect and analyze students’ strategies during problem solving. This study developed a Sankey diagram-based method to visualize process data of multiple-choice items. Such diagram has been widely adopted in industry and ecology to trace flow of information, energy, or resource. Using released items from the 2017 National Assessment of Educational Progress Mathematics Tests, we illustrated how to use such a diagram to elucidate frequent answer formulation patterns of students, their common mistakes, and estimated probabilities of reaching correct/wrong answers at various answering stages. These help reveal the problem solving strategies adopted by students and their underlying cognitive processes. Assessment developers, teachers, and students could use such insights to improve assessments and learning outcomes for confusing concepts.