Analysis of human problem solving drafts: a methodological approach on the example of Rush Hour

AbstractAssessing the quality of a learner’s solution for a given task is an essential step in analyzing a learner’s performance. For a well-defined sequential problem, correctness and optimality of the solution as well as its length provide first simple and reasonable metrics. However, this ignores the fact that there are conceptually different errors that humans make when solving a problem. This work proposes a rule-based system of error categories which is able to classify conceptually different errors with respect to their (assumed) motive. The principles the categories are based on are valid for most well-defined sequential problems and can hence serve as a valuable tool in the analysis of human solutions for such a problem. In this work, the error category system is adapted to the game Rush Hour. We use the category system as a tool for a detailed analysis of 115 human solutions of a Rush Hour game. We found that the most common error type is based on a simple solving heuristic, but mainly occurs in the first half of the solution process. Other error types whose occurrence is numerically less dominant, are still found in the majority of the solutions. However, they occur in very specific game situations. As a first generalization approach of the category system, its application on a further dataset containing 56 different Rush Hour tasks and more than 31,000 human solutions yield promising results.

Return to previous page