Free-form response vs. yes/no-question methodologies in the study of human reasoning


There are two widespread strategies for testing experimentally whether a conclusion follows naively from a sequence of premises. The free-form response strategy (FFR) presents participants with the premises and asks them ``what, if anything, follows?'' In the simplest case, participants' responses are coded as to whether they made the predicted inference. On the yes/no question strategy (YNQ), after presenting the premises, the researcher puts forth a sentence C and asks whether C follows from the premises. We compare the two methodologies with respect to six types of fallacious problems involving propositional connectives from the mental-models literature, to address the question of whether the methodologies are equally valid. We found that the two methodologies overwhelmingly yield identical results. Interestingly, the exceptions we found show that in some cases FFR fails to detect an attractive fallacious conclusion that can be reliably probed with YNQ.

Back to Table of Contents