Extracting Human Face Similarity Judgments: Pairs or Triplets?


Two experimental protocols, pairwise rating and triplet ranking, have been commonly used for eliciting perceptual similarity judgments for visual objects. Pair has the advantage of greater precision, but triplet is potentially cognitive less taxing, thus resulting in less noisy responses. Here, we introduce several information-theoretic measures of how useful the two protocols are for response prediction and parameter estimation. We demonstrate that triplet is significantly better for extracting subject-specific preferences, while the two are comparable across subjects. While the specific conclusions should be interpreted cautiously, the work provides an information-theoretic framework for quantifying how repetitions within and across subjects can help to combat noise in human responses, as well as giving some insight into the nature of similarity representation and response noise in humans. More generally, this work demonstrates that substantial noise and inconsistency corrupt similarity judgments, both within- and across-subjects, with consequent implications for experimental design and data interpretation.

Back to Table of Contents