Self-directed information sampling--the ability to collect information that one expects to be useful--has been shown to improve the efficiency of concept acquisition for both human and machine learners. However, little is known about how people decide which information is worth learning about. In this study, we examine self-directed learning in a relatively complex rule learning task that gave participants the ability to "design and test" stimuli they wanted to learn about. On a subset of trials we recorded participants' uncertainty about how to classify the item they had just designed. Analyses of these uncertainty judgments show that people prefer gathering information about items that help refine one rule at a time (i.e., those that fall close to a pairwise category "margin") rather than items that have the highest overall uncertainty across all relevant hypotheses or rules. Our results give new insight into how people gather information to test currently entertained hypotheses in complex problem solving tasks.