This study aims to examine the factors that influence the use of labels when making similarity decisions. We predict that a person’s visual working memory (VWM) ability and implicit beliefs can predict the frequency in which a subject will rely on labels when making similarity decisions. To test this hypothesis, participants completed a VWM task along with a questionnaire to study how they relate to label use. We found a negative trend between VWM ability and use of labels, but a direct relationship between certain implicit beliefs and label use. Implications of these results in relation to label use are also discussed.