We present a novel way of accounting for similarity judgments. Our approach posits that similarity ratings stem from three main sources: familiarity, priming, and inherent perceptual similarity. We present a process model of our approach in the cognitive architecture ACT-R, and match our model's predictions to data collected from a human subject experiment which involved simple perceptual stimuli. Familiarity accounts for rising ratings over time; priming accounts for asymmetric effects that arise when the stimuli are shown with different frequencies. Pure perceptual similarity also predicts trends in the results. Overall, our model matched the data with R^2 of 0.99.