Consonants and vowels differ in the extent to which they are perceived categorically. We use a Bayesian model of speech perception to explore factors that might cause this difference. Simulations show that perception of vowels, fricatives, and stop consonants can all be captured under a single model in which listeners use their knowledge of phonetic categories to infer the sound that a speaker intended. This suggests that the differences in the way we perceive vowels and consonants, when viewed at the computational level, can be explained as parametric variation within a single framework.