The present study explores learning phonological alternations that contain exceptions. Participants were exposed to a back/round vowel harmony pattern in which a regular suffix followed harmony, varying between /e/ and /o/ depending on the back/round phonetic features of the stem, and an exceptional suffix that was always /o/ regardless of the features of the stem vowel. Participants in Experiment 1 learned the behavior of both suffixes, but performance for the non-alternating suffix was higher when the suffix happened to adhere to vowel harmony. In Experiment 2, participants were exposed only to the same suffixes as Experiment 1, but the non-alternating suffix only appeared in harmonic contexts, creating ambiguity between exceptionality and alternation. Participants only correctly selected the non-alternating suffix when it appeared in a harmonic context. This suggests that learners are biased towards alternating harmony patterns, but require concrete evidence of non-alternation to learn the non-alternating suffix.