The relationship between handedness and mathematical abilities is controversial. Whilst some researchers have claimed that left-handers are gifted in mathematics and strong right-handers perform the worst in mathematical tasks, it has been more recently proposed that mixed-handers are actually the most disadvantaged group. To disentangle these discrepancies, we conducted five experiments in several Italian schools (total participants: N = 2,308) involving students of different ages (6 to 17 years) and a range of mathematical tasks. The results showed that (a) the percentage of variance in mathematics scores explained by handedness was moderate (about 5%) but statistically significant, and (b) the shape of the relationship between handedness and mathematical ability depended on age, task, and gender. We concluded that the different outcomes reported in the literature probably reflected the dissimilarities between the studies about the above variables. Therefore, a more comprehensive model is needed, which explains how these variables interact.