Interlocutors sometimes repeat each other’s representational hand gestures. We investigated if this is a case of direct mimicry of form, or whether perceiving a gesture gives rise to a semantic representation, which subsequently informs gesture production. For this we used an interactive route description task, in which a confederate’s gestures indicated the route in either the vertical or the horizontal plane and either with one or four fingers extended as an index. We found that perceiving vertical gestures led to an increase not only in participants’ production of vertical gestures, but also in their use of one finger as an index, suggesting that seeing vertical gestures caused participants to think of the route as on a map, which led them to point with one finger (as is common on a map) rather than four. Our results support the notion that repetition of meaningful gesture forms results from converging semantic representations.