The aim of the present study was to determine the differential contributions of the action observation network (AON) and the social neural network (SNN) to the experience of naturalness in observed dyadic social interactions. To this end, we used short animation sequences displaying social interactions between two virtual characters and systematically manipulated kinematic features of the social dynamics. A group of 21 male participants rated the “naturalness” of the observed scenes on a four-point scale while undergoing fMRI. Using the ratings of each participant as a parametric modulation of their general neural response to the stimuli, we found that an increase in naturalness experience was associated with higher activations in the AON. The SNN was preferentially recruited with a decrease in naturalness experience. This indicates that understanding familiar interactions involves an automatic kinematic processing of intentionality, while interactions perceived as artificial require higher-level inferential processing.