There are certain universal properties of language that are taken to be definitional to the concept of language itself, such as the arbitrary relationship between sounds and meanings of words. Another possibility is that these “design features” of language may instead be the expressed consequences of general purpose learning constraints within the cognitive system learning the language. To test this, generations of an inverse model learning to map between sounds and meanings of words was tested. In this model, learning to associate phonology to semantics influences the model’s production of phonology from semantics, and phonological productions of one model were used as input to the next generation. Over generations of the model’s learning, the language became easier to acquire, and demonstrated increased arbitrariness of mappings between phonology and semantics. The iterative modelling demonstrated that design features of natural language can spontaneously emerge in a general purpose learning system.