An influential view of the nature of the language system is that of an evolved biological system in which a set of rules is combined with a lexicon that contains the words of the language together with a representation of their context. Alternative views, usually based on connectionist modeling, attempt to explain the structure of language on the basis of complex associative processes. Here I put forward a third view that stresses experience-dependent structural development of the brain circuits supporting language as a core principle of the organization of the language system. On this view, embodied in a recent neuroconstructivist neural network of past tense development and processing, initial domain-general predispositions enable the development of functionally specialized brain structures through interactions between experience-dependent brain development and statistical learning in a structured environment. Together, these processes shape a biological adult language system that appears to separate into distinct mechanism for processing rules and exceptions, whereas in reality those subsystems co-develop and interact closely. This view puts experience-dependent brain development in response to a specific language environment at the heart of understanding not only language development but adult language processing as well.