From at least two months onwards, infants can form perceptual categories. During the first year of life, object knowledge develops from the ability to represent individual object features to representing correlations between attributes and to integrate information from different sources. At the end of the first year, these representations are shaped by labels, opening the way to conceptual knowledge. Here, we review the development of object knowledge and object categorization over the first year of life. We then present an artificial neural network model that models the transition from early perceptual categorization to categories mediated by labels. The model informs a current debate on the role of labels in object categorization by suggesting that although labels do not act as object features they nevertheless affect perceived similarity of perceptually distinct objects sharing the same label. The model presents the first step of an integrated account from early perceptual categorization to language-based concept learning.