A growing body of recent research suggests that verbal categories, particularly labels, impact categorization and perception. These findings are commonly interpreted as demonstrating the involvement of language on cognition, however, whether these assumptions hold true for grammatical structures has yet to be investigated. In the present study, we investigated the extent to which linguistic information, namely, grammatical gender categories structures cognition to subsequently influence categorical judgments and perception. In a non-verbal categorization task, French-English bilinguals and monolingual English speakers made gender-associated judgments about a set of image pairs while event-related potentials were recorded. The image sets were composed of an object paired with either a female or male face, wherein the object was manipulated for their conceptual gender relatedness and grammatical gender congruency to the sex of the following target face. The results showed that grammatical gender modulated the N1 and P2/VPP, as well as the N300 exclusively for the French-English bilinguals, indicating the inclusion of language in the mechanisms associated with attentional bias and categorization. In contrast, conceptual gender information impacted the monolingual English speakers in the later N300 time window given the absence of a comparable grammatical feature. Such effects of grammatical categories in the early perceptual stream have not been found before, and further provide grounds to suggest that language shapes perception.