Nestedness is widely observed in natural metacommunities, but its underlying mechanisms are still poorly understood. The distribution of habitats in the landscape and differences in dispersal rates of distinct insect taxa can determine the nestedness of the metacommunity. Here, we evaluated how species habitat specialization contributes to metacommunity nestedness in insect groups with different dispersal capacities in a mountaintop landscape in south-eastern Brazil. We sampled ants, butterflies and dung beetles in two main habitats, naturally fragmented forest islands and a grassland matrix (campo rupestre), during both dry and rainy seasons. We classified species according to their degree of habitat specialization (generalists or specialists) based on the relative frequencies and abundances between these two contrasting habitats. Forty of 211 species were classified as habitat specialists, seven as habitat generalists. It was not possible to classify the remaining species. The metacommunity was nested in structure, with habitat generalist species contributing more to nestedness than habitat specialists. Nonetheless, habitat distribution in the landscape did not affect the nestedness of the metacommunity. Our findings reveal that species sorting (for habitat specialists) and mass effects (for habitat generalists) are concurrent processes in the mountaintop forest-grassland mosaic. Our study helps to advance our understanding of the differences in the distribution of generalist and specialist species in a tropical mountaintop landscape and improves our ability to predict and manage the increasingly adverse effects of changes in land use and climate on metacommunities and ecosystem functions.