There is widespread concern that cessation of grazing in historically grazed ecosystems is causing biotic homogenization and biodiversity loss. We used 12 montane grassland sites along an 800 km north–south gradient across the UK, to test whether cessation of grazing affects local α- and β-diversity of below-ground food webs. We show cessation of grazing leads to strongly decreased α-diversity of most groups of soil microbes and fauna, particularly of relatively rare taxa. By contrast, the β-diversity varied between groups of soil organisms. While most soil microbial communities exhibited increased homogenization after cessation of grazing, we observed decreased homogenization for soil fauna after cessation of grazing. Overall, our results indicate that exclusion of domesticated herbivores from historically grazed montane grasslands has far-ranging negative consequences for diversity of below-ground food webs. This underscores the importance of grazers for maintaining the diversity of below-ground communities, which play a central role in ecosystem functioning.