People often get lost in buildings, including but not limited to libraries, hospitals, conference centers, and shopping malls. There are at least three contributing factors: the spatial structure of the building, the cognitive maps that users construct as they navigate, and the strategies and spatial abilities of the building users. The goal of this article is to discuss recent research on each of these factors and to argue for an integrative framework that encompasses these factors and their intersections, focusing on the correspondence between the building and the cognitive map, the completeness of the cognitive map as a function of the strategies and individual abilities of the users, the compatibility between the building and the strategies and individual abilities of the users, and complexity that emerges from the intersection of all three factors. We end with an illustrative analysis in which we apply this integrative framework to difficulty in way-finding.