Complex buildings frequently present a challenge to users' understanding, which may affect wayfinding as well as appreciation of the building's structure. In this paper we focus on the building's diagram, a representation by the building's architect that captures its main 'idea'. Motivated by the intuition that a building may be easier to understand if its conceptual diagram can be clearly and easily described, we explored perceivers' descriptions of such diagrams' features. We asked students of Language and students of Architecture to write about the buildings represented in a variety of diagrams, and then repeated the task for photographs of the actual buildings. Using Cognitive Discourse Analysis, we aimed to create a first qualitative exploration of the linguistic and conceptual patterns that are associated with the perception of diagrams and images of complex buildings. Among other factors, results show how perception of the diagram's meaning is fundamentally affected by subject expertise. Linguistic patterns demonstrate the ways in which written descriptions reflect observers' understanding and concepts of building representations, providing a starting point for future studies which may address the possible relationship between the verbalisability of a diagram and the legibility of a building.