As wireless devices increasingly becoming ubiquitous, spontaneous interaction opportunities are accordingly becoming common. Although spontaneous interaction research has been ongoing for many years, a key question of spontaneous interaction still remains open: how can users associate devices in a natural, intuitive and secure manner. Over the past decade, researchers have demonstrated a plethora of device association techniques. Nonetheless, users were often neglected during design or they were only involved in a later stage, like testing. We argue that since device association is performed by end-users, usability factors need to be considered in the early stage of design. In this article, we present a categorisation of factors that influence the usability of device association. This paper aims at providing a framework that informs researchers and system designers of the considerations needed when designing or adopting an association technique.