Determining the handedness of visually presented stimuli is thought to involve two separate stages—a rapid, implicit recognition of laterality followed by a confirmatory mental rotation of the matching hand. In two studies, we explore the role of the dominant and non-dominant hands in this process. In Experiment 1, participants judged stimulus laterality with either their left or right hand held behind their back or with both hands resting in the lap. The variation in reactions times across these conditions reveals that both hands play a role in hand laterality judgments, with the hand which is not involved in the mental rotation stage causing some interference, slowing down mental rotations and making them more accurate. While this interference occurs for both lateralities in right-handed people, it occurs for the dominant hand only in left-handers. This is likely due to left-handers’ greater reliance on the initial, visual recognition stage than on the later, mental rotation stage, particularly when judging hands from the non-dominant laterality. Participants’ own judgments of whether the stimuli were ‘self’ and ‘other’ hands in Experiment 2 suggest a difference in strategy for hands seen from an egocentric and allocentric perspective, with a combined visuo-sensorimotor strategy for the former and a visual only strategy for the latter. This result is discussed with reference to recent brain imaging research showing that the extrastriate body area distinguishes between bodies and body parts in egocentric and allocentric perspective.