Time and space are intimately related, but what is the real nature of this relationship? Is time mapped metaphorically onto space such that effects are always asymmetric (i.e., space affects time more than time affects space)? Or do the two domains share a common representational format and have the ability to influence each other in a flexible manner (i.e., time can sometimes affect space more than vice versa)? In three experiments, we examined whether spatial representations from haptic perception, a modality of relatively low spatial acuity, would lead the effect of time on space to be substantially stronger than the effect of space on time. Participants touched (but could not see) physical sticks while listening to an auditory note, and then reproduced either the length of the stick or the duration of the note. Judgements of length were affected by concurrent stimulus duration, but not vice versa. When participants were allowed to see as well as touch the sticks, however, the higher acuity of visuohaptic perception caused the effects to converge so length and duration influenced each other to a similar extent. These findings run counter to the spatial metaphor account of time, and rather support the spatial representation account in which time and space share a common representational format and the directionality of space-time interaction depends on the perceptual acuity of the modality used to perceive space.