Neutron stars spin down over time due to a number of energy-loss processes. We provide tantalizing population-based evidence that millisecond pulsars (MSPs) have a minimum ellipticity of epsilon ≈ 10−9 around their spin axis and that, consequently, some spin down mostly through gravitational-wave emission. We discuss the implications of such a minimum ellipticity in terms of the internal magnetic field strengths and nuclear matter composition of neutron stars and show it would result in the Advanced LIGO and Virgo gravitational-wave detectors, or their upgrades, detecting gravitational waves from some known MSPs in the near future.