We study the role of charge fluctuations in the decoherence of Rabi oscillations between spin states ↑↓, ↓↑ of two electrons in a double dot structure. We consider the effects of fluctuations in energy and in the quantum state of the system, both in the classical and quantum limits. The role of state fluctuations is shown to be of leading order at sufficiently high temperature, applicable to actual experiments. At low temperature, the low frequency energy fluctuations are the only dominant contribution.