We study the effect of the exciton fine-structure splitting on the polarization entanglement of photon pairs produced by the biexciton cascade in a quantum dot. Entanglement persists despite separations between the intermediate energy levels of up to 4 mu eV. Measurements show that entanglement of the photon pair is robust to the dephasing of the intermediate exciton state responsible for the first-order coherence time of either single photon. We present a theoretical framework incorporating the effects of spin scattering, background light, and dephasing. We distinguish between the first-order coherence time, and a parameter which we measure for the first time and define as the cross-coherence time.