Visible Light Positioning (VLP) has become an essential candidate for high-accurate positioning; however, its positioning accuracy is usually degraded by the noise in the VLP system. To solve this problem, a novel scheme of noise measurement and mitigation is proposed for VLPbased on the noise measurement from Allan Varianceand the noise mitigation from positioning algorithms such asAdaptive Least Squares (ALSQ)andExtended Kalman Filter (EKF). In this scheme, Allan Varianceis introduced for noise analysis in VLPfor the first time, which provides an efficient method for measuring the white noise in the VLPsystems. Meanwhile, we evaluate our noise reduction method under static testusing ALSQ and dynamic test using EKF. Furthermore, this article carefully discusses the relationship between positioning accuracy and Dilution of Precision (DOP) values. The preliminary field static tests demonstrate that the proposed scheme improves thepositioning accuracy by 16.5% and achieves the accuracy of 137mmwhile dynamic tests show an improvement of 60.4% and achieve the mean positioning accuracyof 153 mm.