Chlorophylls and carotenoids are naturally existing pigments and play a crucial role in the chemical and sensory quality of vegetable oils. These compounds have at the same time substantial positive health impacts and can be useful in detecting adulteration. Therefore, this study evaluated the feasibility of a color sensor (RGB sensor) for predicting the total content of carotenoids and chlorophylls in avocado and olive oils, two of the vegetable oils that have well documented health effects. as well as their total spectrophotometric color (TSC). Different color parameters (RGB, HSV, or L*a*b*) and lighting conditions (white or 395 nm UV light) were compared in order to identify the best analytical condition. The least-square support vector machine (LS-SVM) models exhibited superior performance compared to the multiple linear regression (MLR) models. The use of UV light resulted in an enhanced predictive performance for the total chlorophylls content. In contrast, white lighting was found to be more suitable for the prediction of total carotenoids and TSC. The use of HSV or RGB values demonstrated better performance in predicting total chlorophylls (R² > 0.9, RMSE from 0.99 to 4.13 mg kg−1, and RPD from 4.04 to 3.48). On the other hand, the L*a*b* values demonstrated the highest accuracy in predicting the total carotenoids content (R² > 0.8, RMSE from 0.42 to 0.92 mg kg−1, and RPD from 2.02 to 2.22). In conclusion, this color sensor-based approach has been demonstrated as a cost-effective, accurate, and rapid method for predicting pigment content in vegetable oils, requiring minimal or no sample preparation.