Deep learning (DL) models have great potential to be used for recommender systems (RS) in the Internet of Vehicles (IoV) due to their excellent data processing and learning abilities. To alleviate the problem of privacy leakage in centralized training of DL models, federated learning (FL) is engaged for its advantage of just uploading gradients or parameters of models without raw data sharing. However, the ubiquitous data heterogeneity in IoV brings a big challenge to the stable global convergence of FL. In addition, existing FL schemes with high convergence latency cannot realize the timely personalized content recommendation in IoV. In this paper, a FL scheme with adaptive local aggregation for privacy-aware RS in IoV, named FLRS, is proposed. At first, we analyze the data distribution on each vehicle locally from two aspects: label distribution and feature distribution. After receiving vehicle data distribution information, the edge server selects aggregation collaborators for each vehicle according to the distribution similarity of labels and features. In the model updating stage, the local model on each vehicle is adaptively aggregated with those from its collaborators and then the local training process begins. At last, the performance of FLRS is evaluated through simulation experiments.