Adapting classification models to concept drift is one of the main challenges associated with applying these models in dynamic environments. In particular, the learned concept is not static and may change over time under the influence of varying conditions (i.e. varying context). Unlike existing approaches where only the most recent data are considered for adapting the model, we propose incorporating context awareness into the adaptation process. The goal is to utilise knowledge of relevant context variables to facilitate the selection of more relevant training data. Specifically, we propose to weight each training example based on the degree of similarity with the current context. To detect such similarity, we utilise two approaches: a simple difference between the context variable values and a distribution-based distance metric. The experimental analyses show that such explicit context utilisation results in a more effective data selection strategy and enables to produce more accurate predictions.