Recommender systems profile the preferences of users and then use this information to forecast users' future ratings. One of the most common recommendation approaches is the use of matrix factorisation in which users' past ratings of items (i.e. Movies, books, etc.) are used to capture their affinity to implicit factors. A central limitation of such factorisation is that one cannot consider how a user's preferences for a factor have changed over time. In this paper we present the SemanticSVD++ model that overcomes this limitation by using the semantic categories of recommendation items as prior factors for a given user. We present a model to capture the semantic taste evolution of users over time, and demonstrate how such development is susceptible to global influence dynamics. We explain how the SemanticSVD++ model incorporates such evolution information within a matrix factorisation approach, and empirically demonstrate the improvement in predictive capability that this yields when tested on two independent movie recommendation datasets.