Ensuring that service-oriented systems can adapt quickly and effectively to changes in service quality, business needs and their runtime environment is an increasingly important research problem. However, while considerable research has focused on developing runtime adaptation frameworks for service-oriented systems, there has been little work on assessing how effective the adaptations are. Effective adaptation ensures the system remains relevant in a changing environment and is an accurate reflection of user expectations. One way to address the problem is through validation. Validation allows us to assess how well a recommended adaptation addresses the concerns for which the system is reconfigured and provides us with insights into the nature of problems for which different adaptations are suited. However, the dynamic nature of runtime adaptation and the changeable contexts in which service-oriented systems operate make it difficult to specify appropriate validation mechanisms in advance. This paper describes a novel consumer-centered approach that uses machine learning to continuously validate and refine runtime adaptation in service-oriented systems, through model-based clustering and deep learning.