We address the problem of developing a well-performing and implementable scheduler of users with wireless connection to the base station. The main feature of such real-life systems is that the quality conditions of the user channels are time-varying, which turn into the time-varying transmission rate due to different modulation and coding schemes. We assume that this phenomenon follows a Markovian law and most of the discussion is dedicated to the case of three quality conditions of each user, for which we characterize an optimal index policy and show that threshold policies (of giving higher priority to users with higher transmission rate) are not necessarily optimal. For the general case of arbitrary number of quality conditions we design a scheduler and propose its two practical approximations, and illustrate the performance of the proposed index-based schedulers and existing alternatives in a variety of simulation scenarios.