In this talk we consider queueing systems which are subject to control (e.g. admission control, routing control) by an external decision-maker. These systems can be modelled using Markov Decision Processes (MDPs), and our aim is to determine an optimal "policy" which specifies the most advantageous decisions to be taken. We discuss how the techniques of stochastic dynamic programming can be used to compute optimal policies, and how the "adaptive" methods of reinforcement learning can be used to find near-optimal solutions when the dimensionality of the problem becomes too large.