Many real-world situations involve queueing systems in which customers may abandon if service does not start sufficiently quickly. We study a comprehensive model of multi-class queue scheduling accounting for customer abandonment, with the objective of minimizing the total discounted or time-average sum of linear waiting costs, completion rewards, and abandonment penalties of customers in the system. We assume the service times and abandoning times are exponentially distributed. We solve analytically the case in which there is one server and there are one or two customers in the system and obtain an optimal policy. For the general case, we use the framework of restless bandits to analytically design a novel simple index rule with a natural interpretation. We show that the proposed rule achieves near-optimal or asymptotically optimal performance both in single- and multi-server cases, both in overload and underload regimes, and both in idling and non-idling systems.