In this paper, we present a mathematical model to design the service network, that is the set of origin-destination connections. The resulting model considers both full and empty freight car movements, and takes into account handling costs. More specifically, the model suggests the services to provide, as well as the number of trains and the number and type of cars traveling on each connection. Quality of service, which is measured as total travel time, is established by minimizing the waiting time of cars at intermediate stations. Our approach yields a multi-commodity network design problem with concave arc cost functions. To solve this problem, we implement a tabu search procedure which adopts "perturbing" mechanisms to force the algorithm to explore a larger portion of the feasible region. Computational results on realistic instances show a significant improvement over current practice.