Continuous-time multi-state models are widely used for categorical response data, particularly in the modeling of chronic diseases. However inference is difficult when the process is only observed at discrete time points, with no information about the times or types of events between observation times, unless a Markov assumption is made. This assumption can be limiting as rates of transition between disease states might instead depend on the time since entry into the current state. Such a formulation results in a semi-Markov model. We show that the computational problems associated with fitting semi-Markov models to panel-observed data can be alleviated by considering a class of semi-Markov models with phase-type sojourn distributions. This allows methods for hidden Markov models to be applied. In addition, extensions to models where observed states are subject to classification error are given. The methodology is demonstrated on a dataset relating to development of bronchiolitis obliterans syndrome in post-lung-transplantation patients.