In most learning models, the induction methods that are learnable have low expressive power. The learnability of such methods are proved for some concept classes by assuming that the hypothesis space of the method contains the target concept. However, in real-world practical problems the type of target concept being dealt with is almost always unknown. In medical diagnosis, where mistakes can cause fatal results, it is very important to achieve high recognition rates and the use of more expressive methods are common. However, conventional methods are weak at handling irrelevant information which often appears in medical databases. In this paper, we consider test feature classifiers recently introduced by (Lashkia and Aleshin, 2001. IEEE Trans. Syst. Man Cybern. 31 (4), 643–650), and show that they meet all essential requirements to be of practical use in medical decision making, which are: ability to handle irrelevant attributes, high expressive power, high recognition ability, and ability to generate decisions by a set of rules.