Diseases normally progress through several stages. Therefore, biomarkers corresponding to each stage may exist. To deal with such a multi-category problem, including sample stage prediction and biomarker selection, we propose methods for classification and feature selection. The proposed classification method is based on two schemes: error-correcting output coding (ECOC) and pairwise coupling (PWC). The final decision for a test sample prediction is an integration of these two schemes. The biomarker pattern for distinguishing each disease category from another one is achieved by the development of an extended Markov blanket (EMB) feature selection method.