Motivation: In recent years increasing evidence appeared that breastcancer may not constitute a single disease at the molecular level,but comprises a heterogeneous set of subtypes. This suggests that instead of building a single predictor, better predictors might be constructed that solely target samples of a designated subtype. An unavoidable drawback of developing subtype-specific predictors, however,is that a stratification by subtype drastically reduces the numberof samples available for their construction. It is therefore questionable whether the potential benefit of subtyping can outweigh the drawback of a severe loss in sample size. Factors like unequal class distributions and differences in the number of samples per subtype, further complicate comparisons. Results: We present several evaluation strategies that facilitate a comprehensive comparison between subtype-specific predictors and predictors that do not take subtype information into account. Emphasis lies on careful control of sample size as well as class and subtype distributions. The methodology is applied to a large breast cancer compendium involving over 1500 arrays,using a state-of-the-art subtyping scheme. We show that the resulting subtype-specific predictors outperform those that do not take subtype information into account, especially when taking sample size considerations into account.