The essential task of a motor imagery brain–computer interface (BCI) is to extract the motor imagery-related features from electroencephalogram (EEG) signals for classifying motor intentions. However, the optimal frequency band and time segment for extracting such features differ from subject to subject. In this work, we aim to improve the multi-class classification and to reduce the required EEG channel in motor imagery-based BCI by subject-specific time-frequency selection. Our method is based on a criterion namely Fisher discriminant analysis-type F-score to simultaneously select the optimal frequency band and time segment for multi-class classification. The proposed method uses only few Laplacian EEG channels (C3, Cz and C4) located around the sensorimotor area for classification. Applied to a standard multi-class BCI dataset (BCI competition III dataset IIIa), our method leads to better classification performance and smaller standard deviation across subjects compared to the state-of-art methods. Moreover, adding artifacts contaminated trials to the training dataset does not necessarily deteriorate our classification results, indicating that our method is tolerant to artifacts.