Neoadjuvant radiotherapy, as part of the conventional treatment of rectal cancer, can induce fibrotic tissue formation around the tumor. This complicates the exact determination of the tumor borders during surgery, which might increase the chance of positive resection margins. In a previous ex vivo study, we distinguished tumor tissue from healthy rectal wall and fat with an accuracy of 0.95, using diffuse reflectance spectroscopy (DRS). Since this study did not include fibrosis, the aim of the current ex vivo study was to examine whether differentiation of tumor and fibrosis with DRS is possible. DRS measurements from freshly resected specimen of 16 patients were obtained. In eight patients fibrosis was measured, in the other eight patients tumor was measured. The measurements were performed using a DRS probe with a source-detector distance of 2 mm. The spectra were obtained in the wavelength range of 450-1600 nm. Classification of the measurements was done using a support vector machine (SVM) and a set of features extracted from the spectra. The SVM was evaluated using an eight-fold cross-validation, which was repeated ten times. For all repetitions, the area under the ROC curve was greater than 0.85 (mean = 0.87, STD = 0.02). The mean sensitivity and specificity were 0.85 (STD = 0.03) and 0.88 (STD = 0.01) respectively. It can be concluded that tumor tissue can be distinguished from fibrosis based on spectral features from DRS measurements. The next step will be to conduct an in vivo study, to verify these results during surgery.