There are many different studies that try to use physiological features to determine stress. But there exists a lot of uncertainty about which physiological signals and features are the best classifiers and a lot of discrepancies in classification accuracies exist. This study proposes a novel method for the detection of stress. This method contains a two-layered approach to the stress detection problem. Since most features are influenced by speaking, this study suggests that before stress detection takes place a speaking detection algorithm is used. During this study both ECG and respiration data are used to classify stress. The linear influence of the respiration is removed from the ECG data with orthogonal subspace projection to improve the ECG features. An average classification accuracy of 80% is achieved on the test dataset, and a classification accuracy of 77% is achieved on a second dataset which was obtained with a different experimental setup. This shows the real-world applicability and robustness of the designed algorithm. This study, also shows that the influence of speaking on the features is crucial. In literature, a lot of classifiers for stress incorrectly ignore the influence that speaking has on the classification. Combined with a faulty data acquisition method, this possibly results in classifiers trained on detecting speaking instead of stress. With this newly proposed method, the stress detection algorithm is more robust against the influence of speaking.