In military and civilian operations good situational awareness is a prerequisite to make proper decisions. The situational awareness is among others based upon intelligence, threat analysis and detection, altogether element of the so-called DIM (detection, identification, monitoring) system. In case of low probability / high impact CBRN events it is preferred that high impact decisions are based upon data with known levels of confidence and less on the experience of the personnel. For this purpose operators of detectors should not only receive a detection signal, but also information regarding the related confidence of the information thus obtained. Confidence of a detection event is among others dependent of the environment. However the characteristics related to the confidence are not adequately assessed in material procurement requirements nor in the concomitant acceptance tests. To assess these characteristics properly, CBRN detectors should be evaluated on four interrelated key metrics: response time, false alarm rate, detection confidence and sensitivity. This is accomplished through the determination of so-called Receiver Operator Characteristic (ROC) curves rather than by solely measurement of the limit of detection.