Combining accuracy and prior sensitivity for classifier design under prior uncertainty
TCW Landgrebe (TU Delft - Electrical Engineering, Mathematics and Computer Science)
RPW Duin (TU Delft - Electrical Engineering, Mathematics and Computer Science)
More Info
expand_more
Abstract
Consideringtheclassi¿cationprobleminwhichclasspriorsormisallocationcostsarenotknownprecisely,receiveroperatorcharacteristic(ROC)analysishasbecomeastandardtoolinpatternrecognitionforobtainingintegratedperformancemeasurestocopewiththeuncertainty.Similarly,insituationsinwhichpriorsmayvaryinapplication,theROCcanbeusedtoinspectperformanceovertheexpectedrangeofvariation.InthispaperwearguethateventhoughmeasuressuchastheareaundertheROC(AUC)areusefulinobtaininganintegratedperformancemeasureindependentofthepriors,itmayalsobeimportanttoincorporatethesensitivityacrosstheexpectedprior-range.Weshowthataclassi¿ermayresultinagoodAUCscore,butapoor(large)priorsensitivity,whichmaybeundesirable.Amethodologyisproposedthatcombinesbothaccuracyandsensitivity,providinganewmodelselectioncriterionthatisrelevanttocertainproblems.Experimentsshowthatincorporatingsensitivityisveryimportantinsomerealisticscenarios,leadingtobettermodelselectioninsomecases
No files available
Metadata only record. There are no files for this record.