CC
C.C.I.A. Chen
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
2 records found
1
Large language models (LMs) are increasingly used in critical tasks, making it important that these models can be trusted. The confidence an LM assigns to its prediction is often used to indicate how much trust can be placed in that prediction. However, a high confidence can be incorrectly trusted if it turns out to be incorrect, also known as a high-confident error, or unknown unknown (UU). Blind spots, clusters of UUs, are caused by out-of-distribution data (OOD), bias, covariate shift, or unseen data. Previous work by Lippmann et al. generated samples based on these UUs, however, they selected a random subset of all found UUs. Although this was the first study that could mitigate blind spots without the need for crowd workers or oracles, they did not address the underlying causes of blind spots, such as OOD and bias. Data valuation is the research field that is concerned with valuing samples to improve model performance and can identify OOD data, bias, and noise. This paper proposed to combine the research field of data valuation, with the mitigation of blind spots. We generated synthetic samples using the highest-valued samples and retrained the LM using a Weighted Loss based on Data Values (WLDV). We conducted an extensive evaluation of our approach on four tasks, demonstrating a reduction of UUs by up to 32.7%, while retaining the same level of accuracy. This was the first exploration of combining the research fields of data valuation and blind spot mitigation.
...
Large language models (LMs) are increasingly used in critical tasks, making it important that these models can be trusted. The confidence an LM assigns to its prediction is often used to indicate how much trust can be placed in that prediction. However, a high confidence can be incorrectly trusted if it turns out to be incorrect, also known as a high-confident error, or unknown unknown (UU). Blind spots, clusters of UUs, are caused by out-of-distribution data (OOD), bias, covariate shift, or unseen data. Previous work by Lippmann et al. generated samples based on these UUs, however, they selected a random subset of all found UUs. Although this was the first study that could mitigate blind spots without the need for crowd workers or oracles, they did not address the underlying causes of blind spots, such as OOD and bias. Data valuation is the research field that is concerned with valuing samples to improve model performance and can identify OOD data, bias, and noise. This paper proposed to combine the research field of data valuation, with the mitigation of blind spots. We generated synthetic samples using the highest-valued samples and retrained the LM using a Weighted Loss based on Data Values (WLDV). We conducted an extensive evaluation of our approach on four tasks, demonstrating a reduction of UUs by up to 32.7%, while retaining the same level of accuracy. This was the first exploration of combining the research fields of data valuation and blind spot mitigation.
Wheat is a widely used ingredient for food products. To increase the productionand quality of wheat, the density of ’wheat heads’ in a farm can be studied. Accuratelylocating wheat heads in images can be challenging. A lot of work has taken place insupervised semantic segmentation, but these networks typically require large pixel-wisehuman-annotated labeled data. Gathering this data is tedious and labour intensive.This paper proposes to use the novel unsupervised semantic segmentation model W-Net to solve this problem. To improve the accuracy, we investigated the influence ofthe frequency domain, by pre-processing the training data two different times using acustom filter, based on frequencies found in wheat heads, and a high pass filter.The approach is evaluated on the Global Wheat Head Detection (GWHD) dataset[11]. To compare the accuracy the generated segmentations were mapped to boundingboxes based. The proposed method did not show to be able to generate competing de-tection compared to the baseline method associated with the GWHD dataset, but theGWHD dataset has a different measurement of truth, consisting out bounding boxesinstead of segments which is in the disadvantage for the W-Net.Pre-processing the dataset using the high pass filter did increase the intersection overunion with 1,4% and the deviation of the reconstruction loss was smaller when fre-quency filtering was applied.Although the object detection has a low accuracy, this study showed that some ba-sic wheat head detection can be achieved by using the unsupervised segmentationmethod W-Net and the accuracy can be increased if a high pass filter is applied aspre-processing step.
...
Wheat is a widely used ingredient for food products. To increase the productionand quality of wheat, the density of ’wheat heads’ in a farm can be studied. Accuratelylocating wheat heads in images can be challenging. A lot of work has taken place insupervised semantic segmentation, but these networks typically require large pixel-wisehuman-annotated labeled data. Gathering this data is tedious and labour intensive.This paper proposes to use the novel unsupervised semantic segmentation model W-Net to solve this problem. To improve the accuracy, we investigated the influence ofthe frequency domain, by pre-processing the training data two different times using acustom filter, based on frequencies found in wheat heads, and a high pass filter.The approach is evaluated on the Global Wheat Head Detection (GWHD) dataset[11]. To compare the accuracy the generated segmentations were mapped to boundingboxes based. The proposed method did not show to be able to generate competing de-tection compared to the baseline method associated with the GWHD dataset, but theGWHD dataset has a different measurement of truth, consisting out bounding boxesinstead of segments which is in the disadvantage for the W-Net.Pre-processing the dataset using the high pass filter did increase the intersection overunion with 1,4% and the deviation of the reconstruction loss was smaller when fre-quency filtering was applied.Although the object detection has a low accuracy, this study showed that some ba-sic wheat head detection can be achieved by using the unsupervised segmentationmethod W-Net and the accuracy can be increased if a high pass filter is applied aspre-processing step.