MV
M.E.B.P. Visser
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
2 records found
1
As the amount of information available in the world grows, Information Retrieval (IR) systems have become an integral part of day to day life. They determine what subset of the large pool of information is shown to people. IR algorithms determine which items should be returned in response to a query and rank the results in a ranked list.
Recently, concerns about the fairness of IR algorithms have surfaced. In particular, research is being done into whether IR algorithms are fair to producers, people or organizations that provide the items that are retrieved by IR algorithms. The higher an item is in the ranked list, the more attention it receives from users. This attention translates to benefits for the producers, e.g. fame or financial compensation.
In this thesis we investigate the fairness of IR algorithms in terms of a specific measure for provider fairness: the Expected Exposure Loss (EEL). This measure measures whether the providers of equally relevant items receive the same amount of attention in expectation. EEL was first proposed as part of the 2020 TREC Fair Ranking track (FAIR-TREC), which also provided a matching dataset. We investigate for two IR systems whether they achieve fairness on this dataset. We conduct a failure analysis and propose improvements for both systems.
We find that for a system that always returns the same ranking it is not useful to improve its accuracy, but rather that it benefits most from fairness-aware post-processing. By contrast, a fairness-aware systems does benefit from a higher accuracy, since EEL requires that equally relevant items are treated the same. We note that the generalizability of our investigation is limited due to the small size of the FAIR-TREC 2020 dataset and recommend that a larger dataset be made available. ...
Recently, concerns about the fairness of IR algorithms have surfaced. In particular, research is being done into whether IR algorithms are fair to producers, people or organizations that provide the items that are retrieved by IR algorithms. The higher an item is in the ranked list, the more attention it receives from users. This attention translates to benefits for the producers, e.g. fame or financial compensation.
In this thesis we investigate the fairness of IR algorithms in terms of a specific measure for provider fairness: the Expected Exposure Loss (EEL). This measure measures whether the providers of equally relevant items receive the same amount of attention in expectation. EEL was first proposed as part of the 2020 TREC Fair Ranking track (FAIR-TREC), which also provided a matching dataset. We investigate for two IR systems whether they achieve fairness on this dataset. We conduct a failure analysis and propose improvements for both systems.
We find that for a system that always returns the same ranking it is not useful to improve its accuracy, but rather that it benefits most from fairness-aware post-processing. By contrast, a fairness-aware systems does benefit from a higher accuracy, since EEL requires that equally relevant items are treated the same. We note that the generalizability of our investigation is limited due to the small size of the FAIR-TREC 2020 dataset and recommend that a larger dataset be made available. ...
As the amount of information available in the world grows, Information Retrieval (IR) systems have become an integral part of day to day life. They determine what subset of the large pool of information is shown to people. IR algorithms determine which items should be returned in response to a query and rank the results in a ranked list.
Recently, concerns about the fairness of IR algorithms have surfaced. In particular, research is being done into whether IR algorithms are fair to producers, people or organizations that provide the items that are retrieved by IR algorithms. The higher an item is in the ranked list, the more attention it receives from users. This attention translates to benefits for the producers, e.g. fame or financial compensation.
In this thesis we investigate the fairness of IR algorithms in terms of a specific measure for provider fairness: the Expected Exposure Loss (EEL). This measure measures whether the providers of equally relevant items receive the same amount of attention in expectation. EEL was first proposed as part of the 2020 TREC Fair Ranking track (FAIR-TREC), which also provided a matching dataset. We investigate for two IR systems whether they achieve fairness on this dataset. We conduct a failure analysis and propose improvements for both systems.
We find that for a system that always returns the same ranking it is not useful to improve its accuracy, but rather that it benefits most from fairness-aware post-processing. By contrast, a fairness-aware systems does benefit from a higher accuracy, since EEL requires that equally relevant items are treated the same. We note that the generalizability of our investigation is limited due to the small size of the FAIR-TREC 2020 dataset and recommend that a larger dataset be made available.
Recently, concerns about the fairness of IR algorithms have surfaced. In particular, research is being done into whether IR algorithms are fair to producers, people or organizations that provide the items that are retrieved by IR algorithms. The higher an item is in the ranked list, the more attention it receives from users. This attention translates to benefits for the producers, e.g. fame or financial compensation.
In this thesis we investigate the fairness of IR algorithms in terms of a specific measure for provider fairness: the Expected Exposure Loss (EEL). This measure measures whether the providers of equally relevant items receive the same amount of attention in expectation. EEL was first proposed as part of the 2020 TREC Fair Ranking track (FAIR-TREC), which also provided a matching dataset. We investigate for two IR systems whether they achieve fairness on this dataset. We conduct a failure analysis and propose improvements for both systems.
We find that for a system that always returns the same ranking it is not useful to improve its accuracy, but rather that it benefits most from fairness-aware post-processing. By contrast, a fairness-aware systems does benefit from a higher accuracy, since EEL requires that equally relevant items are treated the same. We note that the generalizability of our investigation is limited due to the small size of the FAIR-TREC 2020 dataset and recommend that a larger dataset be made available.
Bachelor thesis
(2019)
-
Maaike Visser, Stefan van der Heijden, Stas Mironov, Nils Hullegien, Adam Head, Jan S. Rellermeyer
The Organic Rankine Cycle Hybrid Integrated Device (ORCHID) is a small scale power plant that is used to study the fundamental gas dynamic behavior of dense organic fluids, as well as the behavior of turbomachinery. In order to draw accurate conclusions about the raw sensor data generated by the ORCHID one has to know when the system is in steady-state. Currently, determining the steady state over historical data is cumbersome, and difficult to do in real time.
Our application aims to solve the problems with the current information workflow by consolidating the functionality that is currently spread across multiple applications into one main application, as well by offering steady state detection over real-time data. Aside from the lack of steady state detection capabilities, our client indicated that the applications currently in use often lag or crash. Therefore we defined three design goals: Performance, Reliability, and Ease of Use.
The main challenge we encountered during this phase was finding a way to properly connect the different external applications needed to properly process the ORCHID's data. The design goals were continuously referenced during the implementation phase to ensure the quality of our application. Additionally, we used unit, integration, and manual testing. The last category also comprised user tests conducted with our client to ensure that the final product would meet his requirements.
With our final application, we solve the client's main problem: it is now possible to detect whether or not a system is in steady state while an experiment is being conducted. This greatly reduces both the amount of time the client has to invest, as well as the amount of energy needed to conduct a successful experiment. ...
Our application aims to solve the problems with the current information workflow by consolidating the functionality that is currently spread across multiple applications into one main application, as well by offering steady state detection over real-time data. Aside from the lack of steady state detection capabilities, our client indicated that the applications currently in use often lag or crash. Therefore we defined three design goals: Performance, Reliability, and Ease of Use.
The main challenge we encountered during this phase was finding a way to properly connect the different external applications needed to properly process the ORCHID's data. The design goals were continuously referenced during the implementation phase to ensure the quality of our application. Additionally, we used unit, integration, and manual testing. The last category also comprised user tests conducted with our client to ensure that the final product would meet his requirements.
With our final application, we solve the client's main problem: it is now possible to detect whether or not a system is in steady state while an experiment is being conducted. This greatly reduces both the amount of time the client has to invest, as well as the amount of energy needed to conduct a successful experiment. ...
The Organic Rankine Cycle Hybrid Integrated Device (ORCHID) is a small scale power plant that is used to study the fundamental gas dynamic behavior of dense organic fluids, as well as the behavior of turbomachinery. In order to draw accurate conclusions about the raw sensor data generated by the ORCHID one has to know when the system is in steady-state. Currently, determining the steady state over historical data is cumbersome, and difficult to do in real time.
Our application aims to solve the problems with the current information workflow by consolidating the functionality that is currently spread across multiple applications into one main application, as well by offering steady state detection over real-time data. Aside from the lack of steady state detection capabilities, our client indicated that the applications currently in use often lag or crash. Therefore we defined three design goals: Performance, Reliability, and Ease of Use.
The main challenge we encountered during this phase was finding a way to properly connect the different external applications needed to properly process the ORCHID's data. The design goals were continuously referenced during the implementation phase to ensure the quality of our application. Additionally, we used unit, integration, and manual testing. The last category also comprised user tests conducted with our client to ensure that the final product would meet his requirements.
With our final application, we solve the client's main problem: it is now possible to detect whether or not a system is in steady state while an experiment is being conducted. This greatly reduces both the amount of time the client has to invest, as well as the amount of energy needed to conduct a successful experiment.
Our application aims to solve the problems with the current information workflow by consolidating the functionality that is currently spread across multiple applications into one main application, as well by offering steady state detection over real-time data. Aside from the lack of steady state detection capabilities, our client indicated that the applications currently in use often lag or crash. Therefore we defined three design goals: Performance, Reliability, and Ease of Use.
The main challenge we encountered during this phase was finding a way to properly connect the different external applications needed to properly process the ORCHID's data. The design goals were continuously referenced during the implementation phase to ensure the quality of our application. Additionally, we used unit, integration, and manual testing. The last category also comprised user tests conducted with our client to ensure that the final product would meet his requirements.
With our final application, we solve the client's main problem: it is now possible to detect whether or not a system is in steady state while an experiment is being conducted. This greatly reduces both the amount of time the client has to invest, as well as the amount of energy needed to conduct a successful experiment.