MP
M. Pigmans
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
2 records found
1
Improving Adversarial Attacks on Decision Tree Ensembles
Exploring the impact of starting points on attack performance
Most of the adversarial attacks suitable for attacking decision tree ensembles work by doing multiple local searches from randomly selected starting points, around the to be attacked victim. In this thesis we investigate the impact of these starting points on the performance of the attack, and find that the starting points significantly impact the performance: some do much better than others. However, we do find that this is not the case for all attacked points, as there are large differences between points in how difficult they are to attack and for all datasets some points are always optimally attacked.
We compare the baseline randomly selected points to three alternative strategies. First, we try alternate random distributions, playing with both the standard deviation, to create a more narrow cone around the victim point, and mean, creating bimodal distributions further away from the victim point. We find that for some datasets these can give up to $5$-$7\%$ improved performance on subsets of the dataset, but these improvements do not generalize to the remainder of the dataset. In general, as long as the distribution is wide enough to successfully find starting points we do not find a substantial performance change.
Secondly, we try to remove the randomness and attack from a fixed direction. For the simpler datasets we find it is possible for a starting direction to perform better than random starting points, but for larger datasets performance becomes much worse. We also try an attack from all main directions around the victim point, which we find performs much worse than $5$-$20$ times fewer random points.
Lastly, we create an attack strategy where we select the closest points that scored well on previously attacked victims. We find that on smaller test sets this gets outperformed by the baseline, but when we extend the attack and give more possible previously well performing starting points we match or outperform the baseline slightly. ...
We compare the baseline randomly selected points to three alternative strategies. First, we try alternate random distributions, playing with both the standard deviation, to create a more narrow cone around the victim point, and mean, creating bimodal distributions further away from the victim point. We find that for some datasets these can give up to $5$-$7\%$ improved performance on subsets of the dataset, but these improvements do not generalize to the remainder of the dataset. In general, as long as the distribution is wide enough to successfully find starting points we do not find a substantial performance change.
Secondly, we try to remove the randomness and attack from a fixed direction. For the simpler datasets we find it is possible for a starting direction to perform better than random starting points, but for larger datasets performance becomes much worse. We also try an attack from all main directions around the victim point, which we find performs much worse than $5$-$20$ times fewer random points.
Lastly, we create an attack strategy where we select the closest points that scored well on previously attacked victims. We find that on smaller test sets this gets outperformed by the baseline, but when we extend the attack and give more possible previously well performing starting points we match or outperform the baseline slightly. ...
Most of the adversarial attacks suitable for attacking decision tree ensembles work by doing multiple local searches from randomly selected starting points, around the to be attacked victim. In this thesis we investigate the impact of these starting points on the performance of the attack, and find that the starting points significantly impact the performance: some do much better than others. However, we do find that this is not the case for all attacked points, as there are large differences between points in how difficult they are to attack and for all datasets some points are always optimally attacked.
We compare the baseline randomly selected points to three alternative strategies. First, we try alternate random distributions, playing with both the standard deviation, to create a more narrow cone around the victim point, and mean, creating bimodal distributions further away from the victim point. We find that for some datasets these can give up to $5$-$7\%$ improved performance on subsets of the dataset, but these improvements do not generalize to the remainder of the dataset. In general, as long as the distribution is wide enough to successfully find starting points we do not find a substantial performance change.
Secondly, we try to remove the randomness and attack from a fixed direction. For the simpler datasets we find it is possible for a starting direction to perform better than random starting points, but for larger datasets performance becomes much worse. We also try an attack from all main directions around the victim point, which we find performs much worse than $5$-$20$ times fewer random points.
Lastly, we create an attack strategy where we select the closest points that scored well on previously attacked victims. We find that on smaller test sets this gets outperformed by the baseline, but when we extend the attack and give more possible previously well performing starting points we match or outperform the baseline slightly.
We compare the baseline randomly selected points to three alternative strategies. First, we try alternate random distributions, playing with both the standard deviation, to create a more narrow cone around the victim point, and mean, creating bimodal distributions further away from the victim point. We find that for some datasets these can give up to $5$-$7\%$ improved performance on subsets of the dataset, but these improvements do not generalize to the remainder of the dataset. In general, as long as the distribution is wide enough to successfully find starting points we do not find a substantial performance change.
Secondly, we try to remove the randomness and attack from a fixed direction. For the simpler datasets we find it is possible for a starting direction to perform better than random starting points, but for larger datasets performance becomes much worse. We also try an attack from all main directions around the victim point, which we find performs much worse than $5$-$20$ times fewer random points.
Lastly, we create an attack strategy where we select the closest points that scored well on previously attacked victims. We find that on smaller test sets this gets outperformed by the baseline, but when we extend the attack and give more possible previously well performing starting points we match or outperform the baseline slightly.
Teaching Assistant Management Platform
Automating the recruitment and scheduling of teaching assistants
Bachelor thesis
(2018)
-
Max Pigmans, Ruben Keulemans, Geert Habben Jansen, Max van Deursen, Xavier Devroey, Stefan Hugtenburg
The majority of the courses in the Computer Science Bachelor at the Delft University of Technology use so called lab sessions to provide an opportunity for students to ask questions about course material and get feedback on their assignment. In order to optimally support the students, teaching assistants, or TAs, are appointed to assist the lecturer during the lab sessions. With the number of students in the Bachelor quickly growing, the process of manually recruiting students to become a TA and assigning the TAs to lab sessions is becoming infeasible.
This project aims to ease the process of gathering and scheduling TAs. In order to achieve this goal, the Teaching Assistant Management platform, or TAM, has been developed. All parties involved in the process of appointing TAs can use TAM to provide their input. Lecturers can register their courses on TAM, students are able to indicate their interest to help with different courses and representatives from Education and Student Affairs can validate the application of the interested students. Using this input, TAM is able to automatically create a schedule by assigning TAs to lab sessions. To provide an algorithm for the automatic generation of schedules, a model based on the minimumcost max flow problem is created. Due to complications with the implementation of the minimum-cost max flow model, the schedule generation is handled by a linear solver: Gurobi. By modeling the constrains for a schedule to be considered valid, Gurobi is used to process the input of the users into an optimized schedule.
TAM consists of three components: a MySQL database, a backend written using Spring, containing the business logic and the implementation of the scheduler, and a frontend website created with Vue to provide an interface to the users. The frontend and the backend are connected using a REST API.
A unique aspect of the project is the live deployment of TAM. At the end of the fourth week, the first version was deployed, allowing interested students to submit their course preferences. Subsequent features have been deployed iteratively. During the development, muliple problems have been encountered. The team underestimated the time required to learn the new technologies, as well as the time needed to maintain a system in production. Furthermore, configuring Single Sign-On required more time than expected. ...
This project aims to ease the process of gathering and scheduling TAs. In order to achieve this goal, the Teaching Assistant Management platform, or TAM, has been developed. All parties involved in the process of appointing TAs can use TAM to provide their input. Lecturers can register their courses on TAM, students are able to indicate their interest to help with different courses and representatives from Education and Student Affairs can validate the application of the interested students. Using this input, TAM is able to automatically create a schedule by assigning TAs to lab sessions. To provide an algorithm for the automatic generation of schedules, a model based on the minimumcost max flow problem is created. Due to complications with the implementation of the minimum-cost max flow model, the schedule generation is handled by a linear solver: Gurobi. By modeling the constrains for a schedule to be considered valid, Gurobi is used to process the input of the users into an optimized schedule.
TAM consists of three components: a MySQL database, a backend written using Spring, containing the business logic and the implementation of the scheduler, and a frontend website created with Vue to provide an interface to the users. The frontend and the backend are connected using a REST API.
A unique aspect of the project is the live deployment of TAM. At the end of the fourth week, the first version was deployed, allowing interested students to submit their course preferences. Subsequent features have been deployed iteratively. During the development, muliple problems have been encountered. The team underestimated the time required to learn the new technologies, as well as the time needed to maintain a system in production. Furthermore, configuring Single Sign-On required more time than expected. ...
The majority of the courses in the Computer Science Bachelor at the Delft University of Technology use so called lab sessions to provide an opportunity for students to ask questions about course material and get feedback on their assignment. In order to optimally support the students, teaching assistants, or TAs, are appointed to assist the lecturer during the lab sessions. With the number of students in the Bachelor quickly growing, the process of manually recruiting students to become a TA and assigning the TAs to lab sessions is becoming infeasible.
This project aims to ease the process of gathering and scheduling TAs. In order to achieve this goal, the Teaching Assistant Management platform, or TAM, has been developed. All parties involved in the process of appointing TAs can use TAM to provide their input. Lecturers can register their courses on TAM, students are able to indicate their interest to help with different courses and representatives from Education and Student Affairs can validate the application of the interested students. Using this input, TAM is able to automatically create a schedule by assigning TAs to lab sessions. To provide an algorithm for the automatic generation of schedules, a model based on the minimumcost max flow problem is created. Due to complications with the implementation of the minimum-cost max flow model, the schedule generation is handled by a linear solver: Gurobi. By modeling the constrains for a schedule to be considered valid, Gurobi is used to process the input of the users into an optimized schedule.
TAM consists of three components: a MySQL database, a backend written using Spring, containing the business logic and the implementation of the scheduler, and a frontend website created with Vue to provide an interface to the users. The frontend and the backend are connected using a REST API.
A unique aspect of the project is the live deployment of TAM. At the end of the fourth week, the first version was deployed, allowing interested students to submit their course preferences. Subsequent features have been deployed iteratively. During the development, muliple problems have been encountered. The team underestimated the time required to learn the new technologies, as well as the time needed to maintain a system in production. Furthermore, configuring Single Sign-On required more time than expected.
This project aims to ease the process of gathering and scheduling TAs. In order to achieve this goal, the Teaching Assistant Management platform, or TAM, has been developed. All parties involved in the process of appointing TAs can use TAM to provide their input. Lecturers can register their courses on TAM, students are able to indicate their interest to help with different courses and representatives from Education and Student Affairs can validate the application of the interested students. Using this input, TAM is able to automatically create a schedule by assigning TAs to lab sessions. To provide an algorithm for the automatic generation of schedules, a model based on the minimumcost max flow problem is created. Due to complications with the implementation of the minimum-cost max flow model, the schedule generation is handled by a linear solver: Gurobi. By modeling the constrains for a schedule to be considered valid, Gurobi is used to process the input of the users into an optimized schedule.
TAM consists of three components: a MySQL database, a backend written using Spring, containing the business logic and the implementation of the scheduler, and a frontend website created with Vue to provide an interface to the users. The frontend and the backend are connected using a REST API.
A unique aspect of the project is the live deployment of TAM. At the end of the fourth week, the first version was deployed, allowing interested students to submit their course preferences. Subsequent features have been deployed iteratively. During the development, muliple problems have been encountered. The team underestimated the time required to learn the new technologies, as well as the time needed to maintain a system in production. Furthermore, configuring Single Sign-On required more time than expected.