Print Email Facebook Twitter Enriching Machine Learning Model Metadata Title Enriching Machine Learning Model Metadata: Collecting performance metadata through automatic evaluation Author Kant, Henk (TU Delft Electrical Engineering, Mathematics and Computer Science) Contributor Bozzon, A. (mentor) Katsifodimos, A (mentor) Hai, R. (mentor) Li, Z. (mentor) Proksch, S. (graduation committee) Degree granting institution Delft University of Technology Programme Computer Science Date 2023-05-08 Abstract As the sharing of machine learning (ML) models has increased in popularity, more so-called model zoos are created. These repositories facilitate the sharing of models and their metadata, and other people to find and re-use an existing model. However, the metadata provided for models is insufficient, with little focus on practical aspects of a model such as performance and limitations. This leaves model zoos to provide little functinality beyond the sharing of existing models, and potential users unable to find an optimal model for their needs. In this thesis we focus on the reporting of model performance, and aim to answer two questions: (1) What are the limitations of model performance reporting found in current model zoos, and (2) How can we provide rich, comparable performance metadata for these models. As a result, we have created a framework which can be used to create a benchmarking system for ML models. Our framework enables the automatic evaluation of models from any source, and through its design encourages the collection of rich and comparable performance metadata for those models. To demonstrate the effectiveness of our framework, we have created a benchmarking system and evaluated 1215 models from existing model zoos, along with a new interface to view this new metadata, and have enabled new use-cases such as the performance comparison of ML models. Subject Machine learningModel zooModel evaluationBenchmarking To reference this document use: http://resolver.tudelft.nl/uuid:315d5fcd-1255-4d84-bad5-4254ef31f8f0 Part of collection Student theses Document type master thesis Rights © 2023 Henk Kant Files PDF HenkKant_thesis_Enriching ... tadata.pdf 2.27 MB Close viewer /islandora/object/uuid:315d5fcd-1255-4d84-bad5-4254ef31f8f0/datastream/OBJ/view