Ali Khatami
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
5 records found
1
Today's software development ecosystem offers an unprecedented array of quality assurance mechanisms. Continuous integration (CI) platforms, automated testing frameworks, automated static analysis tools, and code review systems are now readily available to developers, particularly in open-source software (OSS) projects hosted on platforms like GitHub. Yet this abundance does not automatically translate into effective adoption. Instead, it introduces a new challenge: the rapid expansion of Software Quality Assurance (SQA) practices and tools has created a complex decision space where teams must determine which practices to adopt, how to configure them effectively, how to integrate them into development workflows, and how to design processes that are relevant for their specific project context. This dissertation aims to construct a holistic understanding of software quality assurance by bringing together different aspects of open-source SQA practices that can inform future decision-making.
We begin by establishing an empirical understanding of how SQA practices are currently employed in the OSS world. Projects tend to adopt practices unevenly across dimensions, while broader investment across SQA practices is more common among mature projects. This raises a deeper question about whether developers actually understand the mechanisms already present in their projects. Our survey of open-source maintainers and contributors confirms this concern, uncovering an awareness gap: while developers broadly know that practices such as CI and testing are in place, their awareness is more limited when asked about specific mechanisms and tools.
To address the awareness gap, we design and evaluate \textsc{RepoInsights}, a software quality assurance analytics dashboard consolidating data from testing, code review, and automated workflow activities. While participants value the overview, they express a need for richer and more fine-grained contextual information. This motivates a deeper investigation into GitHub Actions (GHA), a platform deeply integrated into the GitHub ecosystem and a promising source of such contextual depth.
We investigate GitHub Actions through two complementary lenses, revealing two further findings:
workflow configurations suffer from recurring quality issues that require active maintenance, and configured workflows are not necessarily used in practice, as configuration does not imply actual utilization.
Taken together, our five studies contribute toward a deeper understanding of how software quality assurance operates in open-source software development on GitHub.
The empirical findings and conceptual contributions of this dissertation, including the awareness gap, the configuration–usage gap, and the characterization of developer responses to GHA workflow failures, contribute to the holistic understanding needed to navigate the complex SQA decision space in open-source software development. ...
We begin by establishing an empirical understanding of how SQA practices are currently employed in the OSS world. Projects tend to adopt practices unevenly across dimensions, while broader investment across SQA practices is more common among mature projects. This raises a deeper question about whether developers actually understand the mechanisms already present in their projects. Our survey of open-source maintainers and contributors confirms this concern, uncovering an awareness gap: while developers broadly know that practices such as CI and testing are in place, their awareness is more limited when asked about specific mechanisms and tools.
To address the awareness gap, we design and evaluate \textsc{RepoInsights}, a software quality assurance analytics dashboard consolidating data from testing, code review, and automated workflow activities. While participants value the overview, they express a need for richer and more fine-grained contextual information. This motivates a deeper investigation into GitHub Actions (GHA), a platform deeply integrated into the GitHub ecosystem and a promising source of such contextual depth.
We investigate GitHub Actions through two complementary lenses, revealing two further findings:
workflow configurations suffer from recurring quality issues that require active maintenance, and configured workflows are not necessarily used in practice, as configuration does not imply actual utilization.
Taken together, our five studies contribute toward a deeper understanding of how software quality assurance operates in open-source software development on GitHub.
The empirical findings and conceptual contributions of this dissertation, including the awareness gap, the configuration–usage gap, and the characterization of developer responses to GHA workflow failures, contribute to the holistic understanding needed to navigate the complex SQA decision space in open-source software development. ...
Today's software development ecosystem offers an unprecedented array of quality assurance mechanisms. Continuous integration (CI) platforms, automated testing frameworks, automated static analysis tools, and code review systems are now readily available to developers, particularly in open-source software (OSS) projects hosted on platforms like GitHub. Yet this abundance does not automatically translate into effective adoption. Instead, it introduces a new challenge: the rapid expansion of Software Quality Assurance (SQA) practices and tools has created a complex decision space where teams must determine which practices to adopt, how to configure them effectively, how to integrate them into development workflows, and how to design processes that are relevant for their specific project context. This dissertation aims to construct a holistic understanding of software quality assurance by bringing together different aspects of open-source SQA practices that can inform future decision-making.
We begin by establishing an empirical understanding of how SQA practices are currently employed in the OSS world. Projects tend to adopt practices unevenly across dimensions, while broader investment across SQA practices is more common among mature projects. This raises a deeper question about whether developers actually understand the mechanisms already present in their projects. Our survey of open-source maintainers and contributors confirms this concern, uncovering an awareness gap: while developers broadly know that practices such as CI and testing are in place, their awareness is more limited when asked about specific mechanisms and tools.
To address the awareness gap, we design and evaluate \textsc{RepoInsights}, a software quality assurance analytics dashboard consolidating data from testing, code review, and automated workflow activities. While participants value the overview, they express a need for richer and more fine-grained contextual information. This motivates a deeper investigation into GitHub Actions (GHA), a platform deeply integrated into the GitHub ecosystem and a promising source of such contextual depth.
We investigate GitHub Actions through two complementary lenses, revealing two further findings:
workflow configurations suffer from recurring quality issues that require active maintenance, and configured workflows are not necessarily used in practice, as configuration does not imply actual utilization.
Taken together, our five studies contribute toward a deeper understanding of how software quality assurance operates in open-source software development on GitHub.
The empirical findings and conceptual contributions of this dissertation, including the awareness gap, the configuration–usage gap, and the characterization of developer responses to GHA workflow failures, contribute to the holistic understanding needed to navigate the complex SQA decision space in open-source software development.
We begin by establishing an empirical understanding of how SQA practices are currently employed in the OSS world. Projects tend to adopt practices unevenly across dimensions, while broader investment across SQA practices is more common among mature projects. This raises a deeper question about whether developers actually understand the mechanisms already present in their projects. Our survey of open-source maintainers and contributors confirms this concern, uncovering an awareness gap: while developers broadly know that practices such as CI and testing are in place, their awareness is more limited when asked about specific mechanisms and tools.
To address the awareness gap, we design and evaluate \textsc{RepoInsights}, a software quality assurance analytics dashboard consolidating data from testing, code review, and automated workflow activities. While participants value the overview, they express a need for richer and more fine-grained contextual information. This motivates a deeper investigation into GitHub Actions (GHA), a platform deeply integrated into the GitHub ecosystem and a promising source of such contextual depth.
We investigate GitHub Actions through two complementary lenses, revealing two further findings:
workflow configurations suffer from recurring quality issues that require active maintenance, and configured workflows are not necessarily used in practice, as configuration does not imply actual utilization.
Taken together, our five studies contribute toward a deeper understanding of how software quality assurance operates in open-source software development on GitHub.
The empirical findings and conceptual contributions of this dissertation, including the awareness gap, the configuration–usage gap, and the characterization of developer responses to GHA workflow failures, contribute to the holistic understanding needed to navigate the complex SQA decision space in open-source software development.
The qualitative factor in software testing
A systematic mapping study of qualitative methods
Software testing research has provided metrics on efficiency, error rates, and insights into the effectiveness of testing methodologies and tools. However, these tell only a part of the story. The qualitative dimension, which studies experiences, perceptions, and decision-making processes is crucial, but less prevalent in literature. This study aims to systematically map qualitative research in software testing to consolidate and categorize the methodologies used in qualitative testing research, highlight their importance, and identify patterns, gaps, and future directions. We conducted a systematic mapping study, identifying and analyzing 102 primary studies from 2003 to 2023. We categorized the studies according to research strategies, data collection, and data analysis methods. We identified case studies and grounded theory as the most prevalent research strategies. Researchers primarily used semi-structured interviews and thematic analysis to understand how practitioners work and gather stakeholder perspectives. The subject areas most covered by qualitative studies included software testing processes and risks, and test automation. Areas such as test oracles, and machine learning were underrepresented. We also assessed the quality of reporting and the methodological rigor, emphasizing the challenges and limitations identified during the process. Through this study, we provide a comprehensive overview of qualitative research practices in software testing, revealing trends, gaps, and methodological insights.
...
Software testing research has provided metrics on efficiency, error rates, and insights into the effectiveness of testing methodologies and tools. However, these tell only a part of the story. The qualitative dimension, which studies experiences, perceptions, and decision-making processes is crucial, but less prevalent in literature. This study aims to systematically map qualitative research in software testing to consolidate and categorize the methodologies used in qualitative testing research, highlight their importance, and identify patterns, gaps, and future directions. We conducted a systematic mapping study, identifying and analyzing 102 primary studies from 2003 to 2023. We categorized the studies according to research strategies, data collection, and data analysis methods. We identified case studies and grounded theory as the most prevalent research strategies. Researchers primarily used semi-structured interviews and thematic analysis to understand how practitioners work and gather stakeholder perspectives. The subject areas most covered by qualitative studies included software testing processes and risks, and test automation. Areas such as test oracles, and machine learning were underrepresented. We also assessed the quality of reporting and the methodological rigor, emphasizing the challenges and limitations identified during the process. Through this study, we provide a comprehensive overview of qualitative research practices in software testing, revealing trends, gaps, and methodological insights.
Shaken, Not Stirred
How Developers Like Their Amplified Tests
Test amplification makes systematic changes to existing, manually written tests to provide tests complementary to an automated test suite. We consider developer-centric test amplification, where the developer explores, judges and edits the amplified tests before adding them to their maintained test suite. However, it is as yet unclear which kind of selection and editing steps developers take before including an amplified test into the test suite. In this paper we conduct an open source contribution study, amplifying tests of open source Java projects from GitHub. We report which deficiencies we observe in the amplified tests while manually filtering and editing them to open 39 pull requests with amplified tests. We present a detailed analysis of the maintainer's feedback regarding proposed changes, requested information, and expressed judgment. Our observations provide a basis for practitioners to take an informed decision on whether to adopt developer-centric test amplification. As several of the edits we observe are based on the developer's understanding of the amplified test, we conjecture that developer-centric test amplification should invest in supporting the developer to understand the amplified tests.
...
Test amplification makes systematic changes to existing, manually written tests to provide tests complementary to an automated test suite. We consider developer-centric test amplification, where the developer explores, judges and edits the amplified tests before adding them to their maintained test suite. However, it is as yet unclear which kind of selection and editing steps developers take before including an amplified test into the test suite. In this paper we conduct an open source contribution study, amplifying tests of open source Java projects from GitHub. We report which deficiencies we observe in the amplified tests while manually filtering and editing them to open 39 pull requests with amplified tests. We present a detailed analysis of the maintainer's feedback regarding proposed changes, requested information, and expressed judgment. Our observations provide a basis for practitioners to take an informed decision on whether to adopt developer-centric test amplification. As several of the edits we observe are based on the developer's understanding of the amplified test, we conjecture that developer-centric test amplification should invest in supporting the developer to understand the amplified tests.
To ensure the quality of software systems, software engineers can make use of a variety of quality assurance approaches, for example, software testing, modern code review, automated static analysis, and build automation. Each of these quality assurance practices have been studied in depth in isolation, but there is a clear knowledge gap when it comes to our understanding of how these approaches are being used in conjunction, or not. In our study, we broadly investigate whether and how these quality assurance approaches are being used in conjunction in the development of 1454 popular open source software projects on GitHub. Our study indicates that typically projects do not follow all quality assurance practices together with high intensity. In fact, we only observe weak correlation among some quality assurance practices. In general, our study provides a deeper understanding of how existing quality assurance approaches are currently being used in Java-based open source software development. Besides, we specifically zoom in on the more mature projects in our dataset, and generally we observe that more mature projects are more intense in their application of the quality assurance practices, with more focus on their ASAT usage, and code reviewing, but no strong change in their CI usage.
...
To ensure the quality of software systems, software engineers can make use of a variety of quality assurance approaches, for example, software testing, modern code review, automated static analysis, and build automation. Each of these quality assurance practices have been studied in depth in isolation, but there is a clear knowledge gap when it comes to our understanding of how these approaches are being used in conjunction, or not. In our study, we broadly investigate whether and how these quality assurance approaches are being used in conjunction in the development of 1454 popular open source software projects on GitHub. Our study indicates that typically projects do not follow all quality assurance practices together with high intensity. In fact, we only observe weak correlation among some quality assurance practices. In general, our study provides a deeper understanding of how existing quality assurance approaches are currently being used in Java-based open source software development. Besides, we specifically zoom in on the more mature projects in our dataset, and generally we observe that more mature projects are more intense in their application of the quality assurance practices, with more focus on their ASAT usage, and code reviewing, but no strong change in their CI usage.
Software engineers employ a variety of approaches to ensure the quality of software systems, including software testing, modern code review, automated static analysis, build automation, and continuous integration. To make effective decisions regarding quality assurance (QA), software engineers need to have an awareness of (1) the QA approaches that are in use in a project, and (2) how they are used. Through an exploratory, mixed-methods investigation we set out to better understand the awareness of software engineers in open-source software (OSS) development with regard to QA practices. This involved a largescale survey of 471 maintainers and contributors on GitHub. Our findings indicate that a high-level awareness among the respondents is common, but also that the respondents are less certain about how the practices are adopted; we further consider the perspective of both the contributor and the maintainer.
...
Software engineers employ a variety of approaches to ensure the quality of software systems, including software testing, modern code review, automated static analysis, build automation, and continuous integration. To make effective decisions regarding quality assurance (QA), software engineers need to have an awareness of (1) the QA approaches that are in use in a project, and (2) how they are used. Through an exploratory, mixed-methods investigation we set out to better understand the awareness of software engineers in open-source software (OSS) development with regard to QA practices. This involved a largescale survey of 471 maintainers and contributors on GitHub. Our findings indicate that a high-level awareness among the respondents is common, but also that the respondents are less certain about how the practices are adopted; we further consider the perspective of both the contributor and the maintainer.