Science typically advances in small incremental steps, but in some rare instances it leaps forward. One discovery or invention can change how we see the world around us. Would it not be neat to be able to accurately pinpoint those moments of time in an objective way and thereby investigate science and technology’s progress? In 2016, Russel Funk of the University of Minnesota’s Carlson School of Management and Jason Owen-Smith from the University of Michigan published a measure for exactly this purpose.1 Their so-called consolidation-disruption (CD) index quantifies the extent to which published findings affect the subsequent use of the knowledge on which those findings relied. Worryingly, a widely cited subsequent study applied this measure on patents and scientific publications, finding a slowdown in disruptive progress.2 Thickening the plot, a later preprint attributed the finding to dataset artefacts.3 These studies prompt the need for an efficient way to calculate the CD index on large amounts of openly available data. [...]@en

IN CONTRAST TO physical objects and living things, software doesn’t deteriorate with the passage of time. While we age and our shoes fall apart, digital storage ensures that the software’s bits stay immutable. And yet, software needs substantial maintenance over time, owing to changes in its environment.1 Advancing technology and new requirements prompt us to modernize the software to keep it relevant. Here, I show how these changes happen in practice by describing the evolution and modernization of a burglar alarm security system I first developed a quarter-century ago. […]@en

The C preprocessor, a key element of the language, has become a liability due to its lack of integration with modern language semantics. This column describes the analysis of the C preprocessor usage in the Linux kernel, comprising 20 million lines of code, using the CScout refactoring browser. Processing limitations led to a solution leveraging a supercomputer’s parallel processing capabilities. The analysis divided the kernel’s source files across 32 supercomputer nodes and implemented a binary tournament database merging strategy. Initial efforts revealed multiple difficulties. Resolving them involved several false starts involving recursive SQL statements, an SQLite extension, and the GraphViz connected components tool. After a number of redesigns guided by stress-testing, the analysis finished in just 32 hours rather than a week, using 374 CPU hours and 640 GiB RAM on the supercomputer’s nodes.

@en

Generative AI based on large-language models is significantly impacting software development through IDE assistants, cloud-based APIs, and interactive chatbots for coding assistance. It excels in generating and translating code and data, navigating APIs, and creating boilerplate content, thereby enhancing productivity. However, it is prone to generating inaccurate information (“hallucinations”), erroneous code, and potentially introducing security vulnerabilities. To counter these risks, employing automated analysis tools, conducting rigorous testing, and maintaining a deep understanding of computer science concepts are essential. While generative AI can substantially aid development tasks it is not a replacement for human expertise, especially in understanding complex software, its requirements, and architecture.@en

Effective data processing workflows are crucial in data science, business analytics, and machine learning. Domain-specific tools can be invaluable, but often custom workflows are needed. Key to their success is splitting data and tasks into manageable chunks to enhance reliability, troubleshooting, and parallelization. Avoid monolithic programs; instead, favor modular designs that simplify data management and processing. Utilizing tools like xargs and GNU parallel can leverage multiple cores or hosts efficiently. Logging and documenting your workflow are essential for monitoring progress and understanding the process. Handling data subsets allows for quicker feedback and testing. Prepare for invalid data and system failures by designing processes that can gracefully manage exceptions and ensure results are reproducible and incremental, avoiding over-engineering. Simplify where possible, leveraging powerful, mature Unix tools and focusing optimization efforts on parts of the code responsible for the bulk of runtime costs. Adhere to software engineering practices to maintain the quality and integrity of your workflow, ensuring it remains a reliable asset to your organization.

@en

Effective change management is crucial for businesses heavily reliant on software and services to minimise incidents induced by changes. Unfortunately, in practice it is often difficult to effectively use artificial intelligence for IT Operations (AIOps) to enhance service management, primarily due to inadequate data quality. Establishing reliable links between changes and the induced incidents is crucial for identifying patterns, improving change deployment, identifying high-risk changes, and enhancing incident response. In this research, we investigate the enhancement of traceability between changes and incidents through AIOps methods. Our approach involves a close examination of incident-inducing changes, the replication of methods linking incidents to the changes that caused them, introducing an adapted method, and demonstrating its results using historical data and practical evaluations. Our findings reveal that incident-inducing changes exhibit different characteristics dependent on context. Furthermore, a significant disparity exists between assessments based on historical data and real-world observation, with an increased occurrence of false positives when identifying links between unlabeled changes and incidents. This study highlights the complex nature of identifying links between changes and incidents, emphasising the contextual influence on AIOps method effectiveness. While we are actively working on improving the quality of current data through AIOps approaches, it remains apparent that further measures are necessary to address issues like data imbalances and promote a postmortem culture that brings attention to the value of properly administrating tickets. A better overview of change failure rates contributes to improved risk compliance and reliable change management.

@en

RDBUnit is a unit testing framework designed to test relational database queries, created out of a need for unit testing them while working on software analytics tasks. It is available as a Python package on PyPI and open-source software on GitHub. RDBUnit tests consist of three parts: setup, query, and expected result, with the input and output defined as table contents. The framework utilizes a domain-specific language (DSL) for test specifications, employs a simple parsing mechanism, and uses a class hierarchy for managing database differences. It evaluates test results through SQL code generated and handled by the database engine. RDBUnit supports SQLite, mySQL, and PostgreSQL, and is implemented as a command-line tool suitable for diverse operating systems and continuous integration environments. It has proved beneficial in identifying subtle bugs and facilitating a focused and efficient approach to experimenting with SQL queries, especially in big data scenarios, signifying the assurance provided by unit testing in SQL-centric tasks.

@en

Code refactoring is an essential part of software development, because it reduces technical debt, enhances long-term code sustainability, and enables the implementation of functionality that might have been incompatible with an original design. IDEs automate many refactoring tasks, but they sometimes lack support for specific operations or languages. In such cases, regular expressions offer a powerful alternative, automating tedious tasks, reducing errors, and saving time. This article shares a practical example: extending the CScout refactoring browser to collect metrics on C preprocessor usage, which required addressing widespread cyclic dependencies. The changes were facilitated by the "git-subst"Git extension, which makes global text replacements using regular expressions in Git-managed files. A series of 30 git-subst invocations were automatically generated, again using regular expression replacements. While not a cure-all, regular expressions are invaluable for many refactoring tasks, making them a key skill for software developers.

@en

Machine learning (ML) techniques increase the effectiveness of software engineering (SE) lifecycle activities. We systematically collected, quality-assessed, summarized, and categorized 83 reviews in ML for SE published between 2009 and 2022, covering 6,117 primary studies. The SE areas most tackled with ML are software quality and testing, while human-centered areas appear more challenging for ML. We propose a number of ML for SE research challenges and actions, including conducting further empirical validation and industrial studies on ML, reconsidering deficient SE methods, documenting and automating data collection and pipeline processes, reexamining how industrial practitioners distribute their proprietary data, and implementing incremental ML approaches.

@en

Considerable scientific work involves locating, analyzing, systematizing, and synthesizing other publications, often with the help of online scientific publication databases and search engines. However, use of online sources suffers from a lack of repeatability and transparency, as well as from technical restrictions. Alexandria3k is a Python software package and an associated command-line tool that can populate embedded relational databases with slices from the complete set of several open publication metadata sets. These can then be employed for reproducible processing and analysis through versatile and performant queries. We demonstrate the software’s utility by visualizing the evolution of publications in diverse scientific fields and relationships among them, by outlining scientometric facts associated with COVID-19 research, and by replicating commonly-used bibliometric measures and findings regarding scientific productivity, impact, and disruption.@en

Developers and data scientists often struggle to write command-line inputs, even though graphical interfaces or tools like ChatGPT can assist. The solution? "ai-cli,"an open-source system inspired by GitHub Copilot that converts natural language prompts into executable commands for various Linux command-line tools. By tapping into OpenAI's API, which allows interaction through JSON HTTP requests, "ai-cli"transforms user queries into actionable command-line instructions. However, integrating AI assistance across multiple command-line tools, especially in open source settings, can be complex. Historically, operating systems could mediate, but individual tool functionality and the lack of a unified approach have made centralized integration challenging. The "ai-cli"tool, by bridging this gap through dynamic loading and linking with each program's Readline library API, makes command-line interfaces smarter and more user-friendly, opening avenues for further enhancement and cross-platform applicability.

@en

Existing work on the practical impact of software engineering (SE) research examines industrial relevance rather than adoption of study results, hence the question of how results have been practically applied remains open. To answer this and investigate the outcomes of impactful research, we performed a quantitative and qualitative analysis of 4,354 SE patents citing 1,690 SE papers published in four leading SE venues between 1975–2017. Moreover, we conducted a survey on 475 authors of 593 top-cited and awarded publications, achieving 26% response rate. Overall, researchers have equipped practitioners with various tools, processes, and methods, and improved many existing products. SE practice values knowledge-seeking research and is impacted by diverse cross-disciplinary SE areas. Practitioner-oriented publication venues appear more impactful than researcher-oriented ones, while industry-related tracks in conferences could enhance their impact. Some research works did not reach a wide footprint due to limited funding resources or unfavorable cost-benefit trade-off of the proposed solutions. The need for higher SE research funding could be corroborated through a dedicated empirical study. In general, the assessment of impact is subject to its definition. Therefore, academia and industry could jointly agree on a formal description to set a common ground for subsequent research on the topic.@en

We propose a testing framework for validating static typing procedures in compilers. Our core component is a program generator suitably crafted for producing programs that are likely to trigger typing compiler bugs. One of our main contributions is that our program generator gives rise to transformation-based compiler testing for finding typing bugs. We present two novel approaches (type erasure mutation and type overwriting mutation) that apply targeted transformations to an input program to reveal type inference and soundness compiler bugs respectively. Both approaches are guided by an intra-procedural type inference analysis used to capture type information flow. We implement our techniques as a tool, which we call Hephaestus. The extensibility of Hephaestus enables us to test the compilers of three popular JVM languages: Java, Kotlin, and Groovy. Within nine months of testing, we have found 156 bugs (137 confirmed and 85 fixed) with diverse manifestations and root causes in all the examined compilers. Most of the discovered bugs lie in the heart of many critical components related to static typing, such as type inference.

@en

Context: Software development projects increasingly adopt unit testing as a way to identify and correct program faults early in the construction process. Code that is unit tested should therefore have fewer failures associated with it. Objective: Compare the number of field failures arising in unit tested code against those arising in code that has not been unit tested. Method: We retrieved 2 083 979 crash incident reports associated with the Eclipse integrated development environment project, and processed them to obtain a set of 126 026 unique program failure stack traces associated with a specific popular release. We then run the JaCoCo code test coverage analysis on the same release, obtaining results on the line, instruction, and branch-level coverage of 216 392 methods. We also extracted from the source code the classes that are linked to a corresponding test class so as to limit test code coverage results to 1 263 classes with actual tests. Finally, we correlated unit tests with failures at the level of 9 523 failing tested methods. Results: Unit-tested code does not appear to be associated with fewer failures. Conclusion: Unit testing on its own may not be a sufficient method for preventing program failures.

@en

Context: An excessive number of code smells make a software system hard to evolve and maintain. Machine learning methods, in addition to metric-based and heuristic-based methods, have been recently applied to detect code smells; however, current methods are considered far from mature. Objective: First, explore the feasibility of applying deep learning models to detect smells without extensive feature engineering. Second, investigate the possibility of applying transfer-learning in the context of detecting code smells. Methods: We train smell detection models based on Convolution Neural Networks and Recurrent Neural Networks as their principal hidden layers along with autoencoder models. For the first objective, we perform training and evaluation on C# samples, whereas for the second objective, we train the models from C# code and evaluate the models over Java code samples and vice-versa. Results: We find it feasible to detect smells using deep learning methods though the models’ performance is smell-specific. Our experiments show that transfer-learning is definitely feasible for implementation smells with performance comparable to that of direct-learning. This work opens up a new paradigm to detect code smells by transfer-learning especially for the programming languages where the comprehensive code smell detection tools are not available.

@en

The ICSE 2021 paper titled 'PyCG: Practical Call Graph Generation in Python' comes with a replication package with the purpose of providing open access to (1) our prototype call graph generator, namely PyCG, and (2) the data and scripts that replicate the results of the paper. The Artifact Evaluation Committee found that this package leads to the reproduction of the results outlined in the paper and is openly available1. The replication package contains the following: 1) A Docker image which can be either built manually or downloaded from DockerHub. It contains the source code and installation of PyCG, as well as the installations of two other call graph generators (i.e., Pyan and Depends), which we compare PyCG with. 2) A micro-benchmark suite of 112 Python modules (Section I-A). 3) A macro-benchmark suite of 5 popular Python packages (Section I-B). 4) Python and Bash scripts used to execute PyCG, Pyan and Depends against the micro- and macro-benchmarks and compare the corresponding results.@en

We introduce, what is to the best of our knowledge, the first approach for systematically testing Object-Relational Mapping (ORM) systems. Our approach leverages differential testing to establish a test oracle for ORM-specific bugs. Specifically, we first generate random relational database schemas, set up the respective databases, and then, we query these databases using the APIs of the ORM systems under test. To tackle the challenge that ORMs lack a common input language, we generate queries written in an abstract query language. These abstract queries are translated into concrete, executable ORM queries, which are ultimately used to differentially test the correctness of target implementations. The effectiveness of our method heavily relies on the data inserted to the underlying databases. Therefore, we employ a solver-based approach for producing targeted database records with respect to the constraints of the generated queries. We implement our approach as a tool, called CYNTHIA, which found 28 bugs in five popular ORM systems. The vast majority of these bugs are confirmed (25 / 28), more than half were fixed (20 / 28), and three were marked as release blockers by the corresponding developers.

@en

A model regarding the lifetime of individual source code lines or tokens can estimate maintenance effort, guide preventive maintenance, and, more broadly, identify factors that can improve the efficiency of software development. We present methods and tools that allow tracking of each line’s or token’s birth and death. Through them, we analyze 3.3 billion source code element lifetime events in 89 revision control repositories. Statistical analysis shows that code lines are durable, with a median lifespan of about 2.4 years, and that young lines are more likely to be modified or deleted, following a Weibull distribution with the associated hazard rate decreasing over time. This behavior appears to be independent from specific characteristics of lines or tokens, as we could not determine factors that influence significantly their longevity across projects. The programing language, and developer tenure and experience were not found to be significantly correlated with line or token longevity, while project size and project age showed only a slight correlation.

@en

Acquiring developer-prized practical skills, knowledge, and experiences.@en