The ICSE 2021 paper titled 'PyCG: Practical Call Graph Generation in Python' comes with a replication package with the purpose of providing open access to (1) our prototype call graph generator, namely PyCG, and (2) the data and scripts that replicate the results of the paper. The Artifact Evaluation Committee found that this package leads to the reproduction of the results outlined in the paper and is openly available1. The replication package contains the following: 1) A Docker image which can be either built manually or downloaded from DockerHub. It contains the source code and installation of PyCG, as well as the installations of two other call graph generators (i.e., Pyan and Depends), which we compare PyCG with. 2) A micro-benchmark suite of 112 Python modules (Section I-A). 3) A macro-benchmark suite of 5 popular Python packages (Section I-B). 4) Python and Bash scripts used to execute PyCG, Pyan and Depends against the micro- and macro-benchmarks and compare the corresponding results.@en

Despite the substantial progress in compiler testing, research endeavors have mainly focused on detecting compiler crashes and subtle miscompilations caused by bugs in the implementation of compiler optimizations. Surprisingly, this growing body of work neglects other compiler components, most notably the front-end. In statically-typed programming languages with rich and expressive type systems and modern features, such as type inference or a mix of object-oriented with functional programming features, the process of static typing in compiler front-ends is complicated by a high-density of bugs. Such bugs can lead to the acceptance of incorrect programs (breaking code portability or the type system's soundness), the rejection of correct (e.g. well-typed) programs, and the reporting of misleading errors and warnings. We conduct, what is to the best of our knowledge, the first empirical study for understanding and characterizing typing-related compiler bugs. To do so, we manually study 320 typing-related bugs (along with their fixes and test cases) that are randomly sampled from four mainstream JVM languages, namely Java, Scala, Kotlin, and Groovy. We evaluate each bug in terms of several aspects, including their symptom, root cause, bug fix's size, and the characteristics of the bug-revealing test cases. Some representative observations indicate that: (1) more than half of the typing-related bugs manifest as unexpected compile-time errors: the buggy compiler wrongly rejects semantically correct programs, (2) the majority of typing-related bugs lie in the implementations of the underlying type systems and in other core components related to operations on types, (3) parametric polymorphism is the most pervasive feature in the corresponding test cases, (4) one third of typing-related bugs are triggered by non-compilable programs. We believe that our study opens up a new research direction by driving future researchers to build appropriate methods and techniques for a more holistic testing of compilers.

@en

Incremental and parallel builds are crucial features of modern build systems. Parallelism enables fast builds by running independent tasks simultaneously, while incrementality saves time and computing resources by processing the build operations that were affected by a particular code change. Writing build definitions that lead to error-free incremental and parallel builds is a challenging task. This is mainly because developers are often unable to predict the effects of build operations on the file system and how different build operations interact with each other. Faulty build scripts may seriously degrade the reliability of automated builds, as they cause build failures, and non-deterministic and incorrect outputs. To reason about arbitrary build executions, we present BuildFS, a generally-applicable model that takes into account the specification (as declared in build scripts) and the actual behavior (low-level file system operation) of build operations. We then formally define different types of faults related to incremental and parallel builds in terms of the conditions under which a file system operation violates the specification of a build operation. Our testing approach, which relies on the proposed model, analyzes the execution of single full build, translates it into BuildFS, and uncovers faults by checking for corresponding violations. We evaluate the effectiveness, efficiency, and applicability of our approach by examining 612 Make and Gradle projects. Notably, thanks to our treatment of build executions, our method is the first to handle JVM-oriented build systems. The results indicate that our approach is (1) able to uncover several important issues (247 issues found in 47 open-source projects have been confirmed and fixed by the upstream developers), and (2) much faster than a state-of-the-art tool for Make builds (the median and average speedup is 39X and 74X respectively).

@en

Puppet is a popular computer system configuration management tool. By providing abstractions that model system resources it allows administrators to set up computer systems in a reliable, predictable, and documented fashion. Its use suffers from two potential pitfalls. First, if ordering constraints are not correctly specified whenever a Puppet resource depends on another, the nondeterministic application of resources can lead to race conditions and consequent failures. Second, if a service is not tied to its resources (through the notification construct), the system may operate in a stale state whenever a resource gets modified. Such faults can degrade a computing infrastructure's availability and functionality. We have developed an approach that identifies these issues through the analysis of a Puppet program and its system call trace. Specifically, a formal model for traces allows us to capture the interactions of Puppet resources with the file system. By analyzing these interactions we identify (1) resources that are related to each other (e.g., operate on the same file), and (2) resources that should act as notifiers so that changes are correctly propagated. We then check the relationships from the trace's analysis against the program's dependency graph: a representation containing all the ordering constraints and notifications declared in the program. If a mismatch is detected, our system reports a potential fault. We have evaluated our method on a large set of popular Puppet modules, and discovered 92 previously unknown issues in 33 modules. Performance benchmarking shows that our approach can analyze in seconds real-world configurations with a magnitude measured in thousands of lines and millions of system calls.

@en

Online personal data are rarely, if ever, effectively controlled by the users they concern. Worse, as demonstrated by the numerous leaks reported each week, the organizations that store and process them fail to adequately safeguard the required confidentiality. In this paper, we propose pdguard, a framework that defines prototypes and demonstrates an architecture and an implementation that address both problems. In the context of pdguard, personal data are always stored encrypted as opaque objects. Processing them can only be performed through the pdguard application programming interface (api), under data and action-specific authorizations supplied online by third party agents. Through these agents, end-users can easily and reliably authorize and audit how organizations use their personal data. A static verifier can be employed to identify accidental api misuses. Following a security by design approach, pdguard changes the problem of personal data management from the, apparently, intractable problem of supervising processes, operations, personnel, and a large software stack to that of auditing the applications that use the framework for compliance. We demonstrate the framework’s applicability through a reference implementation, by building a pdguard-based e-shop, and by integrating pdguard into the The Guardian newspaper’s website identity application.

@en

JavaScript is one of the web's key building blocks. It is used by the majority of web sites and it is supported by all modern browsers. We present the first large-scale study of client-side JavaScript code over time. Specifically, we have collected and analyzed a dataset containing daily snapshots of JavaScript code coming from Alexa's Top 10000 web sites (~7.5 GB per day) for nine consecutive months, to study different temporal aspects of web client code. We found that scripts change often; typically every few days, indicating a rapid pace in web applications development. We also found that the lifetime of web sites themselves, measured as the time between JavaScript changes, is also short, in the same time scale. We then performed a qualitative analysis to investigate the nature of the changes that take place. We found that apart from standard changes such as the introduction of new functions, many changes are related to online configuration management. In addition, we examined JavaScript code reuse over time and especially the widespread reliance on third-party libraries. Furthermore, we observed how quality issues evolve by employing established static analysis tools to identify potential software bugs, whose evolution we tracked over time. Our results show that quality issues seem to persist over time, while vulnerable libraries tend to decrease.

@en

This article aims to shed light on notable aspects of the history of computing in Greece, starting from 1920. We cover six areas, including the early days of computing in the country, the transition to the Internet era, the formation of a computer-related educational infrastructure, the evolution of data networks, and the growth of the software and hardware industry. In each area we highlight findings that involve the important role of women in this technological evolution, how the risks taken by particular individuals pushed the boundaries of the field, the slow pace of the universities at the early stages, and the key part of multinational technology companies and state-controlled banks. Apart from numerous reports and publications, our sources also include interviews with individuals who played a major part in the development of computing in Greece.@en

Examining the different characteristics of open-source software in relation to security vulnerabilities, can provide the research community with findings that can lead to the development of more secure systems. We present a dataset where the reported vulnerabilities of 8694 open-source project versions, can be correlated with the corresponding source code and a number of software metrics. The metrics were obtained by analyzing the project's source code via well-established tools. Apart from commonly used metrics (e.g. loc), we also provide data related to modern development trends such as continuous integration and testing. We outline motivational examples based on the dataset we describe.

@en

With a code injection attack (CIA) an attacker can introduce malicious code into a computer program or system that fails to properly encode data that comes from an untrusted source. A CIA can have different forms depending on the execution context of the application and the location of the programming flaw that leads to the attack. Currently, CIAs are considered one of the most damaging classes of application attacks since they can severely affect an organisation's infrastructure and cause financial and reputational damage to it. In this paper we examine and categorize the countermeasures developed to detect the various attack forms. In particular, we identify two distinct categories. The first incorporates static program analysis tools used to eliminate flaws that can lead to such attacks during the development of the system. The second involves the use of dynamic detection safeguards that prevent code injection attacks while the system is in production mode. Our analysis is based on nonfunctional characteristics that are considered critical when creating security mechanisms. Such characteristics involve usability, overhead, implementation dependencies, false positives and false negatives. Our categorization and analysis can help both researchers and practitioners either to develop novel approaches, or use the appropriate mechanisms according to their needs.

@en

Security bugs are critical programming errors that can lead to serious vulnerabilities in software. Examining their behaviour and characteristics within a software ecosystem can provide the research community with data regarding their evolution, persistence and others. We present a dataset that we produced by applying static analysis to the Maven Central Repository (approximately 265GB of data) in order to detect potential security bugs. For our analysis we used FindBugs, a tool that examines Java bytecode to detect numerous types of bugs. The dataset contains the metrics’ results that FindBugs reports for every project version (a JAR) included in the ecosystem. For every version in our data repository, we also store specific metadata, such as the JAR’s size, its dependencies and others. Our dataset can be used to produce interesting research results involving security bugs, as we show in specific examples. @en

Cross-Site Scripting (XSS) is one of the most common web application vulnerabilities. It is therefore sometimes referred to as the “buffer overflow of the web.” Drawing a parallel from the current state of practice in preventing unauthorized native code execution (the typical goal in a code injection), we propose a script whitelisting approach to tame JavaScript-driven XSS attacks. Our scheme involves a transparent script interception layer placed in the browser's JavaScript engine. This layer is designed to detect every script that reaches the browser, from every possible route, and compare it to a list of valid scripts for the site or page being accessed; scripts not on the list are prevented from executing. To avoid the false positives caused by minor syntactic changes (e.g., due to dynamic code generation), our layer uses the concept of contextual fingerprints when comparing scripts. Contextual fingerprints are identifiers that represent specific elements of a script and its execution context. Fingerprints can be easily enriched with new elements, if needed, to enhance the proposed method's robustness. The list can be populated by the website's administrators or a trusted third party. To verify our approach, we have developed a prototype and tested it successfully against an extensive array of attacks that were performed on more than 50 real-world vulnerable web applications. We measured the browsing performance overhead of the proposed solution on eight websites that make heavy use of JavaScript. Our mechanism imposed an average overhead of 11.1% on the execution time of the JavaScript engine. When measured as part of a full browsing session, and for all tested websites, the overhead introduced by our layer was less than 0.05%. When script elements are altered or new scripts are added on the server side, a new fingerprint generation phase is required. To examine the temporal aspect of contextual fingerprints, we performed a short-term and a long-term experiment based on the same websites. The former, showed that in a short period of time (10 days), for seven of eight websites, the majority of valid fingerprints stay the same (more than 92% on average). The latter, though, indicated that, in the long run, the number of fingerprints that do not change is reduced. Both experiments can be seen as one of the first attempts to study the feasibility of a whitelisting approach for the web.@en

Programs draw significant parts of their functionality through the use of Application Programming Interfaces (APIs). Apart from the way developers incorporate APIs in their software, the stability of these programs depends on the design and implementation of the APIs. In this work, we report how we used software telemetry data to analyze the causes of API failures in Android applications. Specifically, we got 4.9 gb worth of crash data that thousands of applications sent to a centralized crash report management service. We processed that data to extract approximately a million stack traces, stitching together parts of chained exceptions, and established heuristic rules to draw the border between applications and the API calls. We examined a set of more than a half million stack traces associated with risky API calls to map the space of the most common application failure reasons. Our findings show that the top ones can be attributed to memory exhaustion, race conditions or deadlocks, and missing or corrupt resources. Given the classes of the crash causes we identified, we recommend API design and implementation choices, such as specific exceptions, default resources, and non-blocking algorithms, that can eliminate common failures. In addition, we argue that development tools like memory analyzers, thread debuggers, and static analyzers can prevent crashes through early code testing and analysis. Finally, some execution platform and framework designs for process and memory management can also eliminate some application crashes.

@en

J% is an extension of the Java programming language that efficiently supports the integration of domain-specific languages. In particular, J% allows the embedding of domain-specific language code into Java programs in a syntax-checked and type-safe manner. This paper presents Js support for the sql language. J% checks the syntax and semantics of sql statements at compile-time. It supports query validation against a database schema or through execution to a live database server. The J% compiler generates code that uses standard jdbc api calls, enhancing runtime efficiency and security against sql injection attacks.

@en

Examining a large number of software artifacts can provide the research community with data regarding quality and design. We present a dataset obtained by statically analyzing 22730 jar files taken from the Maven central archive, which is the de-facto application library repository for the Java ecosystem. For our analysis we used three popular static analysis tools that calculate metrics regarding object-oriented design, program size, and package design. The dataset contains the metrics results that every tool reports for every selected jar of the ecosystem. Our dataset can be used to produce interesting research results, such as measure the domain-specific language usage.

@en

Examining software ecosystems can provide the research community with data regarding artifacts, processes, and communities. We present a dataset obtained from the Maven central repository ecosystem (approximately 265gb of data) by statically analyzing the repository to detect potential software bugs. For our analysis we used FindBugs, a tool that examines Java bytecode to detect numerous types of bugs. The dataset contains the metrics results that FindBugs reports for every project version (a jar) included in the ecosystem. For every version we also stored specific metadata such as the jar's size, its dependencies and others. Our dataset can be used to produce interesting research results, as we show in specific examples.@en

A security vulnerability is a programming error that introduces a potentially exploitable weakness into a computer system. Such a vulnerability can severely affect an organization's infrastructure and cause significant financial damage to it. Hence, one of the basic pursuits in every new software release should be to mitigate such defects. A number of tools and techniques are available for performing vulnerability detection in software written in various programming platforms. One of the most common approaches to identify software vulnerabilities is static analysis [1]. This kind of analysis is performed by automated tools either on the program's source or object code and without actually executing it. However, since the formats in which static analysis tools store and present their results vary wildly, it is typically difficult to utilize many of them in the scope of a project. By automating the process of running a variety of vulnerability detectors and collecting their results in an efficient manner during development, the task of tracking security defects throughout the evolution history of software projects can be simplified. In this paper we present TRACER, a framework to support the development of secure applications by constantly monitoring software projects for vulnerabilities. TRACER simplifies the integration of existing tools that detect software vulnerabilities and promotes their use during development and maintenance. Instead of designing and implementing TRACER from the ground up, we built it on top of the open source Alitheia Core [2] platform, which is designed for facilitating large scale quantitative software engineering studies. While Alitheia Core aims for efficient estimation of the quality of software projects, TRACER was designed with a focus on software security. To support the specific objectives of TRACER, a set of new components was added at each level of the Alitheia Core architecture. These include a model for representing software vulnerabilities, a mechanism for automatic vulnerability detection triggering, a REST API for accessing the analysis results, and an archetype for plug-ins to integrate new vulnerability detection tools in the platform. Like Alitheia Core, TRACER monitors multiple data sources associated with the development of a software project, such as the source code repository and bug tracking system, and automatically analyzes each revision. Therefore it can be used to track security defects throughout the evolution of a project. In most cases, the detection of vulnerabilities on a software artifact involves only two steps: invoking an external tool created for this purpose with specific arguments as required, and evaluating the results it generates. There is a vast number of software vulnerability detection tools available, each one having different operating requirements. Such a tool can be integrated in TRACER by creating a corresponding driver that implements these two steps and stores the results using the data model provided by the platform. Thus we can leverage the functionality provided by existing tools, without duplicating it. Such an external tool driver is called a vulnerability detector plug-in, and it uses the Alitheia Core infrastructure to handle automatic activation, as well as storage and retrieval of results. Each vulnerability detector is associated with the set of vulnerability types it can detect and the different types of software artifacts or programming constructs that it can analyze. This allows the platform to automatically trigger it when needing to check if a software project or artifact is vulnerable to a specific type of attacks or a new artifact is submitted to the system for evaluation. To demonstrate the efficiency and usability of the platform, we have created plug-ins to integrate two different tools for vulnerability detection, namely: FindBugs [3], and Frama-c [4]. The former analyzes applications written in Java, while the latter examines applications written in c. This highlights the fact that our platform does not depend on the programming language used to develop the project that is being analyzed, and that the simplicity of integrating third party tools leads to high levels of expandability of the platform.@en

Software vulnerabilities can severely affect an organization's infrastructure and cause significant financial damage to it. A number of tools and techniques are available for performing vulnerability detection in software written in various programming platforms, in a pursuit to mitigate such defects. However, since the requirements for running such tools and the formats in which they store and present their results vary wildly, it is difficult to utilize many of them in the scope of a project. By simplifying the process of running a variety of vulnerability detectors and collecting their results in an efficient, automated manner during development, the task of tracking security defects throughout the evolution history of software projects is bolstered. In this paper we present tracer, a software framework and platform to support the development of more secure applications by constantly mon- itoring software projects for vulnerabilities. The platform allows the easy integration of existing tools that statically detect software vulnerabilities and promotes their use during software development and maintenance. To demonstrate the efficiency and usability of the platform, we integrated two popular static analysis tools, FindBugs and Frama-c as sample implementations, and report on preliminary results from their use.@en

Background. Security bugs are critical programming errors that can lead to serious vulnerabilities in software. Such bugs may allow an attacker to take over an application, steal data or prevent the application from working at all. Aim. We used the projects stored in the Maven repository to study the characteristics of security bugs individually and in relation to other software bugs. Specifically, we studied the evolution of security bugs through time. In addition, we examined their persistence and their relationship with a) the size of the corresponding version, and b) other bug categories. Method. We analyzed every project version of the Maven repository by using FindBugs, a popular static analysis tool. To see how security bugs evolve over time we took advantage of the repository's project history and dependency data. Results. Our results indicate that there is no simple rule governing the number of security bugs as a project evolves. In particular, we cannot say that across projects security-related defect counts increase or decrease significantly over time. Furthermore, security bugs are not eliminated in a way that is particularly different from the other bugs. In addition, the relation of security bugs with a project's size appears to be different from the relation of the bugs coming from other categories. Finally, even if bugs seem to have similar behaviour, severe security bugs seem to be unassociated with other bug categories. Conclusions. Our findings indicate that further research should be done to analyze the evolution of security bugs. Given the fact that our experiment included only Java projects, similar research could be done for another ecosystem. Finally, the fact that projects have their own idiosyncrasies concerning security bugs, could help us find the common characteristics of the projects where security bugs increase over time.@en

Modern programs depend on apis to implement a significant part of their functionality. Apart from the way developers use apis to build their software, the stability of these programs relies on the apis design and implementation. In this work, we evaluate the reliability of apis, by examining software telemetry data, in the form of stack traces, coming from Android application crashes. We got 4.9 GB worth of crash data that thousands of applications send to a centralized crash report management service. We processed that data to extract approximately a million stack traces, stitching together parts of chained exceptions, and established heuristic rules to draw the border between applications and api calls. We examined 80% of the stack traces to map the space of the most common application failure reasons. Our findings show that the top ones can be attributed to memory exhaustion, race conditions or deadlocks, and missing or corrupt resources. At the same time, a significant number of our stack traces (over 10%) remains unclassified due to generic unchecked exceptions, which do not highlight the problems that lead to crashes. Finally, given the classes of crash causes we found, we argue that api design and implementation improvements, such as specific exceptions, non-blocking algorithms, and default resources, can eliminate common failures.

@en

Purpose - The purpose of this paper is to propose a generic approach that prevents a specific class of code injection attacks (CIAs) in a novel way. Design/methodology/approach - To defend against CIAs this approach involves detecting attacks by using location-specific signatures to validate code statements. The signatures are unique identifiers that represent specific characteristics of a statement's execution. The key property that differentiates the scheme presented in this paper is that these characteristics do not depend entirely on the code statement, but also take into account elements from its execution context. Findings - The approach was applied successfully to defend against attacks targeting structured query language (SQL), XML Path Language and JavaScript with positive results. Originality/value - Despite many countermeasures that have been proposed the number of CIAs has been increasing. Malicious users seem to find new ways to introduce compromised embedded executable code to applications by using a variety of languages and techniques. Hence, a generic approach that defends against such attacks would be a useful countermeasure. This approach can defend attacks that involve both domain-specific languages (e.g. SQL) and general purpose languages (e.g. JavaScript) and can be used both against client-side and server-side attacks.@en