Evaluation systems have been long criticized for abusing and misusing bibliometric indicators. This has created a culture by which academics are constantly exposing their daily work to the standards they are expected to perform. In this study, we investigate whether researchers’ own values and expectations are in line with the expectations of the evaluation system. We conduct a multiple case study of five departments in two Dutch universities to examine how they balance between their own valuation regimes and the evaluation schemes. For this, we combine curriculum analysis with a series of semi-structured interviews. We propose a model to study the diversity of academic activities and apply it to the multiple case study to understand how such diversity is shaped by discipline and career stage. We conclude that the observed misalignment is not only resulting from an abuse of metrics but also by a lack of tools to evaluate performance in a contextualized and adaptable way.

We model the growth of scientific literature related to COVID-19 and forecast the expected growth from 1 June 2021. Considering the significant scientific and financial efforts made by the research community to find solutions to end the COVID-19 pandemic, an unprecedented volume of scientific outputs is being produced. This questions the capacity of scientists, politicians and citizens to maintain infrastructure, digest content and take scientifically informed decisions. A crucial aspect is to make predictions to prepare for such a large corpus of scientific literature. Here we base our predictions on the Autoregressive Integrated Moving Average (ARIMA) and exponential smoothing models using the Dimensions database. This source has the particularity of including in the metadata information on the date in which papers were indexed. We present global predictions, plus predictions in three specific settings: by type of access (Open Access), by domain-specific repository (SSRN and MedRxiv) and by several research fields. We conclude by discussing our findings.

This study investigates the scientific mobility and international collaboration networks in the Middle East and North Africa (MENA) region between 2008 and 2017. By using affiliation metadata available in scientific publications, we analyze international scientific mobility flows and collaboration linkages. Three complementary approaches allow us to obtain a detailed characterization of scientific mobility. First, we uncover the main destinations and origins of mobile scholars for each country. Results reveal geographical, cultural and historical proximities. Cooperation programs also contribute to explain some of the observed flows. Second, we use the academic age. The average academic age of migrant scholars in MENA was about 12.4 years. The academic age group 6–10 years is the most common for both emigrant and immigrant scholars. Immigrants are relatively younger than emigrants, except for Iran, Palestine, Lebanon, and Turkey. Scholars who migrated to Gulf Cooperation Council countries, Jordan, and Morocco were on average younger than emigrants by 1.5 years from the same countries. Third, we analyze gender differences. We observe a clear gender gap: Male scholars represent the largest group of migrants in MENA. We conclude by discussing the policy relevance of the scientific mobility and collaboration aspects.

Responsible indicators are crucial for research assessment and monitoring. Transparency and accuracy of indicators are required to make research assessment fair and ensure reproducibility. However, sometimes it is difficult to conduct or replicate studies based on indicators due to the lack of transparency in conceptualization and operationalization. In this paper, we review the different variants of the Probabilistic Affinity Index (PAI), considering both the conceptual and empirical underpinnings. We begin with a review of the historical development of the indicator and the different alternatives proposed. To demonstrate the utility of the indicator, we demonstrate the application of PAI to identifying preferred partners in scientific collaboration. A streamlined procedure is provided, to demonstrate the variations and appropriate calculations. We then compare the results of implementation for 5 specific countries involved in international scientific collaboration. Despite the different proposals on its calculation, we do not observe large differences between the PAI variants, particularly with respect to country size. As with any indicator, the selection of a particular variant is dependent on the research question. To facilitate appropriate use, we provide recommendations for the use of the indicator given specific contexts.

We present a forecasting analysis on the growth of scientific literature related to COVID-19 expected for 2021. Considering the paramount scientific and financial efforts made by the research community to find solutions to end the COVID-19 pandemic, an unprecedented volume of scientific outputs is being produced. This questions the capacity of scientists, politicians and citizens to maintain infrastructure, digest content and take scientifically informed decisions. A crucial aspect is to make predictions to prepare for such a large corpus of scientific literature. Here we base our predictions on the ARIMA model and use two different data sources: the Dimensions and World Health Organization COVID-19 databases. These two sources have the particularity of including in the metadata information the date in which papers were indexed. We present global predictions, plus predictions in three specific settings: type of access (Open Access), NLM source (PubMed and PMC), and domain-specific repository (SSRN and MedRxiv). We conclude by discussing our findings.

ORCID is a scientific infrastructure created to solve the problem of author name ambiguity. Over the years ORCID has also become a useful source for studying academic activities reported by researchers. Our objective in this research was to use ORCID to analyze one of these research activities: the publication of datasets. We illustrate how the identification of datasets that shared in researchers’ ORCID profiles enables the study of the characteristics of the researchers who have produced them. To explore the relevance of ORCID to study data sharing practices we obtained all ORCID profiles reporting at least one dataset in their "works" list, together with information related to the individual researchers producing the datasets. The retrieved data was organized and analyzed in a SQL database hosted at CWTS. Our results indicate that DataCite is by far the most important data source for providing information about datasets recorded in ORCID. There is also a substantial overlap between DataCite records with other repositories (Figshare, Dryad, and Zenodo). The analysis of the distribution of researchers producing datasets shows that the top six countries with more data producers, also have a relatively higher percentage of people who have produced datasets out of total researchers with datasets than researchers in the total ORCID. By disciplines, researchers that belong to the areas of Natural Sciences and Medicine and Life Sciences are those with the largest amount of reported datasets. Finally, we observed that researchers who have started their PhD around 2015 published their first dataset earlier that those researchers that started their PhD before. The work concludes with some reflections of the possibilities of ORCID as a relevant source for research on data sharing practices.

Research careers are typically envisioned as a single path in which a scientist starts as a member of a team working under the guidance of one or more experienced scientists and, if they are successful, ends with the individual leading their own research group and training future generations of scientists. Here we study the author contribution statements of published research papers in order to explore possible biases and disparities in career trajectories in science. We used Bayesian networks to train a prediction model based on a dataset of 70,694 publications from PLoS journals, which included 347,136 distinct authors and their associated contribution statements. This model was used to predict the contributions of 222,925 authors in 6,236,239 publications, and to apply a robust archetypal analysis to profile scientists across four career stages: junior, early-career, mid-career and late-career. All three of the archetypes we found-leader, specialized, and supporting-were encountered for early-career and mid-career researchers. Junior researchers displayed only two archetypes (specialized, and supporting), as did late-career researchers (leader and supporting). Scientists assigned to the leader and specialized archetypes tended to have longer careers than those assigned to the supporting archetype. We also observed consistent gender bias at all stages: the majority of male scientists belonged to the leader archetype, while the larger proportion of women belonged to the specialized archetype, especially for early-career and mid-career researchers.

In this chapter we build upon Moed’s conceptual contributions on the importance of the policy context when using and interpreting scientometric indicators. We focus on the use of indicators in research evaluation regarding internationalisation policies. The globalisation of higher education presents important challenges to institutions worldwide, which are confronted with tensions derived from the need to respond both to their local necessities and demands while participating in global networks. In this context, indicators have served as measures for monitoring and enforcing internationalisation policies, in many cases interpreting them regardless of the policy context in which they are enforced. We will analyse three examples of indicators related to internationalisation. The first one is about international collaborations, under the assumption that a greater number of internationally co-authored publications will benefit a national science system as it will result in higher citation impact. The second one relates to the promotion of English language as the dominant language of science. The third case analyses how different policy contexts shape the selection and construction of indicators, sometimes in a partial way which does not properly reflect the phenomenon under study. The examples illustrate that the interpretation and policy implications of the ‘same’ S&T indicators differ depending on specific contexts.

The implementation of policies promoting the adoption of an open science (OS) culture must be accompanied by indicators that allow monitoring the uptake of such policies and their potential effects on research publishing and sharing practices. This study presents indicators of open access (OA) at the institutional level for universities worldwide. By combining data from Web of Science, Unpaywall and the Leiden Ranking disambiguation of institutions, we track OA coverage of universities’ output for 963 institutions. This paper presents the methodological challenges, conceptual discrepancies and limitations and discusses further steps needed to move forward the discussion on fostering OA and OS practices and policies.

This paper presents a novel methodological framework by which the effects of globalization on international collaboration can be studied and understood. Using the cosine similarity of the disciplinary and partner profiles of countries by collaboration types it is possible to analyse the effects of globalization and the costs and benefits of an increasing global networked research system.

With the advent of altmetrics, digital traces that go beyond the scientific impact can be tracked. Twitter stands as the most appealing platform for their inspection since it gathers academic and non-academic users that discuss a wide-ranging number of topics. This research aims at developing and proposing a fine-grained classification of social media users based on mapping techniques and clustering methods and compare them with other tentative classifications proposed elsewhere. To do so, online activity of over 1.3 million Twitter users is examined, considering both their overall activity on Twitter as well as their interaction with scientific publications.

As part of altmetrics, social media metrics focuses his attention in the quantitative analysis of mentions from social media to scientific literature. One of the approaches to analysing the interactions produced in social media is to identify discussion topics and the actors involved in it, missing more research focusing in the link between them. The main goal of this paper is to propose a methodology that make possible the detection of research topics and to whom can be of interest either within or outside the scientific realm. To do it we combine users’ networks and overlay mapping of topics to visualize and identify communities of attention, allowing to contextualize the mentions to scientific publications. The dataset used is formed by the union of publications in the field of Microbiology and the mentions received from news media, policy briefs and, mostly, twitter.

This paper presents a first attempt to analyse Open Access integration at the institutional level. For this, we combine information from Unpaywall and the Leiden Ranking to offer basic OA indicators for universities. We calculate the overall number of Open Access publications for 930 universities worldwide. OA indicators are also disaggregated by green, gold and hybrid Open Access. We then explore differences between and within countries and offer a general ranking of universities based on the proportion of their output which is openly accessible.