In the age of machine learning, deep learning and artificial intelligence (AI) are expected to improve our lives. Particularly in the field of medicine and medical imaging, AI can make sense of tens if not hundreds of different parameters and find patterns and correlations that are difficult for humans to process. AI is expected to assist doctors in improving patient care and reducing burden. Despite many papers showing how AI algorithms can match or outperform humans in different domains of medicine, not many have been adopted into practice (Kelly et al., 2019). One of the major challenges is trust and acceptance of AI results. These are important issues that are complex. Confidence, trust, and uncertainty influence the way humans make decisions using AI. AI (deep learning algorithms in particular) is a “black box” to users and even the creators of these algorithms, making it very difficult to adopt. Should humans trust AI? Do humans overly trust AI? This chapter explores the human–AI relationship. It starts with a discussion on trust and human interactions. The expert–apprentice model is described to inform how AI could interact with clinicians. Recent technological developments and experience design aspects are detailed, giving an outline of recommendations for designing explainable AI, or XAI.

Controlled human infections provide opportunities to study the interaction between the immune system and malaria parasites, which is essential for vaccine development. Here, we compared immune signatures of malaria-naive Europeans and of Africans with lifelong malaria exposure using mass cytometry, RNA sequencing and data integration, before and 5 and 11 days after venous inoculation with Plasmodium falciparum sporozoites. We observed differences in immune cell populations, antigen-specific responses and gene expression profiles between Europeans and Africans and among Africans with differing degrees of immunity. Before inoculation, an activated/differentiated state of both innate and adaptive cells, including elevated CD161⁺CD4⁺ T cells and interferon-γ production, predicted Africans capable of controlling parasitemia. After inoculation, the rapidity of the transcriptional response and clusters of CD4⁺ T cells, plasmacytoid dendritic cells and innate T cells were among the features distinguishing Africans capable of controlling parasitemia from susceptible individuals. These findings can guide the development of a vaccine effective in malaria-endemic regions.

Quality assessment of different Magnetic Resonance Fingerprinting (MRF) sequences and their corresponding dictionaries remains an unsolved problem. In this work we present a method in which we approach analysis of MRF dictionaries by performing dimensionality reduction and representing them as low-dimensional point sets (embeddings). Dimensionality reduction was performed using a modification of the t-Distributed Stochastic Neighbor Embedding (t-SNE) algorithm. First, we demonstrated stability of calculated embeddings that allows neglecting the stochastic nature of t-SNE. Next, we proposed and analyzed two algorithms for comparing the embeddings. Finally, we performed two simulations in which we reduced the MRF sequence/dictionary in length or size and analyzed the influence of this reduction on the resulting embedding. We believe that this research can pave the way to development of a software tool for analysis, including better understanding, optimization and comparison, of different MRF sequences.

Hierarchical embeddings, such as HSNE, address critical visual and computational scalability issues of traditional techniques for dimensionality reduction. The improved scalability comes at the cost of the need for increased user interaction for exploration. In this paper, we provide a solution for the interactive visual Focus+Context exploration of such embeddings. We explain how to integrate embedding parts from different levels of detail, corresponding to focus and context groups, in a joint visualization. We devise an according interaction model that relates typical semantic operations on a Focus+Context visualization with the according changes in the level-of-detail-hierarchy of the embedding, including also a mode for comparative Focus+Context exploration and extend HSNE to incorporate the presented interaction model. In order to demonstrate the effectiveness of our approach, we present a use case based on the visual exploration of multi-dimensional images.

Innate lymphoid cells (ILCs) are abundant in mucosal tissues and involved in tissue homeostasis and barrier function. Although several ILC subsets have been identified, it is unknown if additional heterogeneity exists, and their differentiation pathways remain largely unclear. We applied mass cytometry to analyze ILCs in the human fetal intestine and distinguished 34 distinct clusters through a t-SNE-based analysis. A lineage (Lin)^-CD7⁺CD127^-CD45RO⁺CD56⁺ population clustered between the CD127⁺ ILC and natural killer (NK) cell subsets, and expressed diverse levels of Eomes, T-bet, GATA3, and RORγt. By visualizing the dynamics of the t-SNE computation, we identified smooth phenotypic transitions from cells within the Lin^-CD7⁺CD127^-CD45RO⁺CD56⁺ cluster to both the NK cells and CD127⁺ ILCs, revealing potential differentiation trajectories. In functional differentiation assays, the Lin^-CD7⁺CD127^-CD45RO⁺CD56⁺CD8a^- cells could develop into CD45RA⁺ NK cells and CD127⁺RORγt⁺ ILC3-like cells. Thus, we identified a previously unknown intermediate innate subset that can differentiate into ILC3 and NK cells.

A bipartite graph is a powerful abstraction for modeling relationships between two collections. Visualizations of bipartite graphs allow users to understand the mutual relationships between the elements in the two collections, e.g., by identifying clusters of similarly connected elements. However, commonly-used visual representations do not scale for the analysis of large bipartite graphs containing tens of millions of vertices, often resorting to an a-priori clustering of the sets. To address this issue, we present the Who's-Active-On-What-Visualization (WAOW-Vis) that allows for multiscale exploration of a bipartite social-network without imposing an a-priori clustering. To this end, we propose to treat a bipartite graph as a high-dimensional space and we create the WAOW-Vis adapting the multiscale dimensionality-reduction technique HSNE. The application of HSNE for bipartite graph requires several modifications that form the contributions of this work. Given the nature of the problem, a set-based similarity is proposed. For efficient and scalable computations, we use compressed bitmaps to represent sets and we present a novel space partitioning tree to efficiently compute similarities; the Sets Intersection Tree. Finally, we validate WAOW-Vis on several datasets connecting Twitter-users and -streams in different domains: news, computer science and politics. We show how WAOW-Vis is particularly effective in identifying hierarchies of communities among social-media users.

Auto-reactive CD8 T-cells play an important role in the destruction of pancreatic β-cells resulting in type 1 diabetes (T1D). However, the phenotype of these auto-reactive cytolytic CD8 T-cells has not yet been extensively described. We used high-dimensional mass cytometry to phenotype autoantigen- (pre-proinsulin), neoantigen- (insulin-DRIP) and virus-(cytomegalovirus) reactive CD8 T-cells in peripheral blood mononuclear cells (PBMCs) of T1D patients. A panel of 33 monoclonal antibodies was designed to further characterise these cells at the single-cell level. HLA-A2 class I tetramers were used for the detection of antigen-specific CD8 T-cells. Using a novel Hierarchical Stochastic Neighbor Embedding (HSNE) tool (implemented in Cytosplore), we identified 42 clusters within the CD8 T-cell compartment of three T1D patients and revealed profound heterogeneity between individuals, as each patient displayed a distinct cluster distribution. Single-cell analysis of pre-proinsulin, insulin-DRIP and cytomegalovirus-specific CD8 T-cells showed that the detected specificities were heterogeneous between and within patients. These findings emphasize the challenge to define the obscure nature of auto-reactive CD8 T-cells.

Deep neural networks are now rivaling human accuracy in several pattern recognition problems. Compared to traditional classifiers, where features are handcrafted, neural networks learn increasingly complex features directly from the data. Instead of handcrafting the features, it is now the network architecture that is manually engineered. The network architecture parameters such as the number of layers or the number of filters per layer and their interconnections are essential for good performance. Even though basic design guidelines exist, designing a neural network is an iterative trial-and-error process that takes days or even weeks to perform due to the large datasets used for training. In this paper, we present DeepEyes, a Progressive Visual Analytics system that supports the design of neural networks during training. We present novel visualizations, supporting the identification of layers that learned a stable set of patterns and, therefore, are of interest for a detailed analysis. The system facilitates the identification of problems, such as superfluous filters or layers, and information that is not being captured by the network. We demonstrate the effectiveness of our system through multiple use cases, showing how a trained network can be compressed, reshaped and adapted to different problems.

Technological advances in mass spectrometry imaging (MSI) have contributed to growing interest in 3D MSI. However, the large size of 3D MSI data sets has made their efficient analysis and visualization and the identification of informative molecular patterns computationally challenging. Hierarchical stochastic neighbor embedding (HSNE), a nonlinear dimensionality reduction technique that aims at finding hierarchical and multiscale representations of large data sets, is a recent development that enables the analysis of millions of data points, with manageable time and memory complexities. We demonstrate that HSNE can be used to analyze large 3D MSI data sets at full mass spectral and spatial resolution. To benchmark the technique as well as demonstrate its broad applicability, we have analyzed a number of publicly available 3D MSI data sets, recorded from various biological systems and spanning different mass-spectrometry ionization techniques. We demonstrate that HSNE is able to rapidly identify regions of interest within these large high-dimensionality data sets as well as aid the identification of molecular ions that characterize these regions of interest; furthermore, through clearly separating measurement artifacts, the HSNE analysis exhibits a degree of robustness to measurement batch effects, spatially correlated noise, and mass spectral misalignment.

Mass cytometry allows high-resolution dissection of the cellular composition of the immune system. However, the high-dimensionality, large size, and non-linear structure of the data poses considerable challenges for the data analysis. In particular, dimensionality reduction-based techniques like t-SNE offer single-cell resolution but are limited in the number of cells that can be analyzed. Here we introduce Hierarchical Stochastic Neighbor Embedding (HSNE) for the analysis of mass cytometry data sets. HSNE constructs a hierarchy of non-linear similarities that can be interactively explored with a stepwise increase in detail up to the single-cell level. We apply HSNE to a study on gastrointestinal disorders and three other available mass cytometry data sets. We find that HSNE efficiently replicates previous observations and identifies rare cell populations that were previously missed due to downsampling. Thus, HSNE removes the scalability limit of conventional t-SNE analysis, a feature that makes it highly suitable for the analysis of massive high-dimensional data sets.