The low degree of labeling and limited photon count of fluorescent emitters in single molecule localization microscopy results in poor quality images of macro-molecular complexes. Particle fusion provides a single reconstruction with high signal-to-noise ratio by combining many single molecule localization microscopy images of the same structure. The underlying assumption of homogeneity is not always valid, heterogeneity can arise due to geometrical shape variations or distinct conformational states. We introduce a Point Cloud Variational Auto-Encoder that works directly on 2D and 3D localization data, to detect multiple modes of variation in such datasets. The computing time is on the order of a few minutes, enabled by the linear scaling with dataset size, and fast network training in just four epochs. The use of lists of localization data instead of pixelated images leads to just minor differences in computational burden between 2D and 3D cases. With the proposed method, we detected radius variation in 2D Nuclear Pore Complex data, height variations in 3D DNA origami tetrahedron data, and both radius and height variations in 3D Nuclear Pore Complex data. In all cases, the detected variations were on the few nanometer scale.

The gradient-consensus Richardson–Lucy (GC-RL) deconvolution algorithm is a novel approach to contrast restoration in high-resolution microscopy imaging without excessive noise amplification. We evaluate this method in this work, focusing on the impact of the noise level of the input image, of imperfections and deviations in image formation from the ideal incoherent case, and of using undersampled input data. The gain in spectral signal to noise ratio (SSNR) is larger when the noise level of the raw image is lower. We also find a positive impact on SSNR gain from using an optical transfer function (OTF) in the deconvolution that has a decreased spatial frequency response compared to the ideal incoherent OTF. A simple upsampling scheme, embedded in the GC-RL deconvolution, is demonstrated to be suitable for undersampled data. We show our results on simulated data, on fluorescence microscopy data, and on brightfield microscopy data of a breast tumour section. The GC-RL method has potential for whole slide imaging technology to retrieve diagnostically relevant image features at the smallest length scales from scanned images at lower resolution, providing a gain in throughput and data storage requirements.

Single-molecule fluorescence localization with minimum photon flux imaging (MINFLUX) can achieve localization precisions in the small nanometer range or better under suitable conditions. Potentially adverse conditions, such as a fixed fluorescence dipole or optical aberrations, that could cause systematic localization errors, have received little attention up to now. Here, we study these effects in simulation. We find that biases occur for fluorophores with a fixed absorption dipole tilted out of the imaging plane. These become larger (up to about 25% of the diameter of the circle spanned by the doughnut center positions), the larger the tilt angle gets. As a rule of thumb, the spread in bias is smaller than 5 nm in case the dipole orientation is less than 30° out of plane for the typical case of a doughnut probing circle of diameter 100 nm. For freely rotating dipoles, only the primary aberrations, astigmatism and coma, contribute to bias. This bias depends on the position of the fluorophore inside the circular probing area of MINFLUX and can be significantly larger than the localization precision. We show that increasing the number of measurements over the circle from a triangular to a hexagonal pattern is beneficial for reducing bias in all cases. Iterative shrinking of the probing area can eliminate the position-dependent bias completely, but a strong dependence on dipole orientation of the bias at the center of the probing area remains.

Richardson-Lucy (RL) deconvolution optimizes the likelihood of the object estimate for an incoherent imaging system. It can offer an increase in contrast, but converges poorly, and shows enhancement of noise as the iteration progresses. We have discovered the underlying reason for this problematic convergence behaviour using a Cramér Rao Lower Bound (CRLB) analysis. An analytical expression for the CRLB diverges for spatial frequency components that approach the diffraction limit from below. The resulting mean noise variance per pixel diverges for large images. These results imply that a regular optimum of the likelihood does not exist, and that RL deconvolution is necessarily ill-convergent.

Small subcellular organelles orchestrate key cellular functions. How biomolecules are spatially organized within these assemblies is poorly understood. Here, we report an automated super-resolution imaging and analysis workflow that integrates confocal microscopy, morphological object screening, targeted 3D super-resolution STED microscopy and quantitative image analysis. Using this smart microscopy workflow, we target the 3D organization of NEAT1, an architectural RNA that constitutes the structural backbone of paraspeckles, a membraneless nuclear organelle. Using site-specific labeling, morphological sorting and particle averaging, we reconstruct the morphological space of paraspeckles along their development cycle from over 10,000 individual particles. Applying spherical harmonics analysis, we report so-far unknown heterotypes of NEAT1 RNA organization. By integrating multi-positional labeling, we determine the coarse conformation of NEAT1 within the organelle and show that the 3’ end forms a loop-like structure at the surface of the paraspeckle. Our study reveals key structural features of paraspeckle structure and growth, as well as the molecular organization of its scaffolding RNA.

Image quality in single-molecule localization microscopy depends largely on the accuracy and precision of the localizations. While under ideal imaging conditions, the theoretically obtainable precision and accuracy are achieved; in practice, this changes if (field-dependent) aberrations are present. Currently, there is no simple way to measure and incorporate these aberrations into the point-spread function (PSF) fitting; therefore, the aberrations are often taken as constant or neglected altogether. Here we introduce a model-based approach to estimate the field-dependent aberration directly from single-molecule data without a calibration step. This is made possible by using nodal aberration theory to incorporate the field dependency of aberrations into our fully vectorial PSF model. This results in a limited set of aberration fit parameters that can be extracted from the raw frames without a bead calibration measurement, also in retrospect. The software implementation is computationally efficient, enabling the fitting of a full 2D or 3D dataset within a few minutes. We demonstrate our method on 2D and 3D localization data of microtubuli, nuclear pore complexes, and nuclear lamina over fields of view of up to 180 µm and compare it with Gaussian fitting, spline-based fitting, and a deep-learning-based approach.

Structured illumination microscopy (SIM) is a powerful method for high-resolution 3D-imaging that is compatible with standard fluorescence labeling techniques, as it provides optical sectioning as well as an up to twofold improvement of lateral resolution over widefield microscopy by combining illumination pattern diversity with computational reconstruction. We present a quantitative analysis of the image quality of 3D-SIM using the spectral signal-to-noise ratio (SSNR). In particular, we compare conventional woodpile illumination pattern based 3D-SIM, where the pattern is rotated and translated to acquire the set of raw images that is fed into the reconstruction algorithm, to (square or hexagonal) lattice 3D-SIM, where the pattern is only translated to assemble the input set of raw images. It appears that conventional 3D-SIM has better SSNR than the considered cases of lattice 3D-SIM. In addition, we have also analyzed the impact of the relative amplitude, angle of incidence and polarization of the set of illumination plane waves on image quality, and show how two SSNR derived metrics, SSNR volume and SSNR entropy, can be used to optimize these illumination pattern parameters.

Multispectral imaging is an established method to extend the number of colours usable in fluorescence imaging beyond the typical limit of three or four. However, standard approaches are poorly suited to live-cell imaging owing to the need to separate light into many spectral channels, and unmixing algorithms struggle with low signal-to-noise ratio data. Here we introduce an approach for multispectral imaging in live cells that comprises an iterative spectral unmixing algorithm and eight-channel camera-based image-acquisition hardware. This enables the accurate unmixing of low signal-to-noise ratio datasets captured at video rates, while maintaining diffraction-limited spatial resolution. We use this approach on a commercial spinning-disk confocal microscope and a home-built oblique-plane light-sheet microscope to image one to seven spectrally distinct fluorophore species simultaneously, using both fluorescent protein fusions and small-molecule dyes. We further develop protein-binding proteins (minibinders), labelled with organic fluorophores, and use these in combination with our multispectral imaging approach to study the endosomal trafficking of cell-surface receptors at endogenous levels.

We address resolution assessment for (light super-resolution) microscopy imaging. In modalities where imaging is not diffraction limited, correlation between two noise independent images is the standard way to infer the resolution. Here we take away the need for two noise independent images by computationally splitting one image acquisition into two noise independent realizations. This procedure generates two Poisson noise distributed images if the input is Poissonian distributed. As most modern cameras are shot-noise limited this procedure is directly applicable. However, also in the presence of readout noise we can compute the resolution faithfully via a correction factor. We evaluate our method on simulations and experimental data of widefield microscopy, STED microscopy, rescan confocal microscopy, image scanning microscopy, conventional confocal microscopy, and transmission electron microscopy. In all situations we find that using one image instead of two results in the same computed image resolution.

DNA-origami nanostructures have shown promising applications in single molecule localization microscopy. They have become a reference standard for benchmarking and for developing new techniques for nanoscopy. Here, we present a pipeline for quantifying the quality of these nano-structures when imaging multiple instances of them using DNA-PAINT technique. We show on several experimental datasets that these structures can have deformations and that the designed binding sites are not equally accessible for the labelled imager strands during the image acquisition process. These limitations result in non-uniform activation of the sites over the origami pattern when fusing the instances into a single reconstruction.

Correction to: Scientific Reports, published online 16 August 2023 The original version of this Article contained an error in the upper inset of Figure 4, where the atomic model was missing. The original Figure 4 and accompanying legend appear below. (Figure presented.) Overlay of the fluorophore positions from the SMLM particle fusion data (pink) and the SNAP-tag derived from the cryo-EM data (purple). For our overall SMLM emitters (pink), the lateral distance between a unit are 9.1 nm for NR and 10.0 nm for CR. The axial distances between a unit are 2.4 nm for NR and 1.2 nm for CR. The SNAP tags (purple) have lateral distances between a unit of 11.6 nm for NR and 11.5 nm for CR as well as axial distances of 2.5 nm for NR and 2.9 nm for CR. The original Article has been corrected.

Single molecule localization microscopy offers resolution nearly down to the molecular level with specific molecular labelling, and is thereby a promising tool for structural biology. In practice, however, the actual value to this field is limited primarily by incomplete fluorescent labelling of the structure. This missing information can be completed by merging information from many structurally identical particles in a particle fusion approach similar to cryo-EM single-particle analysis. In this paper, we present a data analysis of particle fusion results of fluorescently labelled Nup96 nucleoporins in the Nuclear Pore Complex to show that Nup96 occurs in a spatial arrangement of two rings of 8 units with two Nup96 copies per unit giving a total of 32 Nup96 copies per pore. We use Artificial Intelligence assisted modeling in Alphafold to extend the existing cryo-EM model of Nup96 to accurately pinpoint the positions of the fluorescent labels and show the accuracy of the match between fluorescent and cryo-EM data to be better than 3 nm in-plane and 5 nm out-of-plane.

Fusion of multiple chemically identical complexes, so-called particles, in localization microscopy, can improve the signal-to-noise ratio and overcome under-labeling. To this end, structural homogeneity of the data must be assumed. Biological heterogeneity, however, could be present in the data originating from distinct conformational variations or (continuous) variations in particle shapes. We present a prior-knowledge-free method for detecting continuous structural variations with localization microscopy. Detecting this heterogeneity leads to more faithful fusions and reconstructions of the localization microscopy data as their heterogeneity is taken into account. In experimental datasets, we show the continuous variation of the height of DNA origami tetrahedrons imaged with 3D PAINT and of the radius of Nuclear Pore Complexes imaged in 2D with STORM. In simulation, we study the impact on the heterogeneity detection pipeline of Degree Of Labeling and of structural variations in the form of two independent modes.

Summary: We present a fast particle fusion method for particles imaged with single-molecule localization microscopy. The state-of-the-art approach based on all-to-all registration has proven to work well but its computational cost scales unfavorably with the number of particles N, namely as N2. Our method overcomes this problem and achieves a linear scaling of computational cost with N by making use of the Joint Registration of Multiple Point Clouds (JRMPC) method. Straightforward application of JRMPC fails as mostly locally optimal solutions are found. These usually contain several overlapping clusters that each consist of well-aligned particles, but that have different poses. We solve this issue by repeated runs of JRMPC for different initial conditions, followed by a classification step to identify the clusters, and a connection step to link the different clusters obtained for different initializations. In this way a single well-aligned structure is obtained containing the majority of the particles. Results: We achieve reconstructions of experimental DNA-origami datasets consisting of close to 400 particles within only 10 min on a CPU, with an image resolution of 3.2 nm. In addition, we show artifact-free reconstructions of symmetric structures without making any use of the symmetry. We also demonstrate that the method works well for poor data with a low density of labeling and for 3D data.

Modulation enhanced single-molecule localization microscopy (meSMLM) methods improve the localization precision by using patterned illumination to encode additional position information. Iterative meSMLM (imeSMLM) methods iteratively generate prior information on emitter positions, used to locally improve the localization precision during subsequent iterations. The Cramér-Rao lower bound cannot incorporate prior information to bound the best achievable localization precision because it requires estimators to be unbiased. By treating estimands as random variables with a known prior distribution, the Van Trees inequality (VTI) can be used to bound the best possible localization precision of imeSMLM methods. An imeSMLM method is considered, where the positions of in-plane standing-wave illumination patterns are controlled over the course of multiple iterations. Using the VTI, we analytically approximate a lower bound on the maximum localization precision of imeSMLM methods that make use of standing-wave illumination patterns. In addition, we evaluate the maximally achievable localization precision for different illumination pattern placement strategies using Monte Carlo simulations. We show that in the absence of background and under perfect modulation, the information content of signal photons increases exponentially as a function of the iteration count. However, the information increase is no longer exponential as a function of the iteration count under non-zero background, imperfect modulation, or limited mechanical resolution of the illumination positioning system. As a result, imeSMLM with two iterations reaches at most a fivefold improvement over SMLM at 8 expected background photons per pixel and 95% modulation contrast. Moreover, the information increase from imeSMLM is balanced by a reduced signal photon rate. Therefore, SMLM outperforms imeSMLM when considering an equal measurement time and illumination power per iteration. Finally, the VTI is an excellent tool for the assessment of the performance of illumination control and is therefore the method of choice for optimal design and control of imeSMLM methods.

One of the fundamental limits of classical optical microscopy is the diffraction limit of optical resolution. It results from the finite bandwidth of the optical transfer function (or OTF) of an optical microscope, which restricts the maximum spatial frequencies that are transmitted by a microscope. However, given the frequency support of the OTF, which is fully determined by the used optical hardware, an open and unsolved question is what is the optimal amplitude and phase distribution of spatial frequencies across this support that delivers the "sharpest"possible image. In this paper, we will answer this question and present a general rule how to find the optimal OTF for any given imaging system. We discuss our result in the context of optical microscopy, by considering in particular the cases of wide-field microscopy, confocal image scanning microscopy (ISM), 4pi microscopy, and structured illumination microscopy (SIM). Our results are important for finding optimal deconvolution algorithms for microscopy images, and we demonstrate this experimentally on the example of ISM. They can also serve as a guideline for designing optical systems that deliver best possible images, and can be easily generalized to nonoptical imaging such as telescopic imaging, ultrasound imaging, or magnetic resonance imaging.