Richardson-Lucy (RL) deconvolution optimizes the likelihood of the object estimate for an incoherent imaging system. It can offer an increase in contrast, but converges poorly, and shows enhancement of noise as the iteration progresses. We have discovered the underlying reason for this problematic convergence behaviour using a Cramér Rao Lower Bound (CRLB) analysis. An analytical expression for the CRLB diverges for spatial frequency components that approach the diffraction limit from below. The resulting mean noise variance per pixel diverges for large images. These results imply that a regular optimum of the likelihood does not exist, and that RL deconvolution is necessarily ill-convergent.

Multispectral imaging is an established method to extend the number of colours usable in fluorescence imaging beyond the typical limit of three or four. However, standard approaches are poorly suited to live-cell imaging owing to the need to separate light into many spectral channels, and unmixing algorithms struggle with low signal-to-noise ratio data. Here we introduce an approach for multispectral imaging in live cells that comprises an iterative spectral unmixing algorithm and eight-channel camera-based image-acquisition hardware. This enables the accurate unmixing of low signal-to-noise ratio datasets captured at video rates, while maintaining diffraction-limited spatial resolution. We use this approach on a commercial spinning-disk confocal microscope and a home-built oblique-plane light-sheet microscope to image one to seven spectrally distinct fluorophore species simultaneously, using both fluorescent protein fusions and small-molecule dyes. We further develop protein-binding proteins (minibinders), labelled with organic fluorophores, and use these in combination with our multispectral imaging approach to study the endosomal trafficking of cell-surface receptors at endogenous levels.

Image quality in single-molecule localization microscopy depends largely on the accuracy and precision of the localizations. While under ideal imaging conditions, the theoretically obtainable precision and accuracy are achieved; in practice, this changes if (field-dependent) aberrations are present. Currently, there is no simple way to measure and incorporate these aberrations into the point-spread function (PSF) fitting; therefore, the aberrations are often taken as constant or neglected altogether. Here we introduce a model-based approach to estimate the field-dependent aberration directly from single-molecule data without a calibration step. This is made possible by using nodal aberration theory to incorporate the field dependency of aberrations into our fully vectorial PSF model. This results in a limited set of aberration fit parameters that can be extracted from the raw frames without a bead calibration measurement, also in retrospect. The software implementation is computationally efficient, enabling the fitting of a full 2D or 3D dataset within a few minutes. We demonstrate our method on 2D and 3D localization data of microtubuli, nuclear pore complexes, and nuclear lamina over fields of view of up to 180 µm and compare it with Gaussian fitting, spline-based fitting, and a deep-learning-based approach.

Structured illumination microscopy (SIM) is a powerful method for high-resolution 3D-imaging that is compatible with standard fluorescence labeling techniques, as it provides optical sectioning as well as an up to twofold improvement of lateral resolution over widefield microscopy by combining illumination pattern diversity with computational reconstruction. We present a quantitative analysis of the image quality of 3D-SIM using the spectral signal-to-noise ratio (SSNR). In particular, we compare conventional woodpile illumination pattern based 3D-SIM, where the pattern is rotated and translated to acquire the set of raw images that is fed into the reconstruction algorithm, to (square or hexagonal) lattice 3D-SIM, where the pattern is only translated to assemble the input set of raw images. It appears that conventional 3D-SIM has better SSNR than the considered cases of lattice 3D-SIM. In addition, we have also analyzed the impact of the relative amplitude, angle of incidence and polarization of the set of illumination plane waves on image quality, and show how two SSNR derived metrics, SSNR volume and SSNR entropy, can be used to optimize these illumination pattern parameters.

DNA-origami nanostructures have shown promising applications in single molecule localization microscopy. They have become a reference standard for benchmarking and for developing new techniques for nanoscopy. Here, we present a pipeline for quantifying the quality of these nano-structures when imaging multiple instances of them using DNA-PAINT technique. We show on several experimental datasets that these structures can have deformations and that the designed binding sites are not equally accessible for the labelled imager strands during the image acquisition process. These limitations result in non-uniform activation of the sites over the origami pattern when fusing the instances into a single reconstruction.

We address resolution assessment for (light super-resolution) microscopy imaging. In modalities where imaging is not diffraction limited, correlation between two noise independent images is the standard way to infer the resolution. Here we take away the need for two noise independent images by computationally splitting one image acquisition into two noise independent realizations. This procedure generates two Poisson noise distributed images if the input is Poissonian distributed. As most modern cameras are shot-noise limited this procedure is directly applicable. However, also in the presence of readout noise we can compute the resolution faithfully via a correction factor. We evaluate our method on simulations and experimental data of widefield microscopy, STED microscopy, rescan confocal microscopy, image scanning microscopy, conventional confocal microscopy, and transmission electron microscopy. In all situations we find that using one image instead of two results in the same computed image resolution.

Correction to: Scientific Reports, published online 16 August 2023 The original version of this Article contained an error in the upper inset of Figure 4, where the atomic model was missing. The original Figure 4 and accompanying legend appear below. (Figure presented.) Overlay of the fluorophore positions from the SMLM particle fusion data (pink) and the SNAP-tag derived from the cryo-EM data (purple). For our overall SMLM emitters (pink), the lateral distance between a unit are 9.1 nm for NR and 10.0 nm for CR. The axial distances between a unit are 2.4 nm for NR and 1.2 nm for CR. The SNAP tags (purple) have lateral distances between a unit of 11.6 nm for NR and 11.5 nm for CR as well as axial distances of 2.5 nm for NR and 2.9 nm for CR. The original Article has been corrected.

Single molecule localization microscopy offers resolution nearly down to the molecular level with specific molecular labelling, and is thereby a promising tool for structural biology. In practice, however, the actual value to this field is limited primarily by incomplete fluorescent labelling of the structure. This missing information can be completed by merging information from many structurally identical particles in a particle fusion approach similar to cryo-EM single-particle analysis. In this paper, we present a data analysis of particle fusion results of fluorescently labelled Nup96 nucleoporins in the Nuclear Pore Complex to show that Nup96 occurs in a spatial arrangement of two rings of 8 units with two Nup96 copies per unit giving a total of 32 Nup96 copies per pore. We use Artificial Intelligence assisted modeling in Alphafold to extend the existing cryo-EM model of Nup96 to accurately pinpoint the positions of the fluorescent labels and show the accuracy of the match between fluorescent and cryo-EM data to be better than 3 nm in-plane and 5 nm out-of-plane.

Fusion of multiple chemically identical complexes, so-called particles, in localization microscopy, can improve the signal-to-noise ratio and overcome under-labeling. To this end, structural homogeneity of the data must be assumed. Biological heterogeneity, however, could be present in the data originating from distinct conformational variations or (continuous) variations in particle shapes. We present a prior-knowledge-free method for detecting continuous structural variations with localization microscopy. Detecting this heterogeneity leads to more faithful fusions and reconstructions of the localization microscopy data as their heterogeneity is taken into account. In experimental datasets, we show the continuous variation of the height of DNA origami tetrahedrons imaged with 3D PAINT and of the radius of Nuclear Pore Complexes imaged in 2D with STORM. In simulation, we study the impact on the heterogeneity detection pipeline of Degree Of Labeling and of structural variations in the form of two independent modes.

Combining orientation estimation with localization microscopy opens up the possibility to analyze the underlying orientation of biomolecules on the nanometer scale. Inspired by the recent improvement of the localization precision by shifting excitation patterns (MINFLUX, SIMFLUX), we have adapted the idea towards the modulation of excitation polarization to enhance the orientation precision. For this modality two modes are analyzed: i) normally incident excitation with three polarization steps to retrieve the in-plane angle of emitters and ii) obliquely incident excitation with p-polarization with five different azimuthal angles of incidence to retrieve the full orientation. Firstly, we present a theoretical study of the lower precision limit with a Cramér-Rao bound for these modes. For the oblique incidence mode we find a favorable isotropic orientation precision for all molecular orientations if the polar angle of incidence is equal to arccos ^√2/3 ≈ 35 degrees. Secondly, a simulation study is performed to assess the performance for low signal-to-background ratios and how inaccurate illumination polarization angles affect the outcome. We show that a precision, at the Cramér-Rao bound (CRB) limit, of just 2.4 and 1.6 degrees in the azimuthal and polar angles can be achieved with only 1000 detected signal photons and 10 background photons per pixel (about twice better than reported earlier). Lastly, the alignment and calibration of an optical microscope with polarization control is described in detail. With this microscope a proof-of-principle experiment is carried out, demonstrating an experimental in-plane precision close to the CRB limit for signal photon counts ranging from 400 to 10,000.

Summary: We present a fast particle fusion method for particles imaged with single-molecule localization microscopy. The state-of-the-art approach based on all-to-all registration has proven to work well but its computational cost scales unfavorably with the number of particles N, namely as N2. Our method overcomes this problem and achieves a linear scaling of computational cost with N by making use of the Joint Registration of Multiple Point Clouds (JRMPC) method. Straightforward application of JRMPC fails as mostly locally optimal solutions are found. These usually contain several overlapping clusters that each consist of well-aligned particles, but that have different poses. We solve this issue by repeated runs of JRMPC for different initial conditions, followed by a classification step to identify the clusters, and a connection step to link the different clusters obtained for different initializations. In this way a single well-aligned structure is obtained containing the majority of the particles. Results: We achieve reconstructions of experimental DNA-origami datasets consisting of close to 400 particles within only 10 min on a CPU, with an image resolution of 3.2 nm. In addition, we show artifact-free reconstructions of symmetric structures without making any use of the symmetry. We also demonstrate that the method works well for poor data with a low density of labeling and for 3D data.

Super-resolution fluorescence microscopy can be achieved by image reconstruction after spatially patterned illumination or sequential photo-switching and read-out. Reconstruction algorithms and microscope performance are typically tested using simulated image data, due to a lack of strategies to pattern complex fluorescent patterns with nanoscale dimension control. Here, we report direct electron-beam patterning of fluorescence nanopatterns as calibration standards for super-resolution fluorescence. Patterned regions are identified with both electron microscopy and fluorescence labelling of choice, allowing precise correlation of predefined pattern dimensions, a posteriori obtained electron images, and reconstructed super-resolution images.

Modulation enhanced single-molecule localization microscopy (meSMLM) methods improve the localization precision by using patterned illumination to encode additional position information. Iterative meSMLM (imeSMLM) methods iteratively generate prior information on emitter positions, used to locally improve the localization precision during subsequent iterations. The Cramér-Rao lower bound cannot incorporate prior information to bound the best achievable localization precision because it requires estimators to be unbiased. By treating estimands as random variables with a known prior distribution, the Van Trees inequality (VTI) can be used to bound the best possible localization precision of imeSMLM methods. An imeSMLM method is considered, where the positions of in-plane standing-wave illumination patterns are controlled over the course of multiple iterations. Using the VTI, we analytically approximate a lower bound on the maximum localization precision of imeSMLM methods that make use of standing-wave illumination patterns. In addition, we evaluate the maximally achievable localization precision for different illumination pattern placement strategies using Monte Carlo simulations. We show that in the absence of background and under perfect modulation, the information content of signal photons increases exponentially as a function of the iteration count. However, the information increase is no longer exponential as a function of the iteration count under non-zero background, imperfect modulation, or limited mechanical resolution of the illumination positioning system. As a result, imeSMLM with two iterations reaches at most a fivefold improvement over SMLM at 8 expected background photons per pixel and 95% modulation contrast. Moreover, the information increase from imeSMLM is balanced by a reduced signal photon rate. Therefore, SMLM outperforms imeSMLM when considering an equal measurement time and illumination power per iteration. Finally, the VTI is an excellent tool for the assessment of the performance of illumination control and is therefore the method of choice for optimal design and control of imeSMLM methods.

One of the fundamental limits of classical optical microscopy is the diffraction limit of optical resolution. It results from the finite bandwidth of the optical transfer function (or OTF) of an optical microscope, which restricts the maximum spatial frequencies that are transmitted by a microscope. However, given the frequency support of the OTF, which is fully determined by the used optical hardware, an open and unsolved question is what is the optimal amplitude and phase distribution of spatial frequencies across this support that delivers the "sharpest"possible image. In this paper, we will answer this question and present a general rule how to find the optimal OTF for any given imaging system. We discuss our result in the context of optical microscopy, by considering in particular the cases of wide-field microscopy, confocal image scanning microscopy (ISM), 4pi microscopy, and structured illumination microscopy (SIM). Our results are important for finding optimal deconvolution algorithms for microscopy images, and we demonstrate this experimentally on the example of ISM. They can also serve as a guideline for designing optical systems that deliver best possible images, and can be easily generalized to nonoptical imaging such as telescopic imaging, ultrasound imaging, or magnetic resonance imaging.

Single-molecule localization microscopy has developed into a widely used technique to overcome the diffraction limit and enables 3D localization of single-emitters with nanometer precision. A widely used method to enable 3D encoding is to use a cylindrical lens or a phase mask to engineer the point spread function (PSF). The performance of these PSFs is often assessed by comparing the precision they achieve, ignoring accuracy. Nonetheless, accurate localization is required in many applications, such as multi-plane imaging, measuring and modelling of physical processes based on volumetric data, and 3D particle averaging. However, there are PSF model mismatches in the localization schemes due to how reference PSFs are obtained, look-up-tables are created, or spots are fitted. Currently there is little insight in how these model mismatches give rise to systematic axial localization errors, how large these errors are, and how to mitigate them. In this theoretical and simulation work we use a vector PSF model, which incorporates super-critical angle fluorescence (SAF) and the appropriate aplanatic correction factor, to analyze the errors in z-localization. We introduce theory for defining the focal plane in SAF conditions and analyze the predicted axial errors for an astigmatic PSF, double-helix PSF, and saddle-point PSF. These simulations indicate that the absolute axial biases can be as large as 140 nm, 250 nm, and 120 nm for the astigmatic, saddle-point, and double-helix PSF respectively, with relative errors of more than 50%. Finally, we discuss potential experimental methods to verify these findings and propose a workflow to mitigate these effects.

Super-resolution structured illumination microscopy (SIM) has become a widely used method for biological imaging. Standard reconstruction algorithms, however, are prone to generate noise-specific artifacts that limit their applicability for lower signal-to-noise data. Here we present a physically realistic noise model that explains the structured noise artifact, which we then use to motivate new complementary reconstruction approaches. True-Wiener-filtered SIM optimizes contrast given the available signal-to-noise ratio, and flat-noise SIM fully overcomes the structured noise artifact while maintaining resolving power. Both methods eliminate ad hoc user-adjustable reconstruction parameters in favor of physical parameters, enhancing objectivity. The new reconstructions point to a trade-off between contrast and a natural noise appearance. This trade-off can be partly overcome by further notch filtering but at the expense of a decrease in signal-to-noise ratio. The benefits of the proposed approaches are demonstrated on focal adhesion and tubulin samples in two and three dimensions, and on nanofabricated fluorescent test patterns.