We investigate the enhancement of projection-based phase retrieval algorithms through intermediate phase unwrapping steps. In the phase retrieval problem, convergence to local minima presents a serious obstacle for standard iterative algorithms. Wrong solutions are typically characterized by the presence of phase residues, while the ground truth phase is often residue-free. We present our first results in introducing phase unwrapping as an intermediate step in the projection-based algorithms, demonstrated on simulated and experimental data. The method can reduce convergence time or produce a physically meaningful solution where standard algorithms fail. The proposed method can find applications in microscopy, characterization of precise optical instruments, and wavefront sensing.

High-accuracy calibration of a deformable mirror is important for the wide range of applications where adaptive optics is used in the feedforward mode or for introducing a precise phase diversity term. While phase shifting interferometry (PSI) can be used for such calibration, it requires expensive equipment and/or cannot be performed on-tool due to space limitations. In this presentation, we demonstrate our novel phase retrieval method from several interferograms with introduced arbitrary tilts (phase tilting interferometry) applied to the calibration of a membrane deformable mirror. As the tilts can be introduced manually by simply misaligning the reference mirror, our method represents an inexpensive and easy-to-use alternative to PSI. The method first exploits global information to establish the phase tilt parameters using the Zoom FFT to determine the maxima location in the Fourier spectra of the interferogram differences squared. Then, it retrieves the phase locally using the least-squares approach. To decrease the noise print-through in the estimated mirror response functions, linear regression is used on the phases retrieved for several values of the actuator voltages.

Monitoring manufacturing processes plays an important role in chip production. Current state-of-the-art approaches use the entire surface to classify defects with CNN- or Transformer-based models, resulting in considerable mea-surement costs. Therefore, new advanced techniques are required to reduce the cost of inspection. In this work, we ad-vocate for the reinforcement learning-based feedback loop with a classifier trained with supervised contrastive loss. In contrast to previous works in this manner, our approach is not limited to only one type of defect but can identify multiple defects on one wafer. We tested our algorithm on the publicly available WM-811 k and MixedWM38 datasets, showing a significant reduction in scanning time compared to CNN-based approaches while maintaining similar accu-racy. We demonstrate the reduction of up to 40% in costs as-sociated with wafer scanning in defect classification tasks, even if multiple defects are on the surface. Moreover, we demonstrate that in the multi-defect scenario, the trained model can be directly used to detect outliers, requiring only about 12.5% of the surface to find at least one type of defect.

Wafer map defect recognition is a vital part of the semiconductor manufacturing process that requires a high level of precision. Measurement tools in such manufacturing systems can scan only a small region (patch) of the map at a time. However, this can be resource-intensive and lead to unnecessary additional costs if the full wafer map is measured. Instead, selective sparse measurements of the image save a considerable amount of resources (e.g. scanning time). Therefore, in this work, we propose a feedback loop approach for wafer map defect recognition. The algorithm aims to find sequentially the most informative regions in the image based on previously acquired ones and make a prediction of a defect type by having only these partial observations without scanning the full wafer map. To achieve our goal, we introduce a reinforcement learning-based measurement acquisition process and recurrent neural network-based classifier that takes the sequence of these measurements as an input. Additionally, we employ an ensemble technique to increase the accuracy of the prediction. As a result, we reduce the need for scanned patches by 38% having higher accuracy than the conventional convolutional neural network-based approach on a publicly available WM-811k dataset.

We present a novel problem formulation for model-free data-driven fault diagnosis, in which possible faults are diagnosed simultaneously to identifying the linear time-invariant system. This problem is practically relevant for systems whose model cannot be identified reliably prior to diagnosing possible faults, for instance when operating conditions change over time, when a fault is already present before system identification is carried out, or when the system dynamics change due to the presence of the fault. A computationally attractive solution is proposed by solving the problem using unconstrained convex optimization, where the objective function consists of three terms of which two are non-differentiable. An additional recursive implementation based on a proximal algorithm is presented in order to solve the optimization problem online. The numerical results on a buck converter show the application of the proposed solution both offline and online.

Model-based fault diagnosis for dynamical systems is a sophisticated task due to model inaccuracies, measurement noise and many possible fault scenarios. By presenting faults in terms of a dictionary, the latter obstacle is recently addressed using well-known techniques for recovering sparse information (e.g. lasso). However, current state-of-the-art methods still require accurate models and measurements for adequate diagnosis. In our contribution we address the problem of data-driven fault diagnosis in the sense that the model of the linear time-invariant (LTI) system is unknown in addition to the fault. Moreover, our aim is to diagnose (concurrent) faults while only having input/output data and the fault dictionary. This implies the user simply plugs in the data and specifies the set of possible faults in order to know the active faults together with an estimate of the dynamic model. The problem is formulated within a blind system identification context resulting in computationally efficient solutions based on convex optimization.

We propose to use the State Estimation by Sum-of-Norms Regularisation (STATESON-)algorithm for recovering the tip-sample interaction in high-speed tapping mode atomic force microscopy (AFM). This approach enables accurate sample height estimation for each independent cantilever oscillation period, provided that the tip-sample interaction dominates the noise. The entire course of the cantilever deflection signal is compared to a modelled counterpart in subsequent convex minimisations, such that the sparse tip-sample interaction can be recovered. Afterwards, the sample height is determined using the minimum smoothed cantilever deflection per cantilever oscillation period. Results from simulation experiments are in favour of the proposed approach as it consistently reveals sharp edges in sample height, as opposed to both the conventional and a closely related existing approach. However, the non-processed cantilever deflection provided most accurate sample height estimation. It is recommended to implement the STATESON-algorithm in the form of a filter to use it in feedback control of the scanner and cantilever excitation.

We demonstrate a novel closed-loop input design technique on the detection of particles in an imaging system such as a fluorescence microscope. The probability of misdiagnosis is minimized while constraining the input energy such that for instance phototoxicity is reduced. The key novelty of the closed-loop design is that each next input is designed based on the most recent information. Using updated hypothesis probabilities, the input energy distribution is optimized for detection such that unresolved pixels have increased illumination next image acquisition. As compared to conventional open-loop, the results show that (regions of) particles are diagnosed using less energy in the closed-loop approach. Besides the closed-loop approach being viable for particle detection in fluorescence microscopy measurements, it can be developed further to apply in different areas such as sequential object segmentation for reliable and efficient product inspection in Industry 4.0.

High-NA light sheet illumination can improve the resolution of single-molecule localization microscopy (SMLM) by reducing the background fluorescence. These approaches currently require custom-made sample holders or additional specialized objectives, which makes the sample mounting or the optical system complex and therefore reduces the usability of these approaches. Here, we developed a single-objective lens-inclined light sheet microscope (SOLEIL) that is capable of 2D and 3D SMLM in thick samples. SOLEIL combines oblique illumination with point spread function PSF engineering to enable dSTORM imaging in a wide variety of samples. SOLEIL is compatible with standard sample holders and off-the-shelve optics and standard high NA objectives. To accomplish optimal optical sectioning we show that there is an ideal oblique angle and sheet thickness. Furthermore, to show what optical sectioning delivers for SMLM we benchmark SOLEIL against widefield and HILO microscopy with several biological samples. SOLEIL delivers in 15 μm thick Caco2-BBE cells a 374% higher intensity to background ratio and a 54% improvement in the estimated CRLB compared to widefield illumination, and a 184% higher intensity to background ratio and a 20% improvement in the estimated CRLB compared to HILO illumination.

We have applied a combination of blind deconvolution and deep learning to the processing of Shack-Hartmann images.By using the intensity information contained in spot positions, and the fine structure of the separate images created by the lenslets,we have increased the sensitivity and resolution of the sensor over the limit defined by standard processing of spot displacements only.We also have demonstrated the applicability of the method to wavefront sensing using extended objects as a reference.

Some applications require high level of image-based classification certainty while keeping the total illumination energy as low as possible. Examples are minimally invasive visual inspection in Industry 4.0, and medical imaging systems such as computed tomography, in which the radiation dose should be kept “as low as is reasonably achievable”. We introduce a sequential object recognition scheme aimed at minimizing phototoxicity or bleaching while achieving a predefined level of decision accuracy. The novel online procedure relies on approximate weighted Bhattacharyya coefficients for determination of future inputs. Simulation results on the MNIST handwritten digit database show how the total illumination energy is decreased with respect to a detection scheme using constant illumination.

We investigate the general adjustment of projection-based phase retrieval algorithms for use with saturated data. In the phase retrieval problem, model fidelity of experimental data containing a non-zero background level, fixed pattern noise, or overexposure, often presents a serious obstacle for standard algorithms. Recently, it was shown that overexposure can help to increase the signal-to-noise ratio in AI applications. We present our first results in exploring this direction in the phase retrieval problem, using as an example the Gerchberg-Saxton algorithm with simulated data. The proposed method can find application in microscopy, characterisation of precise optical instruments, and machine vision applications of Industry4.0.

We develop for the first time a mathematical framework in which the class of projection algorithms can be applied to high numerical aperture (NA) phase retrieval. Within this framework, we first analyze the basic steps of solving the high-NA phase retrieval problem by projection algorithms and establish the closed forms of all the relevant projection operators. We then study the geometry of the high-NA phase retrieval problem and the obtained results are subsequently used to establish convergence criteria of projection algorithms in the presence of noise. Making use of the vectorial point-spread-function (PSF) is, on the one hand, the key difference between this paper and the literature of phase retrieval mathematics which deals with the scalar PSF. The results of this paper, on the other hand, can be viewed as extensions of those concerning projection methods for low-NA phase retrieval. Importantly, the improved performance of projection methods over the other classes of phase retrieval algorithms in the low-NA setting now also becomes applicable to the high-NA case. This is demonstrated by the accompanying numerical results which show that available solution approaches for high-NA phase retrieval are outperformed by projection methods.

This paper presents a computationally efficient framework in which a single focal-plane image is used to obtain a high-resolution reconstruction of dynamic aberrations. Assuming small-phase aberrations, a non-linear Kalman filter implementation is developed whose computational complexity scales close to linearly with the number of pixels of the focal-plane camera. The performance of themethod is tested in a simulation of an adaptive optics system, where the small-phase assumption is enforced by considering a closed-loop system that uses a low-resolution wavefront sensor to control a deformable mirror. The results confirmthe computational efficiency of the algorithm and showa large robustness against noise and model uncertainties.

We present the convergence analysis of convex combination of the alternating projection and Douglas–Rachford operators for solving the phase retrieval problem. New convergence criteria for iterations generated by the algorithm are established by applying various schemes of numerical analysis and exploring both physical and mathematical characteristics of the phase retrieval problem. Numerical results demonstrate the advantages of the algorithm over the other widely known projection methods in practically relevant simulations.

Inhomogeneities in the refractive index of a biological microscopy sample can introduce phase aberrations, severely impairing the quality of images. Adaptive optics can be employed to correct for phase aberrations and improve image quality. However, conventional adaptive optics can only correct a single phase aberration for the whole field of view (isoplanatic correction) while, due to the highly heterogeneous nature of biological tissues, the sample induced aberrations in microscopy often vary throughout the field of view (anisoplanatic aberration), limiting significantly the effectiveness of adaptive optics. This paper reports on a new approach for aberration correction in laser scanning confocal microscopy, in which a spatial light modulator is used to generate multiple excitation points in the sample to simultaneously scan different portions of the field of view with completely independent correction, achieving anisoplanatic compensation of sample induced aberrations, in a significantly shorter time compared to sequential isoplanatic correction of multiple image subregions. The method was tested in whole Drosophila brains and in larval Zebrafish, each showing a dramatic improvement in resolution and sharpness when compared to conventional isoplanatic adaptive optics.