Basic problem formulation and objective

General:

Super-resolution structured illumination microscopy (SR-SIM) is a powerful technique in the field of fluorescence microscopy that can surpass the optical diffraction limit. In structured illumination microscopy, the fluorescent samples are excited by a series of sinusoidal patterns with different phases and rotations instead of using uniform illumination. These raw images from different phases and orientations are then propagated to different reconstruction algorithms such as fairSIM or OpenSIM, to get the final two-fold super-resolution SIM image.

Problem:

Many biological cells are vulnerable to high power excitation and as a consequence, they are captured with short exposure times or restricted laser power. Therefore, the resulting images have a low signal-to-noise ratio (SNR). The classical SIM reconstruction algorithms are unable to remove the noise and even introduce noise-based artifacts in the reconstructed super-resolution images.Objective:

The basic objective of this work was to design an end-to-end deep learning workflow that can denoise and reconstruct the noisy raw SIM samples into high-resolution SR-SIM images.

Keywords: Greybox learning

Technical System

The ML solution was developed to generate high SNR super-resolution images from low SNR raw SIM images of biological cells.

Welche sensors are used in this work?

DeltaVision|OMX microscope was used to collect the SIM raw images.

What kind of data is acquired?

Raw SIM images of U2OS osteosarcoma cells, tubulin cytoskeleton labeled with antitubulin-Alexa488.

Data acquisition and handling:

A total of 101 randomly selected fields of view were captured by exposing each field of view to the full laser power of a 488 nm excitation laser. The exposure time was set to 20 ms for each of the 15 raw images (three rotations times five phases) in a single recording.
For each field of view, 200 recordings were carried out in a sequence without time delay between recordings. Over the time course of the 200 recordings, photo bleaching took place. Therefore, the first recordings in such a sequence have a very high SNR, while the last recordings have a very low SNR.

Solution

ML-method:

A modified residual encoding-decoding network (RED-Net) based architecture was used to denoise and reconstruct low SNR raw SIM images into high-quality SR-SIM images.

What are general criteria for decision making? How are different criteria / features weighted?

The decisions are mainly made based on evaluation metrics, such as peak signal to noise ratio (PSNR) and structural similarity index measurement (SSIM) as well as human perception of visual quality.

The complete pipeline of the preRED-fairSIM method. In this pipeline, the raw SIM images [512 × 512 (width × height)] of all phases and orientations are denoised separately with the RED-Net architecture. The denoised SIM images of each phase and orientation are then propagated into the fairSIM software in the form of a stack (15 frames) to reconstruct the super- resolution SIM image (Shah et al., 2021).

Are new features derived that were not measured explicitly (e.g., via combination of measured features)?

Yes, implicitly via feature learning in the deep CNN (RED-Net).

What are general criteria for learning?

Two criteria are applied:

Denoising:
According to the first criterion, the network has to learn to denoise the noisy raw samples of different phases and orientations in the process of creating denoised super-resolution output images.
Reconstruction:
According to the second criterion, the network reconstructs the denoised samples into two-fold super-resolution SIM images which have to show cell structures at a same or better quality level than conventional SIM reconstruction algorithms.

Super-resolution SIM (SR-SIM) images of a test sample (U2OS osteosarcoma cells, tubulin cytoskeleton labeled with anti- tubulin-Alexa488) at high-level noise. Each column represents a different reconstruction approach: fairSIM (first column), SR-REDSIM (second column), U-Net-fairSIM (third column), and RED-fairSIM (fourth column). The fifth column depicts the reconstructed reference images which were generated by applying fairSIM image reconstruction to high SNR image data at noise level 0 (lowest noise level; i.e., timestamp 0). Scale bar: 4 μm.

References:

Zafran Hussain Shah, Marcel Müller, Tung-Cheng Wang, Philip Maurice Scheidig, Axel Schneider, Mark Schüttpelz, Thomas Huser, and Wolfram Schenck. Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images. Photonics Research, 9(5):B168–B181, 2021. (DOI: 10.1364/PRJ.416437).