1_Super-Resolution-AME-TrainingModule

Deep-learning-based denoising and reconstruction of superresolution structured illumination microscopy images¶

Download the code from GitHub here!

Part 1: Training

Motivation and problem description¶

Super-resolution structured illumination microscopy (SR-SIM) can provide up to two times spatial resolution of fluorescently labeled samples. Reconstruction of high-quality SR-SIM images depends critically on the various sinusoidal illumination patterns with high modulation contrast. However, there is a crucial trade-off between the spatial resolution of the raw SIM samples and the low exposure times. When the biological sample is illuminated with low excitation power, the raw SIM samples result in low SNR. To address this critical problem, we demonstrate here the end-to-end deep learning-based denoising and reconstruction of noisy raw SIM images into high-resolution SR-SIM images. The proposed SR-REDSIM method generalizes very well across different noise levels and different microscopic settings. Moreover, this method also proves to be very robust to image reconstruction artifacts.

Solution¶

There are several reconstruction algorithms that convert the raw SIM images into SR-SIM images. However, these algorithms are not able to reconstruct a high-quality super-resolution image from noisy raw SIM images. The SR-REDSIM reconstruction algorithm is a complete Deep Learning based end-to-end method that can be used to reconstruct and denoise the noisy raw SIM images. The complete pipeline of SR-REDSIM is shown in the following figure [1]. From Fig. 1, it can be seen that the stack of 15 raw noisy SIM images (three angles with five phases each) with a size of 512×512 pixels [i.e., the stack dimensions were 15×512×512 (frame, width, height)] is provided as input. The algorithm produces a high-quality SR-SIM image with a size of 1024×1024 pixels as output.

title [1]

Loading module¶

In [1]:

import tensorflow as tf
import os, time, datetime, shutil, pickle,argparse, json
import SRDataGenerator
import sys
from utils import compileDNN, psnr
from SRREDModel import SRREDNet,UNet
import matplotlib.pyplot as plt

Loading and preparation of training data¶

This module will load the training and test sample images. The training and test input images are enclosed in the form of stacks where each stack further contains 15 frames of noisy raw SIM images (i.e., 15 images from 5 different phases and 3 orientations) along with high-quality SR-SIM samples as output or ground-truth images. The high-quality SR-SIM images are reconstructed by using the conventional fairSIM reconstruction algorithm [2]. Once the data is loaded, it is further normalized and reshaped according to the requirements. The normalization function will enforce randomness in the training and test data, however, it is not mandatory as the Keras fit function internally shuffles the training data.

In [2]:

def LoadPreprocessedData(imageDir, randomization):


    print('Loading Data')

    trainxPath= os.path.join(imageDir,'trainx')
    trainyPath=os.path.join(imageDir,'trainy')
    testxPath= os.path.join(imageDir,'trainx')
    testyPath= os.path.join(imageDir,'trainy')

    trainXpath= SRDataGenerator.ExtractPath(trainxPath)
    trainYpath= SRDataGenerator.ExtractPath(trainyPath)
    testXpath= SRDataGenerator.ExtractPath(testxPath)
    testYpath= SRDataGenerator.ExtractPath(testyPath)


    print('Loading Images')
    trainDataX= SRDataGenerator.LoadImages(trainXpath)
    trainDataY= SRDataGenerator.LoadImages(trainYpath)
    testDataX= SRDataGenerator.LoadImages(testXpath)
    testDataY= SRDataGenerator.LoadImages(testYpath)


    # Scale images down to [0,1] for following operations
    trainDataX = SRDataGenerator.ImageNormalization(trainDataX[:5])
    trainDataY = SRDataGenerator.ImageNormalization(trainDataY[:5])

    testDataX = SRDataGenerator.ImageNormalization(testDataX[:5])
    testDataY = SRDataGenerator.ImageNormalization(testDataY[:5])



    if randomization:
        print ('Randomization is done')
        trainDataX, trainDataY = Randomization(trainDataX, trainDataY)
        testDataX, testDataY = Randomization(testDataX, testDataY)
    else:
        print ('Data is not Randomized')


    trainDataX, trainDataY= SRDataGenerator.ReshapeData(trainDataX, trainDataY)

    testDataX, testDataY= SRDataGenerator.ReshapeData(testDataX, testDataY)




    return trainDataX, trainDataY, testDataX, testDataY

In [5]:

trainDataX, trainDataY, testDataX, testDataY=LoadPreprocessedData('/home/shah/Documents/Year-2022/ITSML-Project/data', False)
print (trainDataX[0].shape)
print (trainDataY[0].shape)

Loading Data
Loading Images
Data is not Randomized
(15, 512, 512)
(1, 1024, 1024)

Let's have a look at the stack of noisy raw input images and the reference image. Even the ground truth image exhibits reconstruction artifacts, therefore it is named as "reference image".

title [1]

Super-Resolution model¶

SR-REDSIM is based on a modified version of RED-Net. The SR-REDSIM architecture consists of three blocks: the encoder, the decoder, and the upsampling block. SR-REDSIM contains a total of 44 convolutional and deconvolutional layers with symmetric skip connections. The encoder block is composed of 21 convolutional layers, whereas the decoder contains 21 deconvolutional layers. The upsampling block consists of two deconvolutional layers that perform the upsampling task by adjusting the size of the stride. The SR-REDSIM model provides the best results after training the model for 100 epochs.

title [1]

Training configuration¶

Before loading the model we can change some of the important parameters of the model.

In [6]:

ImageWidth, ImageHeight, channel =512,512,15
NumberofKernels=64
sizeofKernel=3
padding='same'
stride=1
activation='relu'
LOSS_TYPE="mean_squared_error"
#model=SRREDNet(ImageWidth, ImageHeight, channel,NumberofKernels,sizeofKernel,padding,stride, activation)

Training model¶

For the training of the model for this super-resolution task, it is recommended to use multiple GPUs in order to speed up the training process. Normally, the training process for 100 epochs requires 12-15 hours with two Nvidia Tesla P100 graphics cards. The training process can be monitored by inspecting the convergence of the loss function value.

In [7]:

#Build model with GPU only
strategy = tf.distribute.MirroredStrategy()
print('Number of devices: {}'.format(strategy.num_replicas_in_sync))
with strategy.scope():
    model=SRREDNet(512, 512, 15,64,3,padding,stride, activation)
    print('Build model')
    model=model.buildDNN()
    model.summary()
    compileDNN(model, LOSS_TYPE)

print ('shape of training data X',trainDataX[0].shape)
print ('shape of training data Y', trainDataY[0].shape)

trainingModel = model.fit(trainDataX, trainDataY, validation_data=(testDataX, testDataY), batch_size=1, epochs=2, verbose=1, initial_epoch=0)
trainingEvaluation = model.evaluate(trainDataX, trainDataY, batch_size=1, verbose=1)

WARNING:tensorflow:There is non-GPU devices in `tf.distribute.Strategy`, not using nccl allreduce.
Number of devices: 1
Build model
Model: "model_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_2 (InputLayer)            [(None, 15, 512, 512 0
__________________________________________________________________________________________________
conv2d_21 (Conv2D)              (None, 32, 512, 512) 4352        input_2[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_21[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_22[0][0]
__________________________________________________________________________________________________
conv2d_24 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_23[0][0]
__________________________________________________________________________________________________
conv2d_25 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_24[0][0]
__________________________________________________________________________________________________
conv2d_26 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_25[0][0]
__________________________________________________________________________________________________
conv2d_27 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_26[0][0]
__________________________________________________________________________________________________
conv2d_28 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_27[0][0]
__________________________________________________________________________________________________
conv2d_29 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_28[0][0]
__________________________________________________________________________________________________
conv2d_30 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_29[0][0]
__________________________________________________________________________________________________
conv2d_31 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_30[0][0]
__________________________________________________________________________________________________
conv2d_32 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_31[0][0]
__________________________________________________________________________________________________
conv2d_33 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_32[0][0]
__________________________________________________________________________________________________
conv2d_34 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_33[0][0]
__________________________________________________________________________________________________
conv2d_35 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_34[0][0]
__________________________________________________________________________________________________
conv2d_36 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_35[0][0]
__________________________________________________________________________________________________
conv2d_37 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_36[0][0]
__________________________________________________________________________________________________
conv2d_38 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_37[0][0]
__________________________________________________________________________________________________
conv2d_39 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_38[0][0]
__________________________________________________________________________________________________
conv2d_40 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_39[0][0]
__________________________________________________________________________________________________
conv2d_41 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_40[0][0]
__________________________________________________________________________________________________
conv2d_transpose_23 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_41[0][0]
__________________________________________________________________________________________________
add_10 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_23[0][0]
                                                                 conv2d_40[0][0]
__________________________________________________________________________________________________
activation_10 (Activation)      (None, 32, 512, 512) 0           add_10[0][0]
__________________________________________________________________________________________________
conv2d_transpose_24 (Conv2DTran (None, 32, 512, 512) 9248        activation_10[0][0]
__________________________________________________________________________________________________
conv2d_transpose_25 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_24[0][0]
__________________________________________________________________________________________________
add_11 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_25[0][0]
                                                                 conv2d_38[0][0]
__________________________________________________________________________________________________
activation_11 (Activation)      (None, 32, 512, 512) 0           add_11[0][0]
__________________________________________________________________________________________________
conv2d_transpose_26 (Conv2DTran (None, 32, 512, 512) 9248        activation_11[0][0]
__________________________________________________________________________________________________
conv2d_transpose_27 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_26[0][0]
__________________________________________________________________________________________________
add_12 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_27[0][0]
                                                                 conv2d_36[0][0]
__________________________________________________________________________________________________
activation_12 (Activation)      (None, 32, 512, 512) 0           add_12[0][0]
__________________________________________________________________________________________________
conv2d_transpose_28 (Conv2DTran (None, 32, 512, 512) 9248        activation_12[0][0]
__________________________________________________________________________________________________
conv2d_transpose_29 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_28[0][0]
__________________________________________________________________________________________________
add_13 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_29[0][0]
                                                                 conv2d_34[0][0]
__________________________________________________________________________________________________
activation_13 (Activation)      (None, 32, 512, 512) 0           add_13[0][0]
__________________________________________________________________________________________________
conv2d_transpose_30 (Conv2DTran (None, 32, 512, 512) 9248        activation_13[0][0]
__________________________________________________________________________________________________
conv2d_transpose_31 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_30[0][0]
__________________________________________________________________________________________________
add_14 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_31[0][0]
                                                                 conv2d_32[0][0]
__________________________________________________________________________________________________
activation_14 (Activation)      (None, 32, 512, 512) 0           add_14[0][0]
__________________________________________________________________________________________________
conv2d_transpose_32 (Conv2DTran (None, 32, 512, 512) 9248        activation_14[0][0]
__________________________________________________________________________________________________
conv2d_transpose_33 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_32[0][0]
__________________________________________________________________________________________________
add_15 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_33[0][0]
                                                                 conv2d_30[0][0]
__________________________________________________________________________________________________
activation_15 (Activation)      (None, 32, 512, 512) 0           add_15[0][0]
__________________________________________________________________________________________________
conv2d_transpose_34 (Conv2DTran (None, 32, 512, 512) 9248        activation_15[0][0]
__________________________________________________________________________________________________
conv2d_transpose_35 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_34[0][0]
__________________________________________________________________________________________________
add_16 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_35[0][0]
                                                                 conv2d_28[0][0]
__________________________________________________________________________________________________
activation_16 (Activation)      (None, 32, 512, 512) 0           add_16[0][0]
__________________________________________________________________________________________________
conv2d_transpose_36 (Conv2DTran (None, 32, 512, 512) 9248        activation_16[0][0]
__________________________________________________________________________________________________
conv2d_transpose_37 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_36[0][0]
__________________________________________________________________________________________________
add_17 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_37[0][0]
                                                                 conv2d_32[0][0]
__________________________________________________________________________________________________
activation_17 (Activation)      (None, 32, 512, 512) 0           add_17[0][0]
__________________________________________________________________________________________________
conv2d_transpose_38 (Conv2DTran (None, 32, 512, 512) 9248        activation_17[0][0]
__________________________________________________________________________________________________
conv2d_transpose_39 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_38[0][0]
__________________________________________________________________________________________________
add_18 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_39[0][0]
                                                                 conv2d_30[0][0]
__________________________________________________________________________________________________
activation_18 (Activation)      (None, 32, 512, 512) 0           add_18[0][0]
__________________________________________________________________________________________________
conv2d_transpose_40 (Conv2DTran (None, 32, 512, 512) 9248        activation_18[0][0]
__________________________________________________________________________________________________
conv2d_transpose_41 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_40[0][0]
__________________________________________________________________________________________________
add_19 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_41[0][0]
                                                                 conv2d_28[0][0]
__________________________________________________________________________________________________
activation_19 (Activation)      (None, 32, 512, 512) 0           add_19[0][0]
__________________________________________________________________________________________________
conv2d_transpose_42 (Conv2DTran (None, 32, 512, 512) 9248        activation_19[0][0]
__________________________________________________________________________________________________
conv2d_transpose_43 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_42[0][0]
__________________________________________________________________________________________________
conv2d_transpose_44 (Conv2DTran (None, 32, 1024, 102 9248        conv2d_transpose_43[0][0]
__________________________________________________________________________________________________
conv2d_transpose_45 (Conv2DTran (None, 1, 1024, 1024 289         conv2d_transpose_44[0][0]
==================================================================================================
Total params: 393,057
Trainable params: 393,057
Non-trainable params: 0
__________________________________________________________________________________________________
Model: "model_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_2 (InputLayer)            [(None, 15, 512, 512 0
__________________________________________________________________________________________________
conv2d_21 (Conv2D)              (None, 32, 512, 512) 4352        input_2[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_21[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_22[0][0]
__________________________________________________________________________________________________
conv2d_24 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_23[0][0]
__________________________________________________________________________________________________
conv2d_25 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_24[0][0]
__________________________________________________________________________________________________
conv2d_26 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_25[0][0]
__________________________________________________________________________________________________
conv2d_27 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_26[0][0]
__________________________________________________________________________________________________
conv2d_28 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_27[0][0]
__________________________________________________________________________________________________
conv2d_29 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_28[0][0]
__________________________________________________________________________________________________
conv2d_30 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_29[0][0]
__________________________________________________________________________________________________
conv2d_31 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_30[0][0]
__________________________________________________________________________________________________
conv2d_32 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_31[0][0]
__________________________________________________________________________________________________
conv2d_33 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_32[0][0]
__________________________________________________________________________________________________
conv2d_34 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_33[0][0]
__________________________________________________________________________________________________
conv2d_35 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_34[0][0]
__________________________________________________________________________________________________
conv2d_36 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_35[0][0]
__________________________________________________________________________________________________
conv2d_37 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_36[0][0]
__________________________________________________________________________________________________
conv2d_38 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_37[0][0]
__________________________________________________________________________________________________
conv2d_39 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_38[0][0]
__________________________________________________________________________________________________
conv2d_40 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_39[0][0]
__________________________________________________________________________________________________
conv2d_41 (Conv2D)              (None, 32, 512, 512) 9248        conv2d_40[0][0]
__________________________________________________________________________________________________
conv2d_transpose_23 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_41[0][0]
__________________________________________________________________________________________________
add_10 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_23[0][0]
                                                                 conv2d_40[0][0]
__________________________________________________________________________________________________
activation_10 (Activation)      (None, 32, 512, 512) 0           add_10[0][0]
__________________________________________________________________________________________________
conv2d_transpose_24 (Conv2DTran (None, 32, 512, 512) 9248        activation_10[0][0]
__________________________________________________________________________________________________
conv2d_transpose_25 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_24[0][0]
__________________________________________________________________________________________________
add_11 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_25[0][0]
                                                                 conv2d_38[0][0]
__________________________________________________________________________________________________
activation_11 (Activation)      (None, 32, 512, 512) 0           add_11[0][0]
__________________________________________________________________________________________________
conv2d_transpose_26 (Conv2DTran (None, 32, 512, 512) 9248        activation_11[0][0]
__________________________________________________________________________________________________
conv2d_transpose_27 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_26[0][0]
__________________________________________________________________________________________________
add_12 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_27[0][0]
                                                                 conv2d_36[0][0]
__________________________________________________________________________________________________
activation_12 (Activation)      (None, 32, 512, 512) 0           add_12[0][0]
__________________________________________________________________________________________________
conv2d_transpose_28 (Conv2DTran (None, 32, 512, 512) 9248        activation_12[0][0]
__________________________________________________________________________________________________
conv2d_transpose_29 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_28[0][0]
__________________________________________________________________________________________________
add_13 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_29[0][0]
                                                                 conv2d_34[0][0]
__________________________________________________________________________________________________
activation_13 (Activation)      (None, 32, 512, 512) 0           add_13[0][0]
__________________________________________________________________________________________________
conv2d_transpose_30 (Conv2DTran (None, 32, 512, 512) 9248        activation_13[0][0]
__________________________________________________________________________________________________
conv2d_transpose_31 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_30[0][0]
__________________________________________________________________________________________________
add_14 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_31[0][0]
                                                                 conv2d_32[0][0]
__________________________________________________________________________________________________
activation_14 (Activation)      (None, 32, 512, 512) 0           add_14[0][0]
__________________________________________________________________________________________________
conv2d_transpose_32 (Conv2DTran (None, 32, 512, 512) 9248        activation_14[0][0]
__________________________________________________________________________________________________
conv2d_transpose_33 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_32[0][0]
__________________________________________________________________________________________________
add_15 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_33[0][0]
                                                                 conv2d_30[0][0]
__________________________________________________________________________________________________
activation_15 (Activation)      (None, 32, 512, 512) 0           add_15[0][0]
__________________________________________________________________________________________________
conv2d_transpose_34 (Conv2DTran (None, 32, 512, 512) 9248        activation_15[0][0]
__________________________________________________________________________________________________
conv2d_transpose_35 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_34[0][0]
__________________________________________________________________________________________________
add_16 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_35[0][0]
                                                                 conv2d_28[0][0]
__________________________________________________________________________________________________
activation_16 (Activation)      (None, 32, 512, 512) 0           add_16[0][0]
__________________________________________________________________________________________________
conv2d_transpose_36 (Conv2DTran (None, 32, 512, 512) 9248        activation_16[0][0]
__________________________________________________________________________________________________
conv2d_transpose_37 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_36[0][0]
__________________________________________________________________________________________________
add_17 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_37[0][0]
                                                                 conv2d_32[0][0]
__________________________________________________________________________________________________
activation_17 (Activation)      (None, 32, 512, 512) 0           add_17[0][0]
__________________________________________________________________________________________________
conv2d_transpose_38 (Conv2DTran (None, 32, 512, 512) 9248        activation_17[0][0]
__________________________________________________________________________________________________
conv2d_transpose_39 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_38[0][0]
__________________________________________________________________________________________________
add_18 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_39[0][0]
                                                                 conv2d_30[0][0]
__________________________________________________________________________________________________
activation_18 (Activation)      (None, 32, 512, 512) 0           add_18[0][0]
__________________________________________________________________________________________________
conv2d_transpose_40 (Conv2DTran (None, 32, 512, 512) 9248        activation_18[0][0]
__________________________________________________________________________________________________
conv2d_transpose_41 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_40[0][0]
__________________________________________________________________________________________________
add_19 (Add)                    (None, 32, 512, 512) 0           conv2d_transpose_41[0][0]
                                                                 conv2d_28[0][0]
__________________________________________________________________________________________________
activation_19 (Activation)      (None, 32, 512, 512) 0           add_19[0][0]
__________________________________________________________________________________________________
conv2d_transpose_42 (Conv2DTran (None, 32, 512, 512) 9248        activation_19[0][0]
__________________________________________________________________________________________________
conv2d_transpose_43 (Conv2DTran (None, 32, 512, 512) 9248        conv2d_transpose_42[0][0]
__________________________________________________________________________________________________
conv2d_transpose_44 (Conv2DTran (None, 32, 1024, 102 9248        conv2d_transpose_43[0][0]
__________________________________________________________________________________________________
conv2d_transpose_45 (Conv2DTran (None, 1, 1024, 1024 289         conv2d_transpose_44[0][0]
==================================================================================================
Total params: 393,057
Trainable params: 393,057
Non-trainable params: 0
__________________________________________________________________________________________________
shape of training data X (15, 512, 512)
shape of training data Y (1, 1024, 1024)
Train on 3 samples, validate on 3 samples
Epoch 1/2
3/3 [==============================] - 55s 18s/sample - loss: 0.0052 - mean_squared_error: 0.0052 - val_loss: 0.0233 - val_mean_squared_error: 0.0233
Epoch 2/2
3/3 [==============================] - 62s 21s/sample - loss: 0.0159 - mean_squared_error: 0.0159 - val_loss: 0.0051 - val_mean_squared_error: 0.0051
3/1 [==========================================================================================] - 13s 4s/sample - loss: 0.0037 - mean_squared_error: 0.0051

Analysis of the predicted image¶

It can be clearly seen that the trained model generalizes well on the stack of the unseen noisy raw input SIM samples and generates high-resolution SR-SIM images. The predicted image below on the right side shows a very smooth cell structure and does not show any reconstruction artifacts whereas in the reference image in the middle, grainy noise is very prominent in the filaments of the cell structure.

title

Export Train Model¶

Once the training process is finished, the trained model will be stored at the specified location.

In [ ]:

trainingModel.save('path to save model')

Reference¶

[1] Zafran Hussain Shah, Marcel Müller, Tung-Cheng Wang, Philip Maurice Scheidig, Axel Schneider, Mark Schüttpelz, Thomas Huser, and Wolfram Schenck, "Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images," Photon. Res. 9, B168-B181 (2021)

[2] M. Müller, V. Mönkemöller, S. Hennig, W. Hübner, and T. Huser, “Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ,” Nat. Commun. 7, 10980 (2016).