Agenda:
Use Open source implementations (available at GitHub)
• ResNet for face superresolution: Keras library:
• How to use Kaggle kernel
• key advantages
• Pipe line to build Network:
• Using tensor board
• How to save the and restore the weights of network
• tf.keras
• Example code
• Use of tensor board
Generative adversarial network (GAN)
• introduction
• How they work
• objective function
• conclusion
Resources used for Keras:
• https://www.tensorflow.org/guide/keras •
https://github.com/Kulbear/deep-learning- coursera/blob/master/Convolutional%20Neural%20Networks/Keras%20-%20Tutorial%20-%20Happy%20House%20v1.ipynb
Resources used for Generative adversarial networks: • https://www.youtube.com/watch?v=yz6dNf7X7SA&t=654s
• https://towardsdatascience.com/generative-adversarial-networks-explained-34472718707a • https://www.youtube.com/watch?v=5WoItGTWV54
Use Open source implementations (available at GitHub)
• git clone https://github.com/MathiasGruber/SRGAN-Keras.git ResNet:
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial
Network by Ledig - CVPR 2017: SRResnet (ResNet (Residual Network), proposed by He at all in Deep Residual Learning for Image Recognition paper (2015),)
Keras libraray:
The Python Deep Learning library• Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow library.
• It was developed with a focus on enabling fast experimentation.
• Supports both convolutional networks and recurrent networks, as well as combinations of the two.
• Runs seamlessly on CPU and GPU. • To run the keras program:
• Install keras using anaconda environment (virtual machine) • Use Kaggle kernel (https://www.kaggle.com/). No need to install
• Pipeline to build Network:
1. Define network: To build a network
2. Compile network: To configure it for training process 3. Fit network: To fit the model with training data
4. Evaluate network: To evaluate evaluation time for trained model on test data
5. Make predict: predict the output of model on test data provided
• Also see the how to draw loss and accuracy curve using tensor board
• How to save the and restore the weights of network
• tf.keras:
• Import tf.keras : Give the interface to access all features of tensorflow liberay in easier way.
• TensorFlow easier to use without sacrificing flexibility and performance.
• Examples code1: https://www.tensorflow.org/guide/keras import tensorflow as tf
from tensorflow.keras import layers import numpy as np
# training:validation:testing=70:15:15
#For small datasets, use in-memory NumPy arrays to train and evaluate a model. data = np.random.random((2000, 32)) #32=8x4 size of image,
labels = np.random.random((2000, 10)) # 10(0-9 digits) outputs val_data = np.random.random((450, 32))
val_labels = np.random.random((450, 10)) test_data = np.random.random((450, 32)) test_labels = np.random.random((450, 10)) model = tf.keras.Sequential([
# Adds a densely-connected layer with 64 units to the model: layers.Dense(64, activation='relu', input_shape=(32,)), # Add another:
layers.Dense(64, activation='relu'),
# Add a softmax layer with 10 output units: layers.Dense(10, activation='softmax')])
# configure learning process by calling the compile method. Specify the optimizer, loss, and metric model.compile(optimizer=tf.train.AdamOptimizer(0.001),
metrics=['accuracy'])
#The model is "fit" to the training data using the fit method
#Epoch: An epoch is one iteration over the entire input data (this is done in smaller batches).[31(#training example are 62)+ 1(# trainexample are 16 = 2000)]
#Batchsize:the model slices the data into smaller batches and iterates over these batches during training
#Validatation: The model to display the loss and metrics in inference mode for the passed data, at the end of each epoch.
#model.fit(data, labels, epochs=10, batch_size=32,validation_data=(val_data, val_labels)) #A callback is an object passed to a model to customize and extend its behavior during training. callbacks = [tf.keras.callbacks.TensorBoard(log_dir='./myLogs')]
model.fit(data, labels, batch_size=32, epochs=5, callbacks=callbacks, validation_data=(val_data, val_labels)) # Save entire model to a HDF5 file
model.save('my_model500.h5')
# Recreate the exact same model, including weights and optimizer. #model = tf.keras.models.load_model('my_model.h5')
model.evaluate(test_data, test_labels, batch_size=32) #To evaluate performance of trained model
result = model.predict(test_data, batch_size=32) print(result.shape)
print(result.shape[0]) # relate this example with digit recognition????/ and tensor board loss curve, save weights... 2nd and 3rd code example
print(result.shape[1])
Tensorboard: Enable during training call
callbacks = [ # A callback is an object passed to a model to customize and extend its behavior during training # Interrupt training if `val_loss` stops improving for over 2 epochs
tf.keras.callbacks.EarlyStopping(patience=2, monitor='val_loss'), # Write TensorBoard logs to `./logs` directory
tf.keras.callbacks.TensorBoard(log_dir='./logs') ]
model.fit(data, labels, batch_size=32, epochs=5, callbacks=callbacks, validation_data=(val_data, val_labels))
Tensorboard: See the training loss and accuracy graph:
Example code2: https://github.com/keras-team/keras/blob/master/examples/mnist_cnn.py …. … Model=Sequential() model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=input_shape)) model.add(Conv2D(64, (3, 3), activation='relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Dropout(0.25)) model.add(Flatten()) model.add(Dense(128, activation='relu')) model.add(Dropout(0.5)) model.add(Dense(num_classes, activation='softmax')) model.compile(loss=keras.losses.categorical_crossentropy, optimizer=keras.optimizers.Adadelta(),
metrics=['accuracy'])
Example code3: Fashion Mnist (Kaggle kernel): I will share code. Run it using Kaggle kernel.
Generative adversarial Networks (GAN): GAN and its variants(Cycle GAN/Star GANS) in many network architectures like in face SR and many other problems
Face Super-resolution Guided by Facial Component Heatmaps by Xin Yu - ECCV 2018
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Jun-Yan Zhu 2018
1. Cycle GAN results on real data (ours): Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks: Trained Cycle Gan model on our 35 pairs of LR-HR pair dataset for 200epoch (Time of training :30 min).
Introduction
• Unsupervised model use to generate sample from the training distribution (generate the data that mimic/copy the original training dataset)
• The network learns to generate from a training distribution through a 2-player game.
• Use game-theoretic approach for optimization • The two entities are Generator and Discriminator.
• These two adversaries are in constant battle throughout the training process • Best samples
• But can be tricky and unstable to train
How they work:
• A Generator is used to generate fake image (real-looking images) and discriminator try to distinguish between real and fake images.
• The entities/adversaries are in constant battle as one(generator) tries to fool the other(discriminator), while the other tries not to be fooled.
• The input, random noise can be a Gaussian distribution and values can be
sampled from this distribution and fed into the generator network and an image is generated.
• This generated image is compared with a real image by the discriminator and it tries to identify if the given image is fake or real.
• The discriminator tries to maximize the objective function; therefore, we can perform gradient ascent on the objective function.
• The generator tries to minimize the objective function; therefore, we can perform gradient descent on the objective function.
By alternating between gradient ascent and descent, the network can be trained
• Objective Function: Since a game-theoretic approach is taken, our objective function is represented as a minimax function.
• But when applied, it is observed that optimizing the generator objective function does not work so well, this is because when the sample is generated is likely to be classified as fake, the model would like to learn from the gradients, but the gradients turn out to be relatively flat. This makes it difficult for the model to learn.
• Therefore, the generator objective function is changed as below.
• Instead of minimizing the likelihood of discriminator being correct, we maximize the likelihood of discriminator being wrong. Therefore, we perform gradient ascent on generator according to this objective function.
Disadvantages
• GANs are more unstable to train because you have to train two networks from a single backpropagation. [WGAN]
Conclusion
• Generative Adversarial Networks are a recent development and have shown huge promises already.
• It is an active area of research and new variants of GANs are coming up frequently.
• It is observed that in the recent face super resolution algorithm GAN are used as important component of their architecture.