Trjtdtk

I'm fairly new at working with neural networks and think I am making some basic mistake. I am trying to assign simulated images to 5 classes to test, what (if any) networks are helpful for a large data problem we have in our group. I am training the ResNet50 CNN included in Keras on 40000 (256 by 256 pixel greyscale) images. While training loss is improving quickly within the first epoch, validation loss fluctuates wildly in (to me) fairly random manner.

I am trying to use as many high level functions as possible and have ended up with the following code:

from keras.preprocessing.image import ImageDataGenerator
from keras.applications.resnet50 import ResNet50
from keras.models import Model
from keras.layers import GlobalAveragePooling2D, Dense


inputShape = (256, 256,1)
targetSize = (256, 256 )
batchSize = 128

# First, load resnet
base_model = ResNet50(include_top =False,
 weights =None,
 input_shape=inputShape)
x = base_model.output

# modified based on the article https://github.com/priya-dwivedi/
# Deep-Learning/blob/master/resnet_keras/Residual_Networks_yourself.ipynb

x = GlobalAveragePooling2D()(x)
predictions = Dense(5, activation= 'softmax')(x)
model = Model(inputs = base_model.input, outputs = predictions)


# Compile model, might want to change loss and metrics
model.compile(loss='categorical_crossentropy',
 optimizer='adam',
 metrics = ["acc"]
 )

# Define the generators to read the data. Training generator performs data argumentation as well
train_datagen = ImageDataGenerator(
 samplewise_center =True,
 samplewise_std_normalization = True,
 shear_range=0.2,
 zoom_range=0.2,
 horizontal_flip=True)

test_datagen = ImageDataGenerator(
 samplewise_center =True,
 samplewise_std_normalization = True,
 )

train_generator = train_datagen.flow_from_directory(
 'trainingData/FolderBasedTrain256',
 target_size=targetSize,
 color_mode = 'grayscale',
 batch_size=batchSize,
 save_to_dir = 'trainingData/argumentedImages',
 class_mode='categorical')

validation_generator = test_datagen.flow_from_directory(
 'trainingData/FolderBasedValidation256',
 target_size=targetSize,
 batch_size=batchSize,
 color_mode = 'grayscale',
 class_mode='categorical')

# Fit the Modell, saving the history for later plotting
history = model.fit_generator(
 train_generator,
 epochs=20,
 validation_data=validation_generator,
 #steps_per_epoch = 62,
 steps_per_epoch = 31,
 #callbacks=[checkpointer],
 validation_steps=9)

I can always create more images or train longer, but to me, this looks as if something fundamental went wrong somewhere. I would be very gratefull for any and all ideas. Thank you!

EDIT: I was told to stress that validation and training set are both created by exaclty the same simulation routine, so they should be relativly easy to classify

EDIT2: Found the error! My batch size did not fit with the amount of data and the steps by epoch, resulting in the CNN not seeing all the training data. Now everything converges nicely and I can evaluate the choice of modell. Thanks to all contributers

There is nothing fundamentally wrong with your code, but maybe your model is not right for your current toy-problem.

In general, this is typical behavior when training in deep learning. Think about it, your target loss is the training loss, so it is directly affected by the training process and as you said "improving quickly". The validation loss is only affected indirectly, so naturally it will be more volatile in comparison.

When you are training, the model is attempting to estimate the real distribution of the data, however all it got is the distribution of the training dataset to rely on (which is similar but not the same).

Suggestions:

I think that your model is an over-kill for a 256x256 grayscale dataset with just 5 classes (Resnet was designed for ImageNet which contains RGB images from 1000 categories). In current state, the model finds its very easy to memorize the training set and overfit. You should look for models that are meant to be used for MNIST, or at most CIFAR10.

You can insist on this model and attempt to increase regularization, but I'm not sure if it will be enough to prevent the model from overfitting in this case.