Trjtdtk

I am trying to develop a multi-class classification model with keras. The task is similar to MNIST but applied to this character font image dataset. I have 153 classes and around 900k entries. I am using the following model and achieving ~0.24 accuracy. The input data is shaped 20,20,1, and normalized so the values range between 0 and 1. Targets are on the one-hot vector format.

X_input = Input(shape=(20,20,1,))

conv = Conv2D(32, (3, 3), activation='relu', padding='same', kernel_regularizer=l2(0.01))(X_input)

pool = MaxPooling2D(pool_size=(2, 2))(conv)

dropout = Dropout(0.5)(pool)

flat = Flatten()(dropout)

dense1 = Dense(512, activation='relu', kernel_constraint=maxnorm(3), kernel_regularizer=l2(0.01))(flat)

dropout = Dropout(0.5)(dense1)

dense2 = Dense(target_shape, activation='softmax', kernel_regularizer=l2(0.01))(dropout)

model = Model(inputs=X_input, outputs=dense2)

model.compile(loss = 'categorical_crossentropy', optimizer=SGD(lr=0.1), metrics=['accuracy'])

model.fit(X_train, Y_train, epochs=10, batch_size=32)

model.save(model_name)

What can I do to increase my accuracy?

Update

Following Shubham Panchal's suggestions, I've redefined my model to this:

X_input = Input(shape=(20,20,1,))

conv = Conv2D(64, (8, 8), activation='relu', padding='same', kernel_regularizer=l2(0.01))(X_input)

pool = MaxPooling2D(pool_size=(2, 2))(conv)

conv2 = Conv2D(128, (4, 4), activation='relu', padding='same', kernel_regularizer=l2(0.01))(pool)

pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)

flat = Flatten()(pool2)

dense1 = Dense(512, activation='relu', kernel_constraint=maxnorm(3), kernel_regularizer=l2(0.01))(flat)

dropout = Dropout(0.5)(dense1)

dense2 = Dense(target_shape, activation='softmax', kernel_regularizer=l2(0.01))(dropout)

model = Model(inputs=X_input, outputs=dense2)

model.compile(loss = 'categorical_crossentropy', optimizer=Adagrad(lr=0.001, epsilon=None, decay=0.0), metrics=['accuracy'])
model.fit(X_train, Y_train, epochs=10, batch_size=32)

model.save(model_name)

The accuracy increased to ~0.35 which is good but clearly not sufficient. Any other suggestions?

You can implement these measures which work in most use cases :

1. Reduce the SGD learning rate. You can reduce it to 0.001 or 0.0001.


2. Use AdamOptimizer if necessary.


3. Don't use Dropout in the Convolutional layers. It should be used only in the Dense layers.


4. Use more Conv2D layers. Using more such layers would help the model generalise better and interpret spatial data.


5. If ReLU doesn't work and you get lesser accuracy means your CNN may be suffering from dying neurons problem. To overcome this you can use LeakyReLU activation. You will not find it in tf.keras.activations module, you need to implement it manually and set alpha ( slope of line in negative side ) to 0.2.


6. 
   

   You can subsequently decrease the kernel size in Conv2D layers. For example in this case :

   
   
   

   Conv2D( 32, kernel_size=( 4 , 4 ) , strides=strides , activation=activation_func),
   MaxPooling2D(pool_size=pool_size, strides=strides ),
   
   Conv2D( 64, kernel_size=( 3 , 3 ) , strides=strides , activation=activation_func),
   MaxPooling2D(pool_size=pool_size , strides=strides),
   

   
   

Other than these, you can also experiment with BatchNormalization layer or Data Augmentation.