Why the CNN model is not trained?

I am trying to train a CNN model in Matlab to predict the mean value of a random vector (the Matlab code named Test_2 is attached). To further clarify, I am generating a random vector with 10 components (using rand function) for 500 times. Correspondingly, the figure of each vector versus 1:10 is plotted and saved separately. Moreover, the mean value of each of the 500 randomly generated vectors are calculated and saved. Thereafter, the saved images are used as the input file (X) for training (70%), validating (15%) and testing (15%) a CNN model which is supposed to predict the mean value of the mentioned random vectors (Y). However, the RMSE of the model becomes too high. In other words, the model is not trained despite changing its options and parameters. I would be grateful if anyone could kindly advise.

Answer 1

[Maksym Tymchenko]

Hi @Mohsen Abyani,

The problem you are trying to solve is very interesting.

It seems that you are creating images that contain MATLAB figures of 10 random points in the range of [0,2] and then you are trying to teach a network to predict the correct mean of the 10 values plotted given the image of the plot as input.

You can certainly accomplish this using a deep learning network. I have looked at your code and everything seems to be set up correctly, the only thing that needed to be changed for the network to train was the architecture itself.

In your original architecture, you are using convolutional layers, but the network is not downsampling the spatial dimensions enough. In fact, the final fully connected layer receives activations of size 328x437x16x1 which results in a weights matrix of size 1x2293376:

This defeats the point of a convolutional architecture whose goal is to gradually downsample the spatial dimension while incresing the channel dimension.

I have created a more suitable architecture for this task by downsampling the activations gradually using max pooling layers:

This architecture results in a much better training behaviour where the loss goes down gradually.

I have chosen some initial training options in the attached script and achieved an RMSE of 0.25 with a 1 minute training loop.

However, I am confident that you can achieve a much better result with some tuning of the training options.

Please let me know if you have any further questions!

[Mohsen Abyani]

@ Maksym Tymchenko

Dear Maksym,

Many thanks for your helpful answer. As you have correctly mentioned, this issue could be solved by downsampling the activations gradually using max pooling layers.

Kind Regards,

Mohsen