Why "trainNetwork" function works with GoogLeNet but not with other pretrained networks. Why the other pretrained networks need the "trainnet" function

When i try to train a modified pretrained GoogLeNet for binary classification, with the trainnet function as bellow:

[...] =trainnet(resizeTrainImgs,lgraph_1,"crossentropy",opts);

It does not work, It displays the following message:

"Error using trainnet (line 16)

Network must be a dlnetwork or an array of layers."

Answer 1

[Cris LaPierre]

It sounds like you are asking what is the difference between trainnet and trainnetwork. This answer might be helpful: https://www.mathworks.com/matlabcentral/answers/2060029-difference-between-trainnet-and-trainnetwork

A related question might be what is the difference between a dlnetwork and a DAGNetwork.

We would have to see the layers of your network to say for certain, but a big difference is that DAGNetworks returns a prediction while a dlnetwork returns scores. This is why different functions are needed to train them.

For example, the last layer in this example DAG network classifies the image

layers = [
    imageInputLayer([28 28 1])
    
    convolution2dLayer(5,16,'Padding','same')
    batchNormalizationLayer
    reluLayer('Name','relu_1')
    
    convolution2dLayer(3,32,'Padding','same','Stride',2)
    batchNormalizationLayer
    reluLayer
    convolution2dLayer(3,32,'Padding','same')
    batchNormalizationLayer
    reluLayer
    
    additionLayer(2,'Name','add')
    
    averagePooling2dLayer(2,'Stride',2)
    fullyConnectedLayer(10)
    softmaxLayer
    classificationLayer];

while in this dlnetwork example, the last layer returns the scores. The actual predictions are made using minibatchpredict.

layers = [
    imageInputLayer([28 28 1],Normalization="none")

    convolution2dLayer(5,16,Padding="same")
    batchNormalizationLayer
    reluLayer(Name="relu_1")

    convolution2dLayer(3,32,Padding="same",Stride=2)
    batchNormalizationLayer
    reluLayer
    convolution2dLayer(3,32,Padding="same")
    batchNormalizationLayer
    reluLayer

    additionLayer(2,Name="add")

    fullyConnectedLayer(numClasses)
    softmaxLayer(Name="softmax")];

Perhaps the flowchart on this doc page is helpful.

[Aristotle]

Dear Cris LaPierre thank you very much for your answer and the flowchart.

I would like to make my question more specific:

Say we have a GoogLeNet experiment with:

1] Dataset of about 500 Control Images and 500 Case Images.

2] “trainingOptions” with “sgdm”, Learning Rate LR, MaxEpochs ME.

3] “trainNetwork” as training function.

4] “classify” as a classification function.

The result is about 80% accuracy.

Then we have an Inception-v3 experiment with:

1] the same Dataset

2] the same “trainingOptions”

3] “trainnet” as training function with “crossentropy” as Loss Function.

4] “minibatchpredict” as a classification function.

The result is about 50% accuracy.

My question is:

How is it possible for Inception-v3 to give this much lower accuracy result, since it is well known that Inception-v3 is an enhanced, larger, and more accurate version of GoogLeNet? (Both networks are DAG nets)

[Cris LaPierre]

trainnet does not work with DAG Networks, so inception-v3 must be a dlnetwork.

We don't have enough information to answer a question about why your results are different. I would suggest updating your trainnet code to select a network using imagePretrainedNetwork. You can then specify the name of the pretrainined network to be either "googlenet" or "inception-v3". For example,

net = imagePretrainedNetwork("inceptionv3")

There are also several other networks to choose from (ref).

This will allow you to run the exact same code with just the network changed.

[Aristotle]

Dear Cris LaPierre thank you for your prompt answer and suggetion.

I would like to ask:

1] what is the mathematical and data structure difference between a “LayerGraph” object that is produced by the older version (v1) of the Deep Learner Designer and a “dlnetwork” object that is produced by the later version of Deep Learner Designer. Why MATLAB decided this change?

2] whic Loss Function is used by the training function “trainNetwork”?

[Cris LaPierre]

For 1, see my original answer above. Perhaps this answer is also helpful. The notable difference is what the last layer is.

For 2, it depends on the type of model you are training. Perhaps this 2023b doc page is helpful. I got there from here.