Optimization of dimensions of hidden layer in neural network

2 views (last 30 days)
Hello all
I want to optimize the number of neurons in 3 hidden layers which I've used in my neural network. Is there any way (apart from applying 3 nested for loops and checking the test performance for each of them ) so that I can know the optimized dimensions of all the three layers?
My Input vector is (208X200) and target is (5x200).
Please help me!

Accepted Answer

Greg Heath
Greg Heath on 31 May 2014
Edited: Greg Heath on 31 May 2014
There is no a priori way to optimize the number of hidden neurons for 1 hidden layer, much less 3. However, you can get a good estimate for the minimum number of the former via trial and error. Increasing the number of hidden layers tends to reduce the total number of hidden neurons. So, maybe a first step would be to design a single hidden layer model first.
A priori information can help, especially with classification where it is known that each class consists of a number of known subclasses. Then a divide and conquer approach can be followed. I have only used this with elliptical basis functions (most of the time with radial basis functions). A first step in this case could be the clustering of each class into subclasses. I can't say much more without revealing proprietary info.
Both clustering and principal component decompositions help understand the data. Look at those first before determining how to construct a divide and conquer approach.
Also take a look at cascade correlation.
  1 Comment
Greg Heath
Greg Heath on 1 Jun 2014
I just noticed your input dimensions of [ 208 200 ]. If you use the default data division ratios Ntrn = 140.
Do you really expect to get reliable performance when you are trying to define a 208 dimensional space with 140 vectors?
Reduce the input dimensionality and/or get more data.

Sign in to comment.

More Answers (0)

Categories

Find more on Deep Learning Toolbox in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!