Time Series Ahead Prediction in Neural Network, Large Scale Iterative Training

2 views (last 30 days)
I am having trouble in implementing neural network to predict N points ahead. My only feature is previous time.
In my scenario I need to predict 90 points ahead. First how I separated my input and target data manually: For Example:
data_in = [1,2,3,4,5,6,7,8,9,10]; //imagine 1:10 only defines the array index values.
N = 90; %predicted second ahead.
P(:, :) T(:) it could also be(2 theta time) P(:, :) T(:)
[1,2,3,4,5] [5+N] | [1,3,5,7,9] [9+N]
[2,3,4,5,6] [6+N] | [2,4,6,8,10] [10+N]
...
until it reaches to end of the data
For the case I have used newff. input_layer_size = 60;
NodeNum1 =60;
NodeNum2 = 40;
TypeNum = 5;
TF1 = 'tansig'; TF2 = 'tansig'; TF3 = 'tansig';
net = newff(trainSamples.P,trainSamples.T,[NodeNum1,NodeNum2,TypeNum],{TF1 TF2 TF3},'traingdx');
net.trainfcn = 'traingdm' ;
net.trainParam.time = 600;
net.trainParam.mc = 0.95;
net.trainParam.epochs = 2000;
net.trainParam.min_grad = 0;
net.trainParam.max_fail = 1000; %large value for infinity
net.trainParam.lr = 0.1; %default value
//At the beginning of my training I filter it with kalman, normalization into range of [0,1] and after that I shuffled the data. 1) I won't able to train my complete data. First I tried to train complete M data which is around 900,000, which didn't gave me a solution.
2) Secondly I tried iteratively training. But in each iteration the new added data is merged with already trained data. After 20,000 trained data the accuracy start to decreases. First trained 1000 data perfectly fits in training. But after when I start iterativelt merge the new data and continue to training, the training accuracy drops very rapidly 90 to 20. For example.
P = P_test(1:1000) T = T_test(1:1000) counter = 1;
while(1)
net = train(net,P,T, [], [] );%until it reaches to minimum error I train it.
[normTrainOutput] = sim(net,P, [], [] );
P = [ P P(counter*1000:counter*2000)]%iteratively new training portion of the data added.
counter = counter + 1; end
This approach is very slow and after a point it won't give any good resuts.
My third approach was iteratively training; It was similar to previous training but in each iteration, I do only train the 1000 portion of the data, without do any merging with previous trained data.For example when I train first 1000 data until it gets to minimum error which has >95% accuracy. After it has been trained, when I have done the same for the second 1000 portion of the data;it overwrites the weight and the predictor mainly behave as the latest train portion of the data.
> P = P_test(1:1000) T = T_test(1:1000) counter = 1;
while(1)
> net = train(net,P,T, [], [] ); % I did also use adapt()
> [normTrainOutput] = sim(net,P, [], [] );
>
> P = [ P(counter*1000:counter*2000)]%iteratively only 1000 portion of the data is added.
> counter = counter + 1;
end
I wasn't able to successes any of this approaches. In each iteration I also observe that slight change on the alpha completely overwrites to already trained data and more focus onto the currently trained data portion. I won't able to come up with a solution to this problem. In iterative training should I keep the learning rate small and number of epochs as small.
And I couldn't find an efficient way to predict 90 points ahead in time series. Any suggestions that what should I do to do in order to predict N points ahead, any tutorial or link for information.
What is the best way for iterative training? On my second approach when I reach 15 000 of trained data, training size starts suddenly to drop. Iteratively should I change the alpha on run time?
Any suggestion or the things I am doing wrong would be very appreciated.
I also implemented recurrent neural network. But on training for large data I have faced with the same problems.

Accepted Answer

Greg Heath
Greg Heath on 28 Jan 2014
1. You can only predict as far as the target has significant autocorrelation lags and/or the input and target have significant cross-correlation lags.
2. Use the timeseries function narxnet (or it's predecessor) instead of a static function like newff.
3. If lags are not significant over the length of the data set, you many have to design different nets for different intervals of time.
4. I have posted many examples which may be useful. Search in the NEWSGROUP and ANSWERS using
greg narxnet
Hope this helps.
Thank you for formally accepting my answer
  7 Comments
Greg Heath
Greg Heath on 28 Mar 2014
Since you only have one series, use NARNET
FD = 1:5;
net = narnet(FD,H);
help narnet
doc narnet
Use a double-loop to find good values for H and initial weights.
rng(0)
for j = Hmin:dH:Hmax
...
for i = 1:Ntrials
Search for examples
greg narnet Ntrials
Avatar@ Wonderland
Avatar@ Wonderland on 30 Mar 2014
Edited: Avatar@ Wonderland on 31 Mar 2014
> Does Ntrials means that prediction for that point ahead? For each input, my output have to be N points ahead. How can I achieve it?
> On default narnet & narxnet make their prediction for 1 point ahead. for example:
P(:, :) T(:)
[1,2,3,4,5] [6] //output is 6th value
*For may case I have to predict (N=90) points ahead.*
P(:, :) T(:)
[1,2,3,4,5] [95] //output is 95th value
>Do you have any example related to this problem, N point ahead prediction by using narnet?
> On your one of example in this link: http://www.mathworks.com/matlabcentral/answers/77142-future-value-prediction-with-neural-network-method-and-right-input-and-target-format-data Ntrials = 10; so It focuses on to predict 1 point ahead prediction. What kind of change should I make to prediction (N=90) points ahead prediction?
> Double loop take exponentially longer time on larger data sets, when I try 10,000 or larger data points.
Your help and guidence will be very valuable.
Kindly Regards for your time and help.

Sign in to comment.

More Answers (1)

Greg Heath
Greg Heath on 11 Apr 2014
1. Use narnet with 1 hidden layer containing H hidden nodes. 2. Use nncorr to determine which autocorrelation feedback delays should be in the feedback delay input FD.
net = narnet(FD,H);
3. Use the equations for No. of training equations, Ntrneq, and number of unknown weights, Nw, to determine a reasonable range of values of H to try (Hmin:dH:Hmax). It is desirable that Ntrneq >> Nw.
4.For each combination of(FD,H), Ntrials = 10 different designs are obtained because the gradient search for the minimum of mse begins at random intial weights. Typically, some of the random weights times do not lead to an acceptable solution. Therefore, for a single FD and numH = 10 different values of H, you will have 100 designs to choose from.
5. That is, typically, how I design:
Search in The NEWSGROUP and ANSWERS for details using
greg narnet Ntrials.
Use the validation performance to rank designs and the corresponding test performance is your answer. Therefore choose your train/val/test ratio so that Ntest = 90.
  3 Comments
Greg Heath
Greg Heath on 12 Apr 2014
The narnet example default is FD = 1:2.
If you use the autocorrelation function you can find the lags associated with autocorrelations that are significant at, say, the 95% confidence level. If the maximum of these is << 90, then the chances of fulfilling your goal are significantly reduced. On the other hand, using a closed loop design you can use previous predictions to predict further.
However, because of error propagation, there is a limit to the length of time that predictions are good.
So, first order of business is to obtain the autocorrelation function and either determine the lags at which autocorrelations are significant or determine the significance level at lag = 90.
Avatar@ Wonderland
Avatar@ Wonderland on 15 Apr 2014
Edited: Avatar@ Wonderland on 15 Apr 2014
Do you have any example source code related to use autocorrelation lags and/or the input and target have significant cross-correlation lags., in order to solve this problem?
How could I combine narnet and autocorrelation function to predict N=90 points ahead?/

Sign in to comment.

Categories

Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!