LSTM for Battery prediction

Hello everyone,

I am currently working on estimating the lifespan of batteries using an LSTM network. The sequence length varies with each cycle. Unfortunately, the results have been very poor so far, as it seems the network is not reading the entire sequence of data correctly.

Below you will find the code for the neural network and a plot comparing the real data with the LSTM predictions.

I would greatly appreciate your support and advice!

Thank you in advance!

% Example data preparation
cycles = 167;

for i = 1:cycles
    % Fill these with your actual data
    voltageData{i} = normal(cha(i).data.Voltage_measured); 
    currentData{i} = normal(cha(i).data.Current_measured);
    timeData{i} = normal(cha(i).data.Time);
    temperatureData{i} = normal(cha(i).data.Temperature_measured);
    %capacities(i) = ...; % Actual capacity value for the i-th cycle
end

for i = 1:cycles
    X{i,:} = [voltageData{i}; currentData{i}; timeData{i}; temperatureData{i}];
end
Y=normal(capa)';

% Split data into training (70%), validation (15%), and test (15%) sets
TrainAmout = round(0.7 * cycles);
ValAmout = round(0.15 * cycles);
TestAmout = cycles - TrainAmout - ValAmout;

Index = randperm(cycles);
trainIndex = Index(1:TrainAmout);
valIndex = Index(TrainAmout+1:TrainAmout+ValAmout);
testIndex = Index(TrainAmout+ValAmout+1:end);

XTrain = X(trainIndex);
YTrain = Y(trainIndex);
XVal   = X(valIndex);
YVal   = Y(valIndex);
XTest  = X(testIndex);
YTest  = Y(testIndex);

inputSize = 4; % Four input features: voltage, current, time, temperature
numHiddenUnits = 50; % Number of hidden units in the LSTM layer
outputSize = 1; % Single output representing capacity

layers = [ ...
    sequenceInputLayer(inputSize)
    lstmLayer(numHiddenUnits, 'OutputMode', 'last')
    fullyConnectedLayer(outputSize)
    regressionLayer];

% Set training options with validation data
options = trainingOptions('adam', ...
    'MaxEpochs', 100, ...
    'MiniBatchSize', 10, ...
    'InitialLearnRate', 0.01, ...
    'GradientThreshold', 1, ...
    'ValidationData', {XVal, YVal}, ...
    'ValidationFrequency', 30, ...
    'Verbose', false, ...
    'Plots', 'training-progress');


% Train the network
net = trainNetwork(XTrain, YTrain, layers, options);

[Umar]

Hi Hussein,

Sorry to hear about your issues. I can make several adjustments to the code and the model architecture. Firstly, it's important to ensure that the data is properly normalized and preprocessed before being fed into the LSTM network. This can help improve the network's ability to learn from the input sequences.Additionally, considering that the sequence length varies with each cycle, we should incorporate this variability into the model. One way to achieve this is by using a sequence-to-sequence architecture, which allows for variable-length input sequences. Moreover, the model architecture itself can be optimized by adjusting parameters such as the number of hidden units in the LSTM layer, the learning rate, and training options. It may also be beneficial to experiment with different network architectures and hyperparameters to find the configuration that best suits your specific dataset.Here's an updated version of your code with some modifications:

% Example data preparation

cycles = 167; for i = 1:cycles % Fill these with your actual data voltageData{i} = normalize(cha(i).data.Voltage_measured); currentData{i} = normalize(cha(i).data.Current_measured); timeData{i} = normalize(cha(i).data.Time); temperatureData{i} = normalize(cha(i).data.Temperature_measured); end

X = cell(cycles, 1); for i = 1:cycles X{i} = [voltageData{i}; currentData{i}; timeData{i}; temperatureData{i}]; end

Y = normalize(capa)';

% Split data into training (70%), validation (15%), and test (15%) sets TrainAmount = round(0.7 * cycles); ValAmount = round(0.15 * cycles); TestAmount = cycles - TrainAmount - ValAmount; Index = randperm(cycles); trainIndex = Index(1:TrainAmount); valIndex = Index(TrainAmount+1:TrainAmount+ValAmount); testIndex = Index(TrainAmount+ValAmount+1:end);

XTrain = X(trainIndex); YTrain = Y(trainIndex); XVal = X(valIndex); YVal = Y(valIndex); XTest = X(testIndex); YTest = Y(testIndex);

inputSize = 4; % Four input features: voltage, current, time, temperature outputSize = 1; % Single output representing capacity

layers = [ sequenceInputLayer(inputSize) lstmLayer(numHiddenUnits, 'OutputMode', 'sequence') fullyConnectedLayer(outputSize) regressionLayer];

% Set training options with validation data options = trainingOptions('adam', ... 'MaxEpochs', 100, ... 'MiniBatchSize', 10, ... 'InitialLearnRate', 0.01, ... 'GradientThreshold', 1, ... 'ValidationData', {XVal, YVal}, ... 'ValidationFrequency', 30, ... 'Verbose', false, ... 'Plots', 'training-progress');

% Train the network net = trainNetwork(XTrain, YTrain, layers, options);

In this modified code snippet, I've made changes to the data normalization process and adjusted the LSTM layer's `OutputMode` to be `'sequence'` instead of `'last'`. This change allows the network to output a sequence of predictions rather than a single prediction. Please note that these modifications are just a starting point. To further improve your model's performance, you may need to experiment with different configurations and hyperparameters based on your specific dataset and problem domain.

I hope these adjustments help improve the performance of your LSTM network for estimating battery lifespan. If you have any further questions or need additional support, please feel free to ask! Good luck with your project!

[hussein al-yasseri]

thank you for the effort you do to help me and thank you for the advice with the lstmLayer(numHiddenUnits, 'OutputMode', 'sequence')

put my output is one single number. Just an example if have 100 cycles each cycle has sequence data. The output of each cycle is one single number, which describe the current capacity.

Cycle 1

X=[423 x 4]---Y=1.8

Cycle 2

X=[723 x 4]---Y=1.7

Cycle 3

X=[463 x 4]---Y=1.6

Cycle 4

X=[429 x 4]---Y=1.5

Cycle 5

X=[394 x 4]---Y=1.4

...

...

...

so on

I tired to combine your advice with my requirements so i made up the architeurkur like this:

layers = [...

sequenceInputLayer(inputSize)

lstmLayer(numHiddenUnits, 'OutputMode', 'sequence')

lstmLayer(numHiddenUnits, 'OutputMode', 'last')

fullyConnectedLayer(outputSize)

regressionLayer];

but still i face the same problem, which can be seen in the in the training process of the neural network:

and the plot looks almost the same, so i dont know where the problem is, is it the different sequence length or something wrong in the code. I thought maybe my learning rate is too high so i made it smaller but it did n't change anything

[Umar]

Hi Hussein,

What you described could be related to the inconsistency in the sequence lengths of the input data. Neural networks typically require input data of uniform length for efficient training. If the sequences vary significantly in length, it can lead to difficulties in learning patterns effectively. You can can preprocess the input data to ensure uniformity. One common approach is padding sequences to a fixed length or using techniques like masking to handle sequences of different lengths effectively. Here is an example of how you can pad sequences using Matlab:

% Assuming X is your input data with varying sequence lengths

maxSequenceLength = max(cellfun(@numel, X)); X_padded = cellfun(@(x) [x; zeros(maxSequenceLength - numel(x), size(x, 2))], X, 'UniformOutput', false);

By padding the sequences to a uniform length, you can provide consistent input data to the neural network, which can help improve training performance. Hopefully, by optimizing the preprocessing of sequence data, you can enhance the training process and improve the overall performance of your neural network model.

[hussein al-yasseri]

thank you very much, tired it with padding, but it did not impact the results, still the same results.

I read that its recommanded to use attention layer, which kind fix the uneven sequence length.

[Umar]

Hi Hussein,

Glad to know your problem is fixed. Good luck! If you still have further questions, please let me know.

[hussein al-yasseri]

no no i mean it did not work out, i just repost the results

[Umar]

Hi Hussein,

Sorry, What did not work out, could you please provide more details.

[hussein al-yasseri]

I try to do padding for the data with this code:

The data has diffrent sequence length, from 780 until 3900. maybe the gab is huge, i dont know

% Example data preparation
cycles = 167;

for i = 1:cycles
    Dsize(i)=length(cha(i).data.Voltage_measured); 
end
DimSize=max(Dsize);

for i = 1:cycles
    voltageData{i} = [normal(cha(i).data.Voltage_measured) zeros(1,(DimSize-length(cha(i).data.Voltage_measured)))]; 
    currentData{i} = [normal(cha(i).data.Current_measured) zeros(1,(DimSize-length(cha(i).data.Voltage_measured)))];
    timeData{i} = [normal(cha(i).data.Time) zeros(1,(DimSize-length(cha(i).data.Voltage_measured)))];
    temperatureData{i} = [normal(cha(i).data.Temperature_measured) zeros(1,(DimSize-length(cha(i).data.Voltage_measured)))];
end

[Umar]

Hi Hussein,

After analyzing your code, you are using 'length(cha(i).data.Voltage_measured)' instead of 'length(voltageData{i})' when calculating the padding for currentData, timeData, and temperatureData and this might lead to inconsistencies in padding based on voltage data length. My suggestion would be modifying the code to ensure consistency in padding across all data types by using 'DimSize-length(voltageData{i})' instead of 'DimSize-length(cha(i).data.Voltage_measured)'. Overall, your approach to padding data for different sequence lengths is valid.

Please let me know if you have any further questions.