aic

Akaike’s Information Criterion for estimated model

Syntax

value = aic(model)

value = aic(model1,...,modeln)

value = aic(___,measure)

Description

value = aic(model) returns the normalized Akaike's Information Criterion (AIC) value for the estimated model.

value = aic(model1,...,modeln) returns the normalized AIC values for multiple estimated models.

value = aic(___,measure) specifies the type of AIC.

Examples

Compute Normalized Akaike's Information Criterion of Estimated Model

Estimate a transfer function model.

load iddata1 z1;
np = 2;
sys = tfest(z1,np);

Compute the normalized Akaike's Information Criterion value.

value = aic(sys)

value = 0.5453

The value is also computed during model estimation. Alternatively, use the Report property of the model to access this value.

sys.Report.Fit.nAIC

ans = 0.5453

Compute Akaike's Information Criterion Metrics of Estimated Model

Estimate a transfer function model.

load iddata1 z1;
np = 2;
sys = tfest(z1,np);

Compute the normalized Akaike's Information Criterion (AIC) value. This syntax is equivalent to aic_raw = aic(sys).

aic_raw = aic(sys,'nAIC')

aic_raw = 0.5453

Compute the raw AIC value.

aic_raw = aic(sys,'aic')

aic_raw = 1.0150e+03

Compute the sample-size corrected AIC value.

aic_c = aic(sys,'AICc')

aic_c = 1.0153e+03

Compute the Bayesian Information Criteria (BIC) value.

bic = aic(sys,'BIC')

bic = 1.0372e+03

These values are also computed during model estimation. Alternatively, use the Report.Fit property of the model to access these values.

sys.Report.Fit

ans = struct with fields:
    FitPercent: 70.7720
       LossFcn: 1.6575
           MSE: 1.6575
           FPE: 1.7252
           AIC: 1.0150e+03
          AICc: 1.0153e+03
          nAIC: 0.5453
           BIC: 1.0372e+03

Pick Model with Optimal Tradeoff Between Accuracy and Complexity Using AICc Criterion

Estimate multiple Output-Error (OE) models and use the small sample-size corrected Akaike's Information Criterion (AICc) value to pick the one with optimal tradeoff between accuracy and complexity.

Load the estimation data.

load iddata2

Specify model orders varying in 1:4 range.

nf = 1:4;
nb = 1:4;
nk = 0:4;

Estimate OE models with all possible combinations of chosen order ranges.

NN = struc(nf,nb,nk); 
models = cell(size(NN,1),1);
for ct = 1:size(NN,1)
   models{ct} = oe(z2, NN(ct,:));
end

Compute the small sample-size corrected AIC values for the models, and return the smallest value.

V = aic(models{:},'AICc');
[Vmin,I] = min(V);

Return the optimal model that has the smallest AICc value.

models{I}

ans =
Discrete-time OE model: y(t) = [B(z)/F(z)]u(t) + e(t)
  B(z) = 1.067 z^-2                                  
                                                     
  F(z) = 1 - 1.824 z^-1 + 1.195 z^-2 - 0.2307 z^-3   
                                                     
Sample time: 0.1 seconds
  
Parameterization:
   Polynomial orders:   nb=1   nf=3   nk=2
   Number of free coefficients: 4
   Use "polydata", "getpvec", "getcov" for parameters and their uncertainties.

Status:                                     
Estimated using OE on time domain data "z2".
Fit to estimation data: 86.53%              
FPE: 0.9809, MSE: 0.9615                    

Input Arguments

model — Identified model

idtf | idgrey | idpoly | idproc | idss | idnlarx, | idnlhw | idnlgrey

Identified model, specified as one of the following model objects:

• idtf

• idgrey

• idpoly

• idproc

• idss

• idnlarx, except nonlinear ARX model that includes a binary-tree or neural network nonlinearity estimator

• idnlhw

• idnlgrey

measure — Type of AIC

'nAIC' (default) | 'aic' | 'AICc' | 'BIC'

Type of AIC, specified as one of the following values:

• 'nAIC' — Normalized AIC

• 'aic' — Raw AIC

• 'AICc' — Small sample-size corrected AIC

• 'BIC' — Bayesian Information Criteria

See Akaike's Information Criterion (AIC) for more information.

Output Arguments

value — Value of quality metric

scalar | vector

Value of the quality measure, returned as a scalar or vector. For multiple models, value is a row vector where value(k) corresponds to the kth estimated model modelk.

More About

Akaike's Information Criterion (AIC)

Akaike's Information Criterion (AIC) provides a measure of model quality obtained by simulating the situation where the model is tested on a different data set. After computing several different models, you can compare them using this criterion. According to Akaike's theory, the most accurate model has the smallest AIC. If you use the same data set for both model estimation and validation, the fit always improves as you increase the model order and, therefore, the flexibility of the model structure.

Akaike's Information Criterion (AIC) includes the following quality metrics:

• Raw AIC, defined as:

  $$AIC=N*\log \left(\det \left(\frac{1}{N}{\displaystyle \sum _1^N\epsilon \left(t,{\widehat{\theta }}_N\right){\left(\epsilon \left(t,{\widehat{\theta }}_N\right)\right)}^T}\right)\right)+2n_p+N*\left(n_y*\left(\log (2\pi )+1\right)\right)$$

  where:

  • N is the number of values in the estimation data set

  • ε(t) is a n_y-by-1 vector of prediction errors

  • $${\theta }_N$$ represents the estimated parameters

  • n_p is the number of estimated parameters

  • n_y is the number of model outputs

• Small sample-size corrected AIC, defined as:

  $$AICc=AIC+2n_p*\frac{n_p+1}{N-n_p-1}$$

• Normalized AIC, defined as:

  $$nAIC=\log \left(\det \left(\frac{1}{N}{\displaystyle \sum _1^N\epsilon \left(t,{\widehat{\theta }}_N\right){\left(\epsilon \left(t,{\widehat{\theta }}_N\right)\right)}^T}\right)\right)+\frac{2n_p}{N}$$

• Bayesian Information Criteria, defined as:

  $$BIC=N*\log \left(\det \left(\frac{1}{N}{\displaystyle \sum _1^N\epsilon \left(t,{\widehat{\theta }}_N\right){\left(\epsilon \left(t,{\widehat{\theta }}_N\right)\right)}^T}\right)\right)+N*(n_y*\log (2\pi )+1)+n_p*\log (N)$$

Tips

• The software computes and stores all types of Akaike's Information Criterion metrics during model estimation. If you want to access these values, see the Report.Fit property of the model.

References

[1] Ljung, L. System Identification: Theory for the User, Upper Saddle River, NJ, Prentice-Hall PTR, 1999. See sections about the statistical framework for parameter estimation and maximum likelihood method and comparing model structures.