OLS and ReML AR model

MarsBaR FAQ:

If you do ordinary least squares (OLS) analyses at the single-subject level, these two approaches will give you the exact same answer. OLS is the analysis that does not try to correct for auto-correlation in the data. OLS at the single subject level is valid (is not biased), but is likely to be less powerful than the alternative (which is removing the autocorrelation - “whitening”). In practice the difference between using and not using OLS is often small.

Dale AM (1990). HBM Optimal experimental design for event-related fMRI

If we assume that the noise process is zero mean and Gaussian with covariance matrix Cn, an efficient unbiased estimate for the HDRs is provided by the maximum likelihood (ML) solution hˆ ML given by

A promising method for obtaining accurate parameterized estimates of Cn is by fitting the residual errors given by ey=y-Xhˆ OLS, where hˆ OLS =(X^TX)^-1X^Ty is the ordinary least-squares estimate of the HDR, as a first-order autoregressive process plus white noise [Burock, 1998; also Burock and Dale submitted]. This noise estimate can then be used to obtain more efficient HDR estimates and to obtain more accurate p-values for statistical parametric mapping.

Autoregression Model

An autoregression model is a linear regression model that uses lagged variables as input variables.

We could calculate the linear regression model manually using the LinearRegession class in scikit-learn and manually specify the lag input variables to use.

Alternately, the statsmodels library provides an autoregression model that automatically selects an appropriate lag value using statistical tests and trains a linear regression model. It is provided in theAR class.

We can use this model by first creating the model AR() and then calling fit() to train it on our dataset. This returns anARResult object.

Once fit, we can use the model to make a prediction by calling the predict() function for a number of observations in the future. This creates 1 7-day forecast, which is different from the persistence example above.