Assumption violation (important)

If any assumptions are violated it may lead to wrong coefficient estimates, their standard errors and the distributions that we evaluate the estimators with. Therefore we often try to remedy any violations with different techniques like non-linear variables (explained below), until the assumptions are satisfied.

Below we find a list of assumption violations. In the sections following this one some of the terms and procedures mentioned in the list will be elaborated upon

Instrumental variables (IV)

If an independent variable X is corrolated with the error term u, we try to divide the variable into two parts: one term corrolated with the error and one term uncorrolated.

The technique for finding an instrumental variable starts by udentifying some other variable (the IV) which is correlated to the endogenious variable, but is uncorrelated with unobserved effects on the dependent variable (the error term). Then we use Two-stage least squares method (2SLS).

The first stage involves predicting the endogenious variable with the new IV with a linear regression. We can generate predictions for the endogenious variablem where the unobserved effects on the dependent variable is removed. So we use this in the final regression model instead of the endogenious variable. Mathematically it looks like this:

If we have a regression model,

$$Y_i = \beta_0 + \beta_1 X_{1,i} + \beta_2 X_{2,i} + u_i$$

but $X_1$ is endogenious, then we find some other variable, Q, which has $Cov(X_1,Q)\neq 0$, and $Cov(Q, u_i) = 0$. Then we model the endogenious variable $X_1$ with a regression model like

$$X_{1,i} = \lambda_0 + \lambda_1 Q_i + v_i$$

where all of the unobserved effects of $X_1$ "are" in $v_i$. So if we predict $X_i$ with $\hat X_{1,i} = \lambda_0 + \lambda_1 Q_i$, then we can reformulate a new regression model,

$$Y_i = \beta_0 + \beta_1 + \hat X_{1,i} + \beta_2 X_{2,i} + u_i$$

Now $\hat X_{1,i}$ is no longer endogenious by applying an instrumental variable.

Sample selection and Heckman correction

Sample selection occurs when the observations are not made randomly and a specific subset of the population is not in the sample. Heckman correction attempts to remidy this. It is done in two steps:

The first step consists of estimating a selection model to predict the probability of an observation being included in a random sample. This is called a limited dependent variable regression.

The second step involves estimating the main model with a correction variable: The Inverse Mills Variable, which we found in the first model. The purpose of the IMV is to correct for the selection.

Difference-in-Difference (DiD)

Difference-in-Diffrence tries to estimate the causal relationship by comparing a group with a property (experiment group) and a group without a property (Control group).

PCA (Principle component analysis)

Principal component analysis in the context of regression takes the explanatory variables as vectors and finds a new basis of orthogonal vectors. These new vectors can be used as explanatory variables, and they are independent variables that explain variance in the dependent variable. These variables can also be ranked in order of "importance" to the dependent variable, so we can pick the k most important vectors and use them in the regression instead of using them all.

Mathematically PCA is done by finding the eigenvectors of the $X^T\ X$ matrix. The eigenvectors are the new orthogonal explanatory variables, and the effect on the dependent variable is measured by the corresponding eigenvalues.

The t-ratio and t-test

The t-ratio is a test statistic for hypothesis testing. It is defined by the error of a parameter divided by the standard error,

$$t = \frac{\hat \beta - \beta}{SE(\hat \beta)},$$

This is a t-distribution so we follow the t-distribution procedure for hypothesis testing.

The $\beta$ will be indexed in most linear regressions because each estimator will have its own t-ratio, like $t_j = \frac{\hat \beta_j - \beta_j}{SE(\hat \beta_j)}$. Then we can estiate significance for each estimate.

Also note the F-statistic which is not discussed in this text yet. It is used to test regressions under restrictions.

Goodness of fit

We need a statistic that determines how well a model fits the data (goodness of fit). This is the purpose of the $R^2$ statistic. It is the square of the correlation between $y$ and $\hat y$, and it measures how much of the squared error can be explained by the model.

It is the ratio of the explained sum of squares (ESS) and the residual sum of squares (RSS),

$$R^2 = \frac{ESS}{RSS} = \frac{\sum_i (\hat y - \bar y)^2}{\sum_i (y_i - \hat y_i)^2}$$

The purposes of the t-ratio and the R^2 statistic are different. The former measures relevancy of regressor, and the latter measures goodness of fit.

Hypothesis testing by maximum likelihood methods

For now we only concern ourselves with the distributions of the following three statistics as the distribution is important for interpretation (relevant for exam).

• Wald

Let q be the number of restrictions (number of parameters being estiamted), T be the sample size and k the number of parameters. Then the Wald statistic is in epproximately an F-distribution F(m, T-k)

• Lagrange multiplier (LM)

The distribution for the LM statistic is a $\chi_q^2$-distribution.

Generally, the finite Wald statistic is better for small sample sizes, as it accounts for T, the sample size.

Autocorrelation

We've already discussed autocorrelation, but for the following material it is essential to be very familiar with the term. When autocorrelation is present there is a constant correlation between the value of a time series at some time $t_i$ and the value at some later time $t_i + k$ with respect to k. In other words, the correlation is the same for all $t_i$ given that k is the same. There is one autocorelation in a time series for all $k$. Mathematically autocorrelation is defined as,

$$\tau_k = \frac{\mathrm{Cov}(r_t, r_{t-k})}{\sqrt{\mathrm{Var}(r_t)\mathrm{Var}(r_{t-k})}} = \frac{\mathbb{E}[(r_t - \mu)(r_{t-k} - \mu)]}{\sigma^2}$$

The function mapping the integers to the autocovariances of some time series for each integer is the autocovariance function (acf), or mathematically $s \mapsto \gamma_s =\ E\left[(y_t - E[y_t])(y_{t-s}-E[y_{t-s}])\right]\;\; \forall s\in\mathbb{N}$. It is often more useful to use autocorrelations because then the units are normalized to 1. Convenienty the autocorelation is found easily from the autocovariances by, $\tau_k = \frac{\gamma_k}{\gamma_0}$

We also have another autocorrelation function than the typical ACF walled the Partial Autcorrolative Function (PACF). This function messures the correlation between an observation made at some arbitrary time and one made $l$ periods ago without the effect of intermediate lags. So if there is some autocorrelation in lags $a$ and $b$ for some process, the PACF will not include whatever effect this may have on lags $a+b$ in the function. The PACF functions will often have very complicated formulas.

Stationarity

Stationarity generally means that some time-series has no trend appart from seasonality. It has constant variance and constant mean. Mathetatically we have two definitions of stationarity. A weak and a strong definition.

A time series is a strictly (strongly) stationary time seires if,

$$P(y_{t_1} \leq b_1,... ,y_{t_n} \leq b_n) = P(y_{t_1+\lambda} \leq b_1,... ,y_{t_n+\lambda} \leq b_n)$$

i.e. if the probability distribution of a time series is the same with respect to lag.

A process is a weakly stationary time series if all the conditions are satisfied

• $E[y_t] = \mu$ (constant mean)
• $E[(y_t-\mu)(y_t-\mu)] = \sigma^2 < \infty$ (constant variance)
• $E[(y_{t_1}-\mu)(y_{t_2} - \mu)] = \gamma_{t_2-t_1}$ (autocovariance)

EditStationarity example: White noise

White noise are examples of stationary processes in that they have constant mean, constant variace and no non-trivial autocorrelation (meaning that $\tau_0 = 1$). We test for white noise by assuming that the autocorrelation is normally distributed with variance $\sigma^2=\ 1/T$ so that we can do a significance test.

EditNon-stationarity example: Random walk

Random walks are more discussed in this course. They are defined by $y_t = y_{t-1} + u_t$, where u_t is white noise. They are not strictly stationary as the probability distribution of the time series is not constant for any lag. It changes depending on the white noise over many lags. Though, the acf of a random walk is strictly decreasing, so it is more predictable than white noise in many ways.

Seasonality

Seasonality is the property where some value is predicatbly fluctuating within some timeframe.

There are many ways of correcting for seasonality. Seasonal differencing is the most relevant preprocessing technique for this course. If a time series has a lag k seasonality we difference two intervals of size k to find the corrected time series. In other words $Y'_t = Y_t - Y_{t-k}$ If a time series is sampled daily and we have yearly seasonality we will have $k=365$.

Other methods of correcting for seasonality involve different types of preprocessing, feature engineering, machine learning, or using seasonal models like SARIMA.

Characteristics

The folliwing characteristics of financial returns are discussed numerous times in this course:

• Non-stationarity of prices, but stationarity of returns.
• No autocorrelations of returns
• There is autocorrelation of squared returns.
• Volatility clustering
• Fat-tailed distributions (non-normal with large kutosis)
• Leverage effects

We would also like to define the notion of unit roots. If a non-stationary process $y_t$ can be differenced $d$ times to become a stationary process then $y_t \sim I(d)$ (the process has d unit roots). Most financial time series have 1 unit root.

Moving averages models (MA)

A q-th order moving averages model is defined by,

$$y_t = \mu + u_t + \theta_1 \ u_{t-1} + \theta_2 \ u_{t-2} +\dots + \theta_q \ u_{t-q}$$

where $u_i$ are i.i.d random variables with $E[u_t]=0, \ Var(u_t)=\sigma^2$.

Now we study some properties of MA models. They all have expected value $E[y_t] = E[\mu]+ E[u_t + \theta_1 \ u_{t-1} + \theta_2 \ u_{t-2} +\dots + \theta_q \ u_{t-q}] = \mu$. They have variance, $Var(y_t)=\left(1+ \theta_1^2 + \dots + \theta_q^2\right)\ \sigma^2$.

The autocovariances can be calculated as,

$$\gamma_s = (\theta_s + \theta_{s+1}\theta_1 + \theta_{s+1}\theta_2 + \dots + \theta_q \theta_{q-s})\sigma^2$$

for $s<q$, and otherwise the autocorrelation will be 0.

Autoregressive models (AR)

An autoregressive model of order p is defined as,

$$y_t = \mu + \phi_1 y_{t-1} + \phi_2 y_{t-2} + \dots + \phi_p y_{t-p} + u_t$$

Note on notation: Lag operator

Sometimes you might see used the lag operator. It is defined as $L^i y_t = y_{t-i}$. We may also define some function, $\phi$ where,

$\phi(L) = 1-(\phi_1L^1+\dots+\phi_pL^p)$

Then, $\phi(L)y_t = \mu+u_t$ ecnapsulates the AR model as defined above.

ARMA

An ARMA process is a combination between an AR- and an MA process. Mathematically an ARMA(p,q) process is,

$$y_t = \mu + u_t + \phi_1 y_{t-1} + \dots + \phi_p y_{t-p} + \theta_1 u_{t-1} + \dots + \theta_q u_{t-q}$$

or if they are expressed with lag operators,

$$\phi(L)y_t = \mu + \theta(L) u_t$$

The stationary condition of the ARMA model requires that all roots of the characteristic polynomial have roots with absolute value larger than one, as explained above.

Note also that for lags larger than q, the MA order, the model will behave like an AR model when discussing autocorrelations.

ARMA variants

There are many different variants of ARMA models. One of them is called ARIMA, where the added I stands for integrated. An integrated autoregressive model has at least one root (of the characteristic polynomial) inside the unit circle. Therefore we introduce the ARIMA model to difference the data d times to make it satisfy the stationary condition. So, an ARIMA(p,d,q) model is equivalent to an ARMA(p,q) model where the original data has been differences d times.

Forecasting

ARCH and GARCH models

HAR models

To define a HAR model we first need to define realized volatility.

For some time t, there is a consistent estimator for the true latent variance,

$$RV_t^2 = \sum_{t=1}{M} r_{t,i}^2$$

where, $M = 1/\Delta$ and the $\Delta$-period intraday return is,

$$r_{t,i} = \log(P_{t-1+i\times\Delta}) - \log(P_{t-1 + (i-1)\times\Delta}) \ \ \ (\text{i.e. log-returns})$$

The HAR model is then a regression model,

$$RV_t = \beta_0 + \beta_D \ RV_{t-1} + \beta_W \ RV_{t-1,t-5} + \beta_M \ RV_{t-1,t-22}$$

where $RV_{t-1,t-a}$ is the average intraperiod realized volatility between the two times $t-1$ and $t-a$.

MIDAS models

MIDAS is short of Mixed Data Sampling. It is a regression method that may use data which has been sampled at different frequencies. The model is defined as,

$$y_t = X_t'\beta + f\left( {X_{t/S}^H}, \theta, \lambda \right) + \epsilon_t$$

Value-at-Risk (VaR)

The value at risk at level $p\in(0,1)$ of a portfolio X at time 1 is defined as,

$$VaR_p(X) = \min{m : P(m R_0 + X < 0) \leq p}$$

where $R_0$ is the risk free rate. So the value at risk is the capital needed to be invested into risk free assets so that the probability of losing money (discounted) is less than $p$. In other words, it uses quantiles to determine risk.

Note that VaR is not subadditive and is therefore not coherent. Another disadvantage og VaR is that the user has to mute it to time horizon and confidence level. It also does not describe losses larger than the VaR, as it does not describe the tailedness of the distribution.

Expected shortfall tries to remidy this.

Expected Shortfall (ES)

Expected shortfall tries to remidy the fact that VaR metric does not describe losses larger than the VaR. Expected shortfall is therefore the expected value of the loss, given that it is larger than the VaR, defined as

$$ES_p(X) = \frac 1 p \int_0^p VaR_u (X) du$$

Note that ES is a coherent risk measure because it rewards diversification.

Goldfeld-Quandt (GQ) test

The GQ test is a test for heteroscadistity. It is useful for testing the homoscedasticity assumption in regression analysis.

It splits the samplpe length into two subsamples of length T1 and T2. Then we do a regression on both samples and the residual variances are calculated. The null hypotehsis is that the variances are equal.

We have the test statistic

$$GQ = \frac{S_1^2}{S_2^2} \sim F(T_1 - k, T_2 - k)$$

White's test

White's test tries to model the residuals with an auxiliary regression with the same regressors as the original equation. Then we can run significance tests with F-test or $R^2$ test on the regression to find whether it is dependent on the regressors.

ARCH test

The ARCH test detects heteroscedasticity in a regression by estimating an ARCH(q) model on the residuals of the original regression. Then we run an LM test with the residual test statistic

$$LM = T \ R^2 \sim \chi_q^2$$

and the null hypothesis stating that $\gamma_0 = \gamma_1 = \dots = \gamma_q$, i.e. no autocorrelation.

TODO: Write about HQIC.