Multicollinearity and the Variance Inflation Factor

Question

You regress returns on a set of factor exposures and find that the coefficients are huge, unstable, and even flip sign when you add one more month of data — yet the overall fit looks fine. A colleague suspects multicollinearity.

Explain what collinearity among predictors does to an OLS fit. Derive the Variance Inflation Factor and explain how to interpret its magnitude. Describe how you would detect collinearity, and list the remedies — including why ridge regression helps. Use a trading context of highly correlated factor exposures.

Accepted Answer

Multicollinearity is the situation where two or more predictors are highly linearly related, so the columns of the design matrix $X$ are nearly redundant. This is endemic in factor models: value and quality, or momentum and low-volatility, or two vendors' versions of the "same" signal, can be correlated 0.9 or more. The key subtlety is that collinearity does not bias OLS — it wrecks the precision of the coefficients.

What collinearity doesOLS coefficients remain unbiased (collinearity does not violate exogeneity), and the model's overall fit and predictions can still be good. What explodes is the variance of the individual coefficients. Intuitively, if feature $A$ and feature $B$ almost always move together, the regression cannot tell whether the response is being driven by $A$ or by $B$: it can put a large positive weight on $A$ and a large negative weight on $B$ that nearly cancel, and get almost the same fit as the reverse. The data does not pin down the split, so the estimated coefficients are huge, have enormous standard errors, and swing wildly — even flipping sign — when you add or remove a few observations. That is precisely the symptom in the question: good fit, but unstable, sign-flipping coefficients.

Deriving the Variance Inflation FactorThe variance of the OLS coefficient on feature $j$ can be written as \begin{equation} 	ext{Var}(\hat{\beta}_j) = \frac{\sigma^2}{(n-1)\,	ext{Var}(x_j)} \cdot \frac{1}{1 - R_j^2} \end{equation} where $R_j^2$ is the $R$-squared from the auxiliary regression of feature $j$ on all the other features. The first factor is what the variance would be if feature $j$ were uncorrelated with the rest; the second factor, \begin{equation} VIF_j = \frac{1}{1 - R_j^2} \end{equation} is the Variance Inflation Factor — the multiplicative penalty you pay for $j$ being explainable by the other predictors. The logic: if feature $j$ is nearly a linear combination of the others, the auxiliary regression fits it well, $R_j^2 	o 1$, the denominator $1 - R_j^2 	o 0$, and $VIF_j 	o \infty$. If feature $j$ is orthogonal to the rest, $R_j^2 = 0$ and $VIF_j = 1$ (no inflation). The standard error of $\hat{\beta}_j$ is inflated by $\sqrt{VIF_j}$.

Interpreting the magnitude

  
    $VIF_j$$R_j^2$Reading
  
  
    10No collinearity
    ~2.50.60Mild, usually fine
    50.80Worth a look
    100.90Serious collinearity
  

Common rules of thumb flag $VIF > 5$ as a concern and $VIF > 10$ as severe (standard error inflated more than threefold), though the right threshold depends on how much coefficient precision you actually need.

DetectionThree complementary checks:

 	Correlation matrixeyeball pairwise correlations of the features; large entries flag obvious pairs. But this misses multi-way collinearity (a feature collinear with a combination of several others), which is exactly what VIF catches.
 	VIF per featurerun the auxiliary regression for each feature and compute $1/(1-R_j^2)$.
 	Condition numbercompute the ratio of the largest to smallest singular value of the (standardised) $X$. A condition number above ~30 indicates ill-conditioning; this is the most global diagnostic.

Remedies

 	Drop a featureif two exposures are near-duplicates (two momentum signals correlated 0.95), keep the cleaner one. Simplest and often best.
 	Combine featuresaverage the redundant signals into a single composite factor, or otherwise reduce the group to one representative.
 	PCAreplace the correlated block with its leading principal components, which are orthogonal by construction — at the cost of interpretability of the resulting components.
 	Ridge regressionadd an $L_2$ penalty $\lambda \sum \beta_j^2$. This is the cleanest fix when you want to keep all the features. Ridge solves $\hat{\beta} = (X^TX + \lambda I)^{-1} X^T y$; adding $\lambda I$ makes the matrix well-conditioned again even when $X^TX$ is nearly singular, which is precisely the collinear case. It introduces a little bias but slashes the coefficient variance, and instead of giving two collinear factors a huge $+$ and huge $-$ weight, it spreads a moderate, stable weight across the correlated group.

Trading contextFactor exposures are routinely collinear — sector and beta, value and size, two vendors' momentum signals. If you run a naive OLS of returns on all of them, VIFs blow up, coefficients become unstable, and you cannot trust the sign or size of any single factor's loading. You might conclude "momentum is short" simply because it is offsetting a collinear sibling. The disciplined workflow: standardise exposures, compute VIFs, prune or combine near-duplicate factors, and use ridge (or elastic net) so the remaining correlated exposures get stable, sensible loadings — and validate with walk-forward rather than shuffled folds because the collinearity structure itself drifts over time.

Title	Category	Subcategory	Difficulty	Status
Deriving OLS	Machine Learning	Linear Models	Hard	Unlock
The Gauss-Markov Theorem	Machine Learning	Linear Models	Hard	Unlock

The Interview Guide for Quants and Traders

Multicollinearity and the Variance Inflation Factor

What do you think of this page?