Random Forests

A single decision tree overfits. A random forest fixes that by training many trees on slightly different views of the data and averaging them, and the result is one of the most reliable off-the-shelf models for tabular financial data. The idea rests on a precise statistical fact: averaging many noisy, decorrelated estimators slashes variance without raising bias. This lesson explains bagging, the variance formula that makes averaging work, the extra feature randomness that defines a random forest, and the out-of-bag trick that gives you a free validation estimate. We work the variance reduction by hand and tie it back to the bias-variance tradeoff and the interview.