Two common ways to rank feature importance in a tree ensemble are Mean Decrease Impurity (MDI) and Mean Decrease Accuracy (MDA). Explain how each is computed, and give the main weakness of MDI that makes practitioners in finance prefer MDA.

Mean Decrease Impurity (MDI)Every split in a decision tree reduces node impurity (Gini or entropy). MDI credits each feature with the total impurity reduction of all the splits that used it, weighted by the number of samples reaching those nodes, then averages across all trees in the forest. It is computed directly from the fitted trees, so it is essentially free and available the moment training finishes. Mean Decrease Accuracy (MDA)Also called permutation importance. Take a fitted model and a held-out set, record the baseline score (accuracy, AUC, negative log-loss). Then, one feature at a time, randomly permute that feature's column — breaking its relationship with the target while preserving its marginal distribution — and re-score. The drop in performance is the feature's importance. It is model-agnostic and, crucially, measured out-of-sample. The main weakness of MDIMDI is in-sample and structurally biased. Because it is read off the training trees, it rewards features the model used to fit — including noise it overfit — and it is systematically biased toward high-cardinality / continuous features: such features offer many more candidate split points, so they get picked more often and accumulate impurity reduction even when they carry no real signal. MDI also cannot tell you whether a feature helps on unseen data, which is the only thing that matters in trading. MDA sidesteps both issues by working out-of-sample and treating the model as a black box, which is why it is the default in quant research, where the signal-to-noise ratio is tiny and in-sample inflation is fatal. MDA has its own blind spot — the substitution effect with correlated features, where two interchangeable predictors each look unimportant because the other covers for it — but that is addressed by permuting clusters of correlated features together rather than abandoning the method.

Complete Training For Your Trading Interview

Interview Questions Machine Learning MDI versus MDA

Unlock

Sorry, you don't have access to this interview question. Sign up to practice with our wide range of interview questions or use the filters to find free examples!

Python scratchpad

# Run Python right here in your browser
def two_sum(nums, target):
    seen = {}
    for i, n in enumerate(nums):
        if target - n in seen:
            return [seen[target - n], i]
        seen[n] = i
    return []

print(two_sum([2, 7, 11, 15], 9))  # [0, 1]

Run Python in your browser

Write and execute code — numpy, pandas & scikit-learn included — right on the question page.

My Notes

Notes are saved in this browser only. Sign up to keep them in your account.

Title	Category	Subcategory	Difficulty	Status
The Substitution Effect	Machine Learning	Feature Importance	Hard	Unlock
What SHAP Values Actually Compute	Machine Learning	Feature Importance	Hard	Unlock

The Interview Guide for Quants and Traders

MDI versus MDA

What do you think of this page?