One tree overfits. Two hundred vote it down.

A tree tries every column and every cut point, and keeps the split that makes the two groups purest, by Gini impurity or entropy. Then it repeats inside each group. The result reads like a flowchart, which is why trees are easy to explain to a manager.

A deep tree can carve out a leaf for a handful of customers, so it memorises noise, and a slightly different sample gives a very different tree: a single tree has high variance. Limiting the depth helps (depth 3 scored 0.72 on new customers), and so does averaging many trees.

That’s bagging (bootstrap aggregating) plus random columns: a random forest. Each tree overfits in its own way, and the average smooths the quirks out. An honest note from this table: the pruned depth-3 tree (0.72) and plain logistic regression (0.68) did about as well, because the pattern is simple. Always compare with a simple model before reaching for a complex one.

Gradient boosting (XGBoost, LightGBM, CatBoost) adds shallow trees in sequence, each moving the predictions a small step, set by the learning rate. It usually wins on business tables but needs care: too many trees or too big a learning rate overfits, so the number of trees is chosen by early stopping on validation data.

Now the trap: the monthly charge has no effect on churn at all, because the table’s code makes churn depend only on tenure and complaints. It scores high because this kind of importance favours columns with many different values, which give the trees many chances to split on noise. Permutation importance, measured on test data, is the safer check.