Score a node two ways, weigh a split, then watch the greedy search fail in the sandbox, and predict — before you read it off the impurity and search panels — the Gini and entropy of a stated node, the information gain of a clean split, and why a depth-1 tree cannot separate XOR at all.
A decision tree grows by asking, at each node, which single yes/no cut most purifies the two sides. To do that it needs a number for how mixed a node is: Gini and entropy are two such impurity measures, both zero on a pure node and largest on a 50/50 one. The information gain of a split is the drop in impurity it buys, size-weighted across the children — and because the search keeps only the best single cut at each step, it is greedy, which is exactly why XOR defeats it.
Open the sandbox. The first three steps are hand computations you verify against the impurity and search panels — leave the dataset on blobs for those. The last step sets the dataset to XOR and the depth to 1 to watch the greedy search come up empty. Use the Gini criterion throughout.
Criterion -> Gini Dataset -> blobs (XOR for the last step)
Max depth -> 4 Min samples -> 2 Seed -> 7
The impurity panel shows G = 1 - ∑ p_k² and H = -∑ p_k log₂ p_k for each node; the search panel plots the information gain of every candidate cut and marks the peak the tree chooses.
Take a node holding 3 points of one class and 1 of the other. Gini impurity is 1 - ∑ p_k², and here p = (3/4, 1/4). Predict Gini, then confirm it against the impurity panel.
Gini = 1 - ((3/4)² + (1/4)²) = 1 - (9/16 + 1/16) = 0.375. It is below the 0.5 ceiling of a 50/50 node, because a 3:1 node is already leaning toward one class.
Now score the same 3-to-1 node with entropy, -∑ p_k log₂ p_k — the other impurity measure, in bits. Switch the criterion to entropy and read the impurity panel. Predict the value first, then read it.
entropy = -(3/4)log₂(3/4) - (1/4)log₂(1/4) = 0.8113 bits. Note it is well above the Gini value of 0.375: entropy is more sharply peaked, so the two agree on which split is best far more often than on the raw impurity number.
Consider a 50/50 parent node [4, 4] that a single cut splits perfectly into two pure children, [4, 0] and [0, 4]. Gain is the parent impurity minus the size-weighted child impurity. With Gini the parent is 0.5 and both children are 0. Predict the gain, then read the search panel's peak.
gain = 0.5 - (1/2)(0) - (1/2)(0) = 0.5 — the entire parent impurity is removed, the largest gain possible here. This is the peak the greedy search climbs to and the cut it keeps.
Set the dataset to XOR and Max depth to 1, and watch the search panel. On the textbook four-corner XOR, opposite corners share a class, so no single axis-aligned cut separates anything — every candidate gain is essentially 0. Predict the training accuracy of a depth-1 tree, then read it.
The tree makes no split at all (a single leaf) and predicts the majority, so training accuracy is exactly 0.5 — the coin-flip baseline. The greedy search, looking only one cut ahead, never sees that two cuts would solve it. That is the greedy trap; more depth on a smoother XOR is what escapes it.
Everything above is waiting in the sandbox. Grow the tree one level at a time and watch each node's Gini and entropy fall, sweep the search panel to see which cut wins the most gain, and switch datasets to find where the greedy, axis-aligned search struggles.
- Scored a 3-to-1 node with Gini = 0.375
- Scored the same node with entropy = 0.8113 bits — more sharply peaked than Gini
- Found a clean split's information gain = 0.5, the whole parent impurity removed
- Saw a depth-1 tree stuck at accuracy 0.5 on XOR — the greedy trap
- Every value you predicted is the demo's own impurity and gain arithmetic, not a picture