Decision Trees

← Back to Classical ML Algorithms

Tree-structured models that make decisions by recursively splitting data based on feature values. Highly interpretable and the building block for powerful ensemble methods.

Key Properties

• Information Gain
• Gini Impurity

How It Works

At each node, select the feature and threshold that best separates the data. Continue splitting until a stopping criterion is met (max depth, min samples, etc.). Pruning reduces overfitting.

Related

• Ensemble Methods (Random Forest, Gradient Boosting use trees)
• Feature Importance (trees naturally rank features)

---

ml decision-trees