Bias-Variance Tradeoff
← Back to Model Evaluation
The fundamental tension in ML between underfitting (high bias, too simple) and overfitting (high variance, too complex). Finding the right balance is central to model selection and tuning.
Key Properties
- High Bias — model too simple, underfits training data
- High Variance — model too complex, memorizes training data
- Total error = bias^2 + variance + irreducible noise
Related
- Regularization (controls model complexity)
- Ensemble Methods (reduce variance via averaging)
- Train-Validation-Test Split (diagnose via learning curves)