Overfitting
Definition
When a machine learning model learns the training data too well, including its noise and quirks, making it perform poorly on new data.
Use Cases
- Zillow: Home price prediction (Zestimate) using machine learning models trained on historical housing and listing data. — Zillow has publicly described using machine learning for valuation and iterating on models with offline evaluation and backtesting to improve generalization across markets and time periods (a common way to detect and mitigate overfitting). (Improved valuation accuracy over time through model updates and evaluation practices designed to generalize beyond the training data.)
- Netflix: Personalized recommendations to predict what a member is likely to watch next. — Netflix has publicly discussed large-scale recommendation modeling and the use of rigorous offline evaluation plus online A/B testing to ensure models generalize to real user behavior (helping catch overfitting that looks good offline but fails in production). (Better personalization and engagement by validating models on unseen data and real-world experiments rather than relying only on training performance.)
- Google: Image recognition and classification models used across products (e.g., Photos and vision research). — Google research has widely documented techniques such as data augmentation, regularization, and validation on held-out datasets to reduce overfitting in deep learning vision models. (More robust model performance on new images and conditions by reducing reliance on training-set quirks.)
Frequently Asked Questions
- What's the difference between overfitting and underfitting?
- Overfitting happens when a model learns the training data too specifically (including noise), so it performs well on training data but poorly on new data. Underfitting is the opposite: the model is too simple or not trained enough, so it performs poorly on both training and new data.
- When should I worry about overfitting?
- Worry about overfitting when your training metrics are much better than your validation/test metrics, when you have a small dataset, when your model is very complex (e.g., deep networks, high-degree polynomials), or when you have many features compared to the number of examples. In production, a sudden drop in real-world accuracy compared to offline results can also indicate overfitting or data drift.
- How much does overfitting cost?
- Overfitting doesn’t have a direct price tag, but it increases costs through wasted training time (more experiments and longer training), higher cloud compute spend (GPUs/TPUs/CPUs for repeated runs), extra data labeling/collection, and business losses from wrong predictions (e.g., false fraud alerts, missed fraud, poor recommendations). The biggest cost drivers are model complexity, number of training runs, dataset size, and the need for ongoing monitoring and retraining.
Category: ai-ml
Difficulty: intermediate
Related Terms
See Also