Checking for modeling pitfalls and remediation

Examples for each automated check. Each script reproduces a check, and walks through fixes described in Automated checks.

SKD001 & SKD002 - Overfitting and underfitting

SKD001 & SKD002 - Overfitting and underfitting

SKD003 - Inconsistent performance across splits

SKD003 - Inconsistent performance across splits

SKD004 - High class imbalance

SKD004 - High class imbalance

SKD005 - Underrepresented classes

SKD005 - Underrepresented classes

SKD006 - Coefficient interpretation

SKD006 - Coefficient interpretation

SKD007 - MDI biased for high-cardinality features

SKD007 - MDI biased for high-cardinality features

SKD008 - Highly correlated input features

SKD008 - Highly correlated input features

SKD009 - Model performance vs. HistGradientBoosting baseline

SKD009 - Model performance vs. HistGradientBoosting baseline

SKD010 - Model slower than baseline

SKD010 - Model slower than baseline

SKD011 & SKD012 - Golden feature and useless features

SKD011 & SKD012 - Golden feature and useless features

SKD013 - Train-test overlap in time series

SKD013 - Train-test overlap in time series

SKD014 & SKD015: Hyperparameter search pitfalls

SKD014 & SKD015: Hyperparameter search pitfalls

SKD016 - Estimator not tuned

SKD016 - Estimator not tuned