SKD016 - Estimator not tuned

This example walks through mitigations when check SKD016 tips on a plain estimator left at scikit-learn defaults. The check compares initialization parameters against a curated table of high-impact hyperparameters and suggests axes worth tuning.

Mitigations from the Automated checks user guide:

• wrap the estimator in GridSearchCV or RandomizedSearchCV over the suggested parameters,

• or set sensible non-default values manually.

We use the employee salaries dataset (above-median salary as the positive class) with a default tabular_pipeline() classifier. The goal is to move off factory defaults either through search or hand-picked values.

Load the employee salaries dataset

Mixed HR features suit tabular_pipeline. A default tabular_pipeline("classifier") leaves HistGradientBoostingClassifier at sklearn defaults — the setup SKD016 is designed to flag.

from skrub.datasets import fetch_employee_salaries

dataset = fetch_employee_salaries()
X = dataset.X
y_salary = dataset.y.squeeze()
y = (y_salary > y_salary.median()).astype(int).rename("high_earner")

Inspect inputs and the binary target with TableReport.

from skrub import TableReport

TableReport(X)

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

TableReport(y)

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

We use the same stratified split for every comparison.

from skore import TrainTestSplit

splitter = TrainTestSplit(test_size=0.2, random_state=42, stratify=y)

Trigger SKD016 - untuned default pipeline

Defaults are fine for a first look at the table, but they are not a production configuration. SKD016 names the high-impact axes that usually matter first for this estimator family.

import skore
from skrub import tabular_pipeline

report = skore.evaluate(
    tabular_pipeline("classifier"),
    X=X,
    y=y,
    pos_label=1,
    splitter=splitter,
)
report

Pipeline(steps=[('tablevectorizer',
                 TableVectorizer(low_cardinality=ToCategorical())),
                ('histgradientboostingclassifier',
                 HistGradientBoostingClassifier())])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

SKD016 should tip that HistGradientBoostingClassifier remains at defaults. Read which parameters it lists — the next sections search or set those knobs.

report.checks.summarize()

report.metrics.summarize(data_source="both").frame()

	HistGradientBoostingClassifier (train)	HistGradientBoostingClassifier (test)
metric
accuracy	0.988756	0.937161
precision	0.992087	0.938111
recall	0.985366	0.936078
roc_auc	0.999151	0.987118
log_loss	0.054099	0.146457
brier_score	0.012345	0.044194
fit_time	2.044371	2.044371
predict_time	0.782540	0.284502

Wrap the estimator in RandomizedSearchCV

Search the axes SKD016 typically flags for HGB (learning rate, iteration budget, depth, leaf size) instead of accepting sklearn defaults. Once the report wraps a fitted search object, SKD016 clears.

A tuned search can still raise SKD014 or SKD015 if the box is too narrow or incomplete — see that combined example.

from scipy.stats import loguniform, randint
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import RandomizedSearchCV

base_pipeline = tabular_pipeline(HistGradientBoostingClassifier(random_state=42))

param_distributions = {
    "histgradientboostingclassifier__learning_rate": loguniform(1e-2, 2e-1),
    "histgradientboostingclassifier__max_iter": randint(100, 401),
    "histgradientboostingclassifier__max_depth": [3, 5, 8, None],
    "histgradientboostingclassifier__min_samples_leaf": randint(10, 51),
}

tuned_search = RandomizedSearchCV(
    base_pipeline,
    param_distributions=param_distributions,
    n_iter=8,
    cv=3,
    scoring="neg_log_loss",
    random_state=42,
    refit=True,
)

report_tuned = skore.evaluate(
    tuned_search,
    X=X,
    y=y,
    pos_label=1,
    splitter=splitter,
)
report_tuned

RandomizedSearchCV(cv=3,
                   estimator=Pipeline(steps=[('tablevectorizer',
                                              TableVectorizer(low_cardinality=ToCategorical())),
                                             ('histgradientboostingclassifier',
                                              HistGradientBoostingClassifier(random_state=42))]),
                   n_iter=8,
                   param_distributions={'histgradientboostingclassifier__learning_rate': <scipy.stats._distn_infrastructure.rv_continuous_frozen object at 0x7f7...9e0>,
                                        'histgradientboostingclassifier__max_depth': [3,
                                                                                      5,
                                                                                      8,
                                                                                      None],
                                        'histgradientboostingclassifier__max_iter': <scipy.stats._distn_infrastructure.rv_discrete_frozen object at 0x7f782c6387c0>,
                                        'histgradientboostingclassifier__min_samples_leaf': <scipy.stats._distn_infrastructure.rv_discrete_frozen object at 0x7f782c639bf0>},
                   random_state=42, scoring='neg_log_loss')

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

SKD016 should be absent; the report wraps a fitted search object.

report_tuned.checks.summarize(fast_mode=True)

report_tuned.estimator_.best_params_

{'histgradientboostingclassifier__learning_rate': np.float64(0.060099747183803134), 'histgradientboostingclassifier__max_depth': 8, 'histgradientboostingclassifier__max_iter': 221, 'histgradientboostingclassifier__min_samples_leaf': 28}

Set sensible non-default values manually

When a full search is impractical, hand-pick hyperparameters that differ from defaults. SKD016 clears as soon as impactful knobs are no longer factory settings — that is an intentional configuration signal, not proof that the values are optimal. Prefer validated search when you can afford it.

model_manual = tabular_pipeline(
    HistGradientBoostingClassifier(
        learning_rate=0.05,
        max_iter=200,
        max_depth=5,
        min_samples_leaf=20,
        random_state=42,
    )
)

report_manual = skore.evaluate(
    model_manual,
    X=X,
    y=y,
    pos_label=1,
    splitter=splitter,
)
report_manual

Pipeline(steps=[('tablevectorizer',
                 TableVectorizer(low_cardinality=ToCategorical())),
                ('histgradientboostingclassifier',
                 HistGradientBoostingClassifier(learning_rate=0.05, max_depth=5,
                                                max_iter=200,
                                                random_state=42))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

SKD016 should be absent once hyperparameters differ from defaults.

report_manual.checks.summarize(fast_mode=True)

report_manual.metrics.summarize(data_source="both").frame()

	HistGradientBoostingClassifier (train)	HistGradientBoostingClassifier (test)
metric
accuracy	0.969927	0.933369
precision	0.968649	0.931034
recall	0.971274	0.936078
roc_auc	0.996075	0.985018
log_loss	0.096833	0.157725
brier_score	0.025237	0.046415
fit_time	1.383695	1.383695
predict_time	0.469582	0.155745

Compare mitigations

Hold-out metrics for the default pipeline and the hand-tuned one. The RandomizedSearchCV report wraps a search estimator, so its metric table can look different in compare(); we show it in its own cell below.

comparison = skore.compare(
    {
        "default_pipeline": report,
        "hand_tuned_hgb": report_manual,
    }
)
comparison.metrics.summarize(data_source="both").frame()

	default_pipeline (train)	default_pipeline (test)	hand_tuned_hgb (train)	hand_tuned_hgb (test)
metric
accuracy	0.988756	0.937161	0.969927	0.933369
precision	0.992087	0.938111	0.968649	0.931034
recall	0.985366	0.936078	0.971274	0.936078
roc_auc	0.999151	0.987118	0.996075	0.985018
log_loss	0.054099	0.146457	0.096833	0.157725
brier_score	0.012345	0.044194	0.025237	0.046415
fit_time	2.044371	2.044371	1.383695	1.383695
predict_time	0.782540	0.284502	0.469582	0.155745

report_tuned.metrics.summarize(data_source="both").frame()

	RandomizedSearchCV (train)	RandomizedSearchCV (test)
metric
score	-0.048947	-0.150687
accuracy	0.988756	0.937703
precision	0.991551	0.937229
recall	0.985908	0.938245
roc_auc	0.999260	0.986551
log_loss	0.048947	0.150687
brier_score	0.011166	0.045203
fit_time	47.923457	47.923457
predict_time	0.490584	0.166994

Conclusion

SKD016 nudges you off scikit-learn defaults for high-impact estimators. Randomized search and hand-tuned HGB parameters both clear the tip; clearing the check means you left factory settings, not that the model is finished. Pair manual choices with periodic search, and watch SKD014/SKD015 once you wrap a BaseSearchCV.