SKD016 - Estimator not tuned

This example walks through mitigations when check SKD016 tips on a plain estimator left at scikit-learn defaults. The check compares initialization parameters against a curated table of high-impact hyperparameters and suggests axes worth tuning.

Mitigations from the Automated checks user guide:

• wrap the estimator in GridSearchCV or RandomizedSearchCV over the suggested parameters,

• or set sensible non-default values manually.

We use the employee salaries dataset (above-median salary as the positive class) with a default tabular_pipeline() classifier. The goal is to move off factory defaults either through search or hand-picked values.

Load the employee salaries dataset

Mixed HR features suit tabular_pipeline. A default tabular_pipeline("classifier") leaves HistGradientBoostingClassifier at sklearn defaults — the setup SKD016 is designed to flag.

from skrub.datasets import fetch_employee_salaries

dataset = fetch_employee_salaries()
X = dataset.X
y_salary = dataset.y.squeeze()
y = (y_salary > y_salary.median()).astype(int).rename("high_earner")

Inspect inputs and the binary target with TableReport.

from skrub import TableReport

TableReport(X)

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

TableReport(y)

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

We use the same stratified split for every comparison.

from skore import TrainTestSplit

splitter = TrainTestSplit(test_size=0.2, random_state=42, stratify=y)

Trigger SKD016 - untuned default pipeline

Defaults are fine for a first look at the table, but they are not a production configuration. SKD016 names the high-impact axes that usually matter first for this estimator family.

import skore
from skrub import tabular_pipeline

report = skore.evaluate(
    tabular_pipeline("classifier"),
    X=X,
    y=y,
    pos_label=1,
    splitter=splitter,
)
report

Pipeline(steps=[('tablevectorizer',
                 TableVectorizer(low_cardinality=ToCategorical())),
                ('histgradientboostingclassifier',
                 HistGradientBoostingClassifier())])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

SKD016 should tip that HistGradientBoostingClassifier remains at defaults. Read which parameters it lists — the next sections search or set those knobs.

report.checks.summarize()

report.metrics.summarize(data_source="both").frame()

	HistGradientBoostingClassifier (train)	HistGradientBoostingClassifier (test)
metric
accuracy	0.989027	0.937703
precision	0.990487	0.940087
recall	0.987534	0.934995
roc_auc	0.999158	0.986606
log_loss	0.052933	0.150311
brier_score	0.011850	0.044754
fit_time	2.213212	2.213212
predict_time	0.776003	0.230984

Wrap the estimator in RandomizedSearchCV

Search the axes SKD016 typically flags for HGB (learning rate, iteration budget, depth, leaf size) instead of accepting sklearn defaults. Once the report wraps a fitted search object, SKD016 clears.

A tuned search can still raise SKD014 or SKD015 if the box is too narrow or incomplete — see that combined example.

from scipy.stats import loguniform, randint
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.model_selection import RandomizedSearchCV

base_pipeline = tabular_pipeline(HistGradientBoostingClassifier(random_state=42))

param_distributions = {
    "histgradientboostingclassifier__learning_rate": loguniform(1e-2, 2e-1),
    "histgradientboostingclassifier__max_iter": randint(100, 401),
    "histgradientboostingclassifier__max_depth": [3, 5, 8, None],
    "histgradientboostingclassifier__min_samples_leaf": randint(10, 51),
}

tuned_search = RandomizedSearchCV(
    base_pipeline,
    param_distributions=param_distributions,
    n_iter=8,
    cv=3,
    scoring="neg_log_loss",
    random_state=42,
    refit=True,
)

report_tuned = skore.evaluate(
    tuned_search,
    X=X,
    y=y,
    pos_label=1,
    splitter=splitter,
)
report_tuned

RandomizedSearchCV(cv=3,
                   estimator=Pipeline(steps=[('tablevectorizer',
                                              TableVectorizer(low_cardinality=ToCategorical())),
                                             ('histgradientboostingclassifier',
                                              HistGradientBoostingClassifier(random_state=42))]),
                   n_iter=8,
                   param_distributions={'histgradientboostingclassifier__learning_rate': <scipy.stats._distn_infrastructure.rv_continuous_frozen object at 0x7f3...9e0>,
                                        'histgradientboostingclassifier__max_depth': [3,
                                                                                      5,
                                                                                      8,
                                                                                      None],
                                        'histgradientboostingclassifier__max_iter': <scipy.stats._distn_infrastructure.rv_discrete_frozen object at 0x7f35f2dd3130>,
                                        'histgradientboostingclassifier__min_samples_leaf': <scipy.stats._distn_infrastructure.rv_discrete_frozen object at 0x7f35f2dd3f00>},
                   random_state=42, scoring='neg_log_loss')

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

SKD016 should be absent; the report wraps a fitted search object.

report_tuned.checks.summarize(fast_mode=True)

report_tuned.estimator_.best_params_

{'histgradientboostingclassifier__learning_rate': np.float64(0.023927528765580644), 'histgradientboostingclassifier__max_depth': 8, 'histgradientboostingclassifier__max_iter': 269, 'histgradientboostingclassifier__min_samples_leaf': 37}

Set sensible non-default values manually

When a full search is impractical, hand-pick hyperparameters that differ from defaults. SKD016 clears as soon as impactful knobs are no longer factory settings — that is an intentional configuration signal, not proof that the values are optimal. Prefer validated search when you can afford it.

model_manual = tabular_pipeline(
    HistGradientBoostingClassifier(
        learning_rate=0.05,
        max_iter=200,
        max_depth=5,
        min_samples_leaf=20,
        random_state=42,
    )
)

report_manual = skore.evaluate(
    model_manual,
    X=X,
    y=y,
    pos_label=1,
    splitter=splitter,
)
report_manual

Pipeline(steps=[('tablevectorizer',
                 TableVectorizer(low_cardinality=ToCategorical())),
                ('histgradientboostingclassifier',
                 HistGradientBoostingClassifier(learning_rate=0.05, max_depth=5,
                                                max_iter=200,
                                                random_state=42))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

SKD016 should be absent once hyperparameters differ from defaults.

report_manual.checks.summarize(fast_mode=True)

report_manual.metrics.summarize(data_source="both").frame()

	HistGradientBoostingClassifier (train)	HistGradientBoostingClassifier (test)
metric
accuracy	0.970469	0.937161
precision	0.967925	0.932476
recall	0.973171	0.942579
roc_auc	0.996354	0.985750
log_loss	0.094452	0.154871
brier_score	0.024436	0.045679
fit_time	1.429433	1.429433
predict_time	0.469798	0.161528

Compare mitigations

Hold-out metrics for the default pipeline and the hand-tuned one. The RandomizedSearchCV report wraps a search estimator, so its metric table can look different in compare(); we show it in its own cell below.

comparison = skore.compare(
    {
        "default_pipeline": report,
        "hand_tuned_hgb": report_manual,
    }
)
comparison.metrics.summarize(data_source="both").frame()

	default_pipeline (train)	default_pipeline (test)	hand_tuned_hgb (train)	hand_tuned_hgb (test)
metric
accuracy	0.989027	0.937703	0.970469	0.937161
precision	0.990487	0.940087	0.967925	0.932476
recall	0.987534	0.934995	0.973171	0.942579
roc_auc	0.999158	0.986606	0.996354	0.985750
log_loss	0.052933	0.150311	0.094452	0.154871
brier_score	0.011850	0.044754	0.024436	0.045679
fit_time	2.213212	2.213212	1.429433	1.429433
predict_time	0.776003	0.230984	0.469798	0.161528

report_tuned.metrics.summarize(data_source="both").frame()

	RandomizedSearchCV (train)	RandomizedSearchCV (test)
metric
score	-0.090711	-0.153892
accuracy	0.972230	0.933911
precision	0.973120	0.934853
recall	0.971274	0.932828
roc_auc	0.996640	0.986055
log_loss	0.090711	0.153892
brier_score	0.022995	0.045784
fit_time	49.192662	49.192662
predict_time	0.518892	0.175656

Conclusion

SKD016 nudges you off scikit-learn defaults for high-impact estimators. Randomized search and hand-tuned HGB parameters both clear the tip; clearing the check means you left factory settings, not that the model is finished. Pair manual choices with periodic search, and watch SKD014/SKD015 once you wrap a BaseSearchCV.