.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/pitfalls_and_solutions/plot_skd006_coefficient_interpretation.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_pitfalls_and_solutions_plot_skd006_coefficient_interpretation.py: .. _example_skd006_coefficient_interpretation: SKD006 - Coefficient interpretation =================================== :ref:`SKD006 ` is a tip about interpretation. This check flags that on mixed-scale features, raw coefficient magnitudes are not directly comparable across columns, and after standardization they are comparable but no longer expressed in the original feature units. Which path you take depends on what you want to read from the coefficients. See also scikit-learn's example on `linear model coefficient interpretation `_. When reading coefficients under SKD006, think of: - standardizing the inputs when you want coefficients that are comparable across features (see `scale matters `_), - multiplying each coefficient by the feature's standard deviation for an "effect per one std" without refitting, - relying on a scale-invariant ranking such as permutation importance when you need importance that does not depend on coefficient units. We use California housing regression with :class:`~sklearn.linear_model.Ridge` on raw numeric columns. The goal is to interpret effect sizes without mistaking scale differences for importance. .. GENERATED FROM PYTHON SOURCE LINES 31-37 Load the California housing dataset =================================== Median income is measured in 10k USD blocks, latitude in degrees, and population in head counts. Fitting Ridge on these raw columns produces coefficients whose magnitudes reflect units as much as predictive strength. .. GENERATED FROM PYTHON SOURCE LINES 37-43 .. code-block:: Python from skrub.datasets import fetch_california_housing housing = fetch_california_housing() X, y = housing.X, housing.y .. GENERATED FROM PYTHON SOURCE LINES 44-45 :class:`~skrub.TableReport` highlights the range differences across columns. .. GENERATED FROM PYTHON SOURCE LINES 45-50 .. code-block:: Python from skrub import TableReport TableReport(X) .. raw:: html

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 51-54 .. code-block:: Python TableReport(y) .. raw:: html

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 55-57 A single :class:`~skore.TrainTestSplit` with a 20 % test set feeds every evaluation below. .. GENERATED FROM PYTHON SOURCE LINES 57-62 .. code-block:: Python from skore import TrainTestSplit splitter = TrainTestSplit(test_size=0.2, random_state=42) .. GENERATED FROM PYTHON SOURCE LINES 63-69 Trigger SKD006 with Ridge on raw mixed-scale features ===================================================== Ridge with a moderate ``alpha`` fits quickly on unscaled inputs. SKD006 indicates that the features are not on the same scale, so raw coefficient magnitudes should not be read as a ranking of importance. .. GENERATED FROM PYTHON SOURCE LINES 69-81 .. code-block:: Python from sklearn.linear_model import Ridge from skore import evaluate report = evaluate( Ridge(alpha=1.0), X=X, y=y, splitter=splitter, ) report .. raw:: html 
Ridge()
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 82-83 Find ``SKD006`` in the Tips tab below. .. GENERATED FROM PYTHON SOURCE LINES 83-86 .. code-block:: Python report.checks.summarize() .. raw:: html


.. GENERATED FROM PYTHON SOURCE LINES 87-97 Let us inspect coefficients with :meth:`~skore.EstimatorReport.inspection.coefficients`, and compare each coefficient's magnitude to the feature's typical range (or standard deviation): a large coefficient on a small-scale column is not necessarily more important than a small coefficient on a large-scale column such as ``Population``. Unscaled coefficients are still useful on their own: they answer "if I change this feature by one unit in its original scale, how much does the prediction change in target units?" They are misleading regarding feature importance. .. GENERATED FROM PYTHON SOURCE LINES 97-100 .. code-block:: Python coef_display = report.inspection.coefficients() coef_display.frame() .. raw:: html
feature coefficient
0 Intercept -37.019420
1 MedInc 0.448511
2 HouseAge 0.009726
3 AveRooms -0.123014
4 AveBedrms 0.781417
5 Population -0.000002
6 AveOccup -0.003526
7 Latitude -0.419787
8 Longitude -0.433681


.. GENERATED FROM PYTHON SOURCE LINES 101-105 Side by side, the raw coefficient magnitudes and the feature standard deviations tell different stories. For instance, ``Population`` has a tiny coefficient because a one-person change is negligible on a head-count scale, even though the column varies a lot across districts. .. GENERATED FROM PYTHON SOURCE LINES 105-122 .. code-block:: Python import pandas as pd coef = coef_display.frame(include_intercept=False).set_index("feature") feature_std = report.X_train.std().rename("feature std. dev. (train)") df = pd.concat([coef, feature_std], axis=1) axes = df.plot.barh( subplots=True, layout=(1, 2), legend=False, sharey=True, sharex=False, figsize=(10, 4), ) axes[0, 0].set_xlabel("coefficient") _ = axes[0, 1].set_xlabel("std. dev.") .. image-sg:: /auto_examples/pitfalls_and_solutions/images/sphx_glr_plot_skd006_coefficient_interpretation_001.png :alt: coefficient, feature std. dev. (train) :srcset: /auto_examples/pitfalls_and_solutions/images/sphx_glr_plot_skd006_coefficient_interpretation_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 123-133 Scale coefficients by feature standard deviation ================================================ Recall the pitfall above: when the model was trained on features with different dynamic ranges, you cannot compare features by looking at raw coefficients alone. Multiplying each fitted coefficient by its feature standard deviation gives an "effect per one standard deviation" that is comparable across columns without refitting. See scikit-learn's discussion of `scale matters `_. .. GENERATED FROM PYTHON SOURCE LINES 133-138 .. code-block:: Python comparable = coef.copy() comparable["effect per std. dev."] = comparable["coefficient"] * feature_std _ = comparable.plot.barh() .. image-sg:: /auto_examples/pitfalls_and_solutions/images/sphx_glr_plot_skd006_coefficient_interpretation_002.png :alt: plot skd006 coefficient interpretation :srcset: /auto_examples/pitfalls_and_solutions/images/sphx_glr_plot_skd006_coefficient_interpretation_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 139-143 That rescaling erases the surprises from the side-by-side bars above. The large raw ``AveBedrms`` coefficient shrinks once multiplied by its small standard deviation, so it drops in the ranking while other features become, in comparison, more important. .. GENERATED FROM PYTHON SOURCE LINES 145-153 Standardize inputs in a pipeline ================================ :class:`~sklearn.preprocessing.StandardScaler` inside a :class:`~sklearn.pipeline.Pipeline` puts every feature on a common scale at fit time, so coefficients become directly comparable as "effect per one standard deviation." That is useful for ranking features, but you lose statements in original units (for example, "one extra year of age"). .. GENERATED FROM PYTHON SOURCE LINES 153-165 .. code-block:: Python from sklearn.pipeline import Pipeline from sklearn.preprocessing import StandardScaler report_scaled = evaluate( Pipeline([("scale", StandardScaler()), ("ridge", Ridge(alpha=1.0))]), X=X, y=y, splitter=splitter, ) report_scaled .. raw:: html 
Pipeline(steps=[('scale', StandardScaler()), ('ridge', Ridge())])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 166-167 ``SKD006`` is still reported in the Tips tab but the warning message changed. .. GENERATED FROM PYTHON SOURCE LINES 167-170 .. code-block:: Python report_scaled.checks.summarize() .. raw:: html


.. GENERATED FROM PYTHON SOURCE LINES 171-182 The coefficient table is now on a shared scale. Read magnitudes as relative importance after standardization, not as effects per original unit of income, rooms, or population. These values are the same quantity as the post-hoc ``coefficient * feature_std`` column from the previous section: both are an effect per one standard deviation. Fitting with ``StandardScaler`` does that rescaling inside the pipeline; multiplying raw coefficients by the train-set std does it after the fact. Small numerical differences can remain because the scaler's internal mean/std and ``X_train.std()`` are not bit-identical, but the ranking and interpretation should match. .. GENERATED FROM PYTHON SOURCE LINES 182-193 .. code-block:: Python pd.concat( [ report_scaled.inspection.coefficients() .frame(include_intercept=False) .set_index("feature"), comparable["effect per std. dev."], ], axis=1, ) .. raw:: html
coefficient effect per std. dev.
feature
MedInc 0.854327 0.854097
HouseAge 0.122624 0.122571
AveRooms -0.294210 -0.293681
AveBedrms 0.339008 0.338521
Population -0.002282 -0.002303
AveOccup -0.040833 -0.040825
Latitude -0.896168 -0.896944
Longitude -0.869071 -0.869813


.. GENERATED FROM PYTHON SOURCE LINES 194-209 Permutation importance (scale-invariant) ======================================== Scaled coefficients are one way to rank features. Another is permutation importance on held-out data: shuffle one column at a time and measure how much the score drops. That answers questions such as "which features did the model rely on most?" and works for nonlinear models as well. Present it as an alternative to scaled coefficients when you care about predictive reliance rather than a linear effect size. Correlated features can distort both coefficient rankings and permutation importance (credit is shared or shifted between partners). See the :ref:`SKD008 example ` and scikit-learn's note on `misleading values on strongly correlated features `_. .. GENERATED FROM PYTHON SOURCE LINES 209-215 .. code-block:: Python display = report.inspection.permutation_importance( seed=42, n_repeats=5, ) .. GENERATED FROM PYTHON SOURCE LINES 216-218 .. code-block:: Python display.frame() .. raw:: html
data_source metric feature value_mean value_std
0 test r2 MedInc 1.038369 0.025849
1 test r2 HouseAge 0.022603 0.002785
2 test r2 AveRooms 0.236201 0.007711
3 test r2 AveBedrms 0.257610 0.007691
4 test r2 Population 0.000002 0.000039
5 test r2 AveOccup 0.001057 0.000244
6 test r2 Latitude 1.207360 0.028557
7 test r2 Longitude 1.147209 0.020633


.. GENERATED FROM PYTHON SOURCE LINES 219-221 .. code-block:: Python display.plot() .. image-sg:: /auto_examples/pitfalls_and_solutions/images/sphx_glr_plot_skd006_coefficient_interpretation_003.png :alt: Permutation importance of Ridge on test set :srcset: /auto_examples/pitfalls_and_solutions/images/sphx_glr_plot_skd006_coefficient_interpretation_003.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none
.. GENERATED FROM PYTHON SOURCE LINES 222-235 Conclusion ========== SKD006 is a reminder to be careful when interpreting coefficients of linear models. When features are on different scales, linear coefficients mix feature importance with feature magnitude; when features are standardized, coefficients are comparable but no longer in original units. Unscaled coefficients remain useful when the question is "if I change this feature by this much in its own units, how much does the prediction change?" Choose standardization, an effect-per-std rescaling, or a scale-invariant method such as permutation importance depending on the question you want to answer. .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 4.060 seconds) .. _sphx_glr_download_auto_examples_pitfalls_and_solutions_plot_skd006_coefficient_interpretation.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_skd006_coefficient_interpretation.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_skd006_coefficient_interpretation.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_skd006_coefficient_interpretation.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_