.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/use_cases/plot_employee_salaries.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_use_cases_plot_employee_salaries.py: .. _example_use_case_employee_salaries: ============================================== Simplified and structured experiment reporting ============================================== This example shows how to leverage `skore` for structuring useful experiment information allowing to get insights from machine learning experiments. .. GENERATED FROM PYTHON SOURCE LINES 13-15 We set some environment variables to avoid some spurious warnings related to parallelism when using the `TextEncoder` from `skrub` that uses tokenizers. .. GENERATED FROM PYTHON SOURCE LINES 17-21 .. code-block:: Python import os os.environ["TOKENIZERS_PARALLELISM"] = "true" .. GENERATED FROM PYTHON SOURCE LINES 22-24 Loading a non-trivial dataset ============================= .. GENERATED FROM PYTHON SOURCE LINES 26-28 We use a skrub dataset that contains information about employees and their salaries. We will see that this dataset is non-trivial. .. GENERATED FROM PYTHON SOURCE LINES 30-35 .. code-block:: Python from skrub.datasets import fetch_employee_salaries datasets = fetch_employee_salaries() df, y = datasets.X, datasets.y .. GENERATED FROM PYTHON SOURCE LINES 36-38 Let's first have a condensed summary of the input data using a :class:`skrub.TableReport`. .. GENERATED FROM PYTHON SOURCE LINES 40-44 .. code-block:: Python from skrub import TableReport TableReport(df) .. raw:: html

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 45-69 From the table report, we can make the following observations: * Looking at the *Table* tab, we observe that the year related to the ``date_first_hired`` column is also present in the ``date`` column. Hence, we should beware of not creating twice the same feature during the feature engineering. * Looking at the *Stats* tab: - The type of data is heterogeneous: we mainly have categorical and date-related features. - The ``division`` and ``employee_position_title`` features contain a large number of categories. It is something that we should consider in our feature engineering. * Looking at the *Associations* tab, we observe that two features are holding the exact same information: ``department`` and ``department_name``. Hence, during our feature engineering, we could potentially drop one of them if the final predictive model is sensitive to the collinearity. In terms of target and thus the task that we want to solve, we are interested in predicting the salary of an employee given the previous features. We therefore have a regression task at end. .. GENERATED FROM PYTHON SOURCE LINES 71-73 .. code-block:: Python TableReport(y) .. raw:: html

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 74-77 Later in this example, we will show that `skore` stores similar information when a model is trained on a dataset, thus enabling us to get quick insights on the dataset used to train and test the model. .. GENERATED FROM PYTHON SOURCE LINES 79-94 Tree-based model ================ Let's start by creating a tree-based model using some out-of-the-box tools. For feature engineering, we use skrub's :class:`~skrub.TableVectorizer`. To deal with the high cardinality of the categorical features, we use a :class:`~skrub.TextEncoder` that uses a language model and an embedding model to encode the categorical features. Finally, we use a :class:`~sklearn.ensemble.HistGradientBoostingRegressor` as a base estimator, it is a rather robust model. Modelling ^^^^^^^^^ .. GENERATED FROM PYTHON SOURCE LINES 96-106 .. code-block:: Python from skrub import TableVectorizer, TextEncoder from sklearn.ensemble import HistGradientBoostingRegressor from sklearn.pipeline import make_pipeline model = make_pipeline( TableVectorizer(high_cardinality=TextEncoder()), HistGradientBoostingRegressor(), ) model .. raw:: html 
Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(high_cardinality=TextEncoder())),
                    ('histgradientboostingregressor',
                     HistGradientBoostingRegressor())])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 107-112 Evaluation ^^^^^^^^^^ Let us compute the cross-validation report for this model using a :class:`skore.CrossValidationReport`: .. GENERATED FROM PYTHON SOURCE LINES 114-121 .. code-block:: Python from skore import CrossValidationReport hgbt_model_report = CrossValidationReport( estimator=model, X=df, y=y, splitter=5, n_jobs=4 ) hgbt_model_report.help() .. rst-class:: sphx-glr-script-out .. code-block:: none ╭───────────── Tools to diagnose estimator HistGradientBoostingRegressor ──────────────╮ │ CrossValidationReport │ │ ├── .metrics │ │ │ ├── .prediction_error(...) - Plot the prediction error of a regression │ │ │ │ model. │ │ │ ├── .r2(...) (↗︎) - Compute the R² score. │ │ │ ├── .rmse(...) (↘︎) - Compute the root mean squared error. │ │ │ ├── .timings(...) - Get all measured processing times related │ │ │ │ to the estimator. │ │ │ ├── .custom_metric(...) - Compute a custom metric. │ │ │ └── .summarize(...) - Report a set of metrics for our estimator. │ │ ├── .cache_predictions(...) - Cache the predictions for sub-estimators │ │ │ reports. │ │ ├── .clear_cache(...) - Clear the cache. │ │ ├── .get_predictions(...) - Get estimator's predictions. │ │ └── Attributes │ │ ├── .X - The data to fit │ │ ├── .y - The target variable to try to predict in │ │ │ the case of supervised learning │ │ ├── .estimator - Estimator to make the cross-validation │ │ │ report from │ │ ├── .estimator_ - The cloned or copied estimator │ │ ├── .estimator_name_ - The name of the estimator │ │ ├── .estimator_reports_ - The estimator reports for each split │ │ ├── .ml_task - No description available │ │ ├── .n_jobs - Number of jobs to run in parallel │ │ ├── .pos_label - For binary classification, the positive │ │ │ class │ │ ├── .split_indices - No description available │ │ └── .splitter - Determines the cross-validation splitting │ │ strategy │ │ │ │ │ │ Legend: │ │ (↗︎) higher is better (↘︎) lower is better │ ╰──────────────────────────────────────────────────────────────────────────────────────╯ .. GENERATED FROM PYTHON SOURCE LINES 122-126 A report provides a collection of useful information. For instance, it allows to compute on demand the predictions of the model and some performance metrics. Let's cache the predictions of the cross-validated models once and for all. .. GENERATED FROM PYTHON SOURCE LINES 128-130 .. code-block:: Python hgbt_model_report.cache_predictions(n_jobs=4) .. GENERATED FROM PYTHON SOURCE LINES 131-135 Now that the predictions are cached, any request to compute a metric will be performed using the cached predictions and will thus be fast. We can now have a look at the performance of the model with some standard metrics. .. GENERATED FROM PYTHON SOURCE LINES 137-139 .. code-block:: Python hgbt_model_report.metrics.summarize().frame() .. raw:: html
HistGradientBoostingRegressor
mean std
Metric
R² 0.924282 0.017469
RMSE 7991.516753 1195.815810
Fit time (s) 19.122908 5.773977
Predict time (s) 11.673561 13.430201


.. GENERATED FROM PYTHON SOURCE LINES 140-143 Similarly to what we saw in the previous section, the :class:`skore.CrossValidationReport` also stores some information about the dataset used. .. GENERATED FROM PYTHON SOURCE LINES 145-148 .. code-block:: Python data_display = hgbt_model_report.data.analyze() data_display .. raw:: html

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 149-154 The display obtained allows for a quick overview with the same HTML-based view as the :class:`skrub.TableReport` we have seen earlier. In addition, you can access a :meth:`skore.TableReportDisplay.plot` method to have a particular focus on one potential analysis. For instance, we can get a figure representing the correlation matrix of the dataset. .. GENERATED FROM PYTHON SOURCE LINES 156-158 .. code-block:: Python data_display.plot(kind="corr") .. image-sg:: /auto_examples/use_cases/images/sphx_glr_plot_employee_salaries_001.png :alt: Cramer's V Correlation :srcset: /auto_examples/use_cases/images/sphx_glr_plot_employee_salaries_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 159-166 We get the results from some statistical metrics aggregated over the cross-validation splits as well as some performance metrics related to the time it took to train and test the model. The :class:`skore.CrossValidationReport` also provides a way to inspect similar information at the level of each cross-validation split by accessing an :class:`skore.EstimatorReport` for each split. .. GENERATED FROM PYTHON SOURCE LINES 168-171 .. code-block:: Python hgbt_split_1 = hgbt_model_report.estimator_reports_[0] hgbt_split_1.metrics.summarize(indicator_favorability=True).frame() .. raw:: html
HistGradientBoostingRegressor Favorability
Metric
R² 0.922033 (↗︎)
RMSE 8040.374042 (↘︎)
Fit time (s) 21.546139 (↘︎)
Predict time (s) 5.515473 (↘︎)


.. GENERATED FROM PYTHON SOURCE LINES 172-174 The favorability of each metric indicates whether the metric is better when higher or lower. .. GENERATED FROM PYTHON SOURCE LINES 176-182 Linear model ============ Now that we have established a first model that serves as a baseline, we shall proceed to define a quite complex linear model: a pipeline with a complex feature engineering that uses a linear model as the base estimator. .. GENERATED FROM PYTHON SOURCE LINES 184-186 Modelling ^^^^^^^^^ .. GENERATED FROM PYTHON SOURCE LINES 188-236 .. code-block:: Python import numpy as np from sklearn.compose import make_column_transformer from sklearn.pipeline import make_pipeline from sklearn.preprocessing import OneHotEncoder, SplineTransformer from sklearn.linear_model import RidgeCV from skrub import DatetimeEncoder, ToDatetime, DropCols, GapEncoder def periodic_spline_transformer(period, n_splines=None, degree=3): if n_splines is None: n_splines = period n_knots = n_splines + 1 # periodic and include_bias is True return SplineTransformer( degree=degree, n_knots=n_knots, knots=np.linspace(0, period, n_knots).reshape(n_knots, 1), extrapolation="periodic", include_bias=True, ) one_hot_features = ["gender", "department_name", "assignment_category"] datetime_features = "date_first_hired" date_encoder = make_pipeline( ToDatetime(), DatetimeEncoder(resolution="day", add_weekday=True, add_total_seconds=False), DropCols("date_first_hired_year"), ) date_engineering = make_column_transformer( (periodic_spline_transformer(12, n_splines=6), ["date_first_hired_month"]), (periodic_spline_transformer(31, n_splines=15), ["date_first_hired_day"]), (periodic_spline_transformer(7, n_splines=3), ["date_first_hired_weekday"]), ) feature_engineering_date = make_pipeline(date_encoder, date_engineering) preprocessing = make_column_transformer( (feature_engineering_date, datetime_features), (OneHotEncoder(drop="if_binary", handle_unknown="ignore"), one_hot_features), (GapEncoder(n_components=100), "division"), (GapEncoder(n_components=100), "employee_position_title"), ) model = make_pipeline(preprocessing, RidgeCV(alphas=np.logspace(-3, 3, 100))) model .. raw:: html 
Pipeline(steps=[('columntransformer',
                     ColumnTransformer(transformers=[('pipeline',
                                                      Pipeline(steps=[('pipeline',
                                                                       Pipeline(steps=[('todatetime',
                                                                                        ToDatetime()),
                                                                                       ('datetimeencoder',
                                                                                        DatetimeEncoder(add_total_seconds=False,
                                                                                                        add_weekday=True,
                                                                                                        resolution='day')),
                                                                                       ('dropcols',
                                                                                        DropCols(cols='date_first_hired_year'))])),
                                                                      ('columntransformer',
                                                                       ColumnTransformer(transfor...
           4.03701726e+01, 4.64158883e+01, 5.33669923e+01, 6.13590727e+01,
           7.05480231e+01, 8.11130831e+01, 9.32603347e+01, 1.07226722e+02,
           1.23284674e+02, 1.41747416e+02, 1.62975083e+02, 1.87381742e+02,
           2.15443469e+02, 2.47707636e+02, 2.84803587e+02, 3.27454916e+02,
           3.76493581e+02, 4.32876128e+02, 4.97702356e+02, 5.72236766e+02,
           6.57933225e+02, 7.56463328e+02, 8.69749003e+02, 1.00000000e+03])))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 237-250 In the diagram above, we can see what how we performed our feature engineering: * For categorical features, we use two approaches. If the number of categories is relatively small, we use a `OneHotEncoder`. If the number of categories is large, we use a `GapEncoder` that is designed to deal with high cardinality categorical features. * Then, we have another transformation to encode the date features. We first split the date into multiple features (day, month, and year). Then, we apply a periodic spline transformation to each of the date features in order to capture the periodicity of the data. * Finally, we fit a :class:`~sklearn.linear_model.RidgeCV` model. .. GENERATED FROM PYTHON SOURCE LINES 252-258 Evaluation ^^^^^^^^^^ Now, we want to evaluate this linear model via cross-validation (with 5 folds). For that, we use skore's :class:`~skore.CrossValidationReport` to investigate the performance of our model. .. GENERATED FROM PYTHON SOURCE LINES 260-265 .. code-block:: Python linear_model_report = CrossValidationReport( estimator=model, X=df, y=y, splitter=5, n_jobs=4 ) linear_model_report.help() .. rst-class:: sphx-glr-script-out .. code-block:: none ╭──────────────────────── Tools to diagnose estimator RidgeCV ─────────────────────────╮ │ CrossValidationReport │ │ ├── .metrics │ │ │ ├── .prediction_error(...) - Plot the prediction error of a regression │ │ │ │ model. │ │ │ ├── .r2(...) (↗︎) - Compute the R² score. │ │ │ ├── .rmse(...) (↘︎) - Compute the root mean squared error. │ │ │ ├── .timings(...) - Get all measured processing times related │ │ │ │ to the estimator. │ │ │ ├── .custom_metric(...) - Compute a custom metric. │ │ │ └── .summarize(...) - Report a set of metrics for our estimator. │ │ ├── .cache_predictions(...) - Cache the predictions for sub-estimators │ │ │ reports. │ │ ├── .clear_cache(...) - Clear the cache. │ │ ├── .get_predictions(...) - Get estimator's predictions. │ │ └── Attributes │ │ ├── .X - The data to fit │ │ ├── .y - The target variable to try to predict in │ │ │ the case of supervised learning │ │ ├── .estimator - Estimator to make the cross-validation │ │ │ report from │ │ ├── .estimator_ - The cloned or copied estimator │ │ ├── .estimator_name_ - The name of the estimator │ │ ├── .estimator_reports_ - The estimator reports for each split │ │ ├── .ml_task - No description available │ │ ├── .n_jobs - Number of jobs to run in parallel │ │ ├── .pos_label - For binary classification, the positive │ │ │ class │ │ ├── .split_indices - No description available │ │ └── .splitter - Determines the cross-validation splitting │ │ strategy │ │ │ │ │ │ Legend: │ │ (↗︎) higher is better (↘︎) lower is better │ ╰──────────────────────────────────────────────────────────────────────────────────────╯ .. GENERATED FROM PYTHON SOURCE LINES 266-274 We observe that the cross-validation report has detected that we have a regression task at hand and thus provides us with some metrics and plots that make sense with regards to our specific problem at hand. To accelerate any future computation (e.g. of a metric), we cache the predictions of our model once and for all. Note that we do not necessarily need to cache the predictions as the report will compute them on the fly (if not cached) and cache them for us. .. GENERATED FROM PYTHON SOURCE LINES 276-282 .. code-block:: Python import warnings with warnings.catch_warnings(): warnings.simplefilter(action="ignore", category=FutureWarning) linear_model_report.cache_predictions(n_jobs=4) .. GENERATED FROM PYTHON SOURCE LINES 283-284 We can now have a look at the performance of the model with some standard metrics. .. GENERATED FROM PYTHON SOURCE LINES 286-288 .. code-block:: Python linear_model_report.metrics.summarize(indicator_favorability=True).frame() .. raw:: html
RidgeCV Favorability
mean std
Metric
R² 0.763299 0.015638 (↗︎)
RMSE 14161.163787 962.053955 (↘︎)
Fit time (s) 11.387958 2.598647 (↘︎)
Predict time (s) 0.425168 0.007193 (↘︎)


.. GENERATED FROM PYTHON SOURCE LINES 289-294 Comparing the models ==================== Now that we cross-validated our models, we can make some further comparison using the :class:`skore.ComparisonReport`: .. GENERATED FROM PYTHON SOURCE LINES 296-301 .. code-block:: Python from skore import ComparisonReport comparator = ComparisonReport([hgbt_model_report, linear_model_report]) comparator.metrics.summarize(indicator_favorability=True).frame() .. raw:: html
mean std Favorability
Estimator HistGradientBoostingRegressor RidgeCV HistGradientBoostingRegressor RidgeCV
Metric
R² 0.924282 0.763299 0.017469 0.015638 (↗︎)
RMSE 7991.516753 14161.163787 1195.815810 962.053955 (↘︎)
Fit time (s) 19.122908 11.387958 5.773977 2.598647 (↘︎)
Predict time (s) 11.673561 0.425168 13.430201 0.007193 (↘︎)


.. GENERATED FROM PYTHON SOURCE LINES 302-307 In addition, if we forgot to compute a specific metric (e.g. :func:`~sklearn.metrics.mean_absolute_error`), we can easily add it to the report, without re-training the model and even without re-computing the predictions since they are cached internally in the report. This allows us to save some potentially huge computation time. .. GENERATED FROM PYTHON SOURCE LINES 309-321 .. code-block:: Python from sklearn.metrics import get_scorer scoring = ["r2", "rmse", get_scorer("neg_mean_absolute_error")] scoring_kwargs = {"response_method": "predict"} scoring_names = ["R²", "RMSE", "MAE"] comparator.metrics.summarize( scoring=scoring, scoring_kwargs=scoring_kwargs, scoring_names=scoring_names, ).frame() .. raw:: html
mean std
Estimator HistGradientBoostingRegressor RidgeCV HistGradientBoostingRegressor RidgeCV
Metric
R² 0.924282 0.763299 0.017469 0.015638
RMSE 7991.516753 14161.163787 1195.815810 962.053955
MAE 4433.515526 9946.342436 251.650830 299.705241


.. GENERATED FROM PYTHON SOURCE LINES 322-325 Finally, we can even get a deeper understanding by analyzing each split in the :class:`~skore.CrossValidationReport`. Here, we plot the actual-vs-predicted values for each split. .. GENERATED FROM PYTHON SOURCE LINES 327-329 .. code-block:: Python linear_model_report.metrics.prediction_error().plot(kind="actual_vs_predicted") .. image-sg:: /auto_examples/use_cases/images/sphx_glr_plot_employee_salaries_002.png :alt: Prediction Error for RidgeCV :srcset: /auto_examples/use_cases/images/sphx_glr_plot_employee_salaries_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 330-340 Conclusion ========== This example showcased `skore`'s integrated approach to machine learning workflow, from initial data exploration with `TableReport` through model development and evaluation with `CrossValidationReport`. We demonstrated how `skore` automatically captures dataset information and provides efficient caching, enabling quick insights and flexible model comparison. The workflow highlights `skore`'s ability to streamline the entire ML process while maintaining computational efficiency. .. rst-class:: sphx-glr-timing **Total running time of the script:** (2 minutes 14.358 seconds) .. _sphx_glr_download_auto_examples_use_cases_plot_employee_salaries.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_employee_salaries.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_employee_salaries.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_employee_salaries.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_