.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/getting_started/plot_skore_getting_started.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_getting_started_plot_skore_getting_started.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_getting_started_plot_skore_getting_started.py:


.. _example_skore_getting_started:

======================
Skore: getting started
======================

.. GENERATED FROM PYTHON SOURCE LINES 10-27

This getting started guide illustrates how to use skore and why:

#.    Get assistance when developing your machine learning projects to avoid common
      pitfalls and follow recommended practices.

      *   :class:`skore.EstimatorReport`: get an insightful report on your estimator,
          for evaluation and inspection

      *   :class:`skore.CrossValidationReport`: get an insightful report on your
          cross-validation results

      *   :class:`skore.ComparisonReport`: benchmark your skore estimator reports

      *   :func:`skore.train_test_split`: get diagnostics when splitting your data

#.    Track your machine learning results using skore's :class:`~skore.Project`
      (for storage).

.. GENERATED FROM PYTHON SOURCE LINES 29-35

Machine learning evaluation and diagnostics
===========================================

Skore implements new tools or wraps some key scikit-learn class / functions to
automatically provide insights and diagnostics when using them, as a way to
facilitate good practices and avoid common pitfalls.

.. GENERATED FROM PYTHON SOURCE LINES 37-45

Model evaluation with skore
^^^^^^^^^^^^^^^^^^^^^^^^^^^

In order to assist its users when programming, skore has implemented a
:class:`skore.EstimatorReport` class.

Let us load a binary classification dataset and get the estimator report for a
:class:`~sklearn.ensemble.RandomForestClassifier`:

.. GENERATED FROM PYTHON SOURCE LINES 47-62

.. code-block:: Python

    from sklearn.datasets import load_breast_cancer
    from sklearn.ensemble import RandomForestClassifier
    from sklearn.model_selection import train_test_split

    from skore import EstimatorReport

    X, y = load_breast_cancer(return_X_y=True, as_frame=True)
    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)

    rf = RandomForestClassifier(random_state=0)

    rf_report = EstimatorReport(
        rf, X_train=X_train, X_test=X_test, y_train=y_train, y_test=y_test
    )








.. GENERATED FROM PYTHON SOURCE LINES 63-65

Now, we can display the helper to see all the insights that are available to us
(skore detected that we are doing binary classification):

.. GENERATED FROM PYTHON SOURCE LINES 67-69

.. code-block:: Python

    rf_report.help()





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    ╭───────────────── Tools to diagnose estimator RandomForestClassifier ─────────────────╮
    │ EstimatorReport                                                                      │
    │ ├── .metrics                                                                         │
    │ │   ├── .accuracy(...)         (↗︎)     - Compute the accuracy score.                 │
    │ │   ├── .brier_score(...)      (↘︎)     - Compute the Brier score.                    │
    │ │   ├── .log_loss(...)         (↘︎)     - Compute the log loss.                       │
    │ │   ├── .precision(...)        (↗︎)     - Compute the precision score.                │
    │ │   ├── .precision_recall(...)         - Plot the precision-recall curve.            │
    │ │   ├── .recall(...)           (↗︎)     - Compute the recall score.                   │
    │ │   ├── .roc(...)                      - Plot the ROC curve.                         │
    │ │   ├── .roc_auc(...)          (↗︎)     - Compute the ROC AUC score.                  │
    │ │   ├── .timings(...)                  - Get all measured processing times related   │
    │ │   │   to the estimator.                                                            │
    │ │   ├── .custom_metric(...)            - Compute a custom metric.                    │
    │ │   └── .report_metrics(...)           - Report a set of metrics for our estimator.  │
    │ ├── .feature_importance                                                              │
    │ │   ├── .mean_decrease_impurity(...)   - Retrieve the mean decrease impurity (MDI)   │
    │ │   │   of a tree-based model.                                                       │
    │ │   └── .permutation(...)              - Report the permutation feature importance.  │
    │ ├── .cache_predictions(...)            - Cache estimator's predictions.              │
    │ ├── .clear_cache(...)                  - Clear the cache.                            │
    │ ├── .get_predictions(...)              - Get estimator's predictions.                │
    │ └── Attributes                                                                       │
    │     ├── .X_test                        - Testing data                                │
    │     ├── .X_train                       - Training data                               │
    │     ├── .y_test                        - Testing target                              │
    │     ├── .y_train                       - Training target                             │
    │     ├── .estimator_                    - The cloned or copied estimator              │
    │     ├── .estimator_name_               - The name of the estimator                   │
    │     ├── .fit_time_                     - The time taken to fit the estimator, in     │
    │     │   seconds                                                                      │
    │     └── .ml_task                       - No description available                    │
    │                                                                                      │
    │                                                                                      │
    │ Legend:                                                                              │
    │ (↗︎) higher is better (↘︎) lower is better                                             │
    ╰──────────────────────────────────────────────────────────────────────────────────────╯




.. GENERATED FROM PYTHON SOURCE LINES 70-77

.. note::
  This helper is great because:

  -   it enables users to get a glimpse at the API of the different available
      accessors without having to look up the online documentation,
  -   it provides methodological guidance: for example, we easily provide
      several metrics as a way to encourage users looking into them.

.. GENERATED FROM PYTHON SOURCE LINES 79-82

We can evaluate our model using the :meth:`~skore.EstimatorReport.metrics` accessor.
In particular, we can get the report metrics that is computed for us (including the
fit and prediction times):

.. GENERATED FROM PYTHON SOURCE LINES 84-86

.. code-block:: Python

    rf_report.metrics.report_metrics(pos_label=1)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>RandomForestClassifier</th>
        </tr>
        <tr>
          <th>Metric</th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>Precision</th>
          <td>0.988636</td>
        </tr>
        <tr>
          <th>Recall</th>
          <td>0.966667</td>
        </tr>
        <tr>
          <th>ROC AUC</th>
          <td>0.997589</td>
        </tr>
        <tr>
          <th>Brier score</th>
          <td>0.026124</td>
        </tr>
        <tr>
          <th>Fit time (s)</th>
          <td>0.176726</td>
        </tr>
        <tr>
          <th>Predict time (s)</th>
          <td>0.007298</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 87-89

For inspection, we can also retrieve the predictions, on the train set for example
(here we display only the first 10 predictions for conciseness purposes):

.. GENERATED FROM PYTHON SOURCE LINES 91-93

.. code-block:: Python

    rf_report.get_predictions(data_source="train", response_method="predict")[0:10]





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    array([1, 1, 0, 1, 0, 1, 1, 1, 1, 1])



.. GENERATED FROM PYTHON SOURCE LINES 94-95

We can also plot the ROC curve that is generated for us:

.. GENERATED FROM PYTHON SOURCE LINES 97-100

.. code-block:: Python

    roc_plot = rf_report.metrics.roc()
    roc_plot.plot()




.. image-sg:: /auto_examples/getting_started/images/sphx_glr_plot_skore_getting_started_001.png
   :alt: plot skore getting started
   :srcset: /auto_examples/getting_started/images/sphx_glr_plot_skore_getting_started_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 101-104

Furthermore, we can inspect our model using the
:meth:`~skore.EstimatorReport.feature_importance` accessor.
In particular, we can inspect the model using the permutation feature importance:

.. GENERATED FROM PYTHON SOURCE LINES 106-111

.. code-block:: Python

    import matplotlib.pyplot as plt

    rf_report.feature_importance.permutation(seed=0).T.boxplot(vert=False)
    plt.tight_layout()




.. image-sg:: /auto_examples/getting_started/images/sphx_glr_plot_skore_getting_started_002.png
   :alt: plot skore getting started
   :srcset: /auto_examples/getting_started/images/sphx_glr_plot_skore_getting_started_002.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 112-119

.. seealso::

  For more information about the motivation and usage of
  :class:`skore.EstimatorReport`, see the following use cases:

  -   :ref:`example_estimator_report` for model evaluation,
  -   :ref:`example_feature_importance` for model inspection.

.. GENERATED FROM PYTHON SOURCE LINES 121-126

Cross-validation with skore
^^^^^^^^^^^^^^^^^^^^^^^^^^^

skore has also (re-)implemented a :class:`skore.CrossValidationReport` class that
contains several :class:`skore.EstimatorReport`, one for each fold.

.. GENERATED FROM PYTHON SOURCE LINES 128-132

.. code-block:: Python

    from skore import CrossValidationReport

    cv_report = CrossValidationReport(rf, X, y, cv_splitter=5)








.. GENERATED FROM PYTHON SOURCE LINES 133-134

We display the cross-validation report helper:

.. GENERATED FROM PYTHON SOURCE LINES 136-138

.. code-block:: Python

    cv_report.help()





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    ╭───────────────── Tools to diagnose estimator RandomForestClassifier ─────────────────╮
    │ CrossValidationReport                                                                │
    │ ├── .metrics                                                                         │
    │ │   ├── .accuracy(...)         (↗︎)     - Compute the accuracy score.                 │
    │ │   ├── .brier_score(...)      (↘︎)     - Compute the Brier score.                    │
    │ │   ├── .log_loss(...)         (↘︎)     - Compute the log loss.                       │
    │ │   ├── .precision(...)        (↗︎)     - Compute the precision score.                │
    │ │   ├── .precision_recall(...)         - Plot the precision-recall curve.            │
    │ │   ├── .recall(...)           (↗︎)     - Compute the recall score.                   │
    │ │   ├── .roc(...)                      - Plot the ROC curve.                         │
    │ │   ├── .roc_auc(...)          (↗︎)     - Compute the ROC AUC score.                  │
    │ │   ├── .timings(...)                  - Get all measured processing times related   │
    │ │   │   to the estimator.                                                            │
    │ │   ├── .custom_metric(...)            - Compute a custom metric.                    │
    │ │   └── .report_metrics(...)           - Report a set of metrics for our estimator.  │
    │ ├── .cache_predictions(...)            - Cache the predictions for sub-estimators    │
    │ │   reports.                                                                         │
    │ ├── .clear_cache(...)                  - Clear the cache.                            │
    │ ├── .get_predictions(...)              - Get estimator's predictions.                │
    │ └── Attributes                                                                       │
    │     ├── .X                             - The data to fit                             │
    │     ├── .y                             - The target variable to try to predict in    │
    │     │   the case of supervised learning                                              │
    │     ├── .estimator_                    - The cloned or copied estimator              │
    │     ├── .estimator_name_               - The name of the estimator                   │
    │     ├── .estimator_reports_            - The estimator reports for each split        │
    │     └── .n_jobs                        - Number of jobs to run in parallel           │
    │                                                                                      │
    │                                                                                      │
    │ Legend:                                                                              │
    │ (↗︎) higher is better (↘︎) lower is better                                             │
    ╰──────────────────────────────────────────────────────────────────────────────────────╯




.. GENERATED FROM PYTHON SOURCE LINES 139-140

We display the mean and standard deviation for each metric:

.. GENERATED FROM PYTHON SOURCE LINES 142-144

.. code-block:: Python

    cv_report.metrics.report_metrics(pos_label=1)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead tr th {
            text-align: left;
        }

        .dataframe thead tr:last-of-type th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr>
          <th></th>
          <th colspan="2" halign="left">RandomForestClassifier</th>
        </tr>
        <tr>
          <th></th>
          <th>mean</th>
          <th>std</th>
        </tr>
        <tr>
          <th>Metric</th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>Precision</th>
          <td>0.967062</td>
          <td>0.028057</td>
        </tr>
        <tr>
          <th>Recall</th>
          <td>0.974726</td>
          <td>0.018424</td>
        </tr>
        <tr>
          <th>ROC AUC</th>
          <td>0.990675</td>
          <td>0.009470</td>
        </tr>
        <tr>
          <th>Brier score</th>
          <td>0.031268</td>
          <td>0.011754</td>
        </tr>
        <tr>
          <th>Fit time (s)</th>
          <td>0.179168</td>
          <td>0.002411</td>
        </tr>
        <tr>
          <th>Predict time (s)</th>
          <td>0.007134</td>
          <td>0.000055</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 145-146

or by individual fold:

.. GENERATED FROM PYTHON SOURCE LINES 148-150

.. code-block:: Python

    cv_report.metrics.report_metrics(aggregate=None, pos_label=1)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead tr th {
            text-align: left;
        }

        .dataframe thead tr:last-of-type th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr>
          <th></th>
          <th colspan="5" halign="left">RandomForestClassifier</th>
        </tr>
        <tr>
          <th></th>
          <th>Split #0</th>
          <th>Split #1</th>
          <th>Split #2</th>
          <th>Split #3</th>
          <th>Split #4</th>
        </tr>
        <tr>
          <th>Metric</th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>Precision</th>
          <td>0.943662</td>
          <td>0.933333</td>
          <td>1.000000</td>
          <td>0.972603</td>
          <td>0.985714</td>
        </tr>
        <tr>
          <th>Recall</th>
          <td>0.943662</td>
          <td>0.985915</td>
          <td>0.986111</td>
          <td>0.986111</td>
          <td>0.971831</td>
        </tr>
        <tr>
          <th>ROC AUC</th>
          <td>0.987226</td>
          <td>0.976089</td>
          <td>0.999008</td>
          <td>0.992560</td>
          <td>0.998491</td>
        </tr>
        <tr>
          <th>Brier score</th>
          <td>0.045161</td>
          <td>0.041514</td>
          <td>0.018967</td>
          <td>0.029430</td>
          <td>0.021269</td>
        </tr>
        <tr>
          <th>Fit time (s)</th>
          <td>0.177849</td>
          <td>0.176736</td>
          <td>0.182790</td>
          <td>0.178119</td>
          <td>0.180344</td>
        </tr>
        <tr>
          <th>Predict time (s)</th>
          <td>0.007193</td>
          <td>0.007075</td>
          <td>0.007188</td>
          <td>0.007091</td>
          <td>0.007122</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 151-152

We display the ROC curves for each fold:

.. GENERATED FROM PYTHON SOURCE LINES 154-157

.. code-block:: Python

    roc_plot_cv = cv_report.metrics.roc()
    roc_plot_cv.plot()




.. image-sg:: /auto_examples/getting_started/images/sphx_glr_plot_skore_getting_started_003.png
   :alt: plot skore getting started
   :srcset: /auto_examples/getting_started/images/sphx_glr_plot_skore_getting_started_003.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 158-160

We can retrieve the estimator report of a specific fold to investigate further,
for example getting the report metrics for the first fold only:

.. GENERATED FROM PYTHON SOURCE LINES 162-164

.. code-block:: Python

    cv_report.estimator_reports_[0].metrics.report_metrics(pos_label=1)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>RandomForestClassifier</th>
        </tr>
        <tr>
          <th>Metric</th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>Precision</th>
          <td>0.943662</td>
        </tr>
        <tr>
          <th>Recall</th>
          <td>0.943662</td>
        </tr>
        <tr>
          <th>ROC AUC</th>
          <td>0.987226</td>
        </tr>
        <tr>
          <th>Brier score</th>
          <td>0.045161</td>
        </tr>
        <tr>
          <th>Fit time (s)</th>
          <td>0.177849</td>
        </tr>
        <tr>
          <th>Predict time (s)</th>
          <td>0.007193</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 165-169

.. seealso::

  For more information about the motivation and usage of
  :class:`skore.CrossValidationReport`, see :ref:`example_use_case_employee_salaries`.

.. GENERATED FROM PYTHON SOURCE LINES 171-179

Comparing estimator reports
^^^^^^^^^^^^^^^^^^^^^^^^^^^

:class:`skore.ComparisonReport` enables users to compare several estimator reports
(corresponding to several estimators) on a same test set, as in a benchmark of
estimators.

Apart from the previous ``rf_report``, let us define another estimator report:

.. GENERATED FROM PYTHON SOURCE LINES 181-188

.. code-block:: Python

    from sklearn.ensemble import GradientBoostingClassifier

    gb = GradientBoostingClassifier(random_state=0)
    gb_report = EstimatorReport(
        gb, X_train=X_train, X_test=X_test, y_train=y_train, y_test=y_test
    )








.. GENERATED FROM PYTHON SOURCE LINES 189-191

We can conveniently compare our two estimator reports, that were applied to the exact
same test set:

.. GENERATED FROM PYTHON SOURCE LINES 193-197

.. code-block:: Python

    from skore import ComparisonReport

    comparator = ComparisonReport(reports=[rf_report, gb_report])








.. GENERATED FROM PYTHON SOURCE LINES 198-200

As for the :class:`~skore.EstimatorReport` and the
:class:`~skore.CrossValidationReport`, we have a helper:

.. GENERATED FROM PYTHON SOURCE LINES 202-204

.. code-block:: Python

    comparator.help()





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    ╭──────────────────────────── Tools to compare estimators ─────────────────────────────╮
    │ ComparisonReport                                                                     │
    │ ├── .metrics                                                                         │
    │ │   ├── .accuracy(...)         (↗︎)     - Compute the accuracy score.                 │
    │ │   ├── .brier_score(...)      (↘︎)     - Compute the Brier score.                    │
    │ │   ├── .log_loss(...)         (↘︎)     - Compute the log loss.                       │
    │ │   ├── .precision(...)        (↗︎)     - Compute the precision score.                │
    │ │   ├── .precision_recall(...)         - Plot the precision-recall curve.            │
    │ │   ├── .recall(...)           (↗︎)     - Compute the recall score.                   │
    │ │   ├── .roc(...)                      - Plot the ROC curve.                         │
    │ │   ├── .roc_auc(...)          (↗︎)     - Compute the ROC AUC score.                  │
    │ │   ├── .timings(...)                  - Get all measured processing times related   │
    │ │   │   to the different estimators.                                                 │
    │ │   ├── .custom_metric(...)            - Compute a custom metric.                    │
    │ │   └── .report_metrics(...)           - Report a set of metrics for the estimators. │
    │ ├── .cache_predictions(...)            - Cache the predictions for sub-estimators    │
    │ │   reports.                                                                         │
    │ ├── .clear_cache(...)                  - Clear the cache.                            │
    │ ├── .get_predictions(...)              - Get estimator's predictions.                │
    │ └── Attributes                                                                       │
    │     ├── .n_jobs                        - Number of jobs to run in parallel           │
    │     ├── .report_names_                 - The names of the compared estimators        │
    │     └── .reports_                      - :class:`~skore                              │
    │                                                                                      │
    │                                                                                      │
    │ Legend:                                                                              │
    │ (↗︎) higher is better (↘︎) lower is better                                             │
    ╰──────────────────────────────────────────────────────────────────────────────────────╯




.. GENERATED FROM PYTHON SOURCE LINES 205-206

Let us display the result of our benchmark:

.. GENERATED FROM PYTHON SOURCE LINES 208-210

.. code-block:: Python

    comparator.metrics.report_metrics(pos_label=1)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th>Estimator</th>
          <th>RandomForestClassifier</th>
          <th>GradientBoostingClassifier</th>
        </tr>
        <tr>
          <th>Metric</th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>Precision</th>
          <td>0.988636</td>
          <td>0.977528</td>
        </tr>
        <tr>
          <th>Recall</th>
          <td>0.966667</td>
          <td>0.966667</td>
        </tr>
        <tr>
          <th>ROC AUC</th>
          <td>0.997589</td>
          <td>0.997904</td>
        </tr>
        <tr>
          <th>Brier score</th>
          <td>0.026124</td>
          <td>0.021075</td>
        </tr>
        <tr>
          <th>Fit time (s)</th>
          <td>0.176726</td>
          <td>0.442174</td>
        </tr>
        <tr>
          <th>Predict time (s)</th>
          <td>0.007298</td>
          <td>0.001343</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 211-212

Thus, we easily have the result of our benchmark for several recommended metrics.

.. GENERATED FROM PYTHON SOURCE LINES 214-216

Moreover, we can display the ROC curve for the two estimator reports we want to
compare, by superimposing them on the same figure:

.. GENERATED FROM PYTHON SOURCE LINES 218-220

.. code-block:: Python

    comparator.metrics.roc().plot()




.. image-sg:: /auto_examples/getting_started/images/sphx_glr_plot_skore_getting_started_004.png
   :alt: plot skore getting started
   :srcset: /auto_examples/getting_started/images/sphx_glr_plot_skore_getting_started_004.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 221-228

Train-test split with skore
^^^^^^^^^^^^^^^^^^^^^^^^^^^

Skore has implemented a :func:`skore.train_test_split` function that wraps
scikit-learn's :func:`sklearn.model_selection.train_test_split`.

Let us load a dataset containing some time series data:

.. GENERATED FROM PYTHON SOURCE LINES 230-240

.. code-block:: Python

    import pandas as pd
    from skrub.datasets import fetch_employee_salaries

    dataset_employee = fetch_employee_salaries()
    X_employee, y_employee = dataset_employee.X, dataset_employee.y
    X_employee["date_first_hired"] = pd.to_datetime(
        X_employee["date_first_hired"], format="%m/%d/%Y"
    )
    X_employee.head(2)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Downloading 'employee_salaries' from https://github.com/skrub-data/skrub-data-files/raw/refs/heads/main/employee_salaries.zip (attempt 1/3)


.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>gender</th>
          <th>department</th>
          <th>department_name</th>
          <th>division</th>
          <th>assignment_category</th>
          <th>employee_position_title</th>
          <th>date_first_hired</th>
          <th>year_first_hired</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>F</td>
          <td>POL</td>
          <td>Department of Police</td>
          <td>MSB Information Mgmt and Tech Division Records...</td>
          <td>Fulltime-Regular</td>
          <td>Office Services Coordinator</td>
          <td>1986-09-22</td>
          <td>1986</td>
        </tr>
        <tr>
          <th>1</th>
          <td>M</td>
          <td>POL</td>
          <td>Department of Police</td>
          <td>ISB Major Crimes Division Fugitive Section</td>
          <td>Fulltime-Regular</td>
          <td>Master Police Officer</td>
          <td>1988-09-12</td>
          <td>1988</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 241-243

We can observe that there is a ``date_first_hired`` which is time-based.
Now, let us apply :func:`skore.train_test_split` on this data:

.. GENERATED FROM PYTHON SOURCE LINES 245-251

.. code-block:: Python

    import skore

    _ = skore.train_test_split(
        X=X_employee, y=y_employee, random_state=0, shuffle=False, as_dict=True
    )





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    ╭─────────────────────────────── TimeBasedColumnWarning ───────────────────────────────╮
    │ We detected some time-based columns (column "date_first_hired") in your data. We     │
    │ recommend using scikit-learn's TimeSeriesSplit instead of train_test_split.          │
    │ Otherwise you might train on future data to predict the past, or get inflated model  │
    │ performance evaluation because natural drift will not be taken into account.         │
    ╰──────────────────────────────────────────────────────────────────────────────────────╯




.. GENERATED FROM PYTHON SOURCE LINES 252-255

We get a ``TimeBasedColumnWarning`` advising us to use
:class:`sklearn.model_selection.TimeSeriesSplit` instead!
Indeed, we should not shuffle time-ordered data!

.. GENERATED FROM PYTHON SOURCE LINES 257-262

.. seealso::

  More methodological advice is available.
  For more information about the motivation and usage of
  :func:`skore.train_test_split`, see :ref:`example_train_test_split`.

.. GENERATED FROM PYTHON SOURCE LINES 264-269

Tracking: skore project
=======================

Another key feature of skore is its :class:`~skore.Project` that allows us to store
and retrieve items of many types.

.. GENERATED FROM PYTHON SOURCE LINES 271-273

Setup: creating and loading a skore project
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. GENERATED FROM PYTHON SOURCE LINES 275-277

Let us start by creating a skore project directory named ``my_project.skore`` in our
current directory:

.. GENERATED FROM PYTHON SOURCE LINES 279-282

.. code-block:: Python


    my_project = skore.Project("my_project")








.. GENERATED FROM PYTHON SOURCE LINES 292-298

Skore project: storing and retrieving some items
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Now that the project exists, we can store some useful items in it (in the same
directory) using :func:`~skore.Project.put`, with a "universal" key-value convention,
along with some annotations.

.. GENERATED FROM PYTHON SOURCE LINES 300-303

Let us store the accuracy and the estimator report of the random forest using
:meth:`~skore.Project.put`, along with some annotation to help us track our
experiments:

.. GENERATED FROM PYTHON SOURCE LINES 305-310

.. code-block:: Python

    my_project.put("accuracy", rf_report.metrics.accuracy(), note="random forest, float")
    my_project.put(
        "estimator_report", rf_report, note="random forest, skore estimator report"
    )








.. GENERATED FROM PYTHON SOURCE LINES 311-316

.. note ::
  With the skore :func:`~skore.Project.put`, there is no need to remember the API for
  saving or exporting each type of object: ``df.to_csv(...)``, ``plt.savefig(...)``,
  ``np.save(...)``, etc.
  There is also the unified :func:`~skore.Project.get` for loading items.

.. GENERATED FROM PYTHON SOURCE LINES 318-319

We can retrieve the value of an item using :meth:`~skore.Project.get`:

.. GENERATED FROM PYTHON SOURCE LINES 321-323

.. code-block:: Python

    my_project.get("accuracy")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    0.972027972027972



.. GENERATED FROM PYTHON SOURCE LINES 324-325

We can also retrieve the storage date and our annotation:

.. GENERATED FROM PYTHON SOURCE LINES 327-332

.. code-block:: Python

    from pprint import pprint

    accuracies = my_project.get("accuracy", metadata="all")
    pprint(accuracies)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    {'date': '2025-04-30T09:29:22.511490+00:00',
     'note': 'random forest, float',
     'value': 0.972027972027972}




.. GENERATED FROM PYTHON SOURCE LINES 333-338

.. seealso::

  For more information about the functionalities and the different types
  of items that we can store in a skore :class:`~skore.Project`,
  see :ref:`example_working_with_projects`.

.. GENERATED FROM PYTHON SOURCE LINES 340-342

Tracking the history of items
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. GENERATED FROM PYTHON SOURCE LINES 344-346

Now, for the gradient boosting model, let us store the same kinds of items
using the exact same keys, namely ``accuracy`` and ``estimator_report``:

.. GENERATED FROM PYTHON SOURCE LINES 348-355

.. code-block:: Python

    my_project.put(
        "accuracy", gb_report.metrics.accuracy(), note="gradient boosting, float"
    )
    my_project.put(
        "estimator_report", gb_report, note="gradient boosting, skore estimator report"
    )








.. GENERATED FROM PYTHON SOURCE LINES 356-358

Skore does not overwrite items with the same name (key): instead, it stores
their history so that nothing is lost:

.. GENERATED FROM PYTHON SOURCE LINES 360-363

.. code-block:: Python

    accuracies_history = my_project.get("accuracy", metadata="all", version="all")
    pprint(accuracies_history)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    [{'date': '2025-04-30T09:29:22.511490+00:00',
      'note': 'random forest, float',
      'value': 0.972027972027972},
     {'date': '2025-04-30T09:29:22.639013+00:00',
      'note': 'gradient boosting, float',
      'value': 0.965034965034965}]




.. GENERATED FROM PYTHON SOURCE LINES 367-372

.. note::
  These tracking functionalities are very useful to:

  -   never lose some key machine learning metrics,
  -   and observe the evolution over time / runs.

.. GENERATED FROM PYTHON SOURCE LINES 374-378

.. seealso::

  For more functionalities about the tracking of items using their history,
  see :ref:`example_tracking_items`.

.. GENERATED FROM PYTHON SOURCE LINES 380-388

.. admonition:: Stay tuned!

  These are only the initial features: skore is a work in progress and aims to be
  an end-to-end library for data scientists.

  Feedbacks are welcome: please feel free to join our
  `Discord <http://discord.probabl.ai>`_ or
  `create an issue <https://github.com/probabl-ai/skore/issues>`_.


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 4.189 seconds)


.. _sphx_glr_download_auto_examples_getting_started_plot_skore_getting_started.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_skore_getting_started.ipynb <plot_skore_getting_started.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_skore_getting_started.py <plot_skore_getting_started.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_skore_getting_started.zip <plot_skore_getting_started.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_