.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/getting_started/plot_getting_started.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_getting_started_plot_getting_started.py: .. _example_getting_started: ====================== Skore: getting started ====================== This guide illustrates how to use skore through a complete machine learning workflow for binary classification: #. Set up a proper experiment with training and test data #. Develop and evaluate multiple models using cross-validation #. Compare models to select the best one #. Validate the final model on held-out data #. Track and organize your machine learning results Throughout this guide, we will see how skore helps you: * Avoid common pitfalls with smart diagnostics * Quickly get rich insights into model performance * Organize and track your experiments Storing reports in Skore Hub ---------------------------- At the end of this example, we send the reports in Skore Hub (https://skore.probabl.ai/) that is a platform for storing, sharing and exploring your machine learning reports. To run this example and push in your own Skore Hub workspace and project, you can run this example with the following command: .. code-block:: bash WORKSPACE= PROJECT= python plot_getting_started.py In this gallery, we are going to push the different reports into a public workspace. .. GENERATED FROM PYTHON SOURCE LINES 42-50 Setting up our binary classification problem ============================================ Let's start by loading the German credit dataset, a classic binary classification problem where we predict the customer's credit risk ("good" or "bad"). This dataset contains various features about credit applicants, including personal information, credit history, and loan details. .. GENERATED FROM PYTHON SOURCE LINES 52-61 .. code-block:: Python import pandas as pd import skore from sklearn.datasets import fetch_openml from skrub import TableReport german_credit = fetch_openml(data_id=31, as_frame=True, parser="pandas") X, y = german_credit.data, german_credit.target TableReport(german_credit.frame) .. raw:: html

Column	Column name	dtype	Is sorted	Unique values	Mean	Std	Min	Median	Max
0	checking_status	CategoricalDtype	False	4 (0.4%)
1	duration	Int64DType	False	33 (3.3%)	20.9	12.1	4	18	72
2	credit_history	CategoricalDtype	False	5 (0.5%)
3	purpose	CategoricalDtype	False	10 (1.0%)
4	credit_amount	Int64DType	False	921 (92.1%)	3.27e+03	2.82e+03	250	2,320	18,424
5	savings_status	CategoricalDtype	False	5 (0.5%)
6	employment	CategoricalDtype	False	5 (0.5%)
7	installment_commitment	Int64DType	False	4 (0.4%)	2.97	1.12	1	3	4
8	personal_status	CategoricalDtype	False	4 (0.4%)
9	other_parties	CategoricalDtype	False	3 (0.3%)
10	residence_since	Int64DType	False	4 (0.4%)	2.85	1.10	1	3	4
11	property_magnitude	CategoricalDtype	False	4 (0.4%)
12	age	Int64DType	False	53 (5.3%)	35.5	11.4	19	33	75
13	other_payment_plans	CategoricalDtype	False	3 (0.3%)
14	housing	CategoricalDtype	False	3 (0.3%)
15	existing_credits	Int64DType	False	4 (0.4%)	1.41	0.578	1	1	4
16	job	CategoricalDtype	False	4 (0.4%)
17	num_dependents	Int64DType	False	2 (0.2%)	1.16	0.362	1	1	2
18	own_telephone	CategoricalDtype	False	2 (0.2%)
19	foreign_worker	CategoricalDtype	False	2 (0.2%)
20	class	CategoricalDtype	False	2 (0.2%)

Column 1	Column 2	Cramér's V	Pearson's Correlation
property_magnitude	housing	0.553
job	own_telephone	0.426
credit_history	existing_credits	0.378
checking_status	class	0.352
employment	job	0.311
age	num_dependents	0.309	0.118
personal_status	num_dependents	0.284
duration	credit_amount	0.281	0.625
age	housing	0.279
credit_amount	own_telephone	0.278
employment	residence_since	0.261
credit_history	class	0.248
residence_since	housing	0.237
employment	age	0.236
credit_amount	job	0.229
duration	class	0.224
purpose	own_telephone	0.221
duration	foreign_worker	0.216
credit_amount	property_magnitude	0.216
credit_history	other_payment_plans	0.215

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

.. GENERATED FROM PYTHON SOURCE LINES 62-73 Creating our experiment and held-out sets ----------------------------------------- We will use skore's enhanced :func:`~skore.train_test_split` function to create our experiment set and a left-out test set. The experiment set will be used for model development and cross-validation, while the left-out set will only be used at the end to validate our final model. Unlike scikit-learn's :func:`~skore.train_test_split`, skore's version provides helpful diagnostics about potential issues with your data split, such as class imbalance. .. GENERATED FROM PYTHON SOURCE LINES 75-79 .. code-block:: Python X_experiment, X_holdout, y_experiment, y_holdout = skore.train_test_split( X, y, random_state=42 ) .. rst-class:: sphx-glr-script-out .. code-block:: none ╭────────────────────── HighClassImbalanceTooFewExamplesWarning ───────────────────────╮ │ It seems that you have a classification problem with at least one class with fewer │ │ than 100 examples in the test set. In this case, using train_test_split may not be a │ │ good idea because of high variability in the scores obtained on the test set. We │ │ suggest three options to tackle this challenge: you can increase test_size, collect │ │ more data, or use skore's CrossValidationReport with the `splitter` parameter of │ │ your choice. │ ╰──────────────────────────────────────────────────────────────────────────────────────╯ ╭───────────────────────────────── ShuffleTrueWarning ─────────────────────────────────╮ │ We detected that the `shuffle` parameter is set to `True` either explicitly or from │ │ its default value. In case of time-ordered events (even if they are independent), │ │ this will result in inflated model performance evaluation because natural drift will │ │ not be taken into account. We recommend setting the shuffle parameter to `False` in │ │ order to ensure the evaluation process is really representative of your production │ │ release process. │ ╰──────────────────────────────────────────────────────────────────────────────────────╯ .. GENERATED FROM PYTHON SOURCE LINES 80-85 Skore tells us we have class-imbalance issues with our data, which we confirm with the :class:`~skore.TableReport` above by clicking on the "class" column and looking at the class distribution: there are only 300 examples where the target is "bad". The second warning concerns time-ordered data, but our data does not contain time-ordered columns so we can safely ignore it. .. GENERATED FROM PYTHON SOURCE LINES 87-101 Model development with cross-validation ======================================= We will investigate two different families of models using cross-validation. 1. A :class:`~sklearn.linear_model.LogisticRegression` which is a linear model 2. A :class:`~sklearn.ensemble.RandomForestClassifier` which is an ensemble of decision trees. In both cases, we rely on :func:`skrub.tabular_pipeline` to choose the proper preprocessing depending on the kind of model. Cross-validation is necessary to get a more reliable estimate of model performance. skore makes it easy through :class:`skore.CrossValidationReport`. .. GENERATED FROM PYTHON SOURCE LINES 103-109 Model no. 1: Linear regression with preprocessing ------------------------------------------------- Our first model will be a linear model, with automatic preprocessing of non-numeric data. Under the hood, skrub's :class:`~skrub.TableVectorizer` will adapt the preprocessing based on our choice to use a linear model. .. GENERATED FROM PYTHON SOURCE LINES 111-117 .. code-block:: Python from sklearn.linear_model import LogisticRegression from skrub import tabular_pipeline simple_model = tabular_pipeline(LogisticRegression()) simple_model .. raw:: html

Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(datetime=DatetimeEncoder(periodic_encoding='spline'))),
                    ('simpleimputer', SimpleImputer(add_indicator=True)),
                    ('squashingscaler', SquashingScaler(max_absolute_value=5)),
                    ('logisticregression', LogisticRegression())])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

.. GENERATED FROM PYTHON SOURCE LINES 118-122 We now evaluate our model with cross-validation, using :func:`~skore.evaluate` with `splitter=5` to perform 5-fold cross-validation. This returns a :class:`~skore.CrossValidationReport` object, which can be used to access the performance metrics and other information about the model. .. GENERATED FROM PYTHON SOURCE LINES 124-131 .. code-block:: Python from skore import evaluate simple_cv_report = evaluate( simple_model, X_experiment, y_experiment, pos_label="good", splitter=5 ) simple_cv_report .. raw:: html

Metric	mean	std
Accuracy	0.729333	0.050903
Precision	0.785632	0.034982
Recall	0.840934	0.050696
ROC AUC	0.750335	0.056447
Log loss	0.552742	0.073028
Brier score	0.184294	0.026786
Fit time (s)	0.078290	0.001014
Predict time (s)	0.055335	0.017234

Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(datetime=DatetimeEncoder(periodic_encoding='spline'))),
                    ('simpleimputer', SimpleImputer(add_indicator=True)),
                    ('squashingscaler', SquashingScaler(max_absolute_value=5)),
                    ('logisticregression', LogisticRegression())])

Column	Column name	dtype	Is sorted	Unique values	Mean	Std	Min	Median	Max
0	checking_status	CategoricalDtype	False	4 (0.5%)
1	duration	Int64DType	False	31 (4.1%)	21.2	11.8	4	18	60
2	credit_history	CategoricalDtype	False	5 (0.7%)
3	purpose	CategoricalDtype	False	10 (1.3%)
4	credit_amount	Int64DType	False	706 (94.1%)	3.39e+03	2.94e+03	250	2,331	18,424
5	savings_status	CategoricalDtype	False	5 (0.7%)
6	employment	CategoricalDtype	False	5 (0.7%)
7	installment_commitment	Int64DType	False	4 (0.5%)	2.98	1.12	1	3	4
8	personal_status	CategoricalDtype	False	4 (0.5%)
9	other_parties	CategoricalDtype	False	3 (0.4%)
10	residence_since	Int64DType	False	4 (0.5%)	2.85	1.10	1	3	4
11	property_magnitude	CategoricalDtype	False	4 (0.5%)
12	age	Int64DType	False	53 (7.1%)	35.4	11.2	19	33	75
13	other_payment_plans	CategoricalDtype	False	3 (0.4%)
14	housing	CategoricalDtype	False	3 (0.4%)
15	existing_credits	Int64DType	False	4 (0.5%)	1.42	0.578	1	1	4
16	job	CategoricalDtype	False	4 (0.5%)
17	num_dependents	Int64DType	False	2 (0.3%)	1.14	0.349	1	1	2
18	own_telephone	CategoricalDtype	False	2 (0.3%)
19	foreign_worker	CategoricalDtype	False	2 (0.3%)
20	class	CategoricalDtype	False	2 (0.3%)

Please enable javascript

1 issue(s) detected, 2 check(s) ran.

.. GENERATED FROM PYTHON SOURCE LINES 132-140 A report will quickly show important information regarding the performance of the model, the dataset used and the architecture of the model. This information is only a quick overview and one can dig deeper into the report to get more information. Indeed, Skore reports allow to structure the statistical information we look for when experimenting with predictive models. First, the :meth:`~skore.CrossValidationReport.help` method shows us all its available methods and attributes, with the knowledge that our model was trained for classification: .. GENERATED FROM PYTHON SOURCE LINES 142-144 .. code-block:: Python simple_cv_report.help() .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 145-146 For example, we can examine the training data, which excludes the held-out data: .. GENERATED FROM PYTHON SOURCE LINES 148-150 .. code-block:: Python simple_cv_report.data.analyze() .. raw:: html

Column	Column name	dtype	Is sorted	Unique values	Mean	Std	Min	Median	Max
0	checking_status	CategoricalDtype	False	4 (0.5%)
1	duration	Int64DType	False	31 (4.1%)	21.2	11.8	4	18	60
2	credit_history	CategoricalDtype	False	5 (0.7%)
3	purpose	CategoricalDtype	False	10 (1.3%)
4	credit_amount	Int64DType	False	706 (94.1%)	3.39e+03	2.94e+03	250	2,331	18,424
5	savings_status	CategoricalDtype	False	5 (0.7%)
6	employment	CategoricalDtype	False	5 (0.7%)
7	installment_commitment	Int64DType	False	4 (0.5%)	2.98	1.12	1	3	4
8	personal_status	CategoricalDtype	False	4 (0.5%)
9	other_parties	CategoricalDtype	False	3 (0.4%)
10	residence_since	Int64DType	False	4 (0.5%)	2.85	1.10	1	3	4
11	property_magnitude	CategoricalDtype	False	4 (0.5%)
12	age	Int64DType	False	53 (7.1%)	35.4	11.2	19	33	75
13	other_payment_plans	CategoricalDtype	False	3 (0.4%)
14	housing	CategoricalDtype	False	3 (0.4%)
15	existing_credits	Int64DType	False	4 (0.5%)	1.42	0.578	1	1	4
16	job	CategoricalDtype	False	4 (0.5%)
17	num_dependents	Int64DType	False	2 (0.3%)	1.14	0.349	1	1	2
18	own_telephone	CategoricalDtype	False	2 (0.3%)
19	foreign_worker	CategoricalDtype	False	2 (0.3%)
20	class	CategoricalDtype	False	2 (0.3%)

Column 1	Column 2	Cramér's V	Pearson's Correlation
property_magnitude	housing	0.558
job	own_telephone	0.438
credit_history	existing_credits	0.367
checking_status	class	0.348
age	num_dependents	0.322	0.132
employment	job	0.315
credit_amount	own_telephone	0.309
personal_status	num_dependents	0.304
duration	credit_amount	0.291	0.626
age	housing	0.270
duration	own_telephone	0.267
credit_history	class	0.265
employment	residence_since	0.257
purpose	own_telephone	0.253
duration	foreign_worker	0.253
credit_amount	job	0.242
employment	age	0.241
purpose	housing	0.231
duration	class	0.228
credit_amount	foreign_worker	0.227

Please enable javascript

.. GENERATED FROM PYTHON SOURCE LINES 151-153 But we can also quickly get an overview of the performance of our model, using :meth:`~skore.CrossValidationReport.metrics.summarize`: .. GENERATED FROM PYTHON SOURCE LINES 155-158 .. code-block:: Python simple_metrics = simple_cv_report.metrics.summarize() simple_metrics.frame(favorability=True) .. raw:: html

	LogisticRegression		Favorability
	mean	std
Metric
Accuracy	0.729333	0.050903	(↗︎)
Precision	0.785632	0.034982	(↗︎)
Recall	0.840934	0.050696	(↗︎)
ROC AUC	0.750335	0.056447	(↗︎)
Log loss	0.552742	0.073028	(↘︎)
Brier score	0.184294	0.026786	(↘︎)
Fit time (s)	0.078290	0.001014	(↘︎)
Predict time (s)	0.055335	0.017234	(↘︎)

.. GENERATED FROM PYTHON SOURCE LINES 159-163 .. note:: `favorability=True` adds a column showing whether higher or lower metric values are better. .. GENERATED FROM PYTHON SOURCE LINES 165-167 In addition to the summary of metrics, skore provides more advanced statistical information such as the precision-recall curve: .. GENERATED FROM PYTHON SOURCE LINES 169-172 .. code-block:: Python precision_recall = simple_cv_report.metrics.precision_recall() precision_recall.help() .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 173-178 .. note:: The output of :meth:`~skore.CrossValidationReport.metrics.precision_recall` is a :class:`~skore.Display` object. This is a common pattern in skore which allows us to access the information in several ways. .. GENERATED FROM PYTHON SOURCE LINES 180-181 We can visualize the critical information as a plot, with only a few lines of code: .. GENERATED FROM PYTHON SOURCE LINES 183-185 .. code-block:: Python precision_recall.plot() .. rst-class:: sphx-glr-script-out .. code-block:: none

.. GENERATED FROM PYTHON SOURCE LINES 186-187 Or we can access the raw information as a dataframe if additional analysis is needed: .. GENERATED FROM PYTHON SOURCE LINES 189-191 .. code-block:: Python precision_recall.frame() .. raw:: html

	split	threshold	precision	recall
0	0	0.110277	0.700000	1.000000
1	0	0.215387	0.744681	1.000000
2	0	0.238563	0.742857	0.990476
3	0	0.248819	0.741007	0.980952
4	0	0.273961	0.739130	0.971429
...	...	...	...	...
660	4	0.982595	1.000000	0.048077
661	4	0.988545	1.000000	0.038462
662	4	0.989817	1.000000	0.028846
663	4	0.994946	1.000000	0.019231
664	4	0.995636	1.000000	0.009615

665 rows × 4 columns

.. GENERATED FROM PYTHON SOURCE LINES 192-193 As another example, we can plot the confusion matrix with the same consistent API: .. GENERATED FROM PYTHON SOURCE LINES 195-198 .. code-block:: Python confusion_matrix = simple_cv_report.metrics.confusion_matrix() confusion_matrix.plot() .. rst-class:: sphx-glr-script-out .. code-block:: none

.. GENERATED FROM PYTHON SOURCE LINES 199-201 Skore also provides utilities to inspect models. Since our model is a linear model, we can study the importance that it gives to each feature: .. GENERATED FROM PYTHON SOURCE LINES 203-206 .. code-block:: Python coefficients = simple_cv_report.inspection.coefficients() coefficients.frame() .. raw:: html

	feature	coefficient_mean	coefficient_std
0	Intercept	1.221919	0.145022
1	checking_status_0<=X<200	-0.356274	0.116078
2	checking_status_<0	-0.712660	0.098075
3	checking_status_>=200	0.216352	0.220024
4	checking_status_no checking	0.957280	0.117765
5	duration	-0.299406	0.085176
6	credit_history_all paid	-0.456190	0.110763
7	credit_history_critical/other existing credit	0.655752	0.152976
8	credit_history_delayed previously	0.104324	0.117658
9	credit_history_existing paid	-0.267659	0.095892
10	credit_history_no credits/all paid	-0.383078	0.178751
11	purpose_business	-0.042625	0.146252
12	purpose_domestic appliance	-0.128409	0.223069
13	purpose_education	-0.504801	0.218228
14	purpose_furniture/equipment	0.088990	0.063633
15	purpose_new car	-0.410305	0.084358
16	purpose_other	0.069206	0.138734
17	purpose_radio/tv	0.138978	0.075132
18	purpose_repairs	-0.271701	0.122305
19	purpose_retraining	0.448025	0.236986
20	purpose_used car	0.469388	0.116847
21	credit_amount	-0.383071	0.145635
22	savings_status_100<=X<500	-0.010966	0.098748
23	savings_status_500<=X<1000	0.085306	0.085370
24	savings_status_<100	-0.333553	0.109446
25	savings_status_>=1000	0.370625	0.316363
26	savings_status_no known savings	0.186693	0.082206
27	employment_1<=X<4	-0.062535	0.085583
28	employment_4<=X<7	0.161965	0.094966
29	employment_<1	-0.091905	0.094219
30	employment_>=7	0.075963	0.109892
31	employment_unemployed	-0.050175	0.131802
32	installment_commitment	-0.627063	0.178107
33	personal_status_female div/dep/mar	-0.166590	0.081708
34	personal_status_male div/sep	-0.220082	0.171560
35	personal_status_male mar/wid	-0.087428	0.061253
36	personal_status_male single	0.328973	0.116731
37	other_parties_co applicant	-0.122421	0.066232
38	other_parties_guarantor	0.276689	0.046478
39	other_parties_none	-0.154268	0.064498
40	residence_since	-0.070676	0.080168
41	property_magnitude_car	-0.012530	0.106496
42	property_magnitude_life insurance	-0.110668	0.086645
43	property_magnitude_no known property	-0.216614	0.087059
44	property_magnitude_real estate	0.184397	0.084662
45	age	0.445303	0.102326
46	other_payment_plans_bank	-0.128049	0.084283
47	other_payment_plans_none	0.200376	0.060104
48	other_payment_plans_stores	-0.072327	0.111137
49	housing_for free	-0.108021	0.141465
50	housing_own	0.149660	0.058071
51	housing_rent	-0.175394	0.053475
52	existing_credits	-0.293875	0.160032
53	job_high qualif/self emp/mgmt	-0.146369	0.138251
54	job_skilled	-0.069226	0.146688
55	job_unemp/unskilled non res	0.245873	0.372472
56	job_unskilled resident	0.031590	0.083569
57	num_dependents	-0.097292	0.102585
58	own_telephone_yes	0.326305	0.141762
59	foreign_worker_yes	-0.680383	0.183700

.. GENERATED FROM PYTHON SOURCE LINES 207-209 .. code-block:: Python coefficients.plot(select_k=15) .. rst-class:: sphx-glr-script-out .. code-block:: none

.. GENERATED FROM PYTHON SOURCE LINES 210-217 Model no. 2: Random forest -------------------------- Now, we cross-validate a more advanced model using :class:`~sklearn.ensemble.RandomForestClassifier`. Again, we rely on :func:`~skrub.tabular_pipeline` to perform the appropriate preprocessing to use with this model. .. GENERATED FROM PYTHON SOURCE LINES 219-224 .. code-block:: Python from sklearn.ensemble import RandomForestClassifier advanced_model = tabular_pipeline(RandomForestClassifier(random_state=0)) advanced_model .. raw:: html

Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(low_cardinality=OrdinalEncoder(handle_unknown='use_encoded_value',
                                                                    unknown_value=-1))),
                    ('randomforestclassifier',
                     RandomForestClassifier(random_state=0))])

.. GENERATED FROM PYTHON SOURCE LINES 225-230 .. code-block:: Python advanced_cv_report = evaluate( advanced_model, X_experiment, y_experiment, pos_label="good", splitter=5 ) advanced_cv_report .. raw:: html

Metric	mean	std
Accuracy	0.745333	0.032796
Precision	0.779443	0.018644
Recall	0.885037	0.053558
ROC AUC	0.773334	0.034190
Log loss	0.508369	0.028632
Brier score	0.169911	0.010967
Fit time (s)	0.192235	0.000926
Predict time (s)	0.044057	0.000340

Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(low_cardinality=OrdinalEncoder(handle_unknown='use_encoded_value',
                                                                    unknown_value=-1))),
                    ('randomforestclassifier',
                     RandomForestClassifier(random_state=0))])

Column	Column name	dtype	Is sorted	Unique values	Mean	Std	Min	Median	Max
0	checking_status	CategoricalDtype	False	4 (0.5%)
1	duration	Int64DType	False	31 (4.1%)	21.2	11.8	4	18	60
2	credit_history	CategoricalDtype	False	5 (0.7%)
3	purpose	CategoricalDtype	False	10 (1.3%)
4	credit_amount	Int64DType	False	706 (94.1%)	3.39e+03	2.94e+03	250	2,331	18,424
5	savings_status	CategoricalDtype	False	5 (0.7%)
6	employment	CategoricalDtype	False	5 (0.7%)
7	installment_commitment	Int64DType	False	4 (0.5%)	2.98	1.12	1	3	4
8	personal_status	CategoricalDtype	False	4 (0.5%)
9	other_parties	CategoricalDtype	False	3 (0.4%)
10	residence_since	Int64DType	False	4 (0.5%)	2.85	1.10	1	3	4
11	property_magnitude	CategoricalDtype	False	4 (0.5%)
12	age	Int64DType	False	53 (7.1%)	35.4	11.2	19	33	75
13	other_payment_plans	CategoricalDtype	False	3 (0.4%)
14	housing	CategoricalDtype	False	3 (0.4%)
15	existing_credits	Int64DType	False	4 (0.5%)	1.42	0.578	1	1	4
16	job	CategoricalDtype	False	4 (0.5%)
17	num_dependents	Int64DType	False	2 (0.3%)	1.14	0.349	1	1	2
18	own_telephone	CategoricalDtype	False	2 (0.3%)
19	foreign_worker	CategoricalDtype	False	2 (0.3%)
20	class	CategoricalDtype	False	2 (0.3%)

Please enable javascript

1 issue(s) detected, 2 check(s) ran.

.. GENERATED FROM PYTHON SOURCE LINES 231-232 We will now compare this new model with the previous one. .. GENERATED FROM PYTHON SOURCE LINES 234-240 Comparing our models ==================== Now that we have our two models, we need to decide which one should go into production. We can compare them with the :func:`~skore.compare` function that returns a :class:`~skore.ComparisonReport`: .. GENERATED FROM PYTHON SOURCE LINES 242-252 .. code-block:: Python from skore import compare comparison = compare( { "Simple Linear Model": simple_cv_report, "Advanced Pipeline": advanced_cv_report, }, ) comparison .. raw:: html

Metric	Advanced Pipeline	Simple Linear Model	Advanced Pipeline	Simple Linear Model
Accuracy	0.745333	0.729333	0.032796	0.050903
Precision	0.779443	0.785632	0.018644	0.034982
Recall	0.885037	0.840934	0.053558	0.050696
ROC AUC	0.773334	0.750335	0.034190	0.056447
Log loss	0.508369	0.552742	0.028632	0.073028
Brier score	0.169911	0.184294	0.010967	0.026786
Fit time (s)	0.192235	0.078290	0.000926	0.001014
Predict time (s)	0.044057	0.055335	0.000340	0.017234

Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(datetime=DatetimeEncoder(periodic_encoding='spline'))),
                    ('simpleimputer', SimpleImputer(add_indicator=True)),
                    ('squashingscaler', SquashingScaler(max_absolute_value=5)),
                    ('logisticregression', LogisticRegression())])

Please enable javascript

1 issue(s) detected, 2 check(s) ran.

Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(low_cardinality=OrdinalEncoder(handle_unknown='use_encoded_value',
                                                                    unknown_value=-1))),
                    ('randomforestclassifier',
                     RandomForestClassifier(random_state=0))])

Please enable javascript

1 issue(s) detected, 2 check(s) ran.

.. GENERATED FROM PYTHON SOURCE LINES 253-254 This report follows the same API as :class:`~skore.CrossValidationReport`: .. GENERATED FROM PYTHON SOURCE LINES 254-256 .. code-block:: Python comparison.help() .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 257-259 We have access to the same tools to perform statistical analysis and compare both models: .. GENERATED FROM PYTHON SOURCE LINES 259-262 .. code-block:: Python comparison_metrics = comparison.metrics.summarize() comparison_metrics.frame(favorability=True) .. raw:: html

	mean		std		Favorability
Estimator	Advanced Pipeline	Simple Linear Model	Advanced Pipeline	Simple Linear Model
Metric
Accuracy	0.745333	0.729333	0.032796	0.050903	(↗︎)
Precision	0.779443	0.785632	0.018644	0.034982	(↗︎)
Recall	0.885037	0.840934	0.053558	0.050696	(↗︎)
ROC AUC	0.773334	0.750335	0.034190	0.056447	(↗︎)
Log loss	0.508369	0.552742	0.028632	0.073028	(↘︎)
Brier score	0.169911	0.184294	0.010967	0.026786	(↘︎)
Fit time (s)	0.192235	0.078290	0.000926	0.001014	(↘︎)
Predict time (s)	0.044057	0.055335	0.000340	0.017234	(↘︎)

.. GENERATED FROM PYTHON SOURCE LINES 263-265 .. code-block:: Python comparison.metrics.precision_recall().plot() .. rst-class:: sphx-glr-script-out .. code-block:: none

.. GENERATED FROM PYTHON SOURCE LINES 266-271 Based on the previous tables and plots, it seems that the :class:`~sklearn.ensemble.RandomForestClassifier` model has slightly better performance. For the purposes of this guide however, we make the arbitrary choice to deploy the linear model to make a comparison with the coefficients study shown earlier. .. GENERATED FROM PYTHON SOURCE LINES 273-280 Final model evaluation on held-out data ======================================= Now that we have chosen to deploy the linear model, we will train it on the full experiment set and evaluate it on our held-out data: training on more data should help performance and we can also validate that our model generalizes well to new data. This can be done in one step with :meth:`~skore.ComparisonReport.create_estimator_report`. .. GENERATED FROM PYTHON SOURCE LINES 282-288 .. code-block:: Python final_report = comparison.create_estimator_report( report_key="Simple Linear Model", X_test=X_holdout, y_test=y_holdout ) final_report .. raw:: html

Please enable javascript

0 issue(s) detected, 2 check(s) ran.

.. GENERATED FROM PYTHON SOURCE LINES 289-291 This returns a :class:`~skore.EstimatorReport` which has a similar API to the other report classes: .. GENERATED FROM PYTHON SOURCE LINES 293-296 .. code-block:: Python final_metrics = final_report.metrics.summarize() final_metrics.frame() .. raw:: html

	LogisticRegression
Metric
Accuracy	0.764000
Precision	0.808290
Recall	0.876404
ROC AUC	0.809613
Log loss	0.467859
Brier score	0.153900
Fit time (s)	0.079347
Predict time (s)	0.047591

.. GENERATED FROM PYTHON SOURCE LINES 297-299 .. code-block:: Python final_report.metrics.confusion_matrix().plot() .. rst-class:: sphx-glr-script-out .. code-block:: none

.. GENERATED FROM PYTHON SOURCE LINES 300-304 We can easily combine the results of the previous cross-validation together with the evaluation on the held-out dataset, since the two are accessible as dataframes. This way, we can check if our chosen model meets the expectations we set during the experiment phase. .. GENERATED FROM PYTHON SOURCE LINES 306-311 .. code-block:: Python pd.concat( [final_metrics.frame(), simple_cv_report.metrics.summarize().frame()], axis="columns", ) .. raw:: html

	LogisticRegression	(LogisticRegression, mean)	(LogisticRegression, std)
Metric
Accuracy	0.764000	0.729333	0.050903
Precision	0.808290	0.785632	0.034982
Recall	0.876404	0.840934	0.050696
ROC AUC	0.809613	0.750335	0.056447
Log loss	0.467859	0.552742	0.073028
Brier score	0.153900	0.184294	0.026786
Fit time (s)	0.079347	0.078290	0.001014
Predict time (s)	0.047591	0.055335	0.017234

.. GENERATED FROM PYTHON SOURCE LINES 312-314 As expected, our final model gets better performance, likely thanks to the larger training set. .. GENERATED FROM PYTHON SOURCE LINES 316-318 Our final sanity check is to compare the features considered most impactful between our final model and the cross-validation: .. GENERATED FROM PYTHON SOURCE LINES 320-336 .. code-block:: Python final_coefficients = final_report.inspection.coefficients() cv_coefficients = simple_cv_report.inspection.coefficients() features_final_coefficients = final_coefficients.frame(select_k=15)["feature"] features_cv_coefficients = cv_coefficients.frame(select_k=15)["feature"] print( f"Most important features available in both models: " f"{set(features_final_coefficients).intersection(set(features_cv_coefficients))}" ) print( f"Most important features available in final model but not in cross-validation: " f"{set(features_final_coefficients).difference(set(features_cv_coefficients))}" ) .. rst-class:: sphx-glr-script-out .. code-block:: none Most important features available in both models: {'purpose_retraining', 'credit_history_all paid', 'purpose_used car', 'Intercept', 'foreign_worker_yes', 'credit_amount', 'credit_history_no credits/all paid', 'installment_commitment', 'checking_status_no checking', 'checking_status_<0', 'credit_history_critical/other existing credit', 'purpose_education', 'age', 'purpose_new car'} Most important features available in final model but not in cross-validation: {'checking_status_0<=X<200'} .. GENERATED FROM PYTHON SOURCE LINES 337-339 We can further check if there is a drastic difference in the ordering by plotting those features with the largest absolute coefficients. .. GENERATED FROM PYTHON SOURCE LINES 341-344 .. code-block:: Python final_coefficients.plot(select_k=15, sorting_order="descending") cv_coefficients.plot(select_k=15, sorting_order="descending") .. rst-class:: sphx-glr-script-out .. code-block:: none

.. GENERATED FROM PYTHON SOURCE LINES 345-346 They seem very similar, so we are done! .. GENERATED FROM PYTHON SOURCE LINES 348-368 Tracking our work with a skore Project ====================================== Now that we have completed our modeling workflow, we should store our models in a safe place for future work. Indeed, if this research notebook were modified, we would no longer be able to relate the current production model to the code that generated it. We can use a :class:`skore.Project` to keep track of our experiments. This makes it easy to organize, retrieve, and compare models over time. Usually this would be done as you go along the model development, but in the interest of simplicity we kept this until the end. We are using Skore Hub (https://skore.probabl.ai/) to store and review our reports. .. note:: Here, we are using Skore Hub to store and analyze the reports that we computed. Note that you can store reports as well locally using `mode="local"` when creating or loading projects via `skore.Project`. .. GENERATED FROM PYTHON SOURCE LINES 368-375 .. code-block:: Python from skore import login login() .. rst-class:: sphx-glr-script-out .. code-block:: none ╭───────────────────────────────── Login to Skore Hub ─────────────────────────────────╮ │ │ │ Successfully logged in, using API key. │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────╯ .. GENERATED FROM PYTHON SOURCE LINES 410-411 We load or create a hub project: .. GENERATED FROM PYTHON SOURCE LINES 411-414 .. code-block:: Python project = Project(f"{WORKSPACE}/{PROJECT}", mode="hub") .. GENERATED FROM PYTHON SOURCE LINES 415-416 We store our reports with descriptive keys: .. GENERATED FROM PYTHON SOURCE LINES 416-419 .. code-block:: Python project.put("simple_linear_model_cv", simple_cv_report) .. rst-class:: sphx-glr-script-out .. code-block:: none Putting simple_linear_model_cv 0:03:37 Consult your report at https://skore.probabl.ai/skore/example-getting-started-0.16/cross-validations/6636 .. GENERATED FROM PYTHON SOURCE LINES 420-422 .. code-block:: Python project.put("advanced_pipeline_cv", advanced_cv_report) .. rst-class:: sphx-glr-script-out .. code-block:: none Putting advanced_pipeline_cv 0:03:34 Consult your report at https://skore.probabl.ai/skore/example-getting-started-0.16/cross-validations/6642 .. GENERATED FROM PYTHON SOURCE LINES 423-425 In this example, we created a read-only Skore Hub project that you can visit by clicking on the link above and explore the reports. .. GENERATED FROM PYTHON SOURCE LINES 427-428 Now we can retrieve a summary of our stored reports: .. GENERATED FROM PYTHON SOURCE LINES 430-434 .. code-block:: Python summary = project.summarize() # Uncomment the next line to display the widget in an interactive environment: # summary .. GENERATED FROM PYTHON SOURCE LINES 435-447 .. note:: Calling `summary` in a Jupyter notebook cell will show the following parallel coordinate plot to help you select models that you want to retrieve: .. image:: /_static/images/screenshot_getting_started.png :alt: Screenshot of the widget in a Jupyter notebook Each line represents a model, and we can select models by clicking on lines or dragging on metric axes to filter by performance. In the screenshot, we selected only the cross-validation reports; this allows us to retrieve exactly those reports programmatically. .. GENERATED FROM PYTHON SOURCE LINES 449-453 Supposing you selected "Cross-validation" in the "Report type" tab, if you now call :meth:`~skore.project._summary.Summary.reports`, you get only the :class:`~skore.CrossValidationReport` objects, which you can directly put in the form of a :class:`~skore.ComparisonReport`: .. GENERATED FROM PYTHON SOURCE LINES 455-460 .. code-block:: Python new_report = summary.reports(return_as="comparison") new_report.help() .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 465-479 .. admonition:: Stay tuned! This is only the beginning for skore. We welcome your feedback and ideas to make it the best tool for end-to-end data science. Key benefits of using skore in your ML workflow: * Standardized evaluation and comparison of models * Rich visualizations and diagnostics * Organized experiment tracking * Seamless integration with scikit-learn Feel free to join our community on `Discord `_ or `create an issue `_. .. rst-class:: sphx-glr-timing **Total running time of the script:** (7 minutes 49.382 seconds) .. _sphx_glr_download_auto_examples_getting_started_plot_getting_started.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_getting_started.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_getting_started.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_getting_started.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_

	steps steps: list of tuples List of (name of step, estimator) tuples that are to be chained in sequential order. To be compatible with the scikit-learn API, all steps must define `fit`. All non-last steps must also define `transform`. See :ref:`Combining Estimators ` for more details.	[('tablevectorizer', ...), ('simpleimputer', ...), ...]
	transform_input transform_input: list of str, default=None The names of the :term:`metadata` parameters that should be transformed by the pipeline before passing it to the step consuming it. This enables transforming some input arguments to ``fit`` (other than ``X``) to be transformed by the steps of the pipeline up to the step which requires them. Requirement is defined via :ref:`metadata routing `. For instance, this can be used to pass a validation set through the pipeline. You can only set this if metadata routing is enabled, which you can enable using ``sklearn.set_config(enable_metadata_routing=True)``. .. versionadded:: 1.6	None
	memory memory: str or object with the joblib.Memory interface, default=None Used to cache the fitted transformers of the pipeline. The last step will never be cached, even if it is a transformer. By default, no caching is performed. If a string is given, it is the path to the caching directory. Enabling caching triggers a clone of the transformers before fitting. Therefore, the transformer instance given to the pipeline cannot be inspected directly. Use the attribute ``named_steps`` or ``steps`` to inspect estimators within the pipeline. Caching the transformers is advantageous when fitting is time consuming. See :ref:`sphx_glr_auto_examples_neighbors_plot_caching_nearest_neighbors.py` for an example on how to enable caching.	None
	verbose verbose: bool, default=False If True, the time elapsed while fitting each step will be printed as it is completed.	False

	cardinality_threshold	40
	low_cardinality	OneHotEncoder..._output=False)
	high_cardinality	StringEncoder()
	numeric	PassThrough()
	datetime	DatetimeEncod...ding='spline')
	specific_transformers	()
	drop_null_fraction	1.0
	drop_if_constant	False
	drop_if_unique	False
	datetime_format	None
	null_strings	None
	n_jobs	None

	categories categories: 'auto' or a list of array-like, default='auto' Categories (unique values) per feature: - 'auto' : Determine categories automatically from the training data. - list : ``categories[i]`` holds the categories expected in the ith column. The passed categories should not mix strings and numeric values within a single feature, and should be sorted in case of numeric values. The used categories can be found in the ``categories_`` attribute. .. versionadded:: 0.20	'auto'
	drop drop: {'first', 'if_binary'} or an array-like of shape (n_features,), default=None Specifies a methodology to use to drop one of the categories per feature. This is useful in situations where perfectly collinear features cause problems, such as when feeding the resulting data into an unregularized linear regression model. However, dropping one category breaks the symmetry of the original representation and can therefore induce a bias in downstream models, for instance for penalized linear classification or regression models. - None : retain all features (the default). - 'first' : drop the first category in each feature. If only one category is present, the feature will be dropped entirely. - 'if_binary' : drop the first category in each feature with two categories. Features with 1 or more than 2 categories are left intact. - array : ``drop[i]`` is the category in feature ``X[:, i]`` that should be dropped. When `max_categories` or `min_frequency` is configured to group infrequent categories, the dropping behavior is handled after the grouping. .. versionadded:: 0.21 The parameter `drop` was added in 0.21. .. versionchanged:: 0.23 The option `drop='if_binary'` was added in 0.23. .. versionchanged:: 1.1 Support for dropping infrequent categories.	'if_binary'
	sparse_output sparse_output: bool, default=True When ``True``, it returns a :class:`scipy.sparse.csr_matrix`, i.e. a sparse matrix in "Compressed Sparse Row" (CSR) format. .. versionadded:: 1.2 `sparse` was renamed to `sparse_output`	False
	dtype dtype: number type, default=np.float64 Desired dtype of output.	'float32'
	handle_unknown handle_unknown: {'error', 'ignore', 'infrequent_if_exist', 'warn'}, default='error' Specifies the way unknown categories are handled during :meth:`transform`. - 'error' : Raise an error if an unknown category is present during transform. - 'ignore' : When an unknown category is encountered during transform, the resulting one-hot encoded columns for this feature will be all zeros. In the inverse transform, an unknown category will be denoted as None. - 'infrequent_if_exist' : When an unknown category is encountered during transform, the resulting one-hot encoded columns for this feature will map to the infrequent category if it exists. The infrequent category will be mapped to the last position in the encoding. During inverse transform, an unknown category will be mapped to the category denoted `'infrequent'` if it exists. If the `'infrequent'` category does not exist, then :meth:`transform` and :meth:`inverse_transform` will handle an unknown category as with `handle_unknown='ignore'`. Infrequent categories exist based on `min_frequency` and `max_categories`. Read more in the :ref:`User Guide `. - 'warn' : When an unknown category is encountered during transform a warning is issued, and the encoding then proceeds as described for `handle_unknown="infrequent_if_exist"`. .. versionchanged:: 1.1 `'infrequent_if_exist'` was added to automatically handle unknown categories and infrequent categories. .. versionadded:: 1.6 The option `"warn"` was added in 1.6.	'ignore'
	min_frequency min_frequency: int or float, default=None Specifies the minimum frequency below which a category will be considered infrequent. - If `int`, categories with a smaller cardinality will be considered infrequent. - If `float`, categories with a smaller cardinality than `min_frequency * n_samples` will be considered infrequent. .. versionadded:: 1.1 Read more in the :ref:`User Guide `.	None
	max_categories max_categories: int, default=None Specifies an upper limit to the number of output features for each input feature when considering infrequent categories. If there are infrequent categories, `max_categories` includes the category representing the infrequent categories along with the frequent categories. If `None`, there is no limit to the number of output features. .. versionadded:: 1.1 Read more in the :ref:`User Guide `.	None
	feature_name_combiner feature_name_combiner: "concat" or callable, default="concat" Callable with signature `def callable(input_feature, category)` that returns a string. This is used to create feature names to be returned by :meth:`get_feature_names_out`. `"concat"` concatenates encoded feature name and category with `feature + "_" + str(category)`.E.g. feature X with values 1, 6, 7 create feature names `X_1, X_6, X_7`. .. versionadded:: 1.3	'concat'

	n_components	30
	vectorizer	'tfidf'
	ngram_range	(3, ...)
	analyzer	'char_wb'
	stop_words	None
	random_state	None
	vocabulary	None

	missing_values missing_values: int, float, str, np.nan, None or pandas.NA, default=np.nan The placeholder for the missing values. All occurrences of `missing_values` will be imputed. For pandas' dataframes with nullable integer dtypes with missing values, `missing_values` can be set to either `np.nan` or `pd.NA`.	nan
	strategy strategy: str or Callable, default='mean' The imputation strategy. - If "mean", then replace missing values using the mean along each column. Can only be used with numeric data. - If "median", then replace missing values using the median along each column. Can only be used with numeric data. - If "most_frequent", then replace missing using the most frequent value along each column. Can be used with strings or numeric data. If there is more than one such value, only the smallest is returned. - If "constant", then replace missing values with fill_value. Can be used with strings or numeric data. - If an instance of Callable, then replace missing values using the scalar statistic returned by running the callable over a dense 1d array containing non-missing values of each column. .. versionadded:: 0.20 strategy="constant" for fixed value imputation. .. versionadded:: 1.5 strategy=callable for custom value imputation.	'mean'
	fill_value fill_value: str or numerical value, default=None When strategy == "constant", `fill_value` is used to replace all occurrences of missing_values. For string or object data types, `fill_value` must be a string. If `None`, `fill_value` will be 0 when imputing numerical data and "missing_value" for strings or object data types.	None
	copy copy: bool, default=True If True, a copy of X will be created. If False, imputation will be done in-place whenever possible. Note that, in the following cases, a new copy will always be made, even if `copy=False`: - If `X` is not an array of floating values; - If `X` is encoded as a CSR matrix; - If `add_indicator=True`.	True
	add_indicator add_indicator: bool, default=False If True, a :class:`MissingIndicator` transform will stack onto output of the imputer's transform. This allows a predictive estimator to account for missingness despite imputation. If a feature has no missing values at fit/train time, the feature won't appear on the missing indicator even if there are missing values at transform/test time.	True
	keep_empty_features keep_empty_features: bool, default=False If True, features that consist exclusively of missing values when `fit` is called are returned in results when `transform` is called. The imputed value is always `0` except when `strategy="constant"` in which case `fill_value` will be used instead. .. versionadded:: 1.2	False

	resolution	'hour'
	add_weekday	False
	add_total_seconds	True
	add_day_of_year	False
	periodic_encoding	'spline'

	penalty penalty: {'l1', 'l2', 'elasticnet', None}, default='l2' Specify the norm of the penalty: - `None`: no penalty is added; - `'l2'`: add a L2 penalty term and it is the default choice; - `'l1'`: add a L1 penalty term; - `'elasticnet'`: both L1 and L2 penalty terms are added. .. warning:: Some penalties may not work with some solvers. See the parameter `solver` below, to know the compatibility between the penalty and solver. .. versionadded:: 0.19 l1 penalty with SAGA solver (allowing 'multinomial' + L1) .. deprecated:: 1.8 `penalty` was deprecated in version 1.8 and will be removed in 1.10. Use `l1_ratio` instead. `l1_ratio=0` for `penalty='l2'`, `l1_ratio=1` for `penalty='l1'` and `l1_ratio` set to any float between 0 and 1 for `'penalty='elasticnet'`.	'deprecated'
	C C: float, default=1.0 Inverse of regularization strength; must be a positive float. Like in support vector machines, smaller values specify stronger regularization. `C=np.inf` results in unpenalized logistic regression. For a visual example on the effect of tuning the `C` parameter with an L1 penalty, see: :ref:`sphx_glr_auto_examples_linear_model_plot_logistic_path.py`.	1.0
	l1_ratio l1_ratio: float, default=0.0 The Elastic-Net mixing parameter, with `0 <= l1_ratio <= 1`. Setting `l1_ratio=1` gives a pure L1-penalty, setting `l1_ratio=0` a pure L2-penalty. Any value between 0 and 1 gives an Elastic-Net penalty of the form `l1_ratio * L1 + (1 - l1_ratio) * L2`. .. warning:: Certain values of `l1_ratio`, i.e. some penalties, may not work with some solvers. See the parameter `solver` below, to know the compatibility between the penalty and solver. .. versionchanged:: 1.8 Default value changed from None to 0.0. .. deprecated:: 1.8 `None` is deprecated and will be removed in version 1.10. Always use `l1_ratio` to specify the penalty type.	0.0
	dual dual: bool, default=False Dual (constrained) or primal (regularized, see also :ref:`this equation `) formulation. Dual formulation is only implemented for l2 penalty with liblinear solver. Prefer `dual=False` when n_samples > n_features.	False
	tol tol: float, default=1e-4 Tolerance for stopping criteria.	0.0001
	fit_intercept fit_intercept: bool, default=True Specifies if a constant (a.k.a. bias or intercept) should be added to the decision function.	True
	intercept_scaling intercept_scaling: float, default=1 Useful only when the solver `liblinear` is used and `self.fit_intercept` is set to `True`. In this case, `x` becomes `[x, self.intercept_scaling]`, i.e. a "synthetic" feature with constant value equal to `intercept_scaling` is appended to the instance vector. The intercept becomes ``intercept_scaling * synthetic_feature_weight``. .. note:: The synthetic feature weight is subject to L1 or L2 regularization as all other features. To lessen the effect of regularization on synthetic feature weight (and therefore on the intercept) `intercept_scaling` has to be increased.	1
	class_weight class_weight: dict or 'balanced', default=None Weights associated with classes in the form ``{class_label: weight}``. If not given, all classes are supposed to have weight one. The "balanced" mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as ``n_samples / (n_classes * np.bincount(y))``. Note that these weights will be multiplied with sample_weight (passed through the fit method) if sample_weight is specified. .. versionadded:: 0.17 class_weight='balanced'	None
	random_state random_state: int, RandomState instance, default=None Used when ``solver`` == 'sag', 'saga' or 'liblinear' to shuffle the data. See :term:`Glossary ` for details.	None
	solver solver: {'lbfgs', 'liblinear', 'newton-cg', 'newton-cholesky', 'sag', 'saga'}, default='lbfgs' Algorithm to use in the optimization problem. Default is 'lbfgs'. To choose a solver, you might want to consider the following aspects: - 'lbfgs' is a good default solver because it works reasonably well for a wide class of problems. - For :term:`multiclass` problems (`n_classes >= 3`), all solvers except 'liblinear' minimize the full multinomial loss, 'liblinear' will raise an error. - 'newton-cholesky' is a good choice for `n_samples` >> `n_features * n_classes`, especially with one-hot encoded categorical features with rare categories. Be aware that the memory usage of this solver has a quadratic dependency on `n_features * n_classes` because it explicitly computes the full Hessian matrix. - For small datasets, 'liblinear' is a good choice, whereas 'sag' and 'saga' are faster for large ones; - 'liblinear' can only handle binary classification by default. To apply a one-versus-rest scheme for the multiclass setting one can wrap it with the :class:`~sklearn.multiclass.OneVsRestClassifier`. .. warning:: The choice of the algorithm depends on the penalty chosen (`l1_ratio=0` for L2-penalty, `l1_ratio=1` for L1-penalty and `0 < l1_ratio < 1` for Elastic-Net) and on (multinomial) multiclass support: ================= ======================== ====================== solver l1_ratio multinomial multiclass ================= ======================== ====================== 'lbfgs' l1_ratio=0 yes 'liblinear' l1_ratio=1 or l1_ratio=0 no 'newton-cg' l1_ratio=0 yes 'newton-cholesky' l1_ratio=0 yes 'sag' l1_ratio=0 yes 'saga' 0<=l1_ratio<=1 yes ================= ======================== ====================== .. note:: 'sag' and 'saga' fast convergence is only guaranteed on features with approximately the same scale. You can preprocess the data with a scaler from :mod:`sklearn.preprocessing`. .. seealso:: Refer to the :ref:`User Guide ` for more information regarding :class:`LogisticRegression` and more specifically the :ref:`Table ` summarizing solver/penalty supports. .. versionadded:: 0.17 Stochastic Average Gradient (SAG) descent solver. Multinomial support in version 0.18. .. versionadded:: 0.19 SAGA solver. .. versionchanged:: 0.22 The default solver changed from 'liblinear' to 'lbfgs' in 0.22. .. versionadded:: 1.2 newton-cholesky solver. Multinomial support in version 1.6.	'lbfgs'
	max_iter max_iter: int, default=100 Maximum number of iterations taken for the solvers to converge.	100
	verbose verbose: int, default=0 For the liblinear and lbfgs solvers set verbose to any positive number for verbosity.	0
	warm_start warm_start: bool, default=False When set to True, reuse the solution of the previous call to fit as initialization, otherwise, just erase the previous solution. Useless for liblinear solver. See :term:`the Glossary `. .. versionadded:: 0.17 warm_start to support lbfgs, newton-cg, sag, saga solvers.	False
	n_jobs n_jobs: int, default=None Does not have any effect. .. deprecated:: 1.8 `n_jobs` is deprecated in version 1.8 and will be removed in 1.10.	None

	steps steps: list of tuples List of (name of step, estimator) tuples that are to be chained in sequential order. To be compatible with the scikit-learn API, all steps must define `fit`. All non-last steps must also define `transform`. See :ref:`Combining Estimators ` for more details.	[('tablevectorizer', ...), ('randomforestclassifier', ...)]
	transform_input transform_input: list of str, default=None The names of the :term:`metadata` parameters that should be transformed by the pipeline before passing it to the step consuming it. This enables transforming some input arguments to ``fit`` (other than ``X``) to be transformed by the steps of the pipeline up to the step which requires them. Requirement is defined via :ref:`metadata routing `. For instance, this can be used to pass a validation set through the pipeline. You can only set this if metadata routing is enabled, which you can enable using ``sklearn.set_config(enable_metadata_routing=True)``. .. versionadded:: 1.6	None
	memory memory: str or object with the joblib.Memory interface, default=None Used to cache the fitted transformers of the pipeline. The last step will never be cached, even if it is a transformer. By default, no caching is performed. If a string is given, it is the path to the caching directory. Enabling caching triggers a clone of the transformers before fitting. Therefore, the transformer instance given to the pipeline cannot be inspected directly. Use the attribute ``named_steps`` or ``steps`` to inspect estimators within the pipeline. Caching the transformers is advantageous when fitting is time consuming. See :ref:`sphx_glr_auto_examples_neighbors_plot_caching_nearest_neighbors.py` for an example on how to enable caching.	None
	verbose verbose: bool, default=False If True, the time elapsed while fitting each step will be printed as it is completed.	False

	cardinality_threshold	40
	low_cardinality	OrdinalEncode...nown_value=-1)
	high_cardinality	StringEncoder()
	numeric	PassThrough()
	datetime	DatetimeEncoder()
	specific_transformers	()
	drop_null_fraction	1.0
	drop_if_constant	False
	drop_if_unique	False
	datetime_format	None
	null_strings	None
	n_jobs	None

	n_estimators n_estimators: int, default=100 The number of trees in the forest. .. versionchanged:: 0.22 The default value of ``n_estimators`` changed from 10 to 100 in 0.22.	100
	criterion criterion: {"gini", "entropy", "log_loss"}, default="gini" The function to measure the quality of a split. Supported criteria are "gini" for the Gini impurity and "log_loss" and "entropy" both for the Shannon information gain, see :ref:`tree_mathematical_formulation`. Note: This parameter is tree-specific.	'gini'
	max_depth max_depth: int, default=None The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples.	None
	min_samples_split min_samples_split: int or float, default=2 The minimum number of samples required to split an internal node: - If int, then consider `min_samples_split` as the minimum number. - If float, then `min_samples_split` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. .. versionchanged:: 0.18 Added float values for fractions.	2
	min_samples_leaf min_samples_leaf: int or float, default=1 The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least ``min_samples_leaf`` training samples in each of the left and right branches. This may have the effect of smoothing the model, especially in regression. - If int, then consider `min_samples_leaf` as the minimum number. - If float, then `min_samples_leaf` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions.	1
	min_weight_fraction_leaf min_weight_fraction_leaf: float, default=0.0 The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided.	0.0
	max_features max_features: {"sqrt", "log2", None}, int or float, default="sqrt" The number of features to consider when looking for the best split: - If int, then consider `max_features` features at each split. - If float, then `max_features` is a fraction and `max(1, int(max_features * n_features_in_))` features are considered at each split. - If "sqrt", then `max_features=sqrt(n_features)`. - If "log2", then `max_features=log2(n_features)`. - If None, then `max_features=n_features`. .. versionchanged:: 1.1 The default of `max_features` changed from `"auto"` to `"sqrt"`. Note: the search for a split does not stop until at least one valid partition of the node samples is found, even if it requires to effectively inspect more than ``max_features`` features.	'sqrt'
	max_leaf_nodes max_leaf_nodes: int, default=None Grow trees with ``max_leaf_nodes`` in best-first fashion. Best nodes are defined as relative reduction in impurity. If None then unlimited number of leaf nodes.	None
	min_impurity_decrease min_impurity_decrease: float, default=0.0 A node will be split if this split induces a decrease of the impurity greater than or equal to this value. The weighted impurity decrease equation is the following:: N_t / N * (impurity - N_t_R / N_t * right_impurity - N_t_L / N_t * left_impurity) where ``N`` is the total number of samples, ``N_t`` is the number of samples at the current node, ``N_t_L`` is the number of samples in the left child, and ``N_t_R`` is the number of samples in the right child. ``N``, ``N_t``, ``N_t_R`` and ``N_t_L`` all refer to the weighted sum, if ``sample_weight`` is passed. .. versionadded:: 0.19	0.0
	bootstrap bootstrap: bool, default=True Whether bootstrap samples are used when building trees. If False, the whole dataset is used to build each tree.	True
	oob_score oob_score: bool or callable, default=False Whether to use out-of-bag samples to estimate the generalization score. By default, :func:`~sklearn.metrics.accuracy_score` is used. Provide a callable with signature `metric(y_true, y_pred)` to use a custom metric. Only available if `bootstrap=True`. For an illustration of out-of-bag (OOB) error estimation, see the example :ref:`sphx_glr_auto_examples_ensemble_plot_ensemble_oob.py`.	False
	n_jobs n_jobs: int, default=None The number of jobs to run in parallel. :meth:`fit`, :meth:`predict`, :meth:`decision_path` and :meth:`apply` are all parallelized over the trees. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details.	None
	random_state random_state: int, RandomState instance or None, default=None Controls both the randomness of the bootstrapping of the samples used when building trees (if ``bootstrap=True``) and the sampling of the features to consider when looking for the best split at each node (if ``max_features < n_features``). See :term:`Glossary ` for details.	0
	verbose verbose: int, default=0 Controls the verbosity when fitting and predicting.	0
	warm_start warm_start: bool, default=False When set to ``True``, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new forest. See :term:`Glossary ` and :ref:`tree_ensemble_warm_start` for details.	False
	class_weight class_weight: {"balanced", "balanced_subsample"}, dict or list of dicts, default=None Weights associated with classes in the form ``{class_label: weight}``. If not given, all classes are supposed to have weight one. For multi-output problems, a list of dicts can be provided in the same order as the columns of y. Note that for multioutput (including multilabel) weights should be defined for each class of every column in its own dict. For example, for four-class multilabel classification weights should be [{0: 1, 1: 1}, {0: 1, 1: 5}, {0: 1, 1: 1}, {0: 1, 1: 1}] instead of [{1:1}, {2:5}, {3:1}, {4:1}]. The "balanced" mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as ``n_samples / (n_classes * np.bincount(y))`` The "balanced_subsample" mode is the same as "balanced" except that weights are computed based on the bootstrap sample for every tree grown. For multi-output, the weights of each column of y will be multiplied. Note that these weights will be multiplied with sample_weight (passed through the fit method) if sample_weight is specified.	None
	ccp_alpha ccp_alpha: non-negative float, default=0.0 Complexity parameter used for Minimal Cost-Complexity Pruning. The subtree with the largest cost complexity that is smaller than ``ccp_alpha`` will be chosen. By default, no pruning is performed. See :ref:`minimal_cost_complexity_pruning` for details. See :ref:`sphx_glr_auto_examples_tree_plot_cost_complexity_pruning.py` for an example of such pruning. .. versionadded:: 0.22	0.0
	max_samples max_samples: int or float, default=None If bootstrap is True, the number of samples to draw from X to train each base estimator. - If None (default), then draw `X.shape[0]` samples. - If int, then draw `max_samples` samples. - If float, then draw `max(round(n_samples * max_samples), 1)` samples. Thus, `max_samples` should be in the interval `(0.0, 1.0]`. .. versionadded:: 0.22	None
	monotonic_cst monotonic_cst: array-like of int of shape (n_features), default=None Indicates the monotonicity constraint to enforce on each feature. - 1: monotonic increase - 0: no constraint - -1: monotonic decrease If monotonic_cst is None, no constraints are applied. Monotonicity constraints are not supported for: - multiclass classifications (i.e. when `n_classes > 2`), - multioutput classifications (i.e. when `n_outputs_ > 1`), - classifications trained on data with missing values. The constraints hold over the probability of the positive class. Read more in the :ref:`User Guide `. .. versionadded:: 1.4	None

	max_absolute_value	5
	quantile_range	(25.0, ...)

	max_absolute_value	5
	quantile_range	(25.0, ...)

checking_status

duration

credit_history

purpose

credit_amount

savings_status

employment

installment_commitment

personal_status

other_parties

residence_since

property_magnitude

age

other_payment_plans

housing

existing_credits

job

num_dependents

own_telephone

foreign_worker

class

checking_status

duration

credit_history

purpose

credit_amount

savings_status

employment

installment_commitment

personal_status

other_parties

residence_since

property_magnitude

age

other_payment_plans

housing

existing_credits

job

num_dependents

own_telephone

foreign_worker

class

Please enable javascript

checking_status

duration

credit_history

purpose

credit_amount

savings_status

employment

installment_commitment

personal_status

other_parties

residence_since

property_magnitude

age

other_payment_plans

housing

existing_credits

job

num_dependents

own_telephone

foreign_worker

class

Please enable javascript

checking_status

duration

credit_history

purpose

credit_amount

savings_status

employment

installment_commitment

personal_status

other_parties

residence_since

property_magnitude

age

other_payment_plans

housing