.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/pitfalls_and_solutions/plot_skd003_inconsistent_performance.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_pitfalls_and_solutions_plot_skd003_inconsistent_performance.py: .. _example_skd003_inconsistent_performance: SKD003 - Inconsistent performance across splits =============================================== :ref:`SKD003 ` flags folds whose test metrics diverge sharply from the median during a cross-validation evaluation. With a proper splitter this is often a diagnostic check: the data have structure (groups, time, or a corrupted batch) that shuffled cross-validation would hide. Realistic triggers: - a contiguous bad batch of labels or features (mislabelled window, logging bug, schema mix-up), - a much easier or harder group in one test fold under :class:`~sklearn.model_selection.GroupKFold`, - temporal drift under :class:`~sklearn.model_selection.TimeSeriesSplit` (e.g. more ill patients start showing up), - accidental fold imbalance from unshuffled :class:`~sklearn.model_selection.KFold` when prevalence varies along collection order (then shuffle or stratify if that will not appear in production). When structure is real, shuffled cross-validation overestimates performance. SKD003 under a proper split is a good sign. The structure may not be fully fixable: once understood, mute with :func:`~skore.configuration` and consider collecting more data on the hard regime. This notebook walks four beats: artificial corruption, a bad group in test, distribution shift in the last time-series fold, then ignoring SKD003 once the problem is understood. .. GENERATED FROM PYTHON SOURCE LINES 35-40 Load Breast Cancer (two classes) ================================ Let us use the Breast Cancer dataset to show how SKD003 can detect a batch of corrupted labels. .. GENERATED FROM PYTHON SOURCE LINES 40-49 .. code-block:: Python import numpy as np import pandas as pd from sklearn.datasets import load_breast_cancer cancer = load_breast_cancer(as_frame=True) X, y = cancer.data, cancer.target .. GENERATED FROM PYTHON SOURCE LINES 50-53 We can inspect the features and target with :class:`~skrub.TableReport`, and notice that it is a well curated dataset with two balanced classes (they appear in roughly equal proportions). .. GENERATED FROM PYTHON SOURCE LINES 53-58 .. code-block:: Python from skrub import TableReport TableReport(X) .. raw:: html

Column	Column name	dtype	Is sorted	Unique values	Mean	Std	Min	Median	Max
0	mean radius	Float64DType	False	456 (80.1%)	14.1	3.52	6.98	13.4	28.1
1	mean texture	Float64DType	False	479 (84.2%)	19.3	4.30	9.71	18.8	39.3
2	mean perimeter	Float64DType	False	522 (91.7%)	92.0	24.3	43.8	86.2	188.
3	mean area	Float64DType	False	539 (94.7%)	655.	352.	144.	551.	2.50e+03
4	mean smoothness	Float64DType	False	474 (83.3%)	0.0964	0.0141	0.0526	0.0959	0.163
5	mean compactness	Float64DType	False	537 (94.4%)	0.104	0.0528	0.0194	0.0926	0.345
6	mean concavity	Float64DType	False	537 (94.4%)	0.0888	0.0797	0.00	0.0615	0.427
7	mean concave points	Float64DType	False	542 (95.3%)	0.0489	0.0388	0.00	0.0335	0.201
8	mean symmetry	Float64DType	False	432 (75.9%)	0.181	0.0274	0.106	0.179	0.304
9	mean fractal dimension	Float64DType	False	499 (87.7%)	0.0628	0.00706	0.0500	0.0615	0.0974
10	radius error	Float64DType	False	540 (94.9%)	0.405	0.277	0.112	0.324	2.87
11	texture error	Float64DType	False	519 (91.2%)	1.22	0.552	0.360	1.11	4.88
12	perimeter error	Float64DType	False	533 (93.7%)	2.87	2.02	0.757	2.29	22.0
13	area error	Float64DType	False	528 (92.8%)	40.3	45.5	6.80	24.5	542.
14	smoothness error	Float64DType	False	547 (96.1%)	0.00704	0.00300	0.00171	0.00638	0.0311
15	compactness error	Float64DType	False	541 (95.1%)	0.0255	0.0179	0.00225	0.0204	0.135
16	concavity error	Float64DType	False	533 (93.7%)	0.0319	0.0302	0.00	0.0259	0.396
17	concave points error	Float64DType	False	507 (89.1%)	0.0118	0.00617	0.00	0.0109	0.0528
18	symmetry error	Float64DType	False	498 (87.5%)	0.0205	0.00827	0.00788	0.0187	0.0790
19	fractal dimension error	Float64DType	False	545 (95.8%)	0.00379	0.00265	0.000895	0.00319	0.0298
20	worst radius	Float64DType	False	457 (80.3%)	16.3	4.83	7.93	15.0	36.0
21	worst texture	Float64DType	False	511 (89.8%)	25.7	6.15	12.0	25.4	49.5
22	worst perimeter	Float64DType	False	514 (90.3%)	107.	33.6	50.4	97.7	251.
23	worst area	Float64DType	False	544 (95.6%)	881.	569.	185.	686.	4.25e+03
24	worst smoothness	Float64DType	False	411 (72.2%)	0.132	0.0228	0.0712	0.131	0.223
25	worst compactness	Float64DType	False	529 (93.0%)	0.254	0.157	0.0273	0.212	1.06
26	worst concavity	Float64DType	False	539 (94.7%)	0.272	0.209	0.00	0.227	1.25
27	worst concave points	Float64DType	False	492 (86.5%)	0.115	0.0657	0.00	0.0999	0.291
28	worst symmetry	Float64DType	False	500 (87.9%)	0.290	0.0619	0.157	0.282	0.664
29	worst fractal dimension	Float64DType	False	535 (94.0%)	0.0839	0.0181	0.0550	0.0800	0.207

Column 1	Column 2	Cramér's V	Pearson's Correlation
mean radius	mean perimeter	0.846	0.998
mean radius	mean area	0.800	0.987
radius error	perimeter error	0.768	0.973
worst radius	worst perimeter	0.757	0.994
mean perimeter	mean area	0.754	0.987
radius error	area error	0.732	0.952
area error	worst area	0.699	0.811
perimeter error	area error	0.687	0.938
worst radius	worst area	0.676	0.984
worst perimeter	worst area	0.662	0.978
mean area	area error	0.652	0.800
mean perimeter	worst radius	0.643	0.969
mean area	worst radius	0.640	0.963
mean radius	worst radius	0.635	0.970
concavity error	concave points error	0.628	0.772
mean radius	area error	0.626	0.736
concavity error	fractal dimension error	0.610	0.727
worst compactness	worst concavity	0.599	0.892
mean area	worst perimeter	0.596	0.959
mean perimeter	area error	0.594	0.745
mean perimeter	worst perimeter	0.592	0.970
radius error	worst area	0.574	0.752
area error	worst perimeter	0.573	0.761
mean radius	worst perimeter	0.570	0.965
worst compactness	worst fractal dimension	0.568	0.810
mean radius	worst area	0.567	0.941
compactness error	fractal dimension error	0.565	0.803
area error	worst radius	0.561	0.757
mean area	worst area	0.559	0.959
perimeter error	worst area	0.558	0.731
mean texture	worst texture	0.554	0.912
mean perimeter	worst area	0.553	0.942
mean concave points	area error	0.551	0.690
mean concavity	mean concave points	0.549	0.921
compactness error	concavity error	0.538	0.801
mean area	radius error	0.514	0.733
concave points error	fractal dimension error	0.511	0.611
mean concave points	worst concave points	0.507	0.910
mean radius	radius error	0.494	0.679
worst concavity	worst fractal dimension	0.487	0.687
radius error	worst perimeter	0.480	0.720
mean perimeter	mean concave points	0.476	0.851
perimeter error	worst perimeter	0.472	0.721
mean compactness	worst compactness	0.467	0.866
mean area	perimeter error	0.466	0.727
mean fractal dimension	worst fractal dimension	0.465	0.767
mean perimeter	radius error	0.465	0.692
mean concave points	worst perimeter	0.463	0.856
mean area	mean concave points	0.463	0.823
worst symmetry	worst fractal dimension	0.461	0.538
mean smoothness	worst smoothness	0.458	0.805
mean radius	perimeter error	0.457	0.674
mean compactness	mean concavity	0.456	0.883
mean radius	mean concave points	0.454	0.823
mean concavity	worst concave points	0.453	0.861
mean concave points	radius error	0.446	0.698
radius error	worst radius	0.445	0.715
mean concave points	perimeter error	0.444	0.711
compactness error	concave points error	0.443	0.744
mean perimeter	perimeter error	0.439	0.693
mean fractal dimension	fractal dimension error	0.439	0.688
concavity error	worst concavity	0.437	0.663
mean concavity	worst concavity	0.433	0.884
perimeter error	worst radius	0.426	0.697
mean compactness	worst concavity	0.423	0.816
mean compactness	worst concave points	0.423	0.816
mean concavity	concavity error	0.421	0.691
mean concave points	worst area	0.419	0.810
mean concavity	area error	0.416	0.617
mean concave points	worst radius	0.412	0.830
mean compactness	mean concave points	0.412	0.831
worst perimeter	worst concave points	0.408	0.816
symmetry error	worst symmetry	0.406	0.389
worst concavity	worst concave points	0.404	0.855
perimeter error	concave points error	0.401	0.556
worst compactness	worst concave points	0.394	0.801
mean concavity	concave points error	0.394	0.683
mean concavity	radius error	0.394	0.632
mean concavity	worst perimeter	0.393	0.730
mean symmetry	worst symmetry	0.393	0.700
mean compactness	compactness error	0.385	0.739
worst radius	worst concave points	0.385	0.787
mean perimeter	mean concavity	0.384	0.716
mean concavity	perimeter error	0.383	0.660
mean perimeter	worst concave points	0.378	0.771
compactness error	worst compactness	0.377	0.679
mean compactness	worst fractal dimension	0.377	0.687
worst smoothness	worst fractal dimension	0.374	0.618
mean area	mean concavity	0.370	0.686
worst area	worst concave points	0.364	0.747
mean symmetry	symmetry error	0.362	0.449
mean compactness	concave points error	0.362	0.642
mean fractal dimension	concavity error	0.362	0.447
area error	worst concave points	0.360	0.538
mean concavity	worst compactness	0.360	0.755
mean compactness	mean symmetry	0.360	0.603
mean concavity	compactness error	0.359	0.670
mean compactness	perimeter error	0.356	0.549
mean concavity	worst area	0.354	0.676
mean radius	mean concavity	0.352	0.677
mean area	worst concave points	0.351	0.722
mean radius	worst concave points	0.351	0.744
mean concavity	worst radius	0.350	0.688
mean concave points	worst concavity	0.346	0.752
worst compactness	worst symmetry	0.345	0.614
compactness error	worst concavity	0.339	0.639
concave points error	worst concave points	0.338	0.602
mean concave points	concave points error	0.337	0.616
mean smoothness	mean compactness	0.331	0.659
mean smoothness	worst fractal dimension	0.331	0.499
mean compactness	worst symmetry	0.328	0.510
mean fractal dimension	worst smoothness	0.325	0.505
worst smoothness	worst compactness	0.321	0.568
compactness error	worst fractal dimension	0.319	0.591
smoothness error	concave points error	0.318	0.328
mean concavity	fractal dimension error	0.317	0.449
mean concave points	worst compactness	0.316	0.667
mean smoothness	mean fractal dimension	0.316	0.585
smoothness error	concavity error	0.312	0.269
mean compactness	symmetry error	0.312	0.230
texture error	perimeter error	0.311	0.223
compactness error	worst symmetry	0.307	0.278
compactness error	symmetry error	0.307	0.395
radius error	worst concave points	0.301	0.531
perimeter error	worst concave points	0.301	0.555
mean fractal dimension	compactness error	0.300	0.560
concave points error	worst concavity	0.294	0.550
worst smoothness	worst symmetry	0.292	0.494
fractal dimension error	worst fractal dimension	0.290	0.591
mean fractal dimension	worst compactness	0.286	0.459
worst concavity	worst symmetry	0.285	0.533
worst perimeter	worst concavity	0.285	0.618
mean compactness	mean fractal dimension	0.283	0.565
perimeter error	symmetry error	0.283	0.266
mean compactness	concavity error	0.282	0.571
mean compactness	area error	0.280	0.456
mean smoothness	mean symmetry	0.279	0.558
perimeter error	compactness error	0.279	0.416
mean smoothness	worst symmetry	0.278	0.394
mean fractal dimension	worst concavity	0.277	0.346
mean compactness	worst perimeter	0.275	0.590
mean concavity	mean symmetry	0.275	0.501
texture error	concave points error	0.275	0.230
radius error	concave points error	0.274	0.513
worst texture	worst area	0.272	0.346
mean concave points	mean symmetry	0.272	0.462
mean fractal dimension	concave points error	0.271	0.341
mean compactness	radius error	0.271	0.497
mean compactness	worst smoothness	0.269	0.566
concavity error	symmetry error	0.268	0.309
mean compactness	fractal dimension error	0.267	0.507
worst radius	worst concavity	0.266	0.574
smoothness error	fractal dimension error	0.266	0.427
mean perimeter	mean compactness	0.265	0.557
worst perimeter	worst compactness	0.265	0.529
mean concavity	worst fractal dimension	0.262	0.515
mean radius	smoothness error	0.261	-0.223
mean compactness	worst area	0.259	0.510
worst smoothness	worst concave points	0.259	0.548
mean compactness	worst radius	0.257	0.535
mean perimeter	worst concavity	0.256	0.564
mean smoothness	mean concavity	0.255	0.522
smoothness error	compactness error	0.254	0.337
mean smoothness	mean concave points	0.254	0.554
mean perimeter	smoothness error	0.250	-0.203
mean fractal dimension	smoothness error	0.249	0.402
mean symmetry	concave points error	0.248	0.393
worst area	worst concavity	0.248	0.543
mean area	mean compactness	0.247	0.499
texture error	smoothness error	0.246	0.397
mean concave points	concavity error	0.246	0.439
worst concave points	worst fractal dimension	0.245	0.511
mean symmetry	compactness error	0.244	0.422
area error	compactness error	0.244	0.285
smoothness error	symmetry error	0.244	0.414
mean symmetry	radius error	0.244	0.303
area error	worst concavity	0.244	0.385
radius error	compactness error	0.242	0.356
mean smoothness	worst compactness	0.242	0.472
mean concave points	compactness error	0.241	0.490
mean fractal dimension	worst symmetry	0.240	0.334
area error	concave points error	0.240	0.416
mean radius	worst concavity	0.239	0.527
mean radius	mean compactness	0.239	0.506
mean area	worst concavity	0.238	0.513
mean symmetry	worst compactness	0.238	0.473
compactness error	worst concave points	0.237	0.483
mean concave points	worst symmetry	0.237	0.376
mean symmetry	worst concave points	0.236	0.430
concave points error	worst compactness	0.236	0.453
texture error	worst texture	0.234	0.409
concave points error	symmetry error	0.233	0.313
radius error	smoothness error	0.232	0.165
radius error	symmetry error	0.231	0.241
concave points error	worst perimeter	0.231	0.395
mean concave points	symmetry error	0.231	0.0954
symmetry error	fractal dimension error	0.231	0.369
mean concavity	worst symmetry	0.230	0.409
mean perimeter	concave points error	0.230	0.407
mean symmetry	worst concavity	0.230	0.434
worst radius	worst compactness	0.229	0.476
worst concave points	worst symmetry	0.229	0.503
fractal dimension error	worst concavity	0.229	0.380
mean texture	worst area	0.229	0.344
texture error	symmetry error	0.229	0.412
area error	smoothness error	0.228	0.0752
perimeter error	concavity error	0.228	0.362
mean area	smoothness error	0.227	-0.167
worst smoothness	worst concavity	0.226	0.519
mean perimeter	worst compactness	0.225	0.456
mean symmetry	area error	0.225	0.224
worst texture	worst concavity	0.225	0.368
mean smoothness	radius error	0.225	0.301
texture error	fractal dimension error	0.223	0.280
area error	worst texture	0.223	0.196
mean fractal dimension	symmetry error	0.223	0.345
concavity error	worst fractal dimension	0.223	0.439
mean symmetry	mean fractal dimension	0.222	0.480
mean smoothness	worst concavity	0.222	0.435
perimeter error	worst concavity	0.221	0.419
mean smoothness	worst concave points	0.221	0.503
smoothness error	worst smoothness	0.221	0.314
mean concavity	mean fractal dimension	0.220	0.337
smoothness error	worst radius	0.219	-0.231
mean radius	concave points error	0.219	0.376
mean texture	texture error	0.218	0.386
concavity error	worst compactness	0.218	0.485
mean smoothness	area error	0.217	0.247
radius error	worst concavity	0.217	0.381
mean smoothness	symmetry error	0.217	0.201
mean concavity	symmetry error	0.217	0.178
fractal dimension error	worst compactness	0.217	0.390
symmetry error	worst fractal dimension	0.216	0.0781
mean area	concave points error	0.216	0.372
mean symmetry	concavity error	0.216	0.343
symmetry error	worst compactness	0.214	0.0603
smoothness error	worst perimeter	0.214	-0.217
worst area	worst compactness	0.213	0.438
mean radius	mean fractal dimension	0.213	-0.312
mean area	worst compactness	0.213	0.390
mean smoothness	smoothness error	0.212	0.332
concave points error	worst area	0.212	0.342
worst texture	worst compactness	0.212	0.361
mean symmetry	texture error	0.212	0.128
mean symmetry	worst fractal dimension	0.210	0.438
mean symmetry	fractal dimension error	0.208	0.332
mean symmetry	perimeter error	0.207	0.314
mean smoothness	compactness error	0.207	0.319
mean concavity	worst smoothness	0.206	0.449
mean texture	area error	0.206	0.260
concave points error	worst radius	0.205	0.358
mean smoothness	concave points error	0.205	0.381
perimeter error	worst compactness	0.204	0.342
concavity error	worst symmetry	0.203	0.198
mean perimeter	symmetry error	0.203	-0.0816
mean radius	worst compactness	0.202	0.413
texture error	concavity error	0.202	0.195
mean concave points	worst smoothness	0.202	0.453
area error	symmetry error	0.201	0.134
mean concave points	worst fractal dimension	0.200	0.369
area error	worst compactness	0.199	0.283
mean symmetry	smoothness error	0.198	0.187
perimeter error	fractal dimension error	0.198	0.244
mean radius	symmetry error	0.197	-0.104
concavity error	worst concave points	0.196	0.440
mean symmetry	worst smoothness	0.196	0.427
mean texture	worst concavity	0.195	0.301
mean perimeter	concavity error	0.194	0.228
mean concavity	smoothness error	0.193	0.0986
symmetry error	worst concavity	0.192	0.0371
mean texture	mean concavity	0.188	0.302
mean perimeter	mean fractal dimension	0.188	-0.261
mean texture	worst concave points	0.187	0.295
worst radius	worst texture	0.187	0.360
mean texture	worst radius	0.185	0.353
mean smoothness	fractal dimension error	0.183	0.284
texture error	compactness error	0.183	0.232
mean perimeter	compactness error	0.182	0.251
worst texture	worst perimeter	0.182	0.365
radius error	concavity error	0.182	0.332
mean texture	worst perimeter	0.182	0.358
mean fractal dimension	worst radius	0.181	-0.254
mean compactness	texture error	0.181	0.0462
mean area	symmetry error	0.181	-0.0725
concave points error	worst fractal dimension	0.180	0.311
symmetry error	worst perimeter	0.179	-0.104
mean radius	concavity error	0.179	0.194
compactness error	worst perimeter	0.178	0.261
mean texture	radius error	0.178	0.276
mean perimeter	mean symmetry	0.178	0.183
mean perimeter	mean smoothness	0.177	0.207
concavity error	worst perimeter	0.177	0.227
radius error	worst compactness	0.177	0.287
texture error	worst smoothness	0.175	-0.0737
worst perimeter	worst symmetry	0.175	0.269
mean texture	perimeter error	0.175	0.282
mean texture	mean compactness	0.173	0.237
symmetry error	worst radius	0.173	-0.128
mean fractal dimension	worst perimeter	0.173	-0.205
worst texture	worst concave points	0.173	0.360
mean radius	mean texture	0.172	0.324
mean radius	compactness error	0.172	0.206
radius error	texture error	0.172	0.213
mean texture	mean perimeter	0.171	0.330
compactness error	worst smoothness	0.170	0.227
mean concavity	worst texture	0.170	0.300
worst texture	worst fractal dimension	0.170	0.219
mean radius	mean smoothness	0.169	0.171
mean area	compactness error	0.169	0.213
mean radius	worst texture	0.168	0.297
mean fractal dimension	perimeter error	0.168	0.0398
mean area	mean smoothness	0.168	0.177
mean symmetry	worst perimeter	0.167	0.219
mean texture	mean area	0.166	0.321
mean texture	worst compactness	0.165	0.278
mean texture	mean concave points	0.165	0.293
mean radius	mean symmetry	0.165	0.148
mean area	mean fractal dimension	0.165	-0.283
mean smoothness	perimeter error	0.164	0.296
symmetry error	worst smoothness	0.164	-0.0127
symmetry error	worst concave points	0.163	-0.0304
mean compactness	smoothness error	0.162	0.135
mean area	mean symmetry	0.162	0.151
worst texture	worst smoothness	0.161	0.225
mean symmetry	worst radius	0.160	0.186
compactness error	worst radius	0.160	0.205
mean area	worst texture	0.160	0.287
mean perimeter	worst texture	0.160	0.303
smoothness error	worst area	0.160	-0.182
mean compactness	worst texture	0.159	0.248
compactness error	worst texture	0.159	0.143
mean concave points	fractal dimension error	0.159	0.258
fractal dimension error	worst symmetry	0.157	0.111
worst radius	worst symmetry	0.156	0.244
area error	concavity error	0.156	0.271
mean smoothness	worst perimeter	0.155	0.239
mean smoothness	concavity error	0.155	0.248
radius error	worst texture	0.155	0.195
concavity error	worst radius	0.155	0.187
mean concave points	worst texture	0.155	0.293
mean concave points	smoothness error	0.154	0.0277
worst texture	worst symmetry	0.153	0.233
symmetry error	worst area	0.153	-0.110
worst perimeter	worst smoothness	0.153	0.237
perimeter error	worst symmetry	0.153	0.110
fractal dimension error	worst smoothness	0.152	0.171
worst perimeter	worst fractal dimension	0.152	0.139
mean perimeter	texture error	0.152	-0.0868
concave points error	worst smoothness	0.151	0.215
texture error	area error	0.150	0.112
radius error	fractal dimension error	0.150	0.228
mean smoothness	texture error	0.150	0.0684
mean concavity	texture error	0.149	0.0762
mean perimeter	worst symmetry	0.149	0.189
compactness error	worst area	0.148	0.199
mean concave points	mean fractal dimension	0.148	0.167
mean texture	concave points error	0.147	0.164
worst radius	worst smoothness	0.146	0.217
fractal dimension error	worst concave points	0.145	0.215
mean area	concavity error	0.145	0.208
mean area	texture error	0.144	-0.0663
radius error	worst symmetry	0.144	0.0945
worst area	worst symmetry	0.143	0.209
perimeter error	smoothness error	0.142	0.151
mean texture	compactness error	0.142	0.192
texture error	worst perimeter	0.141	-0.102
mean smoothness	worst radius	0.139	0.213
texture error	worst symmetry	0.139	-0.128
mean fractal dimension	worst texture	0.139	-0.0513
texture error	worst concave points	0.138	-0.120
worst radius	worst fractal dimension	0.138	0.0935
concave points error	worst symmetry	0.138	0.143
mean fractal dimension	worst area	0.137	-0.232
mean symmetry	worst area	0.136	0.177
mean perimeter	worst smoothness	0.136	0.151
mean radius	texture error	0.135	-0.0973
mean radius	worst symmetry	0.134	0.164
mean radius	worst smoothness	0.134	0.120
area error	fractal dimension error	0.133	0.127
texture error	worst radius	0.133	-0.112
mean concave points	texture error	0.133	0.0215
mean texture	concavity error	0.132	0.143
mean area	worst symmetry	0.132	0.144
fractal dimension error	worst radius	0.131	-0.0375
area error	worst symmetry	0.130	0.0741
perimeter error	worst texture	0.130	0.200
worst area	worst smoothness	0.130	0.209
mean symmetry	worst texture	0.129	0.0907
mean fractal dimension	worst concave points	0.129	0.175
mean area	worst smoothness	0.129	0.124
worst area	worst fractal dimension	0.129	0.0796
mean fractal dimension	texture error	0.129	0.164
mean radius	fractal dimension error	0.127	-0.0426
mean fractal dimension	radius error	0.126	0.000111
mean perimeter	fractal dimension error	0.126	-0.00552
mean smoothness	worst area	0.125	0.207
radius error	worst smoothness	0.125	0.142
fractal dimension error	worst perimeter	0.125	-0.00100
perimeter error	worst smoothness	0.124	0.130
mean texture	worst smoothness	0.124	0.0775
concavity error	worst area	0.123	0.188
smoothness error	worst concave points	0.122	-0.102
mean texture	worst fractal dimension	0.122	0.119
mean radius	worst fractal dimension	0.121	0.00707
concave points error	worst texture	0.120	0.0867
area error	worst smoothness	0.118	0.125
mean fractal dimension	area error	0.117	-0.0902
mean texture	worst symmetry	0.117	0.105
concavity error	worst smoothness	0.116	0.168
smoothness error	worst fractal dimension	0.116	0.101
mean perimeter	worst fractal dimension	0.116	0.0510
mean area	fractal dimension error	0.116	-0.0199
mean texture	fractal dimension error	0.113	0.0545
smoothness error	worst concavity	0.112	-0.0583
mean texture	mean symmetry	0.110	0.0714
mean texture	mean fractal dimension	0.109	-0.0764
mean texture	mean smoothness	0.108	-0.0234
smoothness error	worst symmetry	0.108	-0.107
fractal dimension error	worst texture	0.107	-0.00320
texture error	worst compactness	0.106	-0.0924
mean area	worst fractal dimension	0.105	0.00374
symmetry error	worst texture	0.104	-0.0775
perimeter error	worst fractal dimension	0.103	0.0854
mean smoothness	worst texture	0.103	0.0361
texture error	worst area	0.101	-0.0832
area error	worst fractal dimension	0.0978	0.0175
concavity error	worst texture	0.0976	0.100
mean texture	symmetry error	0.0971	0.00913
texture error	worst concavity	0.0966	-0.0690
fractal dimension error	worst area	0.0927	-0.0227
texture error	worst fractal dimension	0.0927	-0.0457
radius error	worst fractal dimension	0.0914	0.0496
mean texture	smoothness error	0.0905	0.00661
smoothness error	worst compactness	0.0896	-0.0556
smoothness error	worst texture	0.0894	-0.0747

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

.. GENERATED FROM PYTHON SOURCE LINES 59-62 .. code-block:: Python TableReport(y) .. raw:: html

	target
0	0
1	0
2	0
3	0
4	0

564	0
565	0
566	0
567	0
568	1

Column	Column name	dtype	Is sorted	Null values	Unique values	Mean	Std	Min	Median	Max
0	target	Int64DType	False	0 (0.0%)	2 (0.4%)	0.627	0.484	0	1	1

Please enable javascript

.. GENERATED FROM PYTHON SOURCE LINES 63-72 Artificial corruption: a bad contiguous batch ============================================= Let us permute the labels on the first fifth of the dataset to break any association between `X` and `y`. Later, we will use unshuffled 5-fold cross-validation, so all corrupted rows will land in the same fold. We expect the score of this fold to be low due to this corruption. While we are creating this defect artificially, this is a scenario that can happen in practice due to e.g. a logging bug, a broken sensor, or a merge mix-up. .. GENERATED FROM PYTHON SOURCE LINES 72-78 .. code-block:: Python y_corrupted, n_corrupt = y.copy(), len(y) // 5 rng = np.random.default_rng(seed=0) y_corrupted.iloc[:n_corrupt] = rng.permutation(y_corrupted.iloc[:n_corrupt]) .. GENERATED FROM PYTHON SOURCE LINES 79-81 We will use a default :func:`~skrub.tabular_pipeline` classifier for preprocessing, and a gradient boosting model for prediction. .. GENERATED FROM PYTHON SOURCE LINES 81-88 .. code-block:: Python from sklearn.linear_model import LogisticRegression from skrub import tabular_pipeline model = tabular_pipeline(LogisticRegression()) model .. raw:: html

Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(datetime=DatetimeEncoder(periodic_encoding='spline'))),
                    ('simpleimputer', SimpleImputer(add_indicator=True)),
                    ('squashingscaler', SquashingScaler(max_absolute_value=5)),
                    ('logisticregression', LogisticRegression())])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

.. GENERATED FROM PYTHON SOURCE LINES 89-95 Trigger SKD003: cross-validate on corrupted labels ================================================== We will now evaluate the model on the corrupted labels using unshuffled 5-fold cross-validation and look at non aggregated metrics values, to notice the discrepancy on the first fold. .. GENERATED FROM PYTHON SOURCE LINES 95-102 .. code-block:: Python import skore report = skore.evaluate(model, X=X, y=y_corrupted, pos_label=1, splitter=5) report.metrics.summarize(data_source="test").frame(aggregate=None, flat_index=False) .. raw:: html

estimator	LogisticRegression
split	Split #0	Split #1	Split #2	Split #3	Split #4
metric
accuracy	0.649123	0.859649	0.921053	0.964912	0.946903
precision	0.762712	0.816092	0.888889	0.947368	0.945205
recall	0.633803	1.000000	1.000000	1.000000	0.971831
roc_auc	0.640681	0.877825	0.994709	0.982474	0.989940
log_loss	1.688958	0.418982	0.228212	0.198544	0.217792
brier_score	0.309182	0.117186	0.060775	0.046597	0.054071
fit_time	0.148614	0.140205	0.146258	0.119379	0.120484
predict_time	0.083945	0.078751	0.067064	0.066841	0.065843

.. GENERATED FROM PYTHON SOURCE LINES 103-104 Looking at the checks results, ``SKD003`` correctly flags split #0. .. GENERATED FROM PYTHON SOURCE LINES 104-107 .. code-block:: Python report.checks.summarize(fast_mode=True) .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 108-116 Visualize the variance across splits ==================================== The ROC display of a cross-validation report overlays the splits and summarizes them with a mean AUC and its standard deviation. It is built to show how much a model moves from one split to the next, rather than to identify a given split. Here the curves are widely spread and one of them sinks towards the chance level: this is the instability that ``SKD003`` reported. .. GENERATED FROM PYTHON SOURCE LINES 116-119 .. code-block:: Python report.metrics.roc().plot() .. image-sg:: /auto_examples/pitfalls_and_solutions/images/sphx_glr_plot_skd003_inconsistent_performance_001.png :alt: ROC Curve for LogisticRegression Positive label: 1 Data source: Test set :srcset: /auto_examples/pitfalls_and_solutions/images/sphx_glr_plot_skd003_inconsistent_performance_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none

.. GENERATED FROM PYTHON SOURCE LINES 120-121 If the cause was a fixable bad batch, clean labels clear the outlier split. .. GENERATED FROM PYTHON SOURCE LINES 121-127 .. code-block:: Python report_clean = skore.evaluate(model, X=X, y=y, pos_label=1, splitter=5) report_clean.metrics.summarize(data_source="test").frame( aggregate=None, flat_index=False ) .. raw:: html

estimator	LogisticRegression
split	Split #0	Split #1	Split #2	Split #3	Split #4
metric
accuracy	0.991228	0.964912	0.991228	0.973684	0.991150
precision	0.986111	0.958904	0.986301	0.972603	1.000000
recall	1.000000	0.985915	1.000000	0.986111	0.985915
roc_auc	0.996725	0.994759	0.999008	0.988095	1.000000
log_loss	0.068116	0.088164	0.057122	0.098018	0.055142
brier_score	0.017134	0.028177	0.016488	0.021536	0.012032
fit_time	0.111174	0.113374	0.110567	0.111760	0.110291
predict_time	0.062849	0.063302	0.062700	0.062844	0.062409

.. GENERATED FROM PYTHON SOURCE LINES 128-130 .. code-block:: Python report_clean.checks.summarize(fast_mode=True) .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 131-140 Bad group in the test fold ========================== When observations carry a group identifier, e.g. the medical center where patient data were collected, a grouped splitter keeps each group on one side of every split. If one group is much harder or easier to predict, the fold that tests it will look like an outlier and ``SKD003`` will fire. That is expected: the splitter did its job. Shuffled cross-validation would smear the difficult group across folds, hiding the gap and overestimating performance. .. GENERATED FROM PYTHON SOURCE LINES 140-154 .. code-block:: Python import skrub from sklearn.model_selection import GroupKFold n_batches = 10 batch_id = np.minimum(np.arange(len(X)) // (len(X) // n_batches), n_batches - 1) y_batch = y.copy() bad_batch = batch_id == 0 rng_batch = np.random.default_rng(seed=1) y_batch.iloc[bad_batch] = rng_batch.choice(y.unique(), size=int(bad_batch.sum())) df_batch = X.assign(batch_id=batch_id, target=y_batch) .. GENERATED FROM PYTHON SOURCE LINES 155-160 :class:`~sklearn.model_selection.GroupKFold` needs the group vector at split time, so we use a skrub :class:`~skrub.DataOp` to attach it to the data. :meth:`~skrub.DataOp.skb.mark_as_X` accepts a ``cv`` argument and ``split_kwargs`` for group ids. The resulting learner carries its own cross-validation scheme, so :func:`~skore.evaluate` needs no ``splitter``. .. GENERATED FROM PYTHON SOURCE LINES 160-170 .. code-block:: Python data = skrub.var("data", df_batch) groups = data["batch_id"] X_op = data.drop(columns=["batch_id", "target"]).skb.mark_as_X( cv=GroupKFold(n_splits=5), split_kwargs={"groups": groups}, ) y_op = data["target"].skb.mark_as_y() learner = X_op.skb.apply(model, y=y_op).skb.make_learner() .. GENERATED FROM PYTHON SOURCE LINES 171-176 .. code-block:: Python report_grouped = skore.evaluate(learner, data={"data": df_batch}, pos_label=1) report_grouped.metrics.summarize(data_source="test").frame( aggregate=None, flat_index=False ) .. raw:: html

estimator	SkrubLearner
split	Split #0	Split #1	Split #2	Split #3	Split #4
metric
score	0.768595	0.955357	0.946429	0.973214	0.937500
accuracy	0.768595	0.955357	0.946429	0.973214	0.937500
precision	0.885246	0.948052	0.921053	0.962025	0.925926
recall	0.720000	0.986486	1.000000	1.000000	0.986842
roc_auc	0.811304	0.992888	0.993878	0.996345	0.978436
log_loss	1.552004	0.173870	0.171785	0.121842	0.173842
brier_score	0.212717	0.043370	0.047659	0.027993	0.046154
fit_time	0.149480	0.152578	0.196513	0.196123	0.194578
predict_time	0.093719	0.121204	0.122192	0.123455	0.124356

.. GENERATED FROM PYTHON SOURCE LINES 177-178 Looking at the checks results, ``SKD003`` correctly flags split #0. .. GENERATED FROM PYTHON SOURCE LINES 178-181 .. code-block:: Python report_grouped.checks.summarize(fast_mode=True) .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 182-195 Distribution shift in the last time-series fold =============================================== Under :class:`~sklearn.model_selection.TimeSeriesSplit`, later windows can diverge from the training data. Consider a medical setting where diagnoses are collected over time: a sudden influx of ill patients near the end of the study shifts the class distribution. Earlier folds look strong because the model was trained on a balanced population, but the last fold drops as the positive class becomes rare. SKD003 fires, which is expected from chronological cross-validation and not a reason to reshuffle time. We reuse the same breast-cancer dataset, attach a fake timestamp, and reduce the prevalence of the positive class in the last test window. .. GENERATED FROM PYTHON SOURCE LINES 195-210 .. code-block:: Python from sklearn.model_selection import TimeSeriesSplit n_splits_time = 5 last_test_start = len(X) - (len(X) // (n_splits_time + 1)) y_time = y.copy() rng_time = np.random.default_rng(seed=2) y_time.iloc[last_test_start:] = rng_time.choice( [0, 1], size=len(X) - last_test_start, p=[0.95, 0.05] ) timestamps = pd.date_range("2020-01-01", periods=len(X), freq="D") df_time = X.assign(timestamp=timestamps, target=y_time) .. GENERATED FROM PYTHON SOURCE LINES 211-213 As for the grouped section, we use a :class:`~skrub.DataOp` to declare the time-series split directly on the data. .. GENERATED FROM PYTHON SOURCE LINES 213-221 .. code-block:: Python data_time = skrub.var("data_time", df_time) X_time_op = data_time.drop(columns=["timestamp", "target"]).skb.mark_as_X( cv=TimeSeriesSplit(n_splits=n_splits_time), ) y_time_op = data_time["target"].skb.mark_as_y() learner_time = X_time_op.skb.apply(model, y=y_time_op).skb.make_learner() .. GENERATED FROM PYTHON SOURCE LINES 222-227 .. code-block:: Python report_time = skore.evaluate(learner_time, data={"data_time": df_time}, pos_label=1) report_time.metrics.summarize(data_source="test").frame( aggregate=None, flat_index=False ) .. raw:: html

estimator	SkrubLearner
split	Split #0	Split #1	Split #2	Split #3	Split #4
metric
score	0.968085	0.957447	0.968085	0.978723	0.276596
accuracy	0.968085	0.957447	0.968085	0.978723	0.276596
precision	0.967213	0.977778	0.985507	1.000000	0.042857
recall	0.983333	0.936170	0.971429	0.972973	0.750000
roc_auc	0.994118	0.995926	0.986905	1.000000	0.397222
log_loss	0.103022	0.095178	0.090876	0.067273	3.835778
brier_score	0.027234	0.027583	0.021152	0.016549	0.664764
fit_time	0.196051	0.198472	0.174604	0.168912	0.200764
predict_time	0.130211	0.119480	0.095341	0.114247	0.118238

.. GENERATED FROM PYTHON SOURCE LINES 228-231 Split #4 underperforms because the positive class is now rare in that window. SKD003 here is expected from chronological cross-validation, not a reason to reshuffle time. .. GENERATED FROM PYTHON SOURCE LINES 231-234 .. code-block:: Python report_time.checks.summarize(fast_mode=True) .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 235-242 When the problem is understood: mute SKD003 =========================================== Groups and drift are data properties: investigate, collect more labels on the hard regime if needed, but folds may never look uniform. Once SKD003 is expected, mute it with :func:`~skore.configuration` (or ``ignore=["SKD003"]`` on one summarize call). .. GENERATED FROM PYTHON SOURCE LINES 242-247 .. code-block:: Python with skore.configuration(ignore_checks=["SKD003"]): muted = report_time.checks.summarize(fast_mode=True) muted .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 248-256 Side note: if there is no group or time structure, but an unshuffled :class:`~sklearn.model_selection.KFold` still creates imbalanced folds because class prevalence varies along the collection order (for example a sensor failed for part of the dump), shuffling or stratifying is appropriate *when you are sure that irregularity is accidental and will not appear in production*. That case is the exception where changing the splitter to smooth folds is the right fix; do not use it to hide real groups or time drift. .. GENERATED FROM PYTHON SOURCE LINES 258-269 Conclusion ========== SKD003 is a reminder to inspect unstable cross-validation folds. With a proper splitter, firing often means the evaluation exposed a bad batch, a hard group, or temporal shift. Fix what you can (for example a corrupted label window). When the structure is intrinsic, keep the honest splitter, document the outlier regime, mute SKD003 via configuration, and collect more data on that regime if you need better coverage. Avoid shuffled cross-validation as a way to make the check disappear when groups or time are real. See also :ref:`SKD013 ` for chronological train/test overlap on hold-out reports. .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 25.918 seconds) .. _sphx_glr_download_auto_examples_pitfalls_and_solutions_plot_skd003_inconsistent_performance.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_skd003_inconsistent_performance.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_skd003_inconsistent_performance.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_skd003_inconsistent_performance.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_

	mean radius	mean texture	mean perimeter	mean area	mean smoothness	mean compactness	mean concavity	mean concave points	mean symmetry	mean fractal dimension	radius error	texture error	perimeter error	area error	smoothness error	compactness error	concavity error	concave points error	symmetry error	fractal dimension error	worst radius	worst texture	worst perimeter	worst area	worst smoothness	worst compactness	worst concavity	worst concave points	worst symmetry	worst fractal dimension
	mean radius	mean texture	mean perimeter	mean area	mean smoothness	mean compactness	mean concavity	mean concave points	mean symmetry	mean fractal dimension	radius error	texture error	perimeter error	area error	smoothness error	compactness error	concavity error	concave points error	symmetry error	fractal dimension error	worst radius	worst texture	worst perimeter	worst area	worst smoothness	worst compactness	worst concavity	worst concave points	worst symmetry	worst fractal dimension
0	18.0	10.4	123.	1.00e+03	0.118	0.278	0.300	0.147	0.242	0.0787	1.09	0.905	8.59	153.	0.00640	0.0490	0.0537	0.0159	0.0300	0.00619	25.4	17.3	185.	2.02e+03	0.162	0.666	0.712	0.265	0.460	0.119
1	20.6	17.8	133.	1.33e+03	0.0847	0.0786	0.0869	0.0702	0.181	0.0567	0.543	0.734	3.40	74.1	0.00522	0.0131	0.0186	0.0134	0.0139	0.00353	25.0	23.4	159.	1.96e+03	0.124	0.187	0.242	0.186	0.275	0.0890
2	19.7	21.2	130.	1.20e+03	0.110	0.160	0.197	0.128	0.207	0.0600	0.746	0.787	4.58	94.0	0.00615	0.0401	0.0383	0.0206	0.0225	0.00457	23.6	25.5	152.	1.71e+03	0.144	0.424	0.450	0.243	0.361	0.0876
3	11.4	20.4	77.6	386.	0.142	0.284	0.241	0.105	0.260	0.0974	0.496	1.16	3.44	27.2	0.00911	0.0746	0.0566	0.0187	0.0596	0.00921	14.9	26.5	98.9	568.	0.210	0.866	0.687	0.258	0.664	0.173
4	20.3	14.3	135.	1.30e+03	0.100	0.133	0.198	0.104	0.181	0.0588	0.757	0.781	5.44	94.4	0.0115	0.0246	0.0569	0.0188	0.0176	0.00511	22.5	16.7	152.	1.58e+03	0.137	0.205	0.400	0.163	0.236	0.0768

564	21.6	22.4	142.	1.48e+03	0.111	0.116	0.244	0.139	0.173	0.0562	1.18	1.26	7.67	159.	0.0103	0.0289	0.0520	0.0245	0.0111	0.00424	25.4	26.4	166.	2.03e+03	0.141	0.211	0.411	0.222	0.206	0.0712
565	20.1	28.2	131.	1.26e+03	0.0978	0.103	0.144	0.0979	0.175	0.0553	0.765	2.46	5.20	99.0	0.00577	0.0242	0.0395	0.0168	0.0190	0.00250	23.7	38.2	155.	1.73e+03	0.117	0.192	0.322	0.163	0.257	0.0664
566	16.6	28.1	108.	858.	0.0846	0.102	0.0925	0.0530	0.159	0.0565	0.456	1.07	3.42	48.5	0.00590	0.0373	0.0473	0.0156	0.0132	0.00389	19.0	34.1	127.	1.12e+03	0.114	0.309	0.340	0.142	0.222	0.0782
567	20.6	29.3	140.	1.26e+03	0.118	0.277	0.351	0.152	0.240	0.0702	0.726	1.59	5.77	86.2	0.00652	0.0616	0.0712	0.0166	0.0232	0.00619	25.7	39.4	185.	1.82e+03	0.165	0.868	0.939	0.265	0.409	0.124
568	7.76	24.5	47.9	181.	0.0526	0.0436	0.00	0.00	0.159	0.0588	0.386	1.43	2.55	19.1	0.00719	0.00466	0.00	0.00	0.0268	0.00278	9.46	30.4	59.2	269.	0.0900	0.0644	0.00	0.00	0.287	0.0704

	steps steps: list of tuples List of (name of step, estimator) tuples that are to be chained in sequential order. To be compatible with the scikit-learn API, all steps must define `fit`. All non-last steps must also define `transform`. See :ref:`Combining Estimators <combining_estimators>` for more details.	[('tablevectorizer', ...), ('simpleimputer', ...), ...]
	transform_input transform_input: list of str, default=None The names of the :term:`metadata` parameters that should be transformed by the pipeline before passing it to the step consuming it. This enables transforming some input arguments to ``fit`` (other than ``X``) to be transformed by the steps of the pipeline up to the step which requires them. Requirement is defined via :ref:`metadata routing <metadata_routing>`. For instance, this can be used to pass a validation set through the pipeline. You can only set this if metadata routing is enabled, which you can enable using ``sklearn.set_config(enable_metadata_routing=True)``. .. versionadded:: 1.6	None
	memory memory: str or object with the joblib.Memory interface, default=None Used to cache the fitted transformers of the pipeline. The last step will never be cached, even if it is a transformer. By default, no caching is performed. If a string is given, it is the path to the caching directory. Enabling caching triggers a clone of the transformers before fitting. Therefore, the transformer instance given to the pipeline cannot be inspected directly. Use the attribute ``named_steps`` or ``steps`` to inspect estimators within the pipeline. Caching the transformers is advantageous when fitting is time consuming. See :ref:`sphx_glr_auto_examples_neighbors_plot_caching_nearest_neighbors.py` for an example on how to enable caching.	None
	verbose verbose: bool, default=False If True, the time elapsed while fitting each step will be printed as it is completed.	False

	low_cardinality low_cardinality: transformer, "passthrough" or "drop", default=OneHotEncoder instance The transformer for string or categorical columns with strictly fewer than ``cardinality_threshold`` unique values. By default, we use a :class:`~sklearn.preprocessing.OneHotEncoder` that ignores unknown categories and drop one of the transformed columns if the feature contains only 2 categories.	OneHotEncoder..._output=False)
	high_cardinality high_cardinality: transformer, "passthrough" or "drop", default=StringEncoder instance The transformer for string or categorical columns with at least ``cardinality_threshold`` unique values. The default is a :class:`~skrub.StringEncoder` with 30 components (30 output columns for each input). .. versionchanged:: 0.6.0 The default ``high_cardinality`` encoder has been changed from :class:`~skrub.GapEncoder` to :class:`~skrub.StringEncoder`.	StringEncoder()
	numeric numeric: transformer, "passthrough" or "drop", default="passthrough" The transformer for numeric columns (floats, ints, booleans).	PassThrough()
	datetime datetime: transformer, "passthrough" or "drop", default=DatetimeEncoder instance The transformer for date and datetime columns. By default, we use a :class:`~skrub.DatetimeEncoder`.	DatetimeEncod...ding='spline')
	cardinality_threshold cardinality_threshold: int, default=40 String and categorical columns with a number of unique values strictly smaller than this threshold are handled by the transformer ``low_cardinality``, the rest are handled by the transformer ``high_cardinality``.	40
	specific_transformers specific_transformers: list of (transformer, list of column names) pairs, default=() Override the categories above for the given columns and force using the specified transformer. This disables any preprocessing usually done by the TableVectorizer; the columns are passed to the transformer without any modification. A column is not allowed to appear twice in ``specific_transformers``. Consider wrapping the ``TableVectorizer`` in :class:`~skrub.ApplyToCols` to select or exclude specific columns from the processing. Alternatively, the :ref:`skrub Data Ops <user_guide_data_ops_index>` allows for more complex pre-processing.	()
	drop_null_fraction drop_null_fraction: float or None, default=1.0 Fraction of null above which the column is dropped. If `drop_null_fraction` is set to ``1.0``, the column is dropped if it contains only nulls or NaNs (this is the default behavior). If `drop_null_fraction` is a number in ``[0.0, 1.0)``, the column is dropped if the fraction of nulls is strictly larger than `drop_null_fraction`. If `drop_null_fraction` is ``None``, this selection is disabled: no columns are dropped based on the number of null values they contain.	1.0
	drop_if_constant drop_if_constant: bool, default=False If set to true, drop columns that contain a single unique value. Note that missing values are considered as one additional distinct value.	False
	drop_if_unique drop_if_unique: bool, default=False If set to true, drop columns that contain only unique values, i.e., the number of unique values is equal to the number of rows in the column. Numeric columns are never dropped. .. deprecated:: 0.9.0 This functionality can drop informative columns and is unlikely to be of use in practice. It is therefore deprecated and will be removed in a future version.	False
	datetime_format datetime_format: str, default=None The format to use when parsing dates. If None, the format is inferred.	None
	null_strings null_strings: str or sequence of str, default=None Additional strings to consider as null values, beyond the default list.	None
	n_jobs n_jobs: int, default=None Number of jobs to run in parallel. ``None`` means 1 unless in a joblib ``parallel_backend`` context. ``-1`` means using all processors.	None

	periodic_encoding periodic_encoding: 'circular', 'spline', or None, default=None Add periodic features with different granularities. Add periodic features using either trigonometric (``circular``) or ``spline`` encoding. If ``None``, no periodic encoding is applied.	'spline'
	resolution resolution: str or None, default="hour" If a string, extract up to this resolution. Must be "year", "month", "day", "hour", "minute", "second", "microsecond", or "nanosecond". For example, ``resolution="day"`` generates the features "year", "month", and "day" only. If the input column contains dates with no time information, time features ("hour", "minute", … ) are never extracted. If ``None``, the features listed above are not extracted (but day of the week and total seconds may still be extracted, see below).	'hour'
	add_weekday add_weekday: bool, default=False Extract the day of the week as a numerical feature from 1 (Monday) to 7 (Sunday).	False
	add_total_seconds add_total_seconds: bool, default=True Add the total number of seconds since the Unix epoch (00:00:00 UTC on 1 January 1970).	True
	add_day_of_year add_day_of_year: bool, default=False Add the day of year (ordinal day) as an integer in the range 1 to 365 (or 366 in the case of leap years). January 1st will be day 1, December 31st will be day 365 on non-leap years.	False

	drop drop: {'first', 'if_binary'} or an array-like of shape (n_features,), default=None Specifies a methodology to use to drop one of the categories per feature. This is useful in situations where perfectly collinear features cause problems, such as when feeding the resulting data into an unregularized linear regression model. However, dropping one category breaks the symmetry of the original representation and can therefore induce a bias in downstream models, for instance for penalized linear classification or regression models. - None : retain all features (the default). - 'first' : drop the first category in each feature. If only one category is present, the feature will be dropped entirely. - 'if_binary' : drop the first category in each feature with two categories. Features with 1 or more than 2 categories are left intact. - array : ``drop[i]`` is the category in feature ``X[:, i]`` that should be dropped. When `max_categories` or `min_frequency` is configured to group infrequent categories, the dropping behavior is handled after the grouping. .. versionadded:: 0.21 The parameter `drop` was added in 0.21. .. versionchanged:: 0.23 The option `drop='if_binary'` was added in 0.23. .. versionchanged:: 1.1 Support for dropping infrequent categories.	'if_binary'
	sparse_output sparse_output: bool, default=True When ``True``, it returns a SciPy sparse matrix/array in "Compressed Sparse Row" (CSR) format. .. versionadded:: 1.2 `sparse` was renamed to `sparse_output`	False
	dtype dtype: number type, default=np.float64 Desired dtype of output.	'float32'
	handle_unknown handle_unknown: {'error', 'ignore', 'infrequent_if_exist', 'warn'}, default='error' Specifies the way unknown categories are handled during :meth:`transform`. - 'error' : Raise an error if an unknown category is present during transform. - 'ignore' : When an unknown category is encountered during transform, the resulting one-hot encoded columns for this feature will be all zeros. In the inverse transform, an unknown category will be denoted as None. - 'infrequent_if_exist' : When an unknown category is encountered during transform, the resulting one-hot encoded columns for this feature will map to the infrequent category if it exists. The infrequent category will be mapped to the last position in the encoding. During inverse transform, an unknown category will be mapped to the category denoted `'infrequent'` if it exists. If the `'infrequent'` category does not exist, then :meth:`transform` and :meth:`inverse_transform` will handle an unknown category as with `handle_unknown='ignore'`. Infrequent categories exist based on `min_frequency` and `max_categories`. Read more in the :ref:`User Guide <encoder_infrequent_categories>`. - 'warn' : When an unknown category is encountered during transform a warning is issued, and the encoding then proceeds as described for `handle_unknown="infrequent_if_exist"`. .. versionchanged:: 1.1 `'infrequent_if_exist'` was added to automatically handle unknown categories and infrequent categories. .. versionadded:: 1.6 The option `"warn"` was added in 1.6.	'ignore'
	categories categories: 'auto' or a list of array-like, default='auto' Categories (unique values) per feature: - 'auto' : Determine categories automatically from the training data. - list : ``categories[i]`` holds the categories expected in the ith column. The passed categories should not mix strings and numeric values within a single feature, and should be sorted in case of numeric values. The used categories can be found in the ``categories_`` attribute. .. versionadded:: 0.20	'auto'
	min_frequency min_frequency: int or float, default=None Specifies the minimum frequency below which a category will be considered infrequent. - If `int`, categories with a smaller cardinality will be considered infrequent. - If `float`, categories with a smaller cardinality than `min_frequency * n_samples` will be considered infrequent. .. versionadded:: 1.1 Read more in the :ref:`User Guide <encoder_infrequent_categories>`.	None
	max_categories max_categories: int, default=None Specifies an upper limit to the number of output features for each input feature when considering infrequent categories. If there are infrequent categories, `max_categories` includes the category representing the infrequent categories along with the frequent categories. If `None`, there is no limit to the number of output features. .. versionadded:: 1.1 Read more in the :ref:`User Guide <encoder_infrequent_categories>`.	None
	feature_name_combiner feature_name_combiner: "concat" or callable, default="concat" Callable with signature `def callable(input_feature, category)` that returns a string. This is used to create feature names to be returned by :meth:`get_feature_names_out`. `"concat"` concatenates encoded feature name and category with `feature + "_" + str(category)`.E.g. feature X with values 1, 6, 7 create feature names `X_1, X_6, X_7`. .. versionadded:: 1.3	'concat'

	n_components n_components: int, default=30 Number of components to be used for the singular value decomposition (SVD). Must be a positive integer.	30
	vectorizer vectorizer: str, "tfidf" or "hashing", default="tfidf" Vectorizer to apply to the strings, either `tfidf` or `hashing` for scikit-learn TfidfVectorizer or HashingVectorizer respectively.	'tfidf'
	ngram_range ngram_range: tuple of (int, int) pairs, default=(3,4) The lower and upper boundary of the range of n-values for different n-grams to be extracted. All values of n such that min_n <= n <= max_n will be used. For example an ``ngram_range`` of ``(1, 1)`` means only unigrams, ``(1, 2)`` means unigrams and bigrams, and ``(2, 2)`` means only bigrams.	(3, ...)
	analyzer analyzer: str, "char", "word" or "char_wb", default="char_wb" Whether the feature should be made of word or character n-grams. Option ``char_wb`` creates character n-grams only from text inside word boundaries; n-grams at the edges of words are padded with space.	'char_wb'
	stop_words stop_words: {'english'}, list, default=None If 'english', a built-in stop word list for English is used. There are several known issues with 'english' and you should consider an alternative (see `Using stop words <https://scikit-learn.org/stable/modules/feature_extraction.html#using-stop-words>`_). If a list, that list is assumed to contain stop words, all of which will be removed from the resulting tokens. Only applies if ``analyzer == 'word'``. If None, no stop words will be used.	None
	random_state random_state: int, RandomState instance or None, default=None Used during randomized svd. Pass an int for reproducible results across multiple function calls.	None
	vocabulary vocabulary: Mapping or iterable, default=None In case of "tfidf" vectorizer, the vocabulary mapping passed to the vectorizer. Either a Mapping (e.g., a dict) where keys are terms and values are indices in the feature matrix, or an iterable over terms.	None

	add_indicator add_indicator: bool, default=False If True, a :class:`MissingIndicator` transform will stack onto output of the imputer's transform. This allows a predictive estimator to account for missingness despite imputation. If a feature has no missing values at fit/train time, the feature won't appear on the missing indicator even if there are missing values at transform/test time.	True
	missing_values missing_values: int, float, str, np.nan, None or pandas.NA, default=np.nan The placeholder for the missing values. All occurrences of `missing_values` will be imputed. For pandas' dataframes with nullable integer dtypes with missing values, `missing_values` can be set to either `np.nan` or `pd.NA`.	nan
	strategy strategy: str or Callable, default='mean' The imputation strategy. - If "mean", then replace missing values using the mean along each column. Can only be used with numeric data. - If "median", then replace missing values using the median along each column. Can only be used with numeric data. - If "most_frequent", then replace missing using the most frequent value along each column. Can be used with strings or numeric data. If there is more than one such value, only the smallest is returned. - If "constant", then replace missing values with fill_value. Can be used with strings or numeric data. - If an instance of Callable, then replace missing values using the scalar statistic returned by running the callable over a dense 1d array containing non-missing values of each column. .. versionadded:: 0.20 strategy="constant" for fixed value imputation. .. versionadded:: 1.5 strategy=callable for custom value imputation.	'mean'
	fill_value fill_value: str or numerical value, default=None When strategy == "constant", `fill_value` is used to replace all occurrences of missing_values. For string or object data types, `fill_value` must be a string. If `None`, `fill_value` will be 0 when imputing numerical data and "missing_value" for strings or object data types.	None
	copy copy: bool, default=True If True, a copy of X will be created. If False, imputation will be done in-place whenever possible. Note that, in the following cases, a new copy will always be made, even if `copy=False`: - If `X` is not an array of floating values; - If `X` is encoded as a CSR matrix; - If `add_indicator=True`.	True
	keep_empty_features keep_empty_features: bool, default=False If True, features that consist exclusively of missing values when `fit` is called are returned in results when `transform` is called. The imputed value is always `0` except when `strategy="constant"` in which case `fill_value` will be used instead. .. versionadded:: 1.2	False

	max_absolute_value max_absolute_value: float, default=3.0 Maximum absolute value that the transformed data can take.	5
	quantile_range quantile_range: tuple of float, default=(0.25, 0.75) The quantiles used to compute the scaling factor. The first value is the lower quantile and the second value is the upper quantile. The default values are the 25th and 75th percentiles, respectively. The quantiles are used to compute the scaling factor for the robust scaling step. The quantiles are computed from the finite values in the input column. If the two quantiles are equal, the scaling factor is computed from the 0th and 100th percentiles (i.e., the minimum and maximum values of the finite values in the input column).	(25.0, ...)

	penalty penalty: {'l1', 'l2', 'elasticnet', None}, default='l2' Specify the norm of the penalty: - `None`: no penalty is added; - `'l2'`: add an L2 penalty term and it is the default choice; - `'l1'`: add an L1 penalty term; - `'elasticnet'`: both L1 and L2 penalty terms are added. .. warning:: Some penalties may not work with some solvers. See the parameter `solver` below, to know the compatibility between the penalty and solver. .. versionadded:: 0.19 l1 penalty with SAGA solver (allowing 'multinomial' + L1) .. deprecated:: 1.8 `penalty` was deprecated in version 1.8 and will be removed in 1.10. Use `l1_ratio` and `C` instead. `l1_ratio=0` for `penalty='l2'`, `l1_ratio=1` for `penalty='l1'`, `l1_ratio` set to any float between 0 and 1 for `penalty='elasticnet'`, and `C=np.inf` for `penalty=None`.	'deprecated'
	C C: float, default=1.0 Inverse of regularization strength; must be a positive float. Like in support vector machines, smaller values specify stronger regularization. `C=np.inf` results in unpenalized logistic regression. For a visual example on the effect of tuning the `C` parameter with an L1 penalty, see: :ref:`sphx_glr_auto_examples_linear_model_plot_logistic_path.py`.	1.0
	l1_ratio l1_ratio: float, default=0.0 The Elastic-Net mixing parameter, with `0 <= l1_ratio <= 1`. Setting `l1_ratio=1` gives a pure L1-penalty, setting `l1_ratio=0` a pure L2-penalty. Any value between 0 and 1 gives an Elastic-Net penalty of the form `l1_ratio * L1 + (1 - l1_ratio) * L2`. .. warning:: Certain values of `l1_ratio`, i.e. some penalties, may not work with some solvers. See the parameter `solver` below, to know the compatibility between the penalty and solver. .. versionchanged:: 1.8 Default value changed from None to 0.0. .. deprecated:: 1.8 `None` is deprecated and will be removed in version 1.10. Always use `l1_ratio` to specify the penalty type.	0.0
	dual dual: bool, default=False Dual (constrained) or primal (regularized, see also :ref:`this equation <regularized-logistic-loss>`) formulation. Dual formulation is only implemented for l2 penalty with liblinear solver. Prefer `dual=False` when n_samples > n_features.	False
	tol tol: float, default=1e-4 Tolerance for stopping criteria.	0.0001
	fit_intercept fit_intercept: bool, default=True Specifies if a constant (a.k.a. bias or intercept) should be added to the decision function.	True
	intercept_scaling intercept_scaling: float, default=1 Useful only when the solver `liblinear` is used and `self.fit_intercept` is set to `True`. In this case, `x` becomes `[x, self.intercept_scaling]`, i.e. a "synthetic" feature with constant value equal to `intercept_scaling` is appended to the instance vector. The intercept becomes ``intercept_scaling * synthetic_feature_weight``. .. note:: The synthetic feature weight is subject to L1 or L2 regularization as all other features. To lessen the effect of regularization on synthetic feature weight (and therefore on the intercept) `intercept_scaling` has to be increased.	1
	class_weight class_weight: dict or 'balanced', default=None Weights associated with classes in the form ``{class_label: weight}``. If not given, all classes are supposed to have weight one. The "balanced" mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as ``n_samples / (n_classes * np.bincount(y))``. Note that these weights will be multiplied with sample_weight (passed through the fit method) if sample_weight is specified. .. versionadded:: 0.17 class_weight='balanced'	None
	random_state random_state: int, RandomState instance, default=None Used when ``solver`` == 'sag', 'saga' or 'liblinear' to shuffle the data. See :term:`Glossary <random_state>` for details.	None
	solver solver: {'lbfgs', 'liblinear', 'newton-cg', 'newton-cholesky', 'sag', 'saga'}, default='lbfgs' Algorithm to use in the optimization problem. Default is 'lbfgs'. To choose a solver, you might want to consider the following aspects: - 'lbfgs' is a good default solver because it works reasonably well for a wide class of problems. - For :term:`multiclass` problems (`n_classes >= 3`), all solvers except 'liblinear' minimize the full multinomial loss, 'liblinear' will raise an error. - 'newton-cholesky' is a good choice for `n_samples` >> `n_features * n_classes`, especially with one-hot encoded categorical features with rare categories. Be aware that the memory usage of this solver has a quadratic dependency on `n_features * n_classes` because it explicitly computes the full Hessian matrix. - For small datasets, 'liblinear' is a good choice, whereas 'sag' and 'saga' are faster for large ones; - 'liblinear' can only handle binary classification by default. To apply a one-versus-rest scheme for the multiclass setting one can wrap it with the :class:`~sklearn.multiclass.OneVsRestClassifier`. .. warning:: The choice of the algorithm depends on the penalty chosen (`l1_ratio=0` for L2-penalty, `l1_ratio=1` for L1-penalty and `0 < l1_ratio < 1` for Elastic-Net) and on (multinomial) multiclass support: ================= ======================== ====================== solver l1_ratio multinomial multiclass ================= ======================== ====================== 'lbfgs' l1_ratio=0 yes 'liblinear' l1_ratio=1 or l1_ratio=0 no 'newton-cg' l1_ratio=0 yes 'newton-cholesky' l1_ratio=0 yes 'sag' l1_ratio=0 yes 'saga' 0<=l1_ratio<=1 yes ================= ======================== ====================== .. note:: 'sag' and 'saga' fast convergence is only guaranteed on features with approximately the same scale. You can preprocess the data with a scaler from :mod:`sklearn.preprocessing`. .. seealso:: Refer to the :ref:`User Guide <Logistic_regression>` for more information regarding :class:`LogisticRegression` and more specifically the :ref:`Table <logistic_regression_solvers>` summarizing solver/penalty supports. .. versionadded:: 0.17 Stochastic Average Gradient (SAG) descent solver. Multinomial support in version 0.18. .. versionadded:: 0.19 SAGA solver. .. versionchanged:: 0.22 The default solver changed from 'liblinear' to 'lbfgs' in 0.22. .. versionadded:: 1.2 newton-cholesky solver. Multinomial support in version 1.6.	'lbfgs'
	max_iter max_iter: int, default=100 Maximum number of iterations taken for the solvers to converge.	100
	verbose verbose: int, default=0 For the liblinear and lbfgs solvers set verbose to any positive number for verbosity.	0
	warm_start warm_start: bool, default=False When set to True, reuse the solution of the previous call to fit as initialization, otherwise, just erase the previous solution. Useless for liblinear solver. See :term:`the Glossary <warm_start>`. .. versionadded:: 0.17 warm_start to support lbfgs, newton-cg, sag, saga solvers.	False
	n_jobs n_jobs: int, default=None Does not have any effect. .. deprecated:: 1.8 `n_jobs` is deprecated in version 1.8 and will be removed in 1.10.	None

mean radius

mean texture

mean perimeter

mean area

mean smoothness

mean compactness

mean concavity

mean concave points

mean symmetry

mean fractal dimension

radius error

texture error

perimeter error

area error

smoothness error

compactness error

concavity error

concave points error

symmetry error

fractal dimension error

worst radius

worst texture

worst perimeter

worst area

worst smoothness

worst compactness

worst concavity

worst concave points

worst symmetry

worst fractal dimension

mean radius

mean texture

mean perimeter

mean area

mean smoothness

mean compactness

mean concavity

mean concave points

mean symmetry

mean fractal dimension

radius error

texture error

perimeter error

area error

smoothness error

compactness error

concavity error

concave points error

symmetry error

fractal dimension error

worst radius

worst texture

worst perimeter

worst area

worst smoothness

worst compactness

worst concavity

worst concave points

worst symmetry

worst fractal dimension

Please enable javascript

target

target

Please enable javascript