.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/pitfalls_and_solutions/plot_skd005_underrepresented_classes.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_pitfalls_and_solutions_plot_skd005_underrepresented_classes.py: .. _example_skd005_underrepresented_classes: SKD005 - Underrepresented classes ================================= :ref:`SKD005 ` flags a multiclass task when one or more classes each represent less than 10 % of rows. Overall accuracy can look acceptable while rare labels are barely learned. This notebook is about how to work with that rarity once the check fires: we do not try to make SKD005 disappear by reshaping the class histogram. What to do instead: - report absolute counts as well as percentages, - evaluate threshold-free / probabilistic metrics (such as log-loss) before per class precision and accuracy, - collect more rare-class labels when possible, without treating a cleared check as success, - correct for prevalence shift if acquisition oversamples rare types. For binary rare-event tasks (threshold tuning, when ``class_weight`` is a risky shortcut), see :ref:`skd004-high-class-imbalance` and: https://probabl-ai.github.io/calibration-cost-sensitive-learning/content/notebooks/imbalanced_classification.html We take a 10,000-row stratified subsample of Covertype so several forest types fall below 10 %. The goal is to keep natural prevalence visible, judge the multiclass model honestly, then see what extra rare-class rows can do. .. GENERATED FROM PYTHON SOURCE LINES 32-38 Load the Covertype dataset ========================== The full Covertype task has seven forest types. A small stratified subsample keeps frequent classes well represented while types 3-7 drop below 10 % each. We keep the unused rows as a pool for the "more rare-class data" section later. .. GENERATED FROM PYTHON SOURCE LINES 38-56 .. code-block:: Python import pandas as pd from sklearn.datasets import fetch_covtype from sklearn.model_selection import train_test_split df = fetch_covtype(as_frame=True).frame y_full = df["Cover_Type"].astype(str) X_full = df.drop(columns=["Cover_Type"]) X, X_pool, y, y_pool = train_test_split( X_full, y_full, train_size=10_000, stratify=y_full, random_state=42, ) y, y_pool = y.rename("class"), y_pool.rename("class") .. GENERATED FROM PYTHON SOURCE LINES 57-58 Let us inspect the feature matrix with :class:`~skrub.TableReport`. .. GENERATED FROM PYTHON SOURCE LINES 58-63 .. code-block:: Python from skrub import TableReport TableReport(X) .. raw:: html

Column	Column name	dtype	Is sorted	Unique values	Mean	Std	Min	Median	Max
0	Elevation	Float64DType	False	1375 (13.8%)	2.96e+03	280.	1.88e+03	3.00e+03	3.86e+03
1	Aspect	Float64DType	False	361 (3.6%)	157.	112.	0.00	128.	360.
2	Slope	Float64DType	False	50 (0.5%)	14.2	7.48	0.00	13.0	51.0
3	Horizontal_Distance_To_Hydrology	Float64DType	False	367 (3.7%)	270.	213.	0.00	218.	1.34e+03
4	Vertical_Distance_To_Hydrology	Float64DType	False	418 (4.2%)	47.6	59.8	-152.	30.0	544.
5	Horizontal_Distance_To_Roadways	Float64DType	False	3489 (34.9%)	2.34e+03	1.55e+03	0.00	2.01e+03	7.04e+03
6	Hillshade_9am	Float64DType	False	170 (1.7%)	212.	27.2	0.00	217.	254.
7	Hillshade_Noon	Float64DType	False	126 (1.3%)	223.	19.5	100.	226.	254.
8	Hillshade_3pm	Float64DType	False	235 (2.4%)	143.	38.5	0.00	143.	248.
9	Horizontal_Distance_To_Fire_Points	Float64DType	False	2938 (29.4%)	1.97e+03	1.32e+03	0.00	1.68e+03	6.93e+03
10	Wilderness_Area_0	Float64DType	False	2 (< 0.1%)	0.445	0.497	0.00	0.00	1.00
11	Wilderness_Area_1	Float64DType	False	2 (< 0.1%)	0.0491	0.216	0.00	0.00	1.00
12	Wilderness_Area_2	Float64DType	False	2 (< 0.1%)	0.441	0.497	0.00	0.00	1.00
13	Wilderness_Area_3	Float64DType	False	2 (< 0.1%)	0.0650	0.247	0.00	0.00	1.00
14	Soil_Type_0	Float64DType	False	2 (< 0.1%)	0.00530	0.0726	0.00	0.00	1.00
15	Soil_Type_1	Float64DType	False	2 (< 0.1%)	0.0140	0.117	0.00	0.00	1.00
16	Soil_Type_2	Float64DType	False	2 (< 0.1%)	0.00680	0.0822	0.00	0.00	1.00
17	Soil_Type_3	Float64DType	False	2 (< 0.1%)	0.0220	0.147	0.00	0.00	1.00
18	Soil_Type_4	Float64DType	False	2 (< 0.1%)	0.00300	0.0547	0.00	0.00	1.00
19	Soil_Type_5	Float64DType	False	2 (< 0.1%)	0.0124	0.111	0.00	0.00	1.00
20	Soil_Type_6	Float64DType	False	2 (< 0.1%)	0.000200	0.0141	0.00	0.00	1.00
21	Soil_Type_7	Float64DType	False	2 (< 0.1%)	0.000400	0.0200	0.00	0.00	1.00
22	Soil_Type_8	Float64DType	False	2 (< 0.1%)	0.00160	0.0400	0.00	0.00	1.00
23	Soil_Type_9	Float64DType	False	2 (< 0.1%)	0.0551	0.228	0.00	0.00	1.00
24	Soil_Type_10	Float64DType	False	2 (< 0.1%)	0.0230	0.150	0.00	0.00	1.00
25	Soil_Type_11	Float64DType	False	2 (< 0.1%)	0.0518	0.222	0.00	0.00	1.00
26	Soil_Type_12	Float64DType	False	2 (< 0.1%)	0.0318	0.175	0.00	0.00	1.00
27	Soil_Type_13	Float64DType	False	2 (< 0.1%)	0.00140	0.0374	0.00	0.00	1.00
28	Soil_Type_14	Float64DType	True	1 (< 0.1%)	0.00	0.00
29	Soil_Type_15	Float64DType	False	2 (< 0.1%)	0.00520	0.0719	0.00	0.00	1.00
30	Soil_Type_16	Float64DType	False	2 (< 0.1%)	0.00610	0.0779	0.00	0.00	1.00
31	Soil_Type_17	Float64DType	False	2 (< 0.1%)	0.00440	0.0662	0.00	0.00	1.00
32	Soil_Type_18	Float64DType	False	2 (< 0.1%)	0.00620	0.0785	0.00	0.00	1.00
33	Soil_Type_19	Float64DType	False	2 (< 0.1%)	0.0145	0.120	0.00	0.00	1.00
34	Soil_Type_20	Float64DType	False	2 (< 0.1%)	0.00190	0.0435	0.00	0.00	1.00
35	Soil_Type_21	Float64DType	False	2 (< 0.1%)	0.0560	0.230	0.00	0.00	1.00
36	Soil_Type_22	Float64DType	False	2 (< 0.1%)	0.0991	0.299	0.00	0.00	1.00
37	Soil_Type_23	Float64DType	False	2 (< 0.1%)	0.0360	0.186	0.00	0.00	1.00
38	Soil_Type_24	Float64DType	False	2 (< 0.1%)	0.000600	0.0245	0.00	0.00	1.00
39	Soil_Type_25	Float64DType	False	2 (< 0.1%)	0.00560	0.0746	0.00	0.00	1.00
40	Soil_Type_26	Float64DType	False	2 (< 0.1%)	0.00160	0.0400	0.00	0.00	1.00
41	Soil_Type_27	Float64DType	False	2 (< 0.1%)	0.00160	0.0400	0.00	0.00	1.00
42	Soil_Type_28	Float64DType	False	2 (< 0.1%)	0.195	0.396	0.00	0.00	1.00
43	Soil_Type_29	Float64DType	False	2 (< 0.1%)	0.0526	0.223	0.00	0.00	1.00
44	Soil_Type_30	Float64DType	False	2 (< 0.1%)	0.0447	0.207	0.00	0.00	1.00
45	Soil_Type_31	Float64DType	False	2 (< 0.1%)	0.0890	0.285	0.00	0.00	1.00
46	Soil_Type_32	Float64DType	False	2 (< 0.1%)	0.0819	0.274	0.00	0.00	1.00
47	Soil_Type_33	Float64DType	False	2 (< 0.1%)	0.00330	0.0574	0.00	0.00	1.00
48	Soil_Type_34	Float64DType	False	2 (< 0.1%)	0.00360	0.0599	0.00	0.00	1.00
49	Soil_Type_35	Float64DType	False	2 (< 0.1%)	0.000400	0.0200	0.00	0.00	1.00
50	Soil_Type_36	Float64DType	False	2 (< 0.1%)	0.000400	0.0200	0.00	0.00	1.00
51	Soil_Type_37	Float64DType	False	2 (< 0.1%)	0.0217	0.146	0.00	0.00	1.00
52	Soil_Type_38	Float64DType	False	2 (< 0.1%)	0.0239	0.153	0.00	0.00	1.00
53	Soil_Type_39	Float64DType	False	2 (< 0.1%)	0.0157	0.124	0.00	0.00	1.00

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

.. GENERATED FROM PYTHON SOURCE LINES 64-66 Clicking the column in the target's `TableReport` brings a class histogram that shows that the classes are not evenly distributed. .. GENERATED FROM PYTHON SOURCE LINES 66-69 .. code-block:: Python TableReport(y) .. raw:: html

	class
333,153	2
73,090	1
559,685	2
221,753	2
428,256	2

300,351	2
135,386	2
132,505	2
494,080	2
105,915	1

Column	Column name	dtype	Is sorted	Null values	Unique values	Mean	Std	Min	Median	Max
0	class	ObjectDType	False	0 (0.0%)	7 (< 0.1%)

Please enable javascript

.. GENERATED FROM PYTHON SOURCE LINES 70-74 Let us also look at absolute counts: type 4 is under 1 % with only a few dozen rows in this subsample: even a good multiclass model has little to learn from there. Types such as 6 are also under the 10 % SKD005 bar, but with a few hundred rows they are less hopeless. .. GENERATED FROM PYTHON SOURCE LINES 74-79 .. code-block:: Python shares = y.value_counts(normalize=True).sort_index() counts = y.value_counts().sort_index() pd.concat([shares.round(4), counts], axis=1) .. raw:: html

	proportion	count
class
1	0.3646	3646
2	0.4876	4876
3	0.0615	615
4	0.0047	47
5	0.0164	164
6	0.0299	299
7	0.0353	353

.. GENERATED FROM PYTHON SOURCE LINES 80-86 Trigger SKD005: default classifier on imbalanced classes ======================================================== A default gradient boosting classifier does not change label counts. We ignore SKD008 to avoid correlated-feature warnings from Covertype's constant soil one-hots. .. GENERATED FROM PYTHON SOURCE LINES 86-102 .. code-block:: Python import skore from sklearn.ensemble import HistGradientBoostingClassifier from skore import TrainTestSplit splitter = TrainTestSplit(test_size=0.2, random_state=42, stratify=y) classifier = HistGradientBoostingClassifier(random_state=42) report = skore.evaluate( classifier, X=X, y=y, splitter=splitter, ) report .. raw:: html

Metric	Label	Average	HistGradientBoostingClassifier
Accuracy			0.738000
Precision	1		0.765805
Precision	2		0.783453
Precision	3		0.674419
Precision	4		0.250000
Precision	5		0.270270
Precision	6		0.372881
Precision	7		0.650000
Precision		macro	0.538118
Recall	1		0.731139
Recall	2		0.786667
Recall	3		0.707317
Recall	4		0.555556
Recall	5		0.303030
Recall	6		0.366667
Recall	7		0.732394
Recall		macro	0.597538
ROC AUC	1		0.862669
ROC AUC	2		0.833460
ROC AUC	3		0.901625
ROC AUC	4		0.881271
ROC AUC	5		0.776972
ROC AUC	6		0.853458
ROC AUC	7		0.951255
ROC AUC		macro	0.865816
Log loss			2.991535
Fit time (s)			5.212368
Predict time (s)			0.099938

HistGradientBoostingClassifier(random_state=42)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

HistGradientBoostingClassifier

?Documentation for HistGradientBoostingClassifieriFitted

Parameters

	random_state random_state: int, RandomState instance or None, default=None Pseudo-random number generator to control the subsampling in the binning process, and the train/validation data split if early stopping is enabled. Pass an int for reproducible output across multiple function calls. See :term:`Glossary <random_state>`.	42
	loss loss: {'log_loss'}, default='log_loss' The loss function to use in the boosting process. For binary classification problems, 'log_loss' is also known as logistic loss, binomial deviance or binary crossentropy. Internally, the model fits one tree per boosting iteration and uses the logistic sigmoid function (expit) as inverse link function to compute the predicted positive class probability. For multiclass classification problems, 'log_loss' is also known as multinomial deviance or categorical crossentropy. Internally, the model fits one tree per boosting iteration and per class and uses the softmax function as inverse link function to compute the predicted probabilities of the classes.	'log_loss'
	learning_rate learning_rate: float, default=0.1 The learning rate, also known as shrinkage. This is used as a multiplicative factor for the leaves values. Use ``1`` for no shrinkage.	0.1
	max_iter max_iter: int, default=100 The maximum number of iterations of the boosting process, i.e. the maximum number of trees for binary classification. For multiclass classification, `n_classes` trees per iteration are built.	100
	max_leaf_nodes max_leaf_nodes: int or None, default=31 The maximum number of leaves for each tree. Must be strictly greater than 1. If None, there is no maximum limit.	31
	max_depth max_depth: int or None, default=None The maximum depth of each tree. The depth of a tree is the number of edges to go from the root to the deepest leaf. Depth isn't constrained by default.	None
	min_samples_leaf min_samples_leaf: int, default=20 The minimum number of samples per leaf. For small datasets with less than a few hundred samples, it is recommended to lower this value since only very shallow trees would be built.	20
	l2_regularization l2_regularization: float, default=0 The L2 regularization parameter penalizing leaves with small hessians. Use ``0`` for no regularization (default).	0.0
	max_features max_features: float, default=1.0 Proportion of randomly chosen features in each and every node split. This is a form of regularization, smaller values make the trees weaker learners and might prevent overfitting. If interaction constraints from `interaction_cst` are present, only allowed features are taken into account for the subsampling. .. versionadded:: 1.4	1.0
	max_bins max_bins: int, default=255 The maximum number of bins to use for non-missing values. Before training, each feature of the input array `X` is binned into integer-valued bins, which allows for a much faster training stage. Features with a small number of unique values may use less than ``max_bins`` bins. In addition to the ``max_bins`` bins, one more bin is always reserved for missing values. Must be no larger than 255.	255
	categorical_features categorical_features: array-like of {bool, int, str} of shape (n_features) or shape (n_categorical_features,), default='from_dtype' Indicates the categorical features. - None : no feature will be considered categorical. - boolean array-like : boolean mask indicating categorical features. - integer array-like : integer indices indicating categorical features. - str array-like: names of categorical features (assuming the training data has feature names). - `"from_dtype"`: dataframe columns with dtype "Categorical" and "Enum" are considered to be categorical features. The input must be a dataframe that is supported by narwhals (or supports it): :func:`narwhals.from_native` must work. This is the case, for instance, for pandas and polars DataFrames. For each categorical feature, there must be at most `max_bins` unique categories. Negative values for categorical features encoded as numeric dtypes are treated as missing values. All categorical values are converted to floating point numbers. This means that categorical values of 1.0 and 1 are treated as the same category. Read more in the :ref:`User Guide <categorical_support_gbdt>`. .. versionadded:: 0.24 .. versionchanged:: 1.2 Added support for feature names. .. versionchanged:: 1.4 Added `"from_dtype"` option. .. versionchanged:: 1.6 The default value changed from `None` to `"from_dtype"`.	'from_dtype'
	monotonic_cst monotonic_cst: array-like of int of shape (n_features) or dict, default=None Monotonic constraint to enforce on each feature are specified using the following integer values: - 1: monotonic increase - 0: no constraint - -1: monotonic decrease If a dict with str keys, map feature to monotonic constraints by name. If an array, the features are mapped to constraints by position. See :ref:`monotonic_cst_features_names` for a usage example. The constraints are only valid for binary classifications and hold over the probability of the positive class. Read more in the :ref:`User Guide <monotonic_cst_gbdt>`. .. versionadded:: 0.23 .. versionchanged:: 1.2 Accept dict of constraints with feature names as keys.	None
	interaction_cst interaction_cst: {"pairwise", "no_interactions"} or sequence of lists/tuples/sets of int, default=None Specify interaction constraints, the sets of features which can interact with each other in child node splits. Each item specifies the set of feature indices that are allowed to interact with each other. If there are more features than specified in these constraints, they are treated as if they were specified as an additional set. The strings "pairwise" and "no_interactions" are shorthands for allowing only pairwise or no interactions, respectively. For instance, with 5 features in total, `interaction_cst=[{0, 1}]` is equivalent to `interaction_cst=[{0, 1}, {2, 3, 4}]`, and specifies that each branch of a tree will either only split on features 0 and 1 or only split on features 2, 3 and 4. See :ref:`this example<ice-vs-pdp>` on how to use `interaction_cst`. .. versionadded:: 1.2	None
	warm_start warm_start: bool, default=False When set to ``True``, reuse the solution of the previous call to fit and add more estimators to the ensemble. For results to be valid, the estimator should be re-trained on the same data only. See :term:`the Glossary <warm_start>`.	False
	early_stopping early_stopping: 'auto' or bool, default='auto' If 'auto', early stopping is enabled if the sample size is larger than 10000 or if `X_val` and `y_val` are passed to `fit`. If True, early stopping is enabled, otherwise early stopping is disabled. .. versionadded:: 0.23	'auto'
	scoring scoring: str or callable or None, default='loss' Scoring method to use for early stopping. Only used if `early_stopping` is enabled. Options: - str: see :ref:`scoring_string_names` for options. - callable: a scorer callable object (e.g., function) with signature ``scorer(estimator, X, y)``. See :ref:`scoring_callable` for details. - `None`: :ref:`accuracy <accuracy_score>` is used. - 'loss': early stopping is checked w.r.t the loss value.	'loss'
	validation_fraction validation_fraction: int or float or None, default=0.1 Proportion (or absolute size) of training data to set aside as validation data for early stopping. If None, early stopping is done on the training data. The value is ignored if either early stopping is not performed, e.g. `early_stopping=False`, or if `X_val` and `y_val` are passed to fit.	0.1
	n_iter_no_change n_iter_no_change: int, default=10 Used to determine when to "early stop". The fitting process is stopped when none of the last ``n_iter_no_change`` scores are better than the ``n_iter_no_change - 1`` -th-to-last one, up to some tolerance. Only used if early stopping is performed.	10
	tol tol: float, default=1e-7 The absolute tolerance to use when comparing scores. The higher the tolerance, the more likely we are to early stop: higher tolerance means that it will be harder for subsequent iterations to be considered an improvement upon the reference score.	1e-07
	verbose verbose: int, default=0 The verbosity level. If not zero, print some information about the fitting process. ``1`` prints only summary info, ``2`` prints info per iteration.	0
	class_weight class_weight: dict or 'balanced', default=None Weights associated with classes in the form `{class_label: weight}`. If not given, all classes are supposed to have weight one. The "balanced" mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as `n_samples / (n_classes * np.bincount(y))`. Note that these weights will be multiplied with sample_weight (passed through the fit method) if `sample_weight` is specified. .. versionadded:: 1.2	None

Fitted attributes

Name	Type	Value
classes_ classes_: array, shape = (n_classes,) Class labels.	ndarray[object](7,)	['1','2','3',...,'5','6','7']
do_early_stopping_ do_early_stopping_: bool Indicates whether early stopping is used during training.	bool	False
feature_names_in_ feature_names_in_: ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0	ndarray[object](54,)	['Elevation','Aspect','Slope',...,'Soil_Type_37','Soil_Type_38', 'Soil_Type_39']
is_categorical_ is_categorical_: ndarray, shape (n_features, ) or None Boolean mask for the categorical features. ``None`` if there are no categorical features.	NoneType	None
n_features_in_ n_features_in_: int Number of features seen during :term:`fit`. .. versionadded:: 0.24	int	54
n_iter_ n_iter_: int The number of iterations as selected by early stopping, depending on the `early_stopping` parameter. Otherwise it corresponds to max_iter.	int	100
n_trees_per_iteration_ n_trees_per_iteration_: int The number of tree that are built at each iteration. This is equal to 1 for binary classification, and to ``n_classes`` for multiclass classification.	int	7
train_score_ train_score_: ndarray, shape (n_iter_+1,) The scores at each iteration on the training data. The first entry is the score of the ensemble before the first iteration. Scores are computed according to the ``scoring`` parameter. If ``scoring`` is not 'loss', scores are computed on a subset of at most 10 000 samples. Empty if no early stopping.	ndarray[float64](0,)	[]
validation_score_ validation_score_: ndarray, shape (n_iter_+1,) The scores at each iteration on the held-out validation data. The first entry is the score of the ensemble before the first iteration. Scores are computed according to the ``scoring`` parameter. Empty if no early stopping or if ``validation_fraction`` is None.	ndarray[float64](0,)	[]

Column	Column name	dtype	Is sorted	Unique values	Mean	Std	Min	Median	Max
0	Elevation	Float64DType	False	1375 (13.8%)	2.96e+03	280.	1.88e+03	3.00e+03	3.86e+03
1	Aspect	Float64DType	False	361 (3.6%)	157.	112.	0.00	128.	360.
2	Slope	Float64DType	False	50 (0.5%)	14.2	7.48	0.00	13.0	51.0
3	Horizontal_Distance_To_Hydrology	Float64DType	False	367 (3.7%)	270.	213.	0.00	218.	1.34e+03
4	Vertical_Distance_To_Hydrology	Float64DType	False	418 (4.2%)	47.6	59.8	-152.	30.0	544.
5	Horizontal_Distance_To_Roadways	Float64DType	False	3489 (34.9%)	2.34e+03	1.55e+03	0.00	2.01e+03	7.04e+03
6	Hillshade_9am	Float64DType	False	170 (1.7%)	212.	27.2	0.00	217.	254.
7	Hillshade_Noon	Float64DType	False	126 (1.3%)	223.	19.5	100.	226.	254.
8	Hillshade_3pm	Float64DType	False	235 (2.4%)	143.	38.5	0.00	143.	248.
9	Horizontal_Distance_To_Fire_Points	Float64DType	False	2938 (29.4%)	1.97e+03	1.32e+03	0.00	1.68e+03	6.93e+03
10	Wilderness_Area_0	Float64DType	False	2 (< 0.1%)	0.445	0.497	0.00	0.00	1.00
11	Wilderness_Area_1	Float64DType	False	2 (< 0.1%)	0.0491	0.216	0.00	0.00	1.00
12	Wilderness_Area_2	Float64DType	False	2 (< 0.1%)	0.441	0.497	0.00	0.00	1.00
13	Wilderness_Area_3	Float64DType	False	2 (< 0.1%)	0.0650	0.247	0.00	0.00	1.00
14	Soil_Type_0	Float64DType	False	2 (< 0.1%)	0.00530	0.0726	0.00	0.00	1.00
15	Soil_Type_1	Float64DType	False	2 (< 0.1%)	0.0140	0.117	0.00	0.00	1.00
16	Soil_Type_2	Float64DType	False	2 (< 0.1%)	0.00680	0.0822	0.00	0.00	1.00
17	Soil_Type_3	Float64DType	False	2 (< 0.1%)	0.0220	0.147	0.00	0.00	1.00
18	Soil_Type_4	Float64DType	False	2 (< 0.1%)	0.00300	0.0547	0.00	0.00	1.00
19	Soil_Type_5	Float64DType	False	2 (< 0.1%)	0.0124	0.111	0.00	0.00	1.00
20	Soil_Type_6	Float64DType	False	2 (< 0.1%)	0.000200	0.0141	0.00	0.00	1.00
21	Soil_Type_7	Float64DType	False	2 (< 0.1%)	0.000400	0.0200	0.00	0.00	1.00
22	Soil_Type_8	Float64DType	False	2 (< 0.1%)	0.00160	0.0400	0.00	0.00	1.00
23	Soil_Type_9	Float64DType	False	2 (< 0.1%)	0.0551	0.228	0.00	0.00	1.00
24	Soil_Type_10	Float64DType	False	2 (< 0.1%)	0.0230	0.150	0.00	0.00	1.00
25	Soil_Type_11	Float64DType	False	2 (< 0.1%)	0.0518	0.222	0.00	0.00	1.00
26	Soil_Type_12	Float64DType	False	2 (< 0.1%)	0.0318	0.175	0.00	0.00	1.00
27	Soil_Type_13	Float64DType	False	2 (< 0.1%)	0.00140	0.0374	0.00	0.00	1.00
28	Soil_Type_14	Float64DType	True	1 (< 0.1%)	0.00	0.00
29	Soil_Type_15	Float64DType	False	2 (< 0.1%)	0.00520	0.0719	0.00	0.00	1.00
30	Soil_Type_16	Float64DType	False	2 (< 0.1%)	0.00610	0.0779	0.00	0.00	1.00
31	Soil_Type_17	Float64DType	False	2 (< 0.1%)	0.00440	0.0662	0.00	0.00	1.00
32	Soil_Type_18	Float64DType	False	2 (< 0.1%)	0.00620	0.0785	0.00	0.00	1.00
33	Soil_Type_19	Float64DType	False	2 (< 0.1%)	0.0145	0.120	0.00	0.00	1.00
34	Soil_Type_20	Float64DType	False	2 (< 0.1%)	0.00190	0.0435	0.00	0.00	1.00
35	Soil_Type_21	Float64DType	False	2 (< 0.1%)	0.0560	0.230	0.00	0.00	1.00
36	Soil_Type_22	Float64DType	False	2 (< 0.1%)	0.0991	0.299	0.00	0.00	1.00
37	Soil_Type_23	Float64DType	False	2 (< 0.1%)	0.0360	0.186	0.00	0.00	1.00
38	Soil_Type_24	Float64DType	False	2 (< 0.1%)	0.000600	0.0245	0.00	0.00	1.00
39	Soil_Type_25	Float64DType	False	2 (< 0.1%)	0.00560	0.0746	0.00	0.00	1.00
40	Soil_Type_26	Float64DType	False	2 (< 0.1%)	0.00160	0.0400	0.00	0.00	1.00
41	Soil_Type_27	Float64DType	False	2 (< 0.1%)	0.00160	0.0400	0.00	0.00	1.00
42	Soil_Type_28	Float64DType	False	2 (< 0.1%)	0.195	0.396	0.00	0.00	1.00
43	Soil_Type_29	Float64DType	False	2 (< 0.1%)	0.0526	0.223	0.00	0.00	1.00
44	Soil_Type_30	Float64DType	False	2 (< 0.1%)	0.0447	0.207	0.00	0.00	1.00
45	Soil_Type_31	Float64DType	False	2 (< 0.1%)	0.0890	0.285	0.00	0.00	1.00
46	Soil_Type_32	Float64DType	False	2 (< 0.1%)	0.0819	0.274	0.00	0.00	1.00
47	Soil_Type_33	Float64DType	False	2 (< 0.1%)	0.00330	0.0574	0.00	0.00	1.00
48	Soil_Type_34	Float64DType	False	2 (< 0.1%)	0.00360	0.0599	0.00	0.00	1.00
49	Soil_Type_35	Float64DType	False	2 (< 0.1%)	0.000400	0.0200	0.00	0.00	1.00
50	Soil_Type_36	Float64DType	False	2 (< 0.1%)	0.000400	0.0200	0.00	0.00	1.00
51	Soil_Type_37	Float64DType	False	2 (< 0.1%)	0.0217	0.146	0.00	0.00	1.00
52	Soil_Type_38	Float64DType	False	2 (< 0.1%)	0.0239	0.153	0.00	0.00	1.00
53	Soil_Type_39	Float64DType	False	2 (< 0.1%)	0.0157	0.124	0.00	0.00	1.00
54	class	ObjectDType	False	7 (< 0.1%)

Please enable javascript

.. GENERATED FROM PYTHON SOURCE LINES 103-104 SKD005 correctly flags classes 3, 4, 5, 6, and 7 as under 10 % of rows. .. GENERATED FROM PYTHON SOURCE LINES 104-107 .. code-block:: Python report.checks.summarize(fast_mode=True, ignore=["SKD008"]) .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 108-115 Accuracy hides rare-class failures ================================== Accuracy alone can look strong when frequent classes dominate the table. Let us however report it with per class precision and log-loss. We see that global accuracy hides rare-class failures, such as for types 4 and 5. .. GENERATED FROM PYTHON SOURCE LINES 115-120 .. code-block:: Python report.metrics.summarize(metric=["accuracy", "precision", "log_loss"]).frame( flat_index=False ) .. rst-class:: sphx-glr-script-out .. code-block:: none metric label accuracy 0.738000 precision 1 0.765805 2 0.783453 3 0.674419 4 0.250000 5 0.270270 6 0.372881 7 0.650000 log_loss 2.991535 Name: HistGradientBoostingClassifier, dtype: float64 .. GENERATED FROM PYTHON SOURCE LINES 121-124 Let us also inspect the confusion matrix. We see that the model has a hard time predicting type 6, often confusing it with types 2 and 3 or when predicting type 7, confusing it with type 1. .. GENERATED FROM PYTHON SOURCE LINES 124-128 .. code-block:: Python report.metrics.confusion_matrix().plot() .. image-sg:: /auto_examples/pitfalls_and_solutions/images/sphx_glr_plot_skd005_underrepresented_classes_001.png :alt: Confusion Matrix Data source: Test set :srcset: /auto_examples/pitfalls_and_solutions/images/sphx_glr_plot_skd005_underrepresented_classes_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none

.. GENERATED FROM PYTHON SOURCE LINES 129-136 More rare-class training data ============================= Extra labels on underrepresented types can help the multiclass model see them more often. Let us keep one fixed test fold with the natural mix, fit on the original train fold, then refit after adding rare-class rows from the pool (classes that were under 10 % in the subsample). .. GENERATED FROM PYTHON SOURCE LINES 136-156 .. code-block:: Python X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42, stratify=y ) rare_labels = shares[shares < 0.10].index pool_rare = y_pool.isin(rare_labels) X_rare_extra, _, y_rare_extra, _ = train_test_split( X_pool.loc[pool_rare], y_pool.loc[pool_rare], train_size=min(5_000, int(pool_rare.sum())), stratify=y_pool.loc[pool_rare], random_state=42, ) X_train_more = pd.concat([X_train, X_rare_extra]) y_train_more = pd.concat([y_train, y_rare_extra]) print("Rare labels added from the pool:", list(rare_labels)) print("Extra rare-class rows added:", len(y_rare_extra)) .. rst-class:: sphx-glr-script-out .. code-block:: none Rare labels added from the pool: ['3', '4', '5', '6', '7'] Extra rare-class rows added: 5000 .. GENERATED FROM PYTHON SOURCE LINES 157-160 .. code-block:: Python print("\nBaseline train counts:") y_train.value_counts().sort_index() .. rst-class:: sphx-glr-script-out .. code-block:: none Baseline train counts: class 1 2917 2 3901 3 492 4 38 5 131 6 239 7 282 Name: count, dtype: int64 .. GENERATED FROM PYTHON SOURCE LINES 161-164 .. code-block:: Python print("\nEnriched train counts:") y_train_more.value_counts().sort_index() .. rst-class:: sphx-glr-script-out .. code-block:: none Enriched train counts: class 1 2917 2 3901 3 2574 4 198 5 684 6 1250 7 1476 Name: count, dtype: int64 .. GENERATED FROM PYTHON SOURCE LINES 165-174 Let us now fit a model on the enriched train set and compare the results with the original model. We can observe that the model on the enriched train set has a better log-loss, accuracy and per-class precision on the common test set. The log-loss is the most importance metric to look at here, as it evaluates the model's predicted probabilities, which give more robust estimate of the model's quality. In contrast, accuracy and per-class precision are computed with hard class predictions, obtained from the argmax of the predicted probabilities, which can hide uncalibrated predictions. .. GENERATED FROM PYTHON SOURCE LINES 174-179 .. code-block:: Python model_less = HistGradientBoostingClassifier(random_state=42).fit(X_train, y_train) report_less = skore.evaluate(model_less, X_test, y_test, splitter="prefit") report_less .. raw:: html

Metric	Label	Average	HistGradientBoostingClassifier
Accuracy			0.738000
Precision	1		0.765805
Precision	2		0.783453
Precision	3		0.674419
Precision	4		0.250000
Precision	5		0.270270
Precision	6		0.372881
Precision	7		0.650000
Precision		macro	0.538118
Recall	1		0.731139
Recall	2		0.786667
Recall	3		0.707317
Recall	4		0.555556
Recall	5		0.303030
Recall	6		0.366667
Recall	7		0.732394
Recall		macro	0.597538
ROC AUC	1		0.862669
ROC AUC	2		0.833460
ROC AUC	3		0.901625
ROC AUC	4		0.881271
ROC AUC	5		0.776972
ROC AUC	6		0.853458
ROC AUC	7		0.951255
ROC AUC		macro	0.865816
Log loss			2.991535
Fit time (s)			NaN
Predict time (s)			0.106975

HistGradientBoostingClassifier(random_state=42)

HistGradientBoostingClassifier

?Documentation for HistGradientBoostingClassifieriFitted

Parameters

	random_state random_state: int, RandomState instance or None, default=None Pseudo-random number generator to control the subsampling in the binning process, and the train/validation data split if early stopping is enabled. Pass an int for reproducible output across multiple function calls. See :term:`Glossary <random_state>`.	42
	loss loss: {'log_loss'}, default='log_loss' The loss function to use in the boosting process. For binary classification problems, 'log_loss' is also known as logistic loss, binomial deviance or binary crossentropy. Internally, the model fits one tree per boosting iteration and uses the logistic sigmoid function (expit) as inverse link function to compute the predicted positive class probability. For multiclass classification problems, 'log_loss' is also known as multinomial deviance or categorical crossentropy. Internally, the model fits one tree per boosting iteration and per class and uses the softmax function as inverse link function to compute the predicted probabilities of the classes.	'log_loss'
	learning_rate learning_rate: float, default=0.1 The learning rate, also known as shrinkage. This is used as a multiplicative factor for the leaves values. Use ``1`` for no shrinkage.	0.1
	max_iter max_iter: int, default=100 The maximum number of iterations of the boosting process, i.e. the maximum number of trees for binary classification. For multiclass classification, `n_classes` trees per iteration are built.	100
	max_leaf_nodes max_leaf_nodes: int or None, default=31 The maximum number of leaves for each tree. Must be strictly greater than 1. If None, there is no maximum limit.	31
	max_depth max_depth: int or None, default=None The maximum depth of each tree. The depth of a tree is the number of edges to go from the root to the deepest leaf. Depth isn't constrained by default.	None
	min_samples_leaf min_samples_leaf: int, default=20 The minimum number of samples per leaf. For small datasets with less than a few hundred samples, it is recommended to lower this value since only very shallow trees would be built.	20
	l2_regularization l2_regularization: float, default=0 The L2 regularization parameter penalizing leaves with small hessians. Use ``0`` for no regularization (default).	0.0
	max_features max_features: float, default=1.0 Proportion of randomly chosen features in each and every node split. This is a form of regularization, smaller values make the trees weaker learners and might prevent overfitting. If interaction constraints from `interaction_cst` are present, only allowed features are taken into account for the subsampling. .. versionadded:: 1.4	1.0
	max_bins max_bins: int, default=255 The maximum number of bins to use for non-missing values. Before training, each feature of the input array `X` is binned into integer-valued bins, which allows for a much faster training stage. Features with a small number of unique values may use less than ``max_bins`` bins. In addition to the ``max_bins`` bins, one more bin is always reserved for missing values. Must be no larger than 255.	255
	categorical_features categorical_features: array-like of {bool, int, str} of shape (n_features) or shape (n_categorical_features,), default='from_dtype' Indicates the categorical features. - None : no feature will be considered categorical. - boolean array-like : boolean mask indicating categorical features. - integer array-like : integer indices indicating categorical features. - str array-like: names of categorical features (assuming the training data has feature names). - `"from_dtype"`: dataframe columns with dtype "Categorical" and "Enum" are considered to be categorical features. The input must be a dataframe that is supported by narwhals (or supports it): :func:`narwhals.from_native` must work. This is the case, for instance, for pandas and polars DataFrames. For each categorical feature, there must be at most `max_bins` unique categories. Negative values for categorical features encoded as numeric dtypes are treated as missing values. All categorical values are converted to floating point numbers. This means that categorical values of 1.0 and 1 are treated as the same category. Read more in the :ref:`User Guide <categorical_support_gbdt>`. .. versionadded:: 0.24 .. versionchanged:: 1.2 Added support for feature names. .. versionchanged:: 1.4 Added `"from_dtype"` option. .. versionchanged:: 1.6 The default value changed from `None` to `"from_dtype"`.	'from_dtype'
	monotonic_cst monotonic_cst: array-like of int of shape (n_features) or dict, default=None Monotonic constraint to enforce on each feature are specified using the following integer values: - 1: monotonic increase - 0: no constraint - -1: monotonic decrease If a dict with str keys, map feature to monotonic constraints by name. If an array, the features are mapped to constraints by position. See :ref:`monotonic_cst_features_names` for a usage example. The constraints are only valid for binary classifications and hold over the probability of the positive class. Read more in the :ref:`User Guide <monotonic_cst_gbdt>`. .. versionadded:: 0.23 .. versionchanged:: 1.2 Accept dict of constraints with feature names as keys.	None
	interaction_cst interaction_cst: {"pairwise", "no_interactions"} or sequence of lists/tuples/sets of int, default=None Specify interaction constraints, the sets of features which can interact with each other in child node splits. Each item specifies the set of feature indices that are allowed to interact with each other. If there are more features than specified in these constraints, they are treated as if they were specified as an additional set. The strings "pairwise" and "no_interactions" are shorthands for allowing only pairwise or no interactions, respectively. For instance, with 5 features in total, `interaction_cst=[{0, 1}]` is equivalent to `interaction_cst=[{0, 1}, {2, 3, 4}]`, and specifies that each branch of a tree will either only split on features 0 and 1 or only split on features 2, 3 and 4. See :ref:`this example<ice-vs-pdp>` on how to use `interaction_cst`. .. versionadded:: 1.2	None
	warm_start warm_start: bool, default=False When set to ``True``, reuse the solution of the previous call to fit and add more estimators to the ensemble. For results to be valid, the estimator should be re-trained on the same data only. See :term:`the Glossary <warm_start>`.	False
	early_stopping early_stopping: 'auto' or bool, default='auto' If 'auto', early stopping is enabled if the sample size is larger than 10000 or if `X_val` and `y_val` are passed to `fit`. If True, early stopping is enabled, otherwise early stopping is disabled. .. versionadded:: 0.23	'auto'
	scoring scoring: str or callable or None, default='loss' Scoring method to use for early stopping. Only used if `early_stopping` is enabled. Options: - str: see :ref:`scoring_string_names` for options. - callable: a scorer callable object (e.g., function) with signature ``scorer(estimator, X, y)``. See :ref:`scoring_callable` for details. - `None`: :ref:`accuracy <accuracy_score>` is used. - 'loss': early stopping is checked w.r.t the loss value.	'loss'
	validation_fraction validation_fraction: int or float or None, default=0.1 Proportion (or absolute size) of training data to set aside as validation data for early stopping. If None, early stopping is done on the training data. The value is ignored if either early stopping is not performed, e.g. `early_stopping=False`, or if `X_val` and `y_val` are passed to fit.	0.1
	n_iter_no_change n_iter_no_change: int, default=10 Used to determine when to "early stop". The fitting process is stopped when none of the last ``n_iter_no_change`` scores are better than the ``n_iter_no_change - 1`` -th-to-last one, up to some tolerance. Only used if early stopping is performed.	10
	tol tol: float, default=1e-7 The absolute tolerance to use when comparing scores. The higher the tolerance, the more likely we are to early stop: higher tolerance means that it will be harder for subsequent iterations to be considered an improvement upon the reference score.	1e-07
	verbose verbose: int, default=0 The verbosity level. If not zero, print some information about the fitting process. ``1`` prints only summary info, ``2`` prints info per iteration.	0
	class_weight class_weight: dict or 'balanced', default=None Weights associated with classes in the form `{class_label: weight}`. If not given, all classes are supposed to have weight one. The "balanced" mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as `n_samples / (n_classes * np.bincount(y))`. Note that these weights will be multiplied with sample_weight (passed through the fit method) if `sample_weight` is specified. .. versionadded:: 1.2	None

Fitted attributes

Name	Type	Value
classes_ classes_: array, shape = (n_classes,) Class labels.	ndarray[object](7,)	['1','2','3',...,'5','6','7']
do_early_stopping_ do_early_stopping_: bool Indicates whether early stopping is used during training.	bool	False
feature_names_in_ feature_names_in_: ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0	ndarray[object](54,)	['Elevation','Aspect','Slope',...,'Soil_Type_37','Soil_Type_38', 'Soil_Type_39']
is_categorical_ is_categorical_: ndarray, shape (n_features, ) or None Boolean mask for the categorical features. ``None`` if there are no categorical features.	NoneType	None
n_features_in_ n_features_in_: int Number of features seen during :term:`fit`. .. versionadded:: 0.24	int	54
n_iter_ n_iter_: int The number of iterations as selected by early stopping, depending on the `early_stopping` parameter. Otherwise it corresponds to max_iter.	int	100
n_trees_per_iteration_ n_trees_per_iteration_: int The number of tree that are built at each iteration. This is equal to 1 for binary classification, and to ``n_classes`` for multiclass classification.	int	7
train_score_ train_score_: ndarray, shape (n_iter_+1,) The scores at each iteration on the training data. The first entry is the score of the ensemble before the first iteration. Scores are computed according to the ``scoring`` parameter. If ``scoring`` is not 'loss', scores are computed on a subset of at most 10 000 samples. Empty if no early stopping.	ndarray[float64](0,)	[]
validation_score_ validation_score_: ndarray, shape (n_iter_+1,) The scores at each iteration on the held-out validation data. The first entry is the score of the ensemble before the first iteration. Scores are computed according to the ``scoring`` parameter. Empty if no early stopping or if ``validation_fraction`` is None.	ndarray[float64](0,)	[]

Column	Column name	dtype	Is sorted	Unique values	Mean	Std	Min	Median	Max
0	Elevation	Float64DType	False	884 (44.2%)	2.97e+03	281.	1.98e+03	3.00e+03	3.85e+03
1	Aspect	Float64DType	False	357 (17.8%)	154.	110.	0.00	128.	359.
2	Slope	Float64DType	False	44 (2.2%)	14.3	7.60	1.00	13.0	46.0
3	Horizontal_Distance_To_Hydrology	Float64DType	False	245 (12.2%)	273.	218.	0.00	218.	1.34e+03
4	Vertical_Distance_To_Hydrology	Float64DType	False	293 (14.6%)	48.9	61.8	-152.	31.0	530.
5	Horizontal_Distance_To_Roadways	Float64DType	False	1429 (71.5%)	2.35e+03	1.53e+03	30.0	2.00e+03	6.50e+03
6	Hillshade_9am	Float64DType	False	138 (6.9%)	212.	26.8	69.0	218.	254.
7	Hillshade_Noon	Float64DType	False	108 (5.4%)	224.	20.1	120.	227.	254.
8	Hillshade_3pm	Float64DType	False	206 (10.3%)	142.	38.6	0.00	143.	248.
9	Horizontal_Distance_To_Fire_Points	Float64DType	False	1266 (63.3%)	1.95e+03	1.30e+03	42.0	1.70e+03	6.75e+03
10	Wilderness_Area_0	Float64DType	False	2 (0.1%)	0.444	0.497	0.00	0.00	1.00
11	Wilderness_Area_1	Float64DType	False	2 (0.1%)	0.0460	0.210	0.00	0.00	1.00
12	Wilderness_Area_2	Float64DType	False	2 (0.1%)	0.446	0.497	0.00	0.00	1.00
13	Wilderness_Area_3	Float64DType	False	2 (0.1%)	0.0640	0.245	0.00	0.00	1.00
14	Soil_Type_0	Float64DType	False	2 (0.1%)	0.00400	0.0631	0.00	0.00	1.00
15	Soil_Type_1	Float64DType	False	2 (0.1%)	0.0140	0.118	0.00	0.00	1.00
16	Soil_Type_2	Float64DType	False	2 (0.1%)	0.00450	0.0669	0.00	0.00	1.00
17	Soil_Type_3	Float64DType	False	2 (0.1%)	0.0200	0.140	0.00	0.00	1.00
18	Soil_Type_4	Float64DType	False	2 (0.1%)	0.00150	0.0387	0.00	0.00	1.00
19	Soil_Type_5	Float64DType	False	2 (0.1%)	0.0140	0.118	0.00	0.00	1.00
20	Soil_Type_6	Float64DType	True	1 (< 0.1%)	0.00	0.00
21	Soil_Type_7	Float64DType	True	1 (< 0.1%)	0.00	0.00
22	Soil_Type_8	Float64DType	True	1 (< 0.1%)	0.00	0.00
23	Soil_Type_9	Float64DType	False	2 (0.1%)	0.0560	0.230	0.00	0.00	1.00
24	Soil_Type_10	Float64DType	False	2 (0.1%)	0.0235	0.152	0.00	0.00	1.00
25	Soil_Type_11	Float64DType	False	2 (0.1%)	0.0470	0.212	0.00	0.00	1.00
26	Soil_Type_12	Float64DType	False	2 (0.1%)	0.0370	0.189	0.00	0.00	1.00
27	Soil_Type_13	Float64DType	False	2 (0.1%)	0.00200	0.0447	0.00	0.00	1.00
28	Soil_Type_14	Float64DType	True	1 (< 0.1%)	0.00	0.00
29	Soil_Type_15	Float64DType	False	2 (0.1%)	0.00400	0.0631	0.00	0.00	1.00
30	Soil_Type_16	Float64DType	False	2 (0.1%)	0.00600	0.0772	0.00	0.00	1.00
31	Soil_Type_17	Float64DType	False	2 (0.1%)	0.00250	0.0499	0.00	0.00	1.00
32	Soil_Type_18	Float64DType	False	2 (0.1%)	0.00700	0.0834	0.00	0.00	1.00
33	Soil_Type_19	Float64DType	False	2 (0.1%)	0.0120	0.109	0.00	0.00	1.00
34	Soil_Type_20	Float64DType	False	2 (0.1%)	0.00200	0.0447	0.00	0.00	1.00
35	Soil_Type_21	Float64DType	False	2 (0.1%)	0.0580	0.234	0.00	0.00	1.00
36	Soil_Type_22	Float64DType	False	2 (0.1%)	0.106	0.308	0.00	0.00	1.00
37	Soil_Type_23	Float64DType	False	2 (0.1%)	0.0365	0.188	0.00	0.00	1.00
38	Soil_Type_24	Float64DType	False	2 (0.1%)	0.00150	0.0387	0.00	0.00	1.00
39	Soil_Type_25	Float64DType	False	2 (0.1%)	0.00700	0.0834	0.00	0.00	1.00
40	Soil_Type_26	Float64DType	False	2 (0.1%)	0.00350	0.0591	0.00	0.00	1.00
41	Soil_Type_27	Float64DType	False	2 (0.1%)	0.000500	0.0224	0.00	0.00	1.00
42	Soil_Type_28	Float64DType	False	2 (0.1%)	0.195	0.396	0.00	0.00	1.00
43	Soil_Type_29	Float64DType	False	2 (0.1%)	0.0525	0.223	0.00	0.00	1.00
44	Soil_Type_30	Float64DType	False	2 (0.1%)	0.0385	0.192	0.00	0.00	1.00
45	Soil_Type_31	Float64DType	False	2 (0.1%)	0.0865	0.281	0.00	0.00	1.00
46	Soil_Type_32	Float64DType	False	2 (0.1%)	0.0870	0.282	0.00	0.00	1.00
47	Soil_Type_33	Float64DType	False	2 (0.1%)	0.00300	0.0547	0.00	0.00	1.00
48	Soil_Type_34	Float64DType	False	2 (0.1%)	0.00250	0.0499	0.00	0.00	1.00
49	Soil_Type_35	Float64DType	False	2 (0.1%)	0.000500	0.0224	0.00	0.00	1.00
50	Soil_Type_36	Float64DType	False	2 (0.1%)	0.000500	0.0224	0.00	0.00	1.00
51	Soil_Type_37	Float64DType	False	2 (0.1%)	0.0210	0.143	0.00	0.00	1.00
52	Soil_Type_38	Float64DType	False	2 (0.1%)	0.0245	0.155	0.00	0.00	1.00
53	Soil_Type_39	Float64DType	False	2 (0.1%)	0.0190	0.137	0.00	0.00	1.00
54	class	ObjectDType	False	7 (0.4%)

Please enable javascript

.. GENERATED FROM PYTHON SOURCE LINES 180-186 .. code-block:: Python model_more = HistGradientBoostingClassifier(random_state=42).fit( X_train_more, y_train_more ) report_more = skore.evaluate(model_more, X_test, y_test, splitter="prefit") report_more .. raw:: html

Metric	Label	Average	HistGradientBoostingClassifier
Accuracy			0.800500
Precision	1		0.807069
Precision	2		0.844681
Precision	3		0.811594
Precision	4		0.700000
Precision	5		0.508772
Precision	6		0.666667
Precision	7		0.609091
Precision		macro	0.706839
Recall	1		0.751715
Recall	2		0.814359
Recall	3		0.910569
Recall	4		0.777778
Recall	5		0.878788
Recall	6		0.733333
Recall	7		0.943662
Recall		macro	0.830029
ROC AUC	1		0.911377
ROC AUC	2		0.911218
ROC AUC	3		0.995781
ROC AUC	4		0.998940
ROC AUC	5		0.988754
ROC AUC	6		0.989227
ROC AUC	7		0.995517
ROC AUC		macro	0.970116
Log loss			0.493421
Fit time (s)			NaN
Predict time (s)			0.089782

HistGradientBoostingClassifier(random_state=42)

HistGradientBoostingClassifier

?Documentation for HistGradientBoostingClassifieriFitted

Parameters

	random_state random_state: int, RandomState instance or None, default=None Pseudo-random number generator to control the subsampling in the binning process, and the train/validation data split if early stopping is enabled. Pass an int for reproducible output across multiple function calls. See :term:`Glossary <random_state>`.	42
	loss loss: {'log_loss'}, default='log_loss' The loss function to use in the boosting process. For binary classification problems, 'log_loss' is also known as logistic loss, binomial deviance or binary crossentropy. Internally, the model fits one tree per boosting iteration and uses the logistic sigmoid function (expit) as inverse link function to compute the predicted positive class probability. For multiclass classification problems, 'log_loss' is also known as multinomial deviance or categorical crossentropy. Internally, the model fits one tree per boosting iteration and per class and uses the softmax function as inverse link function to compute the predicted probabilities of the classes.	'log_loss'
	learning_rate learning_rate: float, default=0.1 The learning rate, also known as shrinkage. This is used as a multiplicative factor for the leaves values. Use ``1`` for no shrinkage.	0.1
	max_iter max_iter: int, default=100 The maximum number of iterations of the boosting process, i.e. the maximum number of trees for binary classification. For multiclass classification, `n_classes` trees per iteration are built.	100
	max_leaf_nodes max_leaf_nodes: int or None, default=31 The maximum number of leaves for each tree. Must be strictly greater than 1. If None, there is no maximum limit.	31
	max_depth max_depth: int or None, default=None The maximum depth of each tree. The depth of a tree is the number of edges to go from the root to the deepest leaf. Depth isn't constrained by default.	None
	min_samples_leaf min_samples_leaf: int, default=20 The minimum number of samples per leaf. For small datasets with less than a few hundred samples, it is recommended to lower this value since only very shallow trees would be built.	20
	l2_regularization l2_regularization: float, default=0 The L2 regularization parameter penalizing leaves with small hessians. Use ``0`` for no regularization (default).	0.0
	max_features max_features: float, default=1.0 Proportion of randomly chosen features in each and every node split. This is a form of regularization, smaller values make the trees weaker learners and might prevent overfitting. If interaction constraints from `interaction_cst` are present, only allowed features are taken into account for the subsampling. .. versionadded:: 1.4	1.0
	max_bins max_bins: int, default=255 The maximum number of bins to use for non-missing values. Before training, each feature of the input array `X` is binned into integer-valued bins, which allows for a much faster training stage. Features with a small number of unique values may use less than ``max_bins`` bins. In addition to the ``max_bins`` bins, one more bin is always reserved for missing values. Must be no larger than 255.	255
	categorical_features categorical_features: array-like of {bool, int, str} of shape (n_features) or shape (n_categorical_features,), default='from_dtype' Indicates the categorical features. - None : no feature will be considered categorical. - boolean array-like : boolean mask indicating categorical features. - integer array-like : integer indices indicating categorical features. - str array-like: names of categorical features (assuming the training data has feature names). - `"from_dtype"`: dataframe columns with dtype "Categorical" and "Enum" are considered to be categorical features. The input must be a dataframe that is supported by narwhals (or supports it): :func:`narwhals.from_native` must work. This is the case, for instance, for pandas and polars DataFrames. For each categorical feature, there must be at most `max_bins` unique categories. Negative values for categorical features encoded as numeric dtypes are treated as missing values. All categorical values are converted to floating point numbers. This means that categorical values of 1.0 and 1 are treated as the same category. Read more in the :ref:`User Guide <categorical_support_gbdt>`. .. versionadded:: 0.24 .. versionchanged:: 1.2 Added support for feature names. .. versionchanged:: 1.4 Added `"from_dtype"` option. .. versionchanged:: 1.6 The default value changed from `None` to `"from_dtype"`.	'from_dtype'
	monotonic_cst monotonic_cst: array-like of int of shape (n_features) or dict, default=None Monotonic constraint to enforce on each feature are specified using the following integer values: - 1: monotonic increase - 0: no constraint - -1: monotonic decrease If a dict with str keys, map feature to monotonic constraints by name. If an array, the features are mapped to constraints by position. See :ref:`monotonic_cst_features_names` for a usage example. The constraints are only valid for binary classifications and hold over the probability of the positive class. Read more in the :ref:`User Guide <monotonic_cst_gbdt>`. .. versionadded:: 0.23 .. versionchanged:: 1.2 Accept dict of constraints with feature names as keys.	None
	interaction_cst interaction_cst: {"pairwise", "no_interactions"} or sequence of lists/tuples/sets of int, default=None Specify interaction constraints, the sets of features which can interact with each other in child node splits. Each item specifies the set of feature indices that are allowed to interact with each other. If there are more features than specified in these constraints, they are treated as if they were specified as an additional set. The strings "pairwise" and "no_interactions" are shorthands for allowing only pairwise or no interactions, respectively. For instance, with 5 features in total, `interaction_cst=[{0, 1}]` is equivalent to `interaction_cst=[{0, 1}, {2, 3, 4}]`, and specifies that each branch of a tree will either only split on features 0 and 1 or only split on features 2, 3 and 4. See :ref:`this example<ice-vs-pdp>` on how to use `interaction_cst`. .. versionadded:: 1.2	None
	warm_start warm_start: bool, default=False When set to ``True``, reuse the solution of the previous call to fit and add more estimators to the ensemble. For results to be valid, the estimator should be re-trained on the same data only. See :term:`the Glossary <warm_start>`.	False
	early_stopping early_stopping: 'auto' or bool, default='auto' If 'auto', early stopping is enabled if the sample size is larger than 10000 or if `X_val` and `y_val` are passed to `fit`. If True, early stopping is enabled, otherwise early stopping is disabled. .. versionadded:: 0.23	'auto'
	scoring scoring: str or callable or None, default='loss' Scoring method to use for early stopping. Only used if `early_stopping` is enabled. Options: - str: see :ref:`scoring_string_names` for options. - callable: a scorer callable object (e.g., function) with signature ``scorer(estimator, X, y)``. See :ref:`scoring_callable` for details. - `None`: :ref:`accuracy <accuracy_score>` is used. - 'loss': early stopping is checked w.r.t the loss value.	'loss'
	validation_fraction validation_fraction: int or float or None, default=0.1 Proportion (or absolute size) of training data to set aside as validation data for early stopping. If None, early stopping is done on the training data. The value is ignored if either early stopping is not performed, e.g. `early_stopping=False`, or if `X_val` and `y_val` are passed to fit.	0.1
	n_iter_no_change n_iter_no_change: int, default=10 Used to determine when to "early stop". The fitting process is stopped when none of the last ``n_iter_no_change`` scores are better than the ``n_iter_no_change - 1`` -th-to-last one, up to some tolerance. Only used if early stopping is performed.	10
	tol tol: float, default=1e-7 The absolute tolerance to use when comparing scores. The higher the tolerance, the more likely we are to early stop: higher tolerance means that it will be harder for subsequent iterations to be considered an improvement upon the reference score.	1e-07
	verbose verbose: int, default=0 The verbosity level. If not zero, print some information about the fitting process. ``1`` prints only summary info, ``2`` prints info per iteration.	0
	class_weight class_weight: dict or 'balanced', default=None Weights associated with classes in the form `{class_label: weight}`. If not given, all classes are supposed to have weight one. The "balanced" mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as `n_samples / (n_classes * np.bincount(y))`. Note that these weights will be multiplied with sample_weight (passed through the fit method) if `sample_weight` is specified. .. versionadded:: 1.2	None

Fitted attributes

Name	Type	Value
classes_ classes_: array, shape = (n_classes,) Class labels.	ndarray[object](7,)	['1','2','3',...,'5','6','7']
do_early_stopping_ do_early_stopping_: bool Indicates whether early stopping is used during training.	bool	True
feature_names_in_ feature_names_in_: ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0	ndarray[object](54,)	['Elevation','Aspect','Slope',...,'Soil_Type_37','Soil_Type_38', 'Soil_Type_39']
is_categorical_ is_categorical_: ndarray, shape (n_features, ) or None Boolean mask for the categorical features. ``None`` if there are no categorical features.	NoneType	None
n_features_in_ n_features_in_: int Number of features seen during :term:`fit`. .. versionadded:: 0.24	int	54
n_iter_ n_iter_: int The number of iterations as selected by early stopping, depending on the `early_stopping` parameter. Otherwise it corresponds to max_iter.	int	100
n_trees_per_iteration_ n_trees_per_iteration_: int The number of tree that are built at each iteration. This is equal to 1 for binary classification, and to ``n_classes`` for multiclass classification.	int	7
train_score_ train_score_: ndarray, shape (n_iter_+1,) The scores at each iteration on the training data. The first entry is the score of the ensemble before the first iteration. Scores are computed according to the ``scoring`` parameter. If ``scoring`` is not 'loss', scores are computed on a subset of at most 10 000 samples. Empty if no early stopping.	ndarray[float64](101,)	[-1.71,-1.37,-1.19,...,-0.15,-0.15,-0.15]
validation_score_ validation_score_: ndarray, shape (n_iter_+1,) The scores at each iteration on the held-out validation data. The first entry is the score of the ensemble before the first iteration. Scores are computed according to the ``scoring`` parameter. Empty if no early stopping or if ``validation_fraction`` is None.	ndarray[float64](101,)	[-1.71,-1.38,-1.22,...,-0.43,-0.43,-0.43]

Column	Column name	dtype	Is sorted	Unique values	Mean	Std	Min	Median	Max
0	Elevation	Float64DType	False	884 (44.2%)	2.97e+03	281.	1.98e+03	3.00e+03	3.85e+03
1	Aspect	Float64DType	False	357 (17.8%)	154.	110.	0.00	128.	359.
2	Slope	Float64DType	False	44 (2.2%)	14.3	7.60	1.00	13.0	46.0
3	Horizontal_Distance_To_Hydrology	Float64DType	False	245 (12.2%)	273.	218.	0.00	218.	1.34e+03
4	Vertical_Distance_To_Hydrology	Float64DType	False	293 (14.6%)	48.9	61.8	-152.	31.0	530.
5	Horizontal_Distance_To_Roadways	Float64DType	False	1429 (71.5%)	2.35e+03	1.53e+03	30.0	2.00e+03	6.50e+03
6	Hillshade_9am	Float64DType	False	138 (6.9%)	212.	26.8	69.0	218.	254.
7	Hillshade_Noon	Float64DType	False	108 (5.4%)	224.	20.1	120.	227.	254.
8	Hillshade_3pm	Float64DType	False	206 (10.3%)	142.	38.6	0.00	143.	248.
9	Horizontal_Distance_To_Fire_Points	Float64DType	False	1266 (63.3%)	1.95e+03	1.30e+03	42.0	1.70e+03	6.75e+03
10	Wilderness_Area_0	Float64DType	False	2 (0.1%)	0.444	0.497	0.00	0.00	1.00
11	Wilderness_Area_1	Float64DType	False	2 (0.1%)	0.0460	0.210	0.00	0.00	1.00
12	Wilderness_Area_2	Float64DType	False	2 (0.1%)	0.446	0.497	0.00	0.00	1.00
13	Wilderness_Area_3	Float64DType	False	2 (0.1%)	0.0640	0.245	0.00	0.00	1.00
14	Soil_Type_0	Float64DType	False	2 (0.1%)	0.00400	0.0631	0.00	0.00	1.00
15	Soil_Type_1	Float64DType	False	2 (0.1%)	0.0140	0.118	0.00	0.00	1.00
16	Soil_Type_2	Float64DType	False	2 (0.1%)	0.00450	0.0669	0.00	0.00	1.00
17	Soil_Type_3	Float64DType	False	2 (0.1%)	0.0200	0.140	0.00	0.00	1.00
18	Soil_Type_4	Float64DType	False	2 (0.1%)	0.00150	0.0387	0.00	0.00	1.00
19	Soil_Type_5	Float64DType	False	2 (0.1%)	0.0140	0.118	0.00	0.00	1.00
20	Soil_Type_6	Float64DType	True	1 (< 0.1%)	0.00	0.00
21	Soil_Type_7	Float64DType	True	1 (< 0.1%)	0.00	0.00
22	Soil_Type_8	Float64DType	True	1 (< 0.1%)	0.00	0.00
23	Soil_Type_9	Float64DType	False	2 (0.1%)	0.0560	0.230	0.00	0.00	1.00
24	Soil_Type_10	Float64DType	False	2 (0.1%)	0.0235	0.152	0.00	0.00	1.00
25	Soil_Type_11	Float64DType	False	2 (0.1%)	0.0470	0.212	0.00	0.00	1.00
26	Soil_Type_12	Float64DType	False	2 (0.1%)	0.0370	0.189	0.00	0.00	1.00
27	Soil_Type_13	Float64DType	False	2 (0.1%)	0.00200	0.0447	0.00	0.00	1.00
28	Soil_Type_14	Float64DType	True	1 (< 0.1%)	0.00	0.00
29	Soil_Type_15	Float64DType	False	2 (0.1%)	0.00400	0.0631	0.00	0.00	1.00
30	Soil_Type_16	Float64DType	False	2 (0.1%)	0.00600	0.0772	0.00	0.00	1.00
31	Soil_Type_17	Float64DType	False	2 (0.1%)	0.00250	0.0499	0.00	0.00	1.00
32	Soil_Type_18	Float64DType	False	2 (0.1%)	0.00700	0.0834	0.00	0.00	1.00
33	Soil_Type_19	Float64DType	False	2 (0.1%)	0.0120	0.109	0.00	0.00	1.00
34	Soil_Type_20	Float64DType	False	2 (0.1%)	0.00200	0.0447	0.00	0.00	1.00
35	Soil_Type_21	Float64DType	False	2 (0.1%)	0.0580	0.234	0.00	0.00	1.00
36	Soil_Type_22	Float64DType	False	2 (0.1%)	0.106	0.308	0.00	0.00	1.00
37	Soil_Type_23	Float64DType	False	2 (0.1%)	0.0365	0.188	0.00	0.00	1.00
38	Soil_Type_24	Float64DType	False	2 (0.1%)	0.00150	0.0387	0.00	0.00	1.00
39	Soil_Type_25	Float64DType	False	2 (0.1%)	0.00700	0.0834	0.00	0.00	1.00
40	Soil_Type_26	Float64DType	False	2 (0.1%)	0.00350	0.0591	0.00	0.00	1.00
41	Soil_Type_27	Float64DType	False	2 (0.1%)	0.000500	0.0224	0.00	0.00	1.00
42	Soil_Type_28	Float64DType	False	2 (0.1%)	0.195	0.396	0.00	0.00	1.00
43	Soil_Type_29	Float64DType	False	2 (0.1%)	0.0525	0.223	0.00	0.00	1.00
44	Soil_Type_30	Float64DType	False	2 (0.1%)	0.0385	0.192	0.00	0.00	1.00
45	Soil_Type_31	Float64DType	False	2 (0.1%)	0.0865	0.281	0.00	0.00	1.00
46	Soil_Type_32	Float64DType	False	2 (0.1%)	0.0870	0.282	0.00	0.00	1.00
47	Soil_Type_33	Float64DType	False	2 (0.1%)	0.00300	0.0547	0.00	0.00	1.00
48	Soil_Type_34	Float64DType	False	2 (0.1%)	0.00250	0.0499	0.00	0.00	1.00
49	Soil_Type_35	Float64DType	False	2 (0.1%)	0.000500	0.0224	0.00	0.00	1.00
50	Soil_Type_36	Float64DType	False	2 (0.1%)	0.000500	0.0224	0.00	0.00	1.00
51	Soil_Type_37	Float64DType	False	2 (0.1%)	0.0210	0.143	0.00	0.00	1.00
52	Soil_Type_38	Float64DType	False	2 (0.1%)	0.0245	0.155	0.00	0.00	1.00
53	Soil_Type_39	Float64DType	False	2 (0.1%)	0.0190	0.137	0.00	0.00	1.00
54	class	ObjectDType	False	7 (0.4%)

Please enable javascript

.. GENERATED FROM PYTHON SOURCE LINES 187-198 .. code-block:: Python comparison_report = skore.compare( { "baseline_train": report_less, "more_rare_class_rows": report_more, } ) comparison_report.metrics.summarize( metric=["accuracy", "precision", "log_loss"], data_source="test", ).frame(flat_index=False) .. raw:: html

	estimator	baseline_train	more_rare_class_rows
metric	label
accuracy		0.738000	0.800500
precision	1	0.765805	0.807069
	2	0.783453	0.844681
	3	0.674419	0.811594
	4	0.250000	0.700000
	5	0.270270	0.508772
	6	0.372881	0.666667
	7	0.650000	0.609091
log_loss		2.991535	0.493421

.. GENERATED FROM PYTHON SOURCE LINES 199-200 Enriching the train set with more rare-class rows also clears SKD005. .. GENERATED FROM PYTHON SOURCE LINES 200-203 .. code-block:: Python report_more.checks.summarize(fast_mode=True, ignore=["SKD008"]) .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 204-211 Collecting more rare-class rows can improve rare-class precision and log-loss on a fixed natural-prevalence test set. That does not mean we should chase a cleared SKD005: if acquisition preferentially samples rare types, the training mix no longer matches the field, and production prevalence may stay low. We can correct for that shift before reading operating metrics. Clearing the check by reshaping the histogram is optional; better rare-class decisions under honest prevalence is the point. .. GENERATED FROM PYTHON SOURCE LINES 213-224 Conclusion ========== SKD005 is a multiclass rarity warning, not a request to rebalance at all costs. We prefer log-loss (and confusion matrices) over accuracy, we report absolute counts, and we can add rare-class labels when we can without treating a silent check as success. For binary rare-event threshold tuning and when ``class_weight`` is a risky shortcut, see :ref:`skd004-high-class-imbalance`. .. rst-class:: sphx-glr-timing **Total running time of the script:** (1 minutes 39.305 seconds) .. _sphx_glr_download_auto_examples_pitfalls_and_solutions_plot_skd005_underrepresented_classes.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_skd005_underrepresented_classes.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_skd005_underrepresented_classes.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_skd005_underrepresented_classes.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_

Elevation

Aspect

Slope

Horizontal_Distance_To_Hydrology

Vertical_Distance_To_Hydrology

Horizontal_Distance_To_Roadways

Hillshade_9am

Hillshade_Noon

Hillshade_3pm

Horizontal_Distance_To_Fire_Points

Wilderness_Area_0

Wilderness_Area_1

Wilderness_Area_2

Wilderness_Area_3

Soil_Type_0

Soil_Type_1

Soil_Type_2

Soil_Type_3

Soil_Type_4

Soil_Type_5

Soil_Type_6

Soil_Type_7

Soil_Type_8

Soil_Type_9

Soil_Type_10

Soil_Type_11

Soil_Type_12

Soil_Type_13

Soil_Type_14

Soil_Type_15

Soil_Type_16

Soil_Type_17

Soil_Type_18

Soil_Type_19

Soil_Type_20

Soil_Type_21

Soil_Type_22

Soil_Type_23

Soil_Type_24

Soil_Type_25

Soil_Type_26

Soil_Type_27

Soil_Type_28

Soil_Type_29

Soil_Type_30

Soil_Type_31

Soil_Type_32

Soil_Type_33

Soil_Type_34

Soil_Type_35

Soil_Type_36

Soil_Type_37

Soil_Type_38

Soil_Type_39

Please enable javascript

class

class

Please enable javascript

Elevation

Aspect

Slope

Horizontal_Distance_To_Hydrology

Vertical_Distance_To_Hydrology

Horizontal_Distance_To_Roadways

Hillshade_9am

Hillshade_Noon

Hillshade_3pm

Horizontal_Distance_To_Fire_Points

Wilderness_Area_0

Wilderness_Area_1

Wilderness_Area_2

Wilderness_Area_3

Soil_Type_0

Soil_Type_1

Soil_Type_2

Soil_Type_3

Soil_Type_4

Soil_Type_5

Soil_Type_6

Soil_Type_7