Movatterモバイル変換

Notes

The$R^2$ score used when callingscore on a regressor usesmultioutput='uniform_average' from version 0.23 to keep consistentwith default value ofr2_score().This influences thescore method of all the multioutputregressors (except forMultiOutputRegressor).

set_fit_request(*,base_margin='$UNCHANGED$',base_margin_eval_set='$UNCHANGED$',eval_set='$UNCHANGED$',feature_weights='$UNCHANGED$',sample_weight='$UNCHANGED$',sample_weight_eval_set='$UNCHANGED$',verbose='$UNCHANGED$',xgb_model='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thefit method.

Parameters:

self (XGBRegressor)
base_margin (bool |None |str)
base_margin_eval_set (bool |None |str)
eval_set (bool |None |str)
feature_weights (bool |None |str)
sample_weight (bool |None |str)
sample_weight_eval_set (bool |None |str)
verbose (bool |None |str)
xgb_model (bool |None |str)

Return type:

XGBRegressor

set_params(**params)

Set the parameters of this estimator. Modification of the sklearn method toallow unknown kwargs. This allows using the full range of xgboostparameters that are not defined as member variables in sklearn gridsearch.

Return type:: self
Parameters:: params (Any)

set_predict_request(*,base_margin='$UNCHANGED$',iteration_range='$UNCHANGED$',output_margin='$UNCHANGED$',validate_features='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thepredict method.

Parameters:

self (XGBRegressor)
base_margin (bool |None |str)
iteration_range (bool |None |str)
output_margin (bool |None |str)
validate_features (bool |None |str)

Return type:

XGBRegressor

set_score_request(*,sample_weight='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thescore method.

Parameters:

self (XGBRegressor)
sample_weight (bool |None |str)

Return type:

XGBRegressor

classxgboost.XGBClassifier(*,objective='binary:logistic',**kwargs)

Bases:ClassifierMixin,XGBModel

Implementation of the scikit-learn API for XGBoost classification.SeeUsing the Scikit-Learn Estimator Interface for more information.

Parameters:

n_estimators (Optional[int]) – Number of boosting rounds.
max_depth (Optional[int]) – Maximum tree depth for base learners.
max_leaves (Optional[int]) – Maximum number of leaves; 0 indicates no limit.
max_bin (Optional[int]) – If using histogram-based algorithm, maximum number of bins per feature
grow_policy (Optional[str]) –
Tree growing policy.
- depthwise: Favors splitting at nodes closest to the node,
- lossguide: Favors splitting at nodes with highest loss change.
learning_rate (Optional[float]) – Boosting learning rate (xgb’s “eta”)
verbosity (Optional[int]) – The degree of verbosity. Valid values are 0 (silent) - 3 (debug).
objective (Union[str,xgboost.sklearn._SklObjWProto,Callable[[Any,Any],Tuple[numpy.ndarray,numpy.ndarray]],NoneType]) –
Specify the learning task and the corresponding learning objective or a customobjective function to be used.
For custom objective, seeCustom Objective and Evaluation Metric andCustom objective and metric for more information, along with the end note forfunction signatures.
booster (Optional[str]) – Specify which booster to use:gbtree,gblinear ordart.
tree_method (Optional[str]) – Specify which tree method to use. Default to auto. If this parameter is set todefault, XGBoost will choose the most conservative option available. It’srecommended to study this option from the parameters documenttree method
n_jobs (Optional[int]) – Number of parallel threads used to run xgboost. When used with otherScikit-Learn algorithms like grid search, you may choose which algorithm toparallelize and balance the threads. Creating thread contention willsignificantly slow down both algorithms.
gamma (Optional[float]) – (min_split_loss) Minimum loss reduction required to make a further partition ona leaf node of the tree.
min_child_weight (Optional[float]) – Minimum sum of instance weight(hessian) needed in a child.
max_delta_step (Optional[float]) – Maximum delta step we allow each tree’s weight estimation to be.
subsample (Optional[float]) – Subsample ratio of the training instance.
sampling_method (Optional[str]) –
Sampling method. Used only by the GPU version ofhist tree method.
- uniform: Select random training instances uniformly.
- gradient_based: Select random training instances with higher probability
  when the gradient and hessian are larger. (cf. CatBoost)
colsample_bytree (Optional[float]) – Subsample ratio of columns when constructing each tree.
colsample_bylevel (Optional[float]) – Subsample ratio of columns for each level.
colsample_bynode (Optional[float]) – Subsample ratio of columns for each split.
reg_alpha (Optional[float]) – L1 regularization term on weights (xgb’s alpha).
reg_lambda (Optional[float]) – L2 regularization term on weights (xgb’s lambda).
scale_pos_weight (Optional[float]) – Balancing of positive and negative weights.
base_score (Optional[float]) – The initial prediction score of all instances, global bias.
random_state (Union[numpy.random.mtrand.RandomState,numpy.random._generator.Generator,int,NoneType]) –
Random number seed.
Note
Using gblinear booster with shotgun updater is nondeterministic asit uses Hogwild algorithm.
missing (float) – Value in the data which needs to be present as a missing value. Default tonumpy.nan.
num_parallel_tree (Optional[int]) – Used for boosting random forest.
monotone_constraints (Union[Dict[str,int],str,NoneType]) – Constraint of variable monotonicity. Seetutorialfor more information.
interaction_constraints (Union[str,List[Tuple[str]],NoneType]) – Constraints for interaction representing permitted interactions. Theconstraints must be specified in the form of a nested list, e.g.[[0,1],[2,3,4]], where each inner list is a group of indices of features that areallowed to interact with each other. Seetutorial for more information
importance_type (Optional[str]) –
The feature importance type for the feature_importances_ property:
- For tree model, it’s either “gain”, “weight”, “cover”, “total_gain” or“total_cover”.
- For linear model, only “weight” is defined and it’s the normalizedcoefficients without bias.
device (Optional[str]) –
Added in version 2.0.0.
Device ordinal, available options arecpu,cuda, andgpu.
validate_parameters (Optional[bool]) – Give warnings for unknown parameter.
enable_categorical (bool) – See the same parameter ofDMatrix for details.
feature_types (Optional[Sequence[str]]) –
Added in version 1.7.0.
Used for specifying feature types without constructing a dataframe. SeeDMatrix for details.
feature_weights (Optional[ArrayLike]) – Weight for each feature, defines the probability of each feature being selectedwhen colsample is being used. All values must be greater than 0, otherwise aValueError is thrown.
max_cat_to_onehot (Optional[int]) –
Added in version 1.6.0.
Note
This parameter is experimental
A threshold for deciding whether XGBoost should use one-hot encoding based splitfor categorical data. When number of categories is lesser than the thresholdthen one-hot encoding is chosen, otherwise the categories will be partitionedinto children nodes. Also,enable_categorical needs to be set to havecategorical feature support. SeeCategorical Data andParameters for Categorical Feature for details.
max_cat_threshold (Optional[int]) –
Added in version 1.7.0.
Note
This parameter is experimental
Maximum number of categories considered for each split. Used only bypartition-based splits for preventing over-fitting. Also,enable_categoricalneeds to be set to have categorical feature support. SeeCategorical Data andParameters for Categorical Feature for details.
multi_strategy (Optional[str]) –
Added in version 2.0.0.
Note
This parameter is working-in-progress.
The strategy used for training multi-target models, including multi-targetregression and multi-class classification. SeeMultiple Outputs formore information.
- one_output_per_tree: One model for each target.
- multi_output_tree: Use multi-target trees.
eval_metric (Union[str,List[Union[str,Callable]],Callable,NoneType]) –
Added in version 1.6.0.
Metric used for monitoring the training result and early stopping. It can be astring or list of strings as names of predefined metric in XGBoost (SeeXGBoost Parameters), one of the metrics insklearn.metrics, or anyother user defined metric that looks likesklearn.metrics.
If custom objective is also provided, then custom metric should implement thecorresponding reverse link function.
Unlike thescoring parameter commonly used in scikit-learn, when a callableobject is provided, it’s assumed to be a cost function and by default XGBoostwill minimize the result during early stopping.
For advanced usage on Early stopping like directly choosing to maximize insteadof minimize, seexgboost.callback.EarlyStopping.
SeeCustom Objective and Evaluation Metric andCustom objective and metric for moreinformation.
```
fromsklearn.datasetsimportload_diabetesfromsklearn.metricsimportmean_absolute_errorX,y=load_diabetes(return_X_y=True)reg=xgb.XGBRegressor(tree_method="hist",eval_metric=mean_absolute_error,)reg.fit(X,y,eval_set=[(X,y)])
```
early_stopping_rounds (Optional[int]) –
Added in version 1.6.0.
- Activates early stopping. Validation metric needs to improve at least once ineveryearly_stopping_rounds round(s) to continue training. Requires atleast one item ineval_set infit().
- If early stopping occurs, the model will have two additional attributes:best_score andbest_iteration. These are used by thepredict() andapply() methods to determine the optimalnumber of trees during inference. If users want to access the full model(including trees built after early stopping), they can specify theiteration_range in these inference methods. In addition, other utilitieslike model plotting can also use the entire model.
- If you prefer to discard the trees afterbest_iteration, consider using thecallback functionxgboost.callback.EarlyStopping.
- If there’s more than one item ineval_set, the last entry will be used forearly stopping. If there’s more than one metric ineval_metric, the lastmetric will be used for early stopping.
callbacks (Optional[List[xgboost.callback.TrainingCallback]]) –
List of callback functions that are applied at end of each iteration.It is possible to use predefined callbacks by usingCallback API.
Note
States in callback are not preserved during training, which means callbackobjects can not be reused for multiple training sessions withoutreinitialization or deepcopy.
```
forparamsinparameters_grid:# be sure to (re)initialize the callbacks before each runcallbacks=[xgb.callback.LearningRateScheduler(custom_rates)]reg=xgboost.XGBRegressor(**params,callbacks=callbacks)reg.fit(X,y)
```
kwargs (Optional[Any]) –
Keyword arguments for XGBoost Booster object. Full documentation of parameterscan be foundhere.Attempting to set a parameter via the constructor args and **kwargsdict simultaneously will result in a TypeError.
Note
**kwargs unsupported by scikit-learn
**kwargs is unsupported by scikit-learn. We do not guaranteethat parameters passed via this argument will interact properlywith scikit-learn.
Note
Custom objective function
A custom objective function can be provided for theobjectiveparameter. In this case, it should have the signatureobjective(y_true,y_pred)->[grad,hess] orobjective(y_true,y_pred,*,sample_weight)->[grad,hess]:
y_true: array_like of shape [n_samples]
The target values
y_pred: array_like of shape [n_samples]
The predicted values
sample_weight :
Optional sample weights.
grad: array_like of shape [n_samples]
The value of the gradient for each sample point.
hess: array_like of shape [n_samples]
The value of the second derivative for each sample point
Note that, if the custom objective produces negative values forthe Hessian, these will be clipped. If the objective is non-convex,one might also consider using the expected Hessian (Fisherinformation) instead.

apply(X,iteration_range=None)

Return the predicted leaf every tree for each sample. If the model is trainedwith early stopping, thenbest_iteration is used automatically.

Parameters:

X (Any) – Input features matrix. SeeSupported data structures for various XGBoost functions for a list of supported types.
iteration_range (Tuple[int |integer,int |integer]|None) – Seepredict().

Returns:

X_leaves – For each datapoint x in X and for each tree, return the index of theleaf x ends up in. Leaves are numbered within[0;2**(self.max_depth+1)), possibly with gaps in the numbering.

Return type:

array_like, shape=[n_samples, n_trees]

propertybest_iteration:int: The best iteration obtained by early stopping. This attribute is 0-based,for instance if the best iteration is the first round, then best_iteration is 0.

propertybest_score:float: The best score obtained by early stopping.

propertycoef_:ndarray

Coefficients property

Note

Coefficients are defined only for linear learners

Coefficients are only defined when the linear model is chosen asbase learner (booster=gblinear). It is not defined for other baselearner types, such as tree learners (booster=gbtree).

Returns:: coef_
Return type:: array of shape[n_features] or[n_classes,n_features]

evals_result()

Return the evaluation results.

The returned evaluation result is a dictionary:

{'validation_0':{'logloss':['0.604835','0.531479']},'validation_1':{'logloss':['0.41965','0.17686']}}

Return type:: evals_result

propertyfeature_importances_:ndarray

Returns:

feature_importances_ (array of shape[n_features] except for multi-class)
linear model, which returns an array with shape(n_features, n_classes)

propertyfeature_names_in_:ndarray: Names of features seen duringfit(). Defined only whenX hasfeature names that are all strings.

fit(X,y,*,sample_weight=None,base_margin=None,eval_set=None,verbose=True,xgb_model=None,sample_weight_eval_set=None,base_margin_eval_set=None,feature_weights=None)

Fit gradient boosting classifier.

Note that callingfit() multiple times will cause the model object to bere-fit from scratch. To resume training from a previous checkpoint, explicitlypassxgb_model argument.

Parameters:

X (Any) –
Input feature matrix. SeeSupported data structures for various XGBoost functions for a list of supported types.
When thetree_method is set tohist, internally, theQuantileDMatrix will be used instead of theDMatrixfor conserving memory. However, this has performance implications when thedevice of input data is not matched with algorithm. For instance, if theinput is a numpy array on CPU butcuda is used for training, then thedata is first processed on CPU then transferred to GPU.
y (Any) – Labels
sample_weight (Any |None) – instance weights
base_margin (Any |None) – Global bias for each instance. SeeIntercept for details.
eval_set (Sequence[Tuple[Any,Any]]|None) – A list of (X, y) tuple pairs to use as validation sets, for whichmetrics will be computed.Validation metrics will help us track the performance of the model.
verbose (bool |int |None) – Ifverbose is True and an evaluation set is used, the evaluation metricmeasured on the validation set is printed to stdout at each boosting stage.Ifverbose is an integer, the evaluation metric is printed at eachverbose boosting stage. The last boosting stage / the boosting stage foundby usingearly_stopping_rounds is also printed.
xgb_model (Booster |str |XGBModel |None) – file name of stored XGBoost model or ‘Booster’ instance XGBoost model to beloaded before training (allows training continuation).
sample_weight_eval_set (Sequence[Any]|None) – A list of the form [L_1, L_2, …, L_n], where each L_i is an array likeobject storing instance weights for the i-th validation set.
base_margin_eval_set (Sequence[Any]|None) – A list of the form [M_1, M_2, …, M_n], where each M_i is an array likeobject storing base margin for the i-th validation set.
feature_weights (Any |None) –
Deprecated since version 3.0.0.
Usefeature_weights in__init__() orset_params()instead.

Return type:

get_booster()

Get the underlying xgboost Booster of this model.

This will raise an exception when fit was not called

Returns:: booster
Return type:: a xgboost booster of underlying model

get_metadata_routing()

Get metadata routing of this object.

Please checkUser Guide on how the routingmechanism works.

Returns:: routing – AMetadataRequest encapsulatingrouting information.
Return type:: MetadataRequest

get_num_boosting_rounds()

Gets the number of xgboost boosting rounds.

Return type:: int

get_params(deep=True)

Get parameters.

Parameters:: deep (bool)
Return type:: Dict[str,Any]

get_xgb_params()

Get xgboost specific parameters.

Return type:: Dict[str,Any]

propertyintercept_:ndarray

Intercept (bias) property

For tree-based model, the returned value is thebase_score.

Returns:: intercept_
Return type:: array of shape(1,) or[n_classes]

load_model(fname)

Load the model from a file or a bytearray.

model.save_model("model.json")model.load_model("model.json")# ormodel.save_model("model.ubj")model.load_model("model.ubj")# orbuf=model.save_raw()model.load_model(buf)

Parameters:: fname (PathLike |bytearray |str) – Input file name or memory buffer(see also save_raw)
Return type:: None

propertyn_features_in_:int: Number of features seen duringfit().

predict(X,*,output_margin=False,validate_features=True,base_margin=None,iteration_range=None)

Note

This function is only thread safe forgbtree anddart.

Parameters:

X (Any) – Data to predict with. SeeSupported data structures for various XGBoost functions for a list of supported types.
output_margin (bool) – Whether to output the raw untransformed margin value.
validate_features (bool) – When this is True, validate that the Booster’s and data’s feature_names areidentical. Otherwise, it is assumed that the feature_names are the same.
base_margin (Any |None) – Global bias for each instance. SeeIntercept for details.
iteration_range (Tuple[int |integer,int |integer]|None) –
Specifies which layer of trees are used in prediction. For example, if arandom forest is trained with 100 rounds. Specifyingiteration_range=(10,20), then only the forests built during [10, 20) (half open set) roundsare used in this prediction.
Added in version 1.4.0.

Return type:

prediction

predict_proba(X,validate_features=True,base_margin=None,iteration_range=None)

Predict the probability of eachX example being of a given class. If themodel is trained with early stopping, thenbest_iteration is usedautomatically. The estimator usesinplace_predict by default and falls back tousingDMatrix if devices between the data and the estimator don’tmatch.

Note

This function is only thread safe forgbtree anddart.

Parameters:

X (Any) – Feature matrix. SeeSupported data structures for various XGBoost functions for a list of supported types.
validate_features (bool) – When this is True, validate that the Booster’s and data’s feature_names areidentical. Otherwise, it is assumed that the feature_names are the same.
base_margin (Any |None) – Global bias for each instance. SeeIntercept for details.
iteration_range (Tuple[int |integer,int |integer]|None) – Specifies which layer of trees are used in prediction. For example, if arandom forest is trained with 100 rounds. Specifyingiteration_range=(10,20), then only the forests built during [10, 20) (half open set) rounds areused in this prediction.

Returns:

a numpy array of shape array-like of shape (n_samples, n_classes) with theprobability of each data example being of a given class.

Return type:

prediction

save_model(fname)

Save the model to a file.

model.save_model("model.json")# ormodel.save_model("model.ubj")

Parameters:: fname (str |PathLike) – Output file name
Return type:: None

score(X,y,sample_weight=None)

Returnaccuracy on provided data and labels.

In multi-label classification, this is the subset accuracywhich is a harsh metric since you require for each sample thateach label set be correctly predicted.

Parameters:

X (array-like ofshape (n_samples,n_features)) – Test samples.
y (array-like ofshape (n_samples,) or(n_samples,n_outputs)) – True labels forX.
sample_weight (array-like ofshape (n_samples,),default=None) – Sample weights.

Returns:

score – Mean accuracy ofself.predict(X) w.r.t.y.

Return type:

Configure whether metadata should be requested to be passed to thefit method.

Parameters:

self (XGBClassifier)
base_margin (bool |None |str)
base_margin_eval_set (bool |None |str)
eval_set (bool |None |str)
feature_weights (bool |None |str)
sample_weight (bool |None |str)
sample_weight_eval_set (bool |None |str)
verbose (bool |None |str)
xgb_model (bool |None |str)

Return type:

set_params(**params)

Return type:: self
Parameters:: params (Any)

set_predict_proba_request(*,base_margin='$UNCHANGED$',iteration_range='$UNCHANGED$',validate_features='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thepredict_proba method.

Parameters:

self (XGBClassifier)
base_margin (bool |None |str)
iteration_range (bool |None |str)
validate_features (bool |None |str)

Return type:

set_predict_request(*,base_margin='$UNCHANGED$',iteration_range='$UNCHANGED$',output_margin='$UNCHANGED$',validate_features='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thepredict method.

Parameters:

self (XGBClassifier)
base_margin (bool |None |str)
iteration_range (bool |None |str)
output_margin (bool |None |str)
validate_features (bool |None |str)

Return type:

set_score_request(*,sample_weight='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thescore method.

Parameters:

self (XGBClassifier)
sample_weight (bool |None |str)

Return type:

classxgboost.XGBRanker(*,objective='rank:ndcg',**kwargs)

Bases:XGBRankerMixIn,XGBModel

Implementation of the Scikit-Learn API for XGBoost Ranking.

SeeLearning to Rank for an introducion.

SeeUsing the Scikit-Learn Estimator Interface for more information.

Parameters:

n_estimators (Optional[int]) – Number of gradient boosted trees. Equivalent to number of boostingrounds.
max_depth (Optional[int]) – Maximum tree depth for base learners.
max_leaves (Optional[int]) – Maximum number of leaves; 0 indicates no limit.
max_bin (Optional[int]) – If using histogram-based algorithm, maximum number of bins per feature
grow_policy (Optional[str]) –
Tree growing policy.
- depthwise: Favors splitting at nodes closest to the node,
- lossguide: Favors splitting at nodes with highest loss change.
learning_rate (Optional[float]) – Boosting learning rate (xgb’s “eta”)
verbosity (Optional[int]) – The degree of verbosity. Valid values are 0 (silent) - 3 (debug).
objective (Union[str,xgboost.sklearn._SklObjWProto,Callable[[Any,Any],Tuple[numpy.ndarray,numpy.ndarray]],NoneType]) –
Specify the learning task and the corresponding learning objective or a customobjective function to be used.
For custom objective, seeCustom Objective and Evaluation Metric andCustom objective and metric for more information, along with the end note forfunction signatures.
booster (Optional[str]) – Specify which booster to use:gbtree,gblinear ordart.
tree_method (Optional[str]) – Specify which tree method to use. Default to auto. If this parameter is set todefault, XGBoost will choose the most conservative option available. It’srecommended to study this option from the parameters documenttree method
n_jobs (Optional[int]) – Number of parallel threads used to run xgboost. When used with otherScikit-Learn algorithms like grid search, you may choose which algorithm toparallelize and balance the threads. Creating thread contention willsignificantly slow down both algorithms.
gamma (Optional[float]) – (min_split_loss) Minimum loss reduction required to make a further partition ona leaf node of the tree.
min_child_weight (Optional[float]) – Minimum sum of instance weight(hessian) needed in a child.
max_delta_step (Optional[float]) – Maximum delta step we allow each tree’s weight estimation to be.
subsample (Optional[float]) – Subsample ratio of the training instance.
sampling_method (Optional[str]) –
Sampling method. Used only by the GPU version ofhist tree method.
- uniform: Select random training instances uniformly.
- gradient_based: Select random training instances with higher probability
  when the gradient and hessian are larger. (cf. CatBoost)
colsample_bytree (Optional[float]) – Subsample ratio of columns when constructing each tree.
colsample_bylevel (Optional[float]) – Subsample ratio of columns for each level.
colsample_bynode (Optional[float]) – Subsample ratio of columns for each split.
reg_alpha (Optional[float]) – L1 regularization term on weights (xgb’s alpha).
reg_lambda (Optional[float]) – L2 regularization term on weights (xgb’s lambda).
scale_pos_weight (Optional[float]) – Balancing of positive and negative weights.
base_score (Optional[float]) – The initial prediction score of all instances, global bias.
random_state (Union[numpy.random.mtrand.RandomState,numpy.random._generator.Generator,int,NoneType]) –
Random number seed.
Note
Using gblinear booster with shotgun updater is nondeterministic asit uses Hogwild algorithm.
missing (float) – Value in the data which needs to be present as a missing value. Default tonumpy.nan.
num_parallel_tree (Optional[int]) – Used for boosting random forest.
monotone_constraints (Union[Dict[str,int],str,NoneType]) – Constraint of variable monotonicity. Seetutorialfor more information.
interaction_constraints (Union[str,List[Tuple[str]],NoneType]) – Constraints for interaction representing permitted interactions. Theconstraints must be specified in the form of a nested list, e.g.[[0,1],[2,3,4]], where each inner list is a group of indices of features that areallowed to interact with each other. Seetutorial for more information
importance_type (Optional[str]) –
The feature importance type for the feature_importances_ property:
- For tree model, it’s either “gain”, “weight”, “cover”, “total_gain” or“total_cover”.
- For linear model, only “weight” is defined and it’s the normalizedcoefficients without bias.
device (Optional[str]) –
Added in version 2.0.0.
Device ordinal, available options arecpu,cuda, andgpu.
validate_parameters (Optional[bool]) – Give warnings for unknown parameter.
enable_categorical (bool) – See the same parameter ofDMatrix for details.
feature_types (Optional[Sequence[str]]) –
Added in version 1.7.0.
Used for specifying feature types without constructing a dataframe. SeeDMatrix for details.
feature_weights (Optional[ArrayLike]) – Weight for each feature, defines the probability of each feature being selectedwhen colsample is being used. All values must be greater than 0, otherwise aValueError is thrown.
max_cat_to_onehot (Optional[int]) –
Added in version 1.6.0.
Note
This parameter is experimental
A threshold for deciding whether XGBoost should use one-hot encoding based splitfor categorical data. When number of categories is lesser than the thresholdthen one-hot encoding is chosen, otherwise the categories will be partitionedinto children nodes. Also,enable_categorical needs to be set to havecategorical feature support. SeeCategorical Data andParameters for Categorical Feature for details.
max_cat_threshold (Optional[int]) –
Added in version 1.7.0.
Note
This parameter is experimental
Maximum number of categories considered for each split. Used only bypartition-based splits for preventing over-fitting. Also,enable_categoricalneeds to be set to have categorical feature support. SeeCategorical Data andParameters for Categorical Feature for details.
multi_strategy (Optional[str]) –
Added in version 2.0.0.
Note
This parameter is working-in-progress.
The strategy used for training multi-target models, including multi-targetregression and multi-class classification. SeeMultiple Outputs formore information.
- one_output_per_tree: One model for each target.
- multi_output_tree: Use multi-target trees.
eval_metric (Union[str,List[Union[str,Callable]],Callable,NoneType]) –
Added in version 1.6.0.
Metric used for monitoring the training result and early stopping. It can be astring or list of strings as names of predefined metric in XGBoost (SeeXGBoost Parameters), one of the metrics insklearn.metrics, or anyother user defined metric that looks likesklearn.metrics.
If custom objective is also provided, then custom metric should implement thecorresponding reverse link function.
Unlike thescoring parameter commonly used in scikit-learn, when a callableobject is provided, it’s assumed to be a cost function and by default XGBoostwill minimize the result during early stopping.
For advanced usage on Early stopping like directly choosing to maximize insteadof minimize, seexgboost.callback.EarlyStopping.
SeeCustom Objective and Evaluation Metric andCustom objective and metric for moreinformation.
```
fromsklearn.datasetsimportload_diabetesfromsklearn.metricsimportmean_absolute_errorX,y=load_diabetes(return_X_y=True)reg=xgb.XGBRegressor(tree_method="hist",eval_metric=mean_absolute_error,)reg.fit(X,y,eval_set=[(X,y)])
```
early_stopping_rounds (Optional[int]) –
Added in version 1.6.0.
- Activates early stopping. Validation metric needs to improve at least once ineveryearly_stopping_rounds round(s) to continue training. Requires atleast one item ineval_set infit().
- If early stopping occurs, the model will have two additional attributes:best_score andbest_iteration. These are used by thepredict() andapply() methods to determine the optimalnumber of trees during inference. If users want to access the full model(including trees built after early stopping), they can specify theiteration_range in these inference methods. In addition, other utilitieslike model plotting can also use the entire model.
- If you prefer to discard the trees afterbest_iteration, consider using thecallback functionxgboost.callback.EarlyStopping.
- If there’s more than one item ineval_set, the last entry will be used forearly stopping. If there’s more than one metric ineval_metric, the lastmetric will be used for early stopping.
callbacks (Optional[List[xgboost.callback.TrainingCallback]]) –
List of callback functions that are applied at end of each iteration.It is possible to use predefined callbacks by usingCallback API.
Note
States in callback are not preserved during training, which means callbackobjects can not be reused for multiple training sessions withoutreinitialization or deepcopy.
```
forparamsinparameters_grid:# be sure to (re)initialize the callbacks before each runcallbacks=[xgb.callback.LearningRateScheduler(custom_rates)]reg=xgboost.XGBRegressor(**params,callbacks=callbacks)reg.fit(X,y)
```
kwargs (Optional[Any]) –
Keyword arguments for XGBoost Booster object. Full documentation of parameterscan be foundhere.Attempting to set a parameter via the constructor args and **kwargsdict simultaneously will result in a TypeError.
Note
**kwargs unsupported by scikit-learn
**kwargs is unsupported by scikit-learn. We do not guaranteethat parameters passed via this argument will interact properlywith scikit-learn.
Note
A custom objective function is currently not supported by XGBRanker.
Note
Query group information is only required for ranking training but notprediction. Multiple groups can be predicted on a single call topredict().
When fitting the model with thegroup parameter, your data need to be sortedby the query group first.group is an array that contains the size of eachquery group.
Similarly, when fitting the model with theqid parameter, the data should besorted according to query index andqid is an array that contains the queryindex for each training sample.
For example, if your original data look like:
qid
label
features
1
0
x_1
1
1
x_2
1
0
x_3
2
0
x_4
2
1
x_5
2
1
x_6
2
1
x_7
thenfit() method can be called with eithergroup array as[3,4]or withqid as[1,1,1,2,2,2,2], that is the qid column. Also, theqid can be a special column of inputX instead of a separated parameter, seefit() for more info.

apply(X,iteration_range=None)

Return the predicted leaf every tree for each sample. If the model is trainedwith early stopping, thenbest_iteration is used automatically.

Parameters:

X (Any) – Input features matrix. SeeSupported data structures for various XGBoost functions for a list of supported types.
iteration_range (Tuple[int |integer,int |integer]|None) – Seepredict().

Returns:

X_leaves – For each datapoint x in X and for each tree, return the index of theleaf x ends up in. Leaves are numbered within[0;2**(self.max_depth+1)), possibly with gaps in the numbering.

Return type:

array_like, shape=[n_samples, n_trees]

propertybest_iteration:int: The best iteration obtained by early stopping. This attribute is 0-based,for instance if the best iteration is the first round, then best_iteration is 0.

propertybest_score:float: The best score obtained by early stopping.

propertycoef_:ndarray

Coefficients property

Note

Coefficients are defined only for linear learners

Coefficients are only defined when the linear model is chosen asbase learner (booster=gblinear). It is not defined for other baselearner types, such as tree learners (booster=gbtree).

Returns:: coef_
Return type:: array of shape[n_features] or[n_classes,n_features]

evals_result()

Return the evaluation results.

The returned evaluation result is a dictionary:

{'validation_0':{'logloss':['0.604835','0.531479']},'validation_1':{'logloss':['0.41965','0.17686']}}

Return type:: evals_result

propertyfeature_importances_:ndarray

Returns:

feature_importances_ (array of shape[n_features] except for multi-class)
linear model, which returns an array with shape(n_features, n_classes)

propertyfeature_names_in_:ndarray: Names of features seen duringfit(). Defined only whenX hasfeature names that are all strings.

fit(X,y,*,group=None,qid=None,sample_weight=None,base_margin=None,eval_set=None,eval_group=None,eval_qid=None,verbose=False,xgb_model=None,sample_weight_eval_set=None,base_margin_eval_set=None,feature_weights=None)

Fit gradient boosting ranker

Note that callingfit() multiple times will cause the model object to bere-fit from scratch. To resume training from a previous checkpoint, explicitlypassxgb_model argument.

Parameters:

X (Any) –
Feature matrix. SeeSupported data structures for various XGBoost functions for a list of supported types.
When this is apandas.DataFrame or acudf.DataFrame,it may contain a special column calledqid for specifying the queryindex. Using a special column is the same as using theqid parameter,except for being compatible with sklearn utility functions likesklearn.model_selection.cross_validation(). The same conventionapplies to theXGBRanker.score() andXGBRanker.predict().
qid
feat_0
feat_1
0
$x_{00}$
$x_{01}$
1
$x_{10}$
$x_{11}$
1
$x_{20}$
$x_{21}$
When thetree_method is set tohist, internally, theQuantileDMatrix will be used instead of theDMatrixfor conserving memory. However, this has performance implications when thedevice of input data is not matched with algorithm. For instance, if theinput is a numpy array on CPU butcuda is used for training, then thedata is first processed on CPU then transferred to GPU.
y (Any) – Labels
group (Any |None) – Size of each query group of training data. Should have as many elements asthe query groups in the training data. If this is set to None, then usermust provide qid.
qid (Any |None) – Query ID for each training sample. Should have the size of n_samples. Ifthis is set to None, then user must provide group or a special column in X.
sample_weight (Any |None) –
Query group weights
Note
Weights are per-group for ranking tasks
In ranking task, one weight is assigned to each query group/id (not eachdata point). This is because we only care about the relative ordering ofdata points within each group, so it doesn’t make sense to assignweights to individual data points.
base_margin (Any |None) – Global bias for each instance. SeeIntercept for details.
eval_set (Sequence[Tuple[Any,Any]]|None) – A list of (X, y) tuple pairs to use as validation sets, for whichmetrics will be computed.Validation metrics will help us track the performance of the model.
eval_group (Sequence[Any]|None) – A list in whicheval_group[i] is the list containing the sizes of allquery groups in thei-th pair ineval_set.
eval_qid (Sequence[Any]|None) – A list in whicheval_qid[i] is the array containing query ID ofi-thpair ineval_set. The special column convention inX applies tovalidation datasets as well.
verbose (bool |int |None) – Ifverbose is True and an evaluation set is used, the evaluation metricmeasured on the validation set is printed to stdout at each boosting stage.Ifverbose is an integer, the evaluation metric is printed at eachverbose boosting stage. The last boosting stage / the boosting stage foundby usingearly_stopping_rounds is also printed.
xgb_model (Booster |str |XGBModel |None) – file name of stored XGBoost model or ‘Booster’ instance XGBoost model to beloaded before training (allows training continuation).
sample_weight_eval_set (Sequence[Any]|None) –
A list of the form [L_1, L_2, …, L_n], where each L_i is a list ofgroup weights on the i-th validation set.
Note
Weights are per-group for ranking tasks
In ranking task, one weight is assigned to each query group (not eachdata point). This is because we only care about the relative ordering ofdata points within each group, so it doesn’t make sense to assignweights to individual data points.
base_margin_eval_set (Sequence[Any]|None) – A list of the form [M_1, M_2, …, M_n], where each M_i is an array likeobject storing base margin for the i-th validation set.
feature_weights (Any |None) – Weight for each feature, defines the probability of each feature beingselected when colsample is being used. All values must be greater than 0,otherwise aValueError is thrown.

Return type:

XGBRanker

get_booster()

Get the underlying xgboost Booster of this model.

This will raise an exception when fit was not called

Returns:: booster
Return type:: a xgboost booster of underlying model

get_metadata_routing()

Get metadata routing of this object.

Please checkUser Guide on how the routingmechanism works.

Returns:: routing – AMetadataRequest encapsulatingrouting information.
Return type:: MetadataRequest

get_num_boosting_rounds()

Gets the number of xgboost boosting rounds.

Return type:: int

get_params(deep=True)

Get parameters.

Parameters:: deep (bool)
Return type:: Dict[str,Any]

get_xgb_params()

Get xgboost specific parameters.

Return type:: Dict[str,Any]

propertyintercept_:ndarray

Intercept (bias) property

For tree-based model, the returned value is thebase_score.

Returns:: intercept_
Return type:: array of shape(1,) or[n_classes]

load_model(fname)

Load the model from a file or a bytearray.

model.save_model("model.json")model.load_model("model.json")# ormodel.save_model("model.ubj")model.load_model("model.ubj")# orbuf=model.save_raw()model.load_model(buf)

Parameters:: fname (PathLike |bytearray |str) – Input file name or memory buffer(see also save_raw)
Return type:: None

propertyn_features_in_:int: Number of features seen duringfit().

predict(X,*,output_margin=False,validate_features=True,base_margin=None,iteration_range=None)

Note

This function is only thread safe forgbtree anddart.

Parameters:

X (Any) – Data to predict with. SeeSupported data structures for various XGBoost functions for a list of supported types.
output_margin (bool) – Whether to output the raw untransformed margin value.
validate_features (bool) – When this is True, validate that the Booster’s and data’s feature_names areidentical. Otherwise, it is assumed that the feature_names are the same.
base_margin (Any |None) – Global bias for each instance. SeeIntercept for details.
iteration_range (Tuple[int |integer,int |integer]|None) –
Specifies which layer of trees are used in prediction. For example, if arandom forest is trained with 100 rounds. Specifyingiteration_range=(10,20), then only the forests built during [10, 20) (half open set) roundsare used in this prediction.
Added in version 1.4.0.

Return type:

prediction

save_model(fname)

Save the model to a file.

model.save_model("model.json")# ormodel.save_model("model.ubj")

Parameters:: fname (str |PathLike) – Output file name
Return type:: None

score(X,y)

Evaluate score for data using the last evaluation metric. If the model istrained with early stopping, thenbest_iteration is usedautomatically.

Parameters:

X (Union[pd.DataFrame,cudf.DataFrame]) – Feature matrix. A DataFrame with a specialqid column.
y (Any) – Labels

Returns:

The result of the first evaluation metric for the ranker.

Return type:

score

set_fit_request(*,base_margin='$UNCHANGED$',base_margin_eval_set='$UNCHANGED$',eval_group='$UNCHANGED$',eval_qid='$UNCHANGED$',eval_set='$UNCHANGED$',feature_weights='$UNCHANGED$',group='$UNCHANGED$',qid='$UNCHANGED$',sample_weight='$UNCHANGED$',sample_weight_eval_set='$UNCHANGED$',verbose='$UNCHANGED$',xgb_model='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thefit method.

Parameters:

self (XGBRanker)
base_margin (bool |None |str)
base_margin_eval_set (bool |None |str)
eval_group (bool |None |str)
eval_qid (bool |None |str)
eval_set (bool |None |str)
feature_weights (bool |None |str)
group (bool |None |str)
qid (bool |None |str)
sample_weight (bool |None |str)
sample_weight_eval_set (bool |None |str)
verbose (bool |None |str)
xgb_model (bool |None |str)

Return type:

XGBRanker

set_params(**params)

Return type:: self
Parameters:: params (Any)

set_predict_request(*,base_margin='$UNCHANGED$',iteration_range='$UNCHANGED$',output_margin='$UNCHANGED$',validate_features='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thepredict method.

Parameters:

self (XGBRanker)
base_margin (bool |None |str)
iteration_range (bool |None |str)
output_margin (bool |None |str)
validate_features (bool |None |str)

Return type:

XGBRanker

classxgboost.XGBRFRegressor(*,learning_rate=1.0,subsample=0.8,colsample_bynode=0.8,reg_lambda=1e-05,**kwargs)

Bases:XGBRegressor

scikit-learn API for XGBoost random forest regression.SeeUsing the Scikit-Learn Estimator Interface for more information.

Parameters:

n_estimators (Optional[int]) – Number of trees in random forest to fit.
max_depth (Optional[int]) – Maximum tree depth for base learners.
max_leaves (Optional[int]) – Maximum number of leaves; 0 indicates no limit.
max_bin (Optional[int]) – If using histogram-based algorithm, maximum number of bins per feature
grow_policy (Optional[str]) –
Tree growing policy.
- depthwise: Favors splitting at nodes closest to the node,
- lossguide: Favors splitting at nodes with highest loss change.
learning_rate (Optional[float]) – Boosting learning rate (xgb’s “eta”)
verbosity (Optional[int]) – The degree of verbosity. Valid values are 0 (silent) - 3 (debug).
objective (Union[str,xgboost.sklearn._SklObjWProto,Callable[[Any,Any],Tuple[numpy.ndarray,numpy.ndarray]],NoneType]) –
Specify the learning task and the corresponding learning objective or a customobjective function to be used.
For custom objective, seeCustom Objective and Evaluation Metric andCustom objective and metric for more information, along with the end note forfunction signatures.
booster (Optional[str]) – Specify which booster to use:gbtree,gblinear ordart.
tree_method (Optional[str]) – Specify which tree method to use. Default to auto. If this parameter is set todefault, XGBoost will choose the most conservative option available. It’srecommended to study this option from the parameters documenttree method
n_jobs (Optional[int]) – Number of parallel threads used to run xgboost. When used with otherScikit-Learn algorithms like grid search, you may choose which algorithm toparallelize and balance the threads. Creating thread contention willsignificantly slow down both algorithms.
gamma (Optional[float]) – (min_split_loss) Minimum loss reduction required to make a further partition ona leaf node of the tree.
min_child_weight (Optional[float]) – Minimum sum of instance weight(hessian) needed in a child.
max_delta_step (Optional[float]) – Maximum delta step we allow each tree’s weight estimation to be.
subsample (Optional[float]) – Subsample ratio of the training instance.
sampling_method (Optional[str]) –
Sampling method. Used only by the GPU version ofhist tree method.
- uniform: Select random training instances uniformly.
- gradient_based: Select random training instances with higher probability
  when the gradient and hessian are larger. (cf. CatBoost)
colsample_bytree (Optional[float]) – Subsample ratio of columns when constructing each tree.
colsample_bylevel (Optional[float]) – Subsample ratio of columns for each level.
colsample_bynode (Optional[float]) – Subsample ratio of columns for each split.
reg_alpha (Optional[float]) – L1 regularization term on weights (xgb’s alpha).
reg_lambda (Optional[float]) – L2 regularization term on weights (xgb’s lambda).
scale_pos_weight (Optional[float]) – Balancing of positive and negative weights.
base_score (Optional[float]) – The initial prediction score of all instances, global bias.
random_state (Union[numpy.random.mtrand.RandomState,numpy.random._generator.Generator,int,NoneType]) –
Random number seed.
Note
Using gblinear booster with shotgun updater is nondeterministic asit uses Hogwild algorithm.
missing (float) – Value in the data which needs to be present as a missing value. Default tonumpy.nan.
num_parallel_tree (Optional[int]) – Used for boosting random forest.
monotone_constraints (Union[Dict[str,int],str,NoneType]) – Constraint of variable monotonicity. Seetutorialfor more information.
interaction_constraints (Union[str,List[Tuple[str]],NoneType]) – Constraints for interaction representing permitted interactions. Theconstraints must be specified in the form of a nested list, e.g.[[0,1],[2,3,4]], where each inner list is a group of indices of features that areallowed to interact with each other. Seetutorial for more information
importance_type (Optional[str]) –
The feature importance type for the feature_importances_ property:
- For tree model, it’s either “gain”, “weight”, “cover”, “total_gain” or“total_cover”.
- For linear model, only “weight” is defined and it’s the normalizedcoefficients without bias.
device (Optional[str]) –
Added in version 2.0.0.
Device ordinal, available options arecpu,cuda, andgpu.
validate_parameters (Optional[bool]) – Give warnings for unknown parameter.
enable_categorical (bool) – See the same parameter ofDMatrix for details.
feature_types (Optional[Sequence[str]]) –
Added in version 1.7.0.
Used for specifying feature types without constructing a dataframe. SeeDMatrix for details.
feature_weights (Optional[ArrayLike]) – Weight for each feature, defines the probability of each feature being selectedwhen colsample is being used. All values must be greater than 0, otherwise aValueError is thrown.
max_cat_to_onehot (Optional[int]) –
Added in version 1.6.0.
Note
This parameter is experimental
A threshold for deciding whether XGBoost should use one-hot encoding based splitfor categorical data. When number of categories is lesser than the thresholdthen one-hot encoding is chosen, otherwise the categories will be partitionedinto children nodes. Also,enable_categorical needs to be set to havecategorical feature support. SeeCategorical Data andParameters for Categorical Feature for details.
max_cat_threshold (Optional[int]) –
Added in version 1.7.0.
Note
This parameter is experimental
Maximum number of categories considered for each split. Used only bypartition-based splits for preventing over-fitting. Also,enable_categoricalneeds to be set to have categorical feature support. SeeCategorical Data andParameters for Categorical Feature for details.
multi_strategy (Optional[str]) –
Added in version 2.0.0.
Note
This parameter is working-in-progress.
The strategy used for training multi-target models, including multi-targetregression and multi-class classification. SeeMultiple Outputs formore information.
- one_output_per_tree: One model for each target.
- multi_output_tree: Use multi-target trees.
eval_metric (Union[str,List[Union[str,Callable]],Callable,NoneType]) –
Added in version 1.6.0.
Metric used for monitoring the training result and early stopping. It can be astring or list of strings as names of predefined metric in XGBoost (SeeXGBoost Parameters), one of the metrics insklearn.metrics, or anyother user defined metric that looks likesklearn.metrics.
If custom objective is also provided, then custom metric should implement thecorresponding reverse link function.
Unlike thescoring parameter commonly used in scikit-learn, when a callableobject is provided, it’s assumed to be a cost function and by default XGBoostwill minimize the result during early stopping.
For advanced usage on Early stopping like directly choosing to maximize insteadof minimize, seexgboost.callback.EarlyStopping.
SeeCustom Objective and Evaluation Metric andCustom objective and metric for moreinformation.
```
fromsklearn.datasetsimportload_diabetesfromsklearn.metricsimportmean_absolute_errorX,y=load_diabetes(return_X_y=True)reg=xgb.XGBRegressor(tree_method="hist",eval_metric=mean_absolute_error,)reg.fit(X,y,eval_set=[(X,y)])
```
early_stopping_rounds (Optional[int]) –
Added in version 1.6.0.
- Activates early stopping. Validation metric needs to improve at least once ineveryearly_stopping_rounds round(s) to continue training. Requires atleast one item ineval_set infit().
- If early stopping occurs, the model will have two additional attributes:best_score andbest_iteration. These are used by thepredict() andapply() methods to determine the optimalnumber of trees during inference. If users want to access the full model(including trees built after early stopping), they can specify theiteration_range in these inference methods. In addition, other utilitieslike model plotting can also use the entire model.
- If you prefer to discard the trees afterbest_iteration, consider using thecallback functionxgboost.callback.EarlyStopping.
- If there’s more than one item ineval_set, the last entry will be used forearly stopping. If there’s more than one metric ineval_metric, the lastmetric will be used for early stopping.
callbacks (Optional[List[xgboost.callback.TrainingCallback]]) –
List of callback functions that are applied at end of each iteration.It is possible to use predefined callbacks by usingCallback API.
Note
States in callback are not preserved during training, which means callbackobjects can not be reused for multiple training sessions withoutreinitialization or deepcopy.
```
forparamsinparameters_grid:# be sure to (re)initialize the callbacks before each runcallbacks=[xgb.callback.LearningRateScheduler(custom_rates)]reg=xgboost.XGBRegressor(**params,callbacks=callbacks)reg.fit(X,y)
```
kwargs (Optional[Any]) –
Keyword arguments for XGBoost Booster object. Full documentation of parameterscan be foundhere.Attempting to set a parameter via the constructor args and **kwargsdict simultaneously will result in a TypeError.
Note
**kwargs unsupported by scikit-learn
**kwargs is unsupported by scikit-learn. We do not guaranteethat parameters passed via this argument will interact properlywith scikit-learn.
Note
Custom objective function
A custom objective function can be provided for theobjectiveparameter. In this case, it should have the signatureobjective(y_true,y_pred)->[grad,hess] orobjective(y_true,y_pred,*,sample_weight)->[grad,hess]:
y_true: array_like of shape [n_samples]
The target values
y_pred: array_like of shape [n_samples]
The predicted values
sample_weight :
Optional sample weights.
grad: array_like of shape [n_samples]
The value of the gradient for each sample point.
hess: array_like of shape [n_samples]
The value of the second derivative for each sample point
Note that, if the custom objective produces negative values forthe Hessian, these will be clipped. If the objective is non-convex,one might also consider using the expected Hessian (Fisherinformation) instead.

apply(X,iteration_range=None)

Return the predicted leaf every tree for each sample. If the model is trainedwith early stopping, thenbest_iteration is used automatically.

Parameters:

X (Any) – Input features matrix. SeeSupported data structures for various XGBoost functions for a list of supported types.
iteration_range (Tuple[int |integer,int |integer]|None) – Seepredict().

Returns:

X_leaves – For each datapoint x in X and for each tree, return the index of theleaf x ends up in. Leaves are numbered within[0;2**(self.max_depth+1)), possibly with gaps in the numbering.

Return type:

array_like, shape=[n_samples, n_trees]

propertybest_iteration:int: The best iteration obtained by early stopping. This attribute is 0-based,for instance if the best iteration is the first round, then best_iteration is 0.

propertybest_score:float: The best score obtained by early stopping.

propertycoef_:ndarray

Coefficients property

Note

Coefficients are defined only for linear learners

Coefficients are only defined when the linear model is chosen asbase learner (booster=gblinear). It is not defined for other baselearner types, such as tree learners (booster=gbtree).

Returns:: coef_
Return type:: array of shape[n_features] or[n_classes,n_features]

evals_result()

Return the evaluation results.

The returned evaluation result is a dictionary:

{'validation_0':{'logloss':['0.604835','0.531479']},'validation_1':{'logloss':['0.41965','0.17686']}}

Return type:: evals_result

propertyfeature_importances_:ndarray

Returns:

feature_importances_ (array of shape[n_features] except for multi-class)
linear model, which returns an array with shape(n_features, n_classes)

propertyfeature_names_in_:ndarray: Names of features seen duringfit(). Defined only whenX hasfeature names that are all strings.

fit(X,y,*,sample_weight=None,base_margin=None,eval_set=None,verbose=True,xgb_model=None,sample_weight_eval_set=None,base_margin_eval_set=None,feature_weights=None)

Fit gradient boosting model.

Note that callingfit() multiple times will cause the model object to bere-fit from scratch. To resume training from a previous checkpoint, explicitlypassxgb_model argument.

Parameters:

X (Any) –
Input feature matrix. SeeSupported data structures for various XGBoost functions for a list of supported types.
When thetree_method is set tohist, internally, theQuantileDMatrix will be used instead of theDMatrixfor conserving memory. However, this has performance implications when thedevice of input data is not matched with algorithm. For instance, if theinput is a numpy array on CPU butcuda is used for training, then thedata is first processed on CPU then transferred to GPU.
y (Any) – Labels
sample_weight (Any |None) – instance weights
base_margin (Any |None) – Global bias for each instance. SeeIntercept for details.
eval_set (Sequence[Tuple[Any,Any]]|None) – A list of (X, y) tuple pairs to use as validation sets, for whichmetrics will be computed.Validation metrics will help us track the performance of the model.
verbose (bool |int |None) – Ifverbose is True and an evaluation set is used, the evaluation metricmeasured on the validation set is printed to stdout at each boosting stage.Ifverbose is an integer, the evaluation metric is printed at eachverbose boosting stage. The last boosting stage / the boosting stage foundby usingearly_stopping_rounds is also printed.
xgb_model (Booster |str |XGBModel |None) – file name of stored XGBoost model or ‘Booster’ instance XGBoost model to beloaded before training (allows training continuation).
sample_weight_eval_set (Sequence[Any]|None) – A list of the form [L_1, L_2, …, L_n], where each L_i is an array likeobject storing instance weights for the i-th validation set.
base_margin_eval_set (Sequence[Any]|None) – A list of the form [M_1, M_2, …, M_n], where each M_i is an array likeobject storing base margin for the i-th validation set.
feature_weights (Any |None) –
Deprecated since version 3.0.0.
Usefeature_weights in__init__() orset_params()instead.

Return type:

get_booster()

Get the underlying xgboost Booster of this model.

This will raise an exception when fit was not called

Returns:: booster
Return type:: a xgboost booster of underlying model

get_metadata_routing()

Get metadata routing of this object.

Please checkUser Guide on how the routingmechanism works.

Returns:: routing – AMetadataRequest encapsulatingrouting information.
Return type:: MetadataRequest

get_num_boosting_rounds()

Gets the number of xgboost boosting rounds.

Return type:: int

get_params(deep=True)

Get parameters.

Parameters:: deep (bool)
Return type:: Dict[str,Any]

get_xgb_params()

Get xgboost specific parameters.

Return type:: Dict[str,Any]

propertyintercept_:ndarray

Intercept (bias) property

For tree-based model, the returned value is thebase_score.

Returns:: intercept_
Return type:: array of shape(1,) or[n_classes]

load_model(fname)

Load the model from a file or a bytearray.

model.save_model("model.json")model.load_model("model.json")# ormodel.save_model("model.ubj")model.load_model("model.ubj")# orbuf=model.save_raw()model.load_model(buf)

Parameters:: fname (PathLike |bytearray |str) – Input file name or memory buffer(see also save_raw)
Return type:: None

propertyn_features_in_:int: Number of features seen duringfit().

predict(X,*,output_margin=False,validate_features=True,base_margin=None,iteration_range=None)

Note

This function is only thread safe forgbtree anddart.

Parameters:

X (Any) – Data to predict with. SeeSupported data structures for various XGBoost functions for a list of supported types.
output_margin (bool) – Whether to output the raw untransformed margin value.
validate_features (bool) – When this is True, validate that the Booster’s and data’s feature_names areidentical. Otherwise, it is assumed that the feature_names are the same.
base_margin (Any |None) – Global bias for each instance. SeeIntercept for details.
iteration_range (Tuple[int |integer,int |integer]|None) –
Specifies which layer of trees are used in prediction. For example, if arandom forest is trained with 100 rounds. Specifyingiteration_range=(10,20), then only the forests built during [10, 20) (half open set) roundsare used in this prediction.
Added in version 1.4.0.

Return type:

prediction

save_model(fname)

Save the model to a file.

model.save_model("model.json")# ormodel.save_model("model.ubj")

Parameters:: fname (str |PathLike) – Output file name
Return type:: None

score(X,y,sample_weight=None)

Returncoefficient of determination on test data.

Parameters:

X (array-like ofshape (n_samples,n_features)) – Test samples. For some estimators this may be a precomputedkernel matrix or a list of generic objects instead with shape(n_samples,n_samples_fitted), wheren_samples_fittedis the number of samples used in the fitting for the estimator.
y (array-like ofshape (n_samples,) or(n_samples,n_outputs)) – True values forX.
sample_weight (array-like ofshape (n_samples,),default=None) – Sample weights.

Returns:

score –$R^2$ ofself.predict(X) w.r.t.y.

Return type:

Notes

Configure whether metadata should be requested to be passed to thefit method.

Parameters:

self (XGBRFRegressor)
base_margin (bool |None |str)
base_margin_eval_set (bool |None |str)
eval_set (bool |None |str)
feature_weights (bool |None |str)
sample_weight (bool |None |str)
sample_weight_eval_set (bool |None |str)
verbose (bool |None |str)
xgb_model (bool |None |str)

Return type:

set_params(**params)

Return type:: self
Parameters:: params (Any)

set_predict_request(*,base_margin='$UNCHANGED$',iteration_range='$UNCHANGED$',output_margin='$UNCHANGED$',validate_features='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thepredict method.

Parameters:

self (XGBRFRegressor)
base_margin (bool |None |str)
iteration_range (bool |None |str)
output_margin (bool |None |str)
validate_features (bool |None |str)

Return type:

set_score_request(*,sample_weight='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thescore method.

Parameters:

self (XGBRFRegressor)
sample_weight (bool |None |str)

Return type:

classxgboost.XGBRFClassifier(*,learning_rate=1.0,subsample=0.8,colsample_bynode=0.8,reg_lambda=1e-05,**kwargs)

Bases:XGBClassifier

scikit-learn API for XGBoost random forest classification.SeeUsing the Scikit-Learn Estimator Interface for more information.

Parameters:

n_estimators (Optional[int]) – Number of trees in random forest to fit.
max_depth (Optional[int]) – Maximum tree depth for base learners.
max_leaves (Optional[int]) – Maximum number of leaves; 0 indicates no limit.
max_bin (Optional[int]) – If using histogram-based algorithm, maximum number of bins per feature
grow_policy (Optional[str]) –
Tree growing policy.
- depthwise: Favors splitting at nodes closest to the node,
- lossguide: Favors splitting at nodes with highest loss change.
learning_rate (Optional[float]) – Boosting learning rate (xgb’s “eta”)
verbosity (Optional[int]) – The degree of verbosity. Valid values are 0 (silent) - 3 (debug).
objective (Union[str,xgboost.sklearn._SklObjWProto,Callable[[Any,Any],Tuple[numpy.ndarray,numpy.ndarray]],NoneType]) –
Specify the learning task and the corresponding learning objective or a customobjective function to be used.
For custom objective, seeCustom Objective and Evaluation Metric andCustom objective and metric for more information, along with the end note forfunction signatures.
booster (Optional[str]) – Specify which booster to use:gbtree,gblinear ordart.
tree_method (Optional[str]) – Specify which tree method to use. Default to auto. If this parameter is set todefault, XGBoost will choose the most conservative option available. It’srecommended to study this option from the parameters documenttree method
n_jobs (Optional[int]) – Number of parallel threads used to run xgboost. When used with otherScikit-Learn algorithms like grid search, you may choose which algorithm toparallelize and balance the threads. Creating thread contention willsignificantly slow down both algorithms.
gamma (Optional[float]) – (min_split_loss) Minimum loss reduction required to make a further partition ona leaf node of the tree.
min_child_weight (Optional[float]) – Minimum sum of instance weight(hessian) needed in a child.
max_delta_step (Optional[float]) – Maximum delta step we allow each tree’s weight estimation to be.
subsample (Optional[float]) – Subsample ratio of the training instance.
sampling_method (Optional[str]) –
Sampling method. Used only by the GPU version ofhist tree method.
- uniform: Select random training instances uniformly.
- gradient_based: Select random training instances with higher probability
  when the gradient and hessian are larger. (cf. CatBoost)
colsample_bytree (Optional[float]) – Subsample ratio of columns when constructing each tree.
colsample_bylevel (Optional[float]) – Subsample ratio of columns for each level.
colsample_bynode (Optional[float]) – Subsample ratio of columns for each split.
reg_alpha (Optional[float]) – L1 regularization term on weights (xgb’s alpha).
reg_lambda (Optional[float]) – L2 regularization term on weights (xgb’s lambda).
scale_pos_weight (Optional[float]) – Balancing of positive and negative weights.
base_score (Optional[float]) – The initial prediction score of all instances, global bias.
random_state (Union[numpy.random.mtrand.RandomState,numpy.random._generator.Generator,int,NoneType]) –
Random number seed.
Note
Using gblinear booster with shotgun updater is nondeterministic asit uses Hogwild algorithm.
missing (float) – Value in the data which needs to be present as a missing value. Default tonumpy.nan.
num_parallel_tree (Optional[int]) – Used for boosting random forest.
monotone_constraints (Union[Dict[str,int],str,NoneType]) – Constraint of variable monotonicity. Seetutorialfor more information.
interaction_constraints (Union[str,List[Tuple[str]],NoneType]) – Constraints for interaction representing permitted interactions. Theconstraints must be specified in the form of a nested list, e.g.[[0,1],[2,3,4]], where each inner list is a group of indices of features that areallowed to interact with each other. Seetutorial for more information
importance_type (Optional[str]) –
The feature importance type for the feature_importances_ property:
- For tree model, it’s either “gain”, “weight”, “cover”, “total_gain” or“total_cover”.
- For linear model, only “weight” is defined and it’s the normalizedcoefficients without bias.
device (Optional[str]) –
Added in version 2.0.0.
Device ordinal, available options arecpu,cuda, andgpu.
validate_parameters (Optional[bool]) – Give warnings for unknown parameter.
enable_categorical (bool) – See the same parameter ofDMatrix for details.
feature_types (Optional[Sequence[str]]) –
Added in version 1.7.0.
Used for specifying feature types without constructing a dataframe. SeeDMatrix for details.
feature_weights (Optional[ArrayLike]) – Weight for each feature, defines the probability of each feature being selectedwhen colsample is being used. All values must be greater than 0, otherwise aValueError is thrown.
max_cat_to_onehot (Optional[int]) –
Added in version 1.6.0.
Note
This parameter is experimental
A threshold for deciding whether XGBoost should use one-hot encoding based splitfor categorical data. When number of categories is lesser than the thresholdthen one-hot encoding is chosen, otherwise the categories will be partitionedinto children nodes. Also,enable_categorical needs to be set to havecategorical feature support. SeeCategorical Data andParameters for Categorical Feature for details.
max_cat_threshold (Optional[int]) –
Added in version 1.7.0.
Note
This parameter is experimental
Maximum number of categories considered for each split. Used only bypartition-based splits for preventing over-fitting. Also,enable_categoricalneeds to be set to have categorical feature support. SeeCategorical Data andParameters for Categorical Feature for details.
multi_strategy (Optional[str]) –
Added in version 2.0.0.
Note
This parameter is working-in-progress.
The strategy used for training multi-target models, including multi-targetregression and multi-class classification. SeeMultiple Outputs formore information.
- one_output_per_tree: One model for each target.
- multi_output_tree: Use multi-target trees.
eval_metric (Union[str,List[Union[str,Callable]],Callable,NoneType]) –
Added in version 1.6.0.
Metric used for monitoring the training result and early stopping. It can be astring or list of strings as names of predefined metric in XGBoost (SeeXGBoost Parameters), one of the metrics insklearn.metrics, or anyother user defined metric that looks likesklearn.metrics.
If custom objective is also provided, then custom metric should implement thecorresponding reverse link function.
Unlike thescoring parameter commonly used in scikit-learn, when a callableobject is provided, it’s assumed to be a cost function and by default XGBoostwill minimize the result during early stopping.
For advanced usage on Early stopping like directly choosing to maximize insteadof minimize, seexgboost.callback.EarlyStopping.
SeeCustom Objective and Evaluation Metric andCustom objective and metric for moreinformation.
```
fromsklearn.datasetsimportload_diabetesfromsklearn.metricsimportmean_absolute_errorX,y=load_diabetes(return_X_y=True)reg=xgb.XGBRegressor(tree_method="hist",eval_metric=mean_absolute_error,)reg.fit(X,y,eval_set=[(X,y)])
```
early_stopping_rounds (Optional[int]) –
Added in version 1.6.0.
- Activates early stopping. Validation metric needs to improve at least once ineveryearly_stopping_rounds round(s) to continue training. Requires atleast one item ineval_set infit().
- If early stopping occurs, the model will have two additional attributes:best_score andbest_iteration. These are used by thepredict() andapply() methods to determine the optimalnumber of trees during inference. If users want to access the full model(including trees built after early stopping), they can specify theiteration_range in these inference methods. In addition, other utilitieslike model plotting can also use the entire model.
- If you prefer to discard the trees afterbest_iteration, consider using thecallback functionxgboost.callback.EarlyStopping.
- If there’s more than one item ineval_set, the last entry will be used forearly stopping. If there’s more than one metric ineval_metric, the lastmetric will be used for early stopping.
callbacks (Optional[List[xgboost.callback.TrainingCallback]]) –
List of callback functions that are applied at end of each iteration.It is possible to use predefined callbacks by usingCallback API.
Note
States in callback are not preserved during training, which means callbackobjects can not be reused for multiple training sessions withoutreinitialization or deepcopy.
```
forparamsinparameters_grid:# be sure to (re)initialize the callbacks before each runcallbacks=[xgb.callback.LearningRateScheduler(custom_rates)]reg=xgboost.XGBRegressor(**params,callbacks=callbacks)reg.fit(X,y)
```
kwargs (Optional[Any]) –
Keyword arguments for XGBoost Booster object. Full documentation of parameterscan be foundhere.Attempting to set a parameter via the constructor args and **kwargsdict simultaneously will result in a TypeError.
Note
**kwargs unsupported by scikit-learn
**kwargs is unsupported by scikit-learn. We do not guaranteethat parameters passed via this argument will interact properlywith scikit-learn.
Note
Custom objective function
A custom objective function can be provided for theobjectiveparameter. In this case, it should have the signatureobjective(y_true,y_pred)->[grad,hess] orobjective(y_true,y_pred,*,sample_weight)->[grad,hess]:
y_true: array_like of shape [n_samples]
The target values
y_pred: array_like of shape [n_samples]
The predicted values
sample_weight :
Optional sample weights.
grad: array_like of shape [n_samples]
The value of the gradient for each sample point.
hess: array_like of shape [n_samples]
The value of the second derivative for each sample point
Note that, if the custom objective produces negative values forthe Hessian, these will be clipped. If the objective is non-convex,one might also consider using the expected Hessian (Fisherinformation) instead.

apply(X,iteration_range=None)

Return the predicted leaf every tree for each sample. If the model is trainedwith early stopping, thenbest_iteration is used automatically.

Parameters:

X (Any) – Input features matrix. SeeSupported data structures for various XGBoost functions for a list of supported types.
iteration_range (Tuple[int |integer,int |integer]|None) – Seepredict().

Returns:

X_leaves – For each datapoint x in X and for each tree, return the index of theleaf x ends up in. Leaves are numbered within[0;2**(self.max_depth+1)), possibly with gaps in the numbering.

Return type:

array_like, shape=[n_samples, n_trees]

propertybest_iteration:int: The best iteration obtained by early stopping. This attribute is 0-based,for instance if the best iteration is the first round, then best_iteration is 0.

propertybest_score:float: The best score obtained by early stopping.

propertycoef_:ndarray

Coefficients property

Note

Coefficients are defined only for linear learners

Coefficients are only defined when the linear model is chosen asbase learner (booster=gblinear). It is not defined for other baselearner types, such as tree learners (booster=gbtree).

Returns:: coef_
Return type:: array of shape[n_features] or[n_classes,n_features]

evals_result()

Return the evaluation results.

The returned evaluation result is a dictionary:

{'validation_0':{'logloss':['0.604835','0.531479']},'validation_1':{'logloss':['0.41965','0.17686']}}

Return type:: evals_result

propertyfeature_importances_:ndarray

Returns:

feature_importances_ (array of shape[n_features] except for multi-class)
linear model, which returns an array with shape(n_features, n_classes)

propertyfeature_names_in_:ndarray: Names of features seen duringfit(). Defined only whenX hasfeature names that are all strings.

fit(X,y,*,sample_weight=None,base_margin=None,eval_set=None,verbose=True,xgb_model=None,sample_weight_eval_set=None,base_margin_eval_set=None,feature_weights=None)

Fit gradient boosting classifier.

Note that callingfit() multiple times will cause the model object to bere-fit from scratch. To resume training from a previous checkpoint, explicitlypassxgb_model argument.

Parameters:

X (Any) –
Input feature matrix. SeeSupported data structures for various XGBoost functions for a list of supported types.
When thetree_method is set tohist, internally, theQuantileDMatrix will be used instead of theDMatrixfor conserving memory. However, this has performance implications when thedevice of input data is not matched with algorithm. For instance, if theinput is a numpy array on CPU butcuda is used for training, then thedata is first processed on CPU then transferred to GPU.
y (Any) – Labels
sample_weight (Any |None) – instance weights
base_margin (Any |None) – Global bias for each instance. SeeIntercept for details.
eval_set (Sequence[Tuple[Any,Any]]|None) – A list of (X, y) tuple pairs to use as validation sets, for whichmetrics will be computed.Validation metrics will help us track the performance of the model.
verbose (bool |int |None) – Ifverbose is True and an evaluation set is used, the evaluation metricmeasured on the validation set is printed to stdout at each boosting stage.Ifverbose is an integer, the evaluation metric is printed at eachverbose boosting stage. The last boosting stage / the boosting stage foundby usingearly_stopping_rounds is also printed.
xgb_model (Booster |str |XGBModel |None) – file name of stored XGBoost model or ‘Booster’ instance XGBoost model to beloaded before training (allows training continuation).
sample_weight_eval_set (Sequence[Any]|None) – A list of the form [L_1, L_2, …, L_n], where each L_i is an array likeobject storing instance weights for the i-th validation set.
base_margin_eval_set (Sequence[Any]|None) – A list of the form [M_1, M_2, …, M_n], where each M_i is an array likeobject storing base margin for the i-th validation set.
feature_weights (Any |None) –
Deprecated since version 3.0.0.
Usefeature_weights in__init__() orset_params()instead.

Return type:

get_booster()

Get the underlying xgboost Booster of this model.

This will raise an exception when fit was not called

Returns:: booster
Return type:: a xgboost booster of underlying model

get_metadata_routing()

Get metadata routing of this object.

Please checkUser Guide on how the routingmechanism works.

Returns:: routing – AMetadataRequest encapsulatingrouting information.
Return type:: MetadataRequest

get_num_boosting_rounds()

Gets the number of xgboost boosting rounds.

Return type:: int

get_params(deep=True)

Get parameters.

Parameters:: deep (bool)
Return type:: Dict[str,Any]

get_xgb_params()

Get xgboost specific parameters.

Return type:: Dict[str,Any]

propertyintercept_:ndarray

Intercept (bias) property

For tree-based model, the returned value is thebase_score.

Returns:: intercept_
Return type:: array of shape(1,) or[n_classes]

load_model(fname)

Load the model from a file or a bytearray.

model.save_model("model.json")model.load_model("model.json")# ormodel.save_model("model.ubj")model.load_model("model.ubj")# orbuf=model.save_raw()model.load_model(buf)

Parameters:: fname (PathLike |bytearray |str) – Input file name or memory buffer(see also save_raw)
Return type:: None

propertyn_features_in_:int: Number of features seen duringfit().

predict(X,*,output_margin=False,validate_features=True,base_margin=None,iteration_range=None)

Note

This function is only thread safe forgbtree anddart.

Parameters:

X (Any) – Data to predict with. SeeSupported data structures for various XGBoost functions for a list of supported types.
output_margin (bool) – Whether to output the raw untransformed margin value.
validate_features (bool) – When this is True, validate that the Booster’s and data’s feature_names areidentical. Otherwise, it is assumed that the feature_names are the same.
base_margin (Any |None) – Global bias for each instance. SeeIntercept for details.
iteration_range (Tuple[int |integer,int |integer]|None) –
Specifies which layer of trees are used in prediction. For example, if arandom forest is trained with 100 rounds. Specifyingiteration_range=(10,20), then only the forests built during [10, 20) (half open set) roundsare used in this prediction.
Added in version 1.4.0.

Return type:

prediction

predict_proba(X,validate_features=True,base_margin=None,iteration_range=None)

Note

This function is only thread safe forgbtree anddart.

Parameters:

X (Any) – Feature matrix. SeeSupported data structures for various XGBoost functions for a list of supported types.
validate_features (bool) – When this is True, validate that the Booster’s and data’s feature_names areidentical. Otherwise, it is assumed that the feature_names are the same.
base_margin (Any |None) – Global bias for each instance. SeeIntercept for details.
iteration_range (Tuple[int |integer,int |integer]|None) – Specifies which layer of trees are used in prediction. For example, if arandom forest is trained with 100 rounds. Specifyingiteration_range=(10,20), then only the forests built during [10, 20) (half open set) rounds areused in this prediction.

Returns:

a numpy array of shape array-like of shape (n_samples, n_classes) with theprobability of each data example being of a given class.

Return type:

prediction

save_model(fname)

Save the model to a file.

model.save_model("model.json")# ormodel.save_model("model.ubj")

Parameters:: fname (str |PathLike) – Output file name
Return type:: None

score(X,y,sample_weight=None)

Returnaccuracy on provided data and labels.

In multi-label classification, this is the subset accuracywhich is a harsh metric since you require for each sample thateach label set be correctly predicted.

Parameters:

X (array-like ofshape (n_samples,n_features)) – Test samples.
y (array-like ofshape (n_samples,) or(n_samples,n_outputs)) – True labels forX.
sample_weight (array-like ofshape (n_samples,),default=None) – Sample weights.

Returns:

score – Mean accuracy ofself.predict(X) w.r.t.y.

Return type:

Configure whether metadata should be requested to be passed to thefit method.

Parameters:

self (XGBRFClassifier)
base_margin (bool |None |str)
base_margin_eval_set (bool |None |str)
eval_set (bool |None |str)
feature_weights (bool |None |str)
sample_weight (bool |None |str)
sample_weight_eval_set (bool |None |str)
verbose (bool |None |str)
xgb_model (bool |None |str)

Return type:

set_params(**params)

Return type:: self
Parameters:: params (Any)

set_predict_proba_request(*,base_margin='$UNCHANGED$',iteration_range='$UNCHANGED$',validate_features='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thepredict_proba method.

Parameters:

self (XGBRFClassifier)
base_margin (bool |None |str)
iteration_range (bool |None |str)
validate_features (bool |None |str)

Return type:

set_predict_request(*,base_margin='$UNCHANGED$',iteration_range='$UNCHANGED$',output_margin='$UNCHANGED$',validate_features='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thepredict method.

Parameters:

self (XGBRFClassifier)
base_margin (bool |None |str)
iteration_range (bool |None |str)
output_margin (bool |None |str)
validate_features (bool |None |str)

Return type:

set_score_request(*,sample_weight='$UNCHANGED$')

Configure whether metadata should be requested to be passed to thescore method.

Parameters:

self (XGBRFClassifier)
sample_weight (bool |None |str)

Return type:

Plotting API

Plotting Library.

xgboost.plot_importance(booster,*,ax=None,height=0.2,xlim=None,ylim=None,title='Featureimportance',xlabel='Importancescore',ylabel='Features',fmap='',importance_type='weight',max_num_features=None,grid=True,show_values=True,values_format='{v}',**kwargs)

Plot importance based on fitted trees.

Parameters:

booster (XGBModel |Booster |dict) – Booster or XGBModel instance, or dict taken by Booster.get_fscore()
ax (matplotlib Axes) – Target axes instance. If None, new figure and axes will be created.
grid (bool) – Turn the axes grids on or off. Default is True (On).
importance_type (str) –
How the importance is calculated: either “weight”, “gain”, or “cover”
- ”weight” is the number of times a feature appears in a tree
- ”gain” is the average gain of splits which use the feature
- ”cover” is the average coverage of splits which use the featurewhere coverage is defined as the number of samples affected by the split
max_num_features (int |None) – Maximum number of top features displayed on plot. If None, all features will bedisplayed.
height (float) – Bar height, passed to ax.barh()
xlim (tuple |None) – Tuple passed to axes.xlim()
ylim (tuple |None) – Tuple passed to axes.ylim()
title (str) – Axes title. To disable, pass None.
xlabel (str) – X axis title label. To disable, pass None.
ylabel (str) – Y axis title label. To disable, pass None.
fmap (str |PathLike) – The name of feature map file.
show_values (bool) – Show values on plot. To disable, pass False.
values_format (str) – Format string for values. “v” will be replaced by the value of the featureimportance. e.g. Pass “{v:.2f}” in order to limit the number of digits afterthe decimal point to two, for each value printed on the graph.
kwargs (Any) – Other keywords passed to ax.barh()

Returns:

Return type:

matplotlib Axes

xgboost.plot_tree(booster,*,fmap='',num_trees=None,rankdir=None,ax=None,with_stats=False,tree_idx=0,**kwargs)

Plot specified tree.

Parameters:

booster (Booster |XGBModel) – Booster or XGBModel instance
fmap (str (optional)) – The name of feature map file
num_trees (int |None) –
Deprecated since version 3.0.
rankdir (str,default "TB") – Passed to graphviz via graph_attr
ax (matplotlib Axes,default None) – Target axes instance. If None, new figure and axes will be created.
with_stats (bool) –
Added in version 3.0.
Seeto_graphviz().
tree_idx (int) –
Added in version 3.0.
Seeto_graphviz().
kwargs (Any) – Other keywords passed toto_graphviz()

Returns:

Return type:

matplotlib Axes

xgboost.to_graphviz(booster,*,fmap='',num_trees=None,rankdir=None,yes_color=None,no_color=None,condition_node_params=None,leaf_node_params=None,with_stats=False,tree_idx=0,**kwargs)

Convert specified tree to graphviz instance. IPython can automatically plotthe returned graphviz instance. Otherwise, you should call .render() methodof the returned graphviz instance.

Parameters:

booster (Booster |XGBModel) – Booster or XGBModel instance
fmap (str |PathLike) – The name of feature map file
num_trees (int |None) –
Deprecated since version 3.0.
Specify the ordinal number of target tree
rankdir (str |None) – Passed to graphviz via graph_attr
yes_color (str |None) – Edge color when meets the node condition.
no_color (str |None) – Edge color when doesn’t meet the node condition.
condition_node_params (dict |None) –
Condition node configuration for for graphviz. Example:
```
{'shape':'box','style':'filled,rounded','fillcolor':'#78bceb'}
```
leaf_node_params (dict |None) –
Leaf node configuration for graphviz. Example:
```
{'shape':'box','style':'filled','fillcolor':'#e48038'}
```
with_stats (bool) –
Added in version 3.0.
Controls whether the split statistics should be included.
tree_idx (int) –
Added in version 3.0.
Specify the ordinal index of target tree.
kwargs (Any) – Other keywords passed to graphviz graph_attr, e.g.graph[{key}={value}]

Returns:

graph

Return type:

graphviz.Source

Callback API

Callback library containing training routines. SeeCallback Functions for a quick introduction.

classxgboost.callback.TrainingCallback

Interface for training callback.

Added in version 1.3.0.

after_iteration(model,epoch,evals_log)

Run after each iteration. ReturnsTrue when training should stop.

Parameters:

model (Any) – Eeither aBooster object or a CVPack if the cv functionin xgboost is being used.
epoch (int) – The current training iteration.
evals_log (Dict[str,Dict[str,List[float]|List[Tuple[float,float]]]]) –
A dictionary containing the evaluation history:
```
{"data_name":{"metric_name":[0.5,...]}}
```

Return type:

after_training(model)

Run after training is finished.

Parameters:: model (Any)
Return type:: Any

before_iteration(model,epoch,evals_log)

Run before each iteration. Returns True when training should stop. Seeafter_iteration() for details.

Parameters:

model (Any)
epoch (int)
evals_log (Dict[str,Dict[str,List[float]|List[Tuple[float,float]]]])

Return type:

before_training(model)

Run before training starts.

Parameters:: model (Any)
Return type:: Any

classxgboost.callback.EvaluationMonitor(rank=0,period=1,show_stdv=False,logger=<functioncommunicator_print>)

Print the evaluation result at each iteration.

Added in version 1.3.0.

Parameters:

rank (int) – Which worker should be used for printing the result.
period (int) – How many epoches between printing.
show_stdv (bool) – Used in cv to show standard deviation. Users should not specify it.
logger (Callable[[str],None]) – A callable used for logging evaluation result.

after_iteration(model,epoch,evals_log)

Run after each iteration. ReturnsTrue when training should stop.

Parameters:

model (Any) – Eeither aBooster object or a CVPack if the cv functionin xgboost is being used.
epoch (int) – The current training iteration.
evals_log (Dict[str,Dict[str,List[float]|List[Tuple[float,float]]]]) –
A dictionary containing the evaluation history:
```
{"data_name":{"metric_name":[0.5,...]}}
```

Return type:

after_training(model)

Run after training is finished.

Parameters:: model (Any)
Return type:: Any

classxgboost.callback.EarlyStopping(*,rounds,metric_name=None,data_name=None,maximize=None,save_best=False,min_delta=0.0)

Callback function for early stopping

Added in version 1.3.0.

Parameters:

rounds (int) – Early stopping rounds.
metric_name (str |None) – Name of metric that is used for early stopping.
data_name (str |None) – Name of dataset that is used for early stopping.
maximize (bool |None) – Whether to maximize evaluation metric. None means auto (discouraged).
save_best (bool |None) – Whether training should return the best model or the last model. If set toTrue, it will only keep the boosting rounds up to the detected best iteration,discarding the ones that come after. This is only supported with tree methods(notgblinear). Also, thecv function doesn’t return a model, the parameteris not applicable.
min_delta (float) –
Added in version 1.5.0.
Minimum absolute change in score to be qualified as an improvement.

Examples

es=xgboost.callback.EarlyStopping(rounds=2,min_delta=1e-3,save_best=True,maximize=False,data_name="validation_0",metric_name="mlogloss",)clf=xgboost.XGBClassifier(tree_method="hist",device="cuda",callbacks=[es])X,y=load_digits(return_X_y=True)clf.fit(X,y,eval_set=[(X,y)])

after_iteration(model,epoch,evals_log)

Run after each iteration. ReturnsTrue when training should stop.

Parameters:

model (Any) – Eeither aBooster object or a CVPack if the cv functionin xgboost is being used.
epoch (int) – The current training iteration.
evals_log (Dict[str,Dict[str,List[float]|List[Tuple[float,float]]]]) –
A dictionary containing the evaluation history:
```
{"data_name":{"metric_name":[0.5,...]}}
```

Return type:

after_training(model)

Run after training is finished.

Parameters:: model (Any)
Return type:: Any

before_training(model)

Run before training starts.

Parameters:: model (Any)
Return type:: Any

classxgboost.callback.LearningRateScheduler(learning_rates)

Callback function for scheduling learning rate.

Added in version 1.3.0.

Parameters:: learning_rates (Callable[[int],float]|Sequence[float]) – If it’s a callable object, then it should accept an integer parameterepoch and returns the corresponding learning rate. Otherwise itshould be a sequence like list or tuple with the same size of boostingrounds.

after_iteration(model,epoch,evals_log)

Run after each iteration. ReturnsTrue when training should stop.

Parameters:

model (Any) – Eeither aBooster object or a CVPack if the cv functionin xgboost is being used.
epoch (int) – The current training iteration.
evals_log (Dict[str,Dict[str,List[float]|List[Tuple[float,float]]]]) –
A dictionary containing the evaluation history:
```
{"data_name":{"metric_name":[0.5,...]}}
```

Return type:

classxgboost.callback.TrainingCheckPoint(directory,name='model',as_pickle=False,interval=100)

Checkpointing operation. Users are encouraged to create their own callbacks forcheckpoint as XGBoost doesn’t handle distributed file systems. When checkpointing ondistributed systems, be sure to know the rank of the worker to avoid multipleworkers checkpointing to the same place.

Added in version 1.3.0.

Since XGBoost 2.1.0, the default format is changed to UBJSON.

Parameters:

directory (str |PathLike) – Output model directory.
name (str) – pattern of output model file. Models will be saved as name_0.ubj, name_1.ubj,name_2.ubj ….
as_pickle (bool) – When set to True, all training parameters will be saved in pickle format,instead of saving only the model.
interval (int) – Interval of checkpointing. Checkpointing is slow so setting a larger number canreduce performance hit.

after_iteration(model,epoch,evals_log)

Run after each iteration. ReturnsTrue when training should stop.

Parameters:

model (Any) – Eeither aBooster object or a CVPack if the cv functionin xgboost is being used.
epoch (int) – The current training iteration.
evals_log (Dict[str,Dict[str,List[float]|List[Tuple[float,float]]]]) –
A dictionary containing the evaluation history:
```
{"data_name":{"metric_name":[0.5,...]}}
```

Return type: