Multiple Outputs

Added in version 1.6.

Starting from version 1.6, XGBoost has experimental support for multi-output regressionand multi-label classification with Python package. Multi-label classification usuallyrefers to targets that have multiple non-exclusive class labels. For instance, a moviecan be simultaneously classified as both sci-fi and comedy. For detailed explanation ofterminologies related to different multi-output models please refer to thescikit-learn user guide.

Note

As of XGBoost 3.0, the feature is experimental and has limited features. Only thePython package is tested. In addition,glinear is not supported.

Training with One-Model-Per-Target

By default, XGBoost builds one model for each target similar to sklearn meta estimators,with the added benefit of reusing data and other integrated features like SHAP. For aworked example of regression, seeA demo for multi-output regression. For multi-label classification,the binary relevance strategy is used. Inputy should be of shape(n_samples,n_classes) with each column having a value of 0 or 1 to specify whether the sample islabeled as positive for respective class. Given a sample with 3 output classes and 2labels, the correspondingy should be encoded as[1,0,1] with the second classlabeled as negative and the rest labeled as positive. At the moment XGBoost supports onlydense matrix for labels.

fromsklearn.datasetsimportmake_multilabel_classificationimportnumpyasnpX,y=make_multilabel_classification(n_samples=32,n_classes=5,n_labels=3,random_state=0)clf=xgb.XGBClassifier(tree_method="hist")clf.fit(X,y)np.testing.assert_allclose(clf.predict(X),y)

The feature is still under development with limited support from objectives and metrics.

Training with Vector Leaf

Added in version 2.0.

Note

This is still working-in-progress, and most features are missing.

XGBoost can optionally build multi-output trees with the size of leaf equals to the numberof targets when the tree methodhist is used. The behavior can be controlled by themulti_strategy training parameter, which can take the valueone_output_per_tree (thedefault) for building one model per-target ormulti_output_tree for buildingmulti-output trees.

clf=xgb.XGBClassifier(tree_method="hist",multi_strategy="multi_output_tree")

SeeA demo for multi-output regression for a worked example withregression.