ExtraTreesRegressor #

classsklearn.ensemble.ExtraTreesRegressor(n_estimators=100,*,criterion='squared_error',max_depth=None,min_samples_split=2,min_samples_leaf=1,min_weight_fraction_leaf=0.0,max_features=1.0,max_leaf_nodes=None,min_impurity_decrease=0.0,bootstrap=False,oob_score=False,n_jobs=None,random_state=None,verbose=0,warm_start=False,ccp_alpha=0.0,max_samples=None,monotonic_cst=None)[source]#

An extra-trees regressor.

This class implements a meta estimator that fits a number ofrandomized decision trees (a.k.a. extra-trees) on various sub-samplesof the dataset and uses averaging to improve the predictive accuracyand control over-fitting.

This estimator has native support for missing values (NaNs) forrandom splits. During training, a random threshold will be chosento split the non-missing values on. Then the non-missing values will be sentto the left and right child based on the randomly selected threshold, whilethe missing values will also be randomly sent to the left or right child.This is repeated for every feature considered at each split. The best splitamong these is chosen.

Read more in theUser Guide.

Parameters:

n_estimatorsint, default=100

The number of trees in the forest.

Changed in version 0.22:The default value ofn_estimators changed from 10 to 100in 0.22.

criterion{“squared_error”, “absolute_error”, “friedman_mse”, “poisson”}, default=”squared_error”

The function to measure the quality of a split. Supported criteriaare “squared_error” for the mean squared error, which is equal tovariance reduction as feature selection criterion and minimizes the L2loss using the mean of each terminal node, “friedman_mse”, which usesmean squared error with Friedman’s improvement score for potentialsplits, “absolute_error” for the mean absolute error, which minimizesthe L1 loss using the median of each terminal node, and “poisson” whichuses reduction in Poisson deviance to find splits.Training using “absolute_error” is significantly slowerthan when using “squared_error”.

Added in version 0.18:Mean Absolute Error (MAE) criterion.

max_depthint, default=None

The maximum depth of the tree. If None, then nodes are expanded untilall leaves are pure or until all leaves contain less thanmin_samples_split samples.

min_samples_splitint or float, default=2

The minimum number of samples required to split an internal node:

If int, then considermin_samples_split as the minimum number.
If float, thenmin_samples_split is a fraction andceil(min_samples_split*n_samples) are the minimumnumber of samples for each split.

Changed in version 0.18:Added float values for fractions.

min_samples_leafint or float, default=1

The minimum number of samples required to be at a leaf node.A split point at any depth will only be considered if it leaves atleastmin_samples_leaf training samples in each of the left andright branches. This may have the effect of smoothing the model,especially in regression.

If int, then considermin_samples_leaf as the minimum number.
If float, thenmin_samples_leaf is a fraction andceil(min_samples_leaf*n_samples) are the minimumnumber of samples for each node.

Changed in version 0.18:Added float values for fractions.

min_weight_fraction_leaffloat, default=0.0

The minimum weighted fraction of the sum total of weights (of allthe input samples) required to be at a leaf node. Samples haveequal weight when sample_weight is not provided.

max_features{“sqrt”, “log2”, None}, int or float, default=1.0

The number of features to consider when looking for the best split:

If int, then considermax_features features at each split.
If float, thenmax_features is a fraction andmax(1,int(max_features*n_features_in_)) features are considered at eachsplit.
If “sqrt”, thenmax_features=sqrt(n_features).
If “log2”, thenmax_features=log2(n_features).
If None or 1.0, thenmax_features=n_features.

Note

The default of 1.0 is equivalent to bagged trees and morerandomness can be achieved by setting smaller values, e.g. 0.3.

Changed in version 1.1:The default ofmax_features changed from"auto" to 1.0.

Note: the search for a split does not stop until at least onevalid partition of the node samples is found, even if it requires toeffectively inspect more thanmax_features features.

max_leaf_nodesint, default=None

Grow trees withmax_leaf_nodes in best-first fashion.Best nodes are defined as relative reduction in impurity.If None then unlimited number of leaf nodes.

min_impurity_decreasefloat, default=0.0

A node will be split if this split induces a decrease of the impuritygreater than or equal to this value.

The weighted impurity decrease equation is the following:

N_t/N*(impurity-N_t_R/N_t*right_impurity-N_t_L/N_t*left_impurity)

whereN is the total number of samples,N_t is the number ofsamples at the current node,N_t_L is the number of samples in theleft child, andN_t_R is the number of samples in the right child.

N,N_t,N_t_R andN_t_L all refer to the weighted sum,ifsample_weight is passed.

Added in version 0.19.

bootstrapbool, default=False

Whether bootstrap samples are used when building trees. If False, thewhole dataset is used to build each tree.

oob_scorebool or callable, default=False

Whether to use out-of-bag samples to estimate the generalization score.By default,r2_score is used.Provide a callable with signaturemetric(y_true,y_pred) to use acustom metric. Only available ifbootstrap=True.

For an illustration of out-of-bag (OOB) error estimation, see the exampleOOB Errors for Random Forests.

n_jobsint, default=None

The number of jobs to run in parallel.fit,predict,decision_path andapply are all parallelized over thetrees.None means 1 unless in ajoblib.parallel_backendcontext.-1 means using all processors. SeeGlossary for more details.

random_stateint, RandomState instance or None, default=None

Controls 3 sources of randomness:

the bootstrapping of the samples used when building trees(ifbootstrap=True)
the sampling of the features to consider when looking for the bestsplit at each node (ifmax_features<n_features)
the draw of the splits for each of themax_features

SeeGlossary for details.

verboseint, default=0

Controls the verbosity when fitting and predicting.

warm_startbool, default=False

When set toTrue, reuse the solution of the previous call to fitand add more estimators to the ensemble, otherwise, just fit a wholenew forest. SeeGlossary andFitting additional trees for details.

ccp_alphanon-negative float, default=0.0

Complexity parameter used for Minimal Cost-Complexity Pruning. Thesubtree with the largest cost complexity that is smaller thanccp_alpha will be chosen. By default, no pruning is performed. SeeMinimal Cost-Complexity Pruning for details. SeePost pruning decision trees with cost complexity pruningfor an example of such pruning.

Added in version 0.22.

max_samplesint or float, default=None

If bootstrap is True, the number of samples to draw from Xto train each base estimator.

If None (default), then drawX.shape[0] samples.
If int, then drawmax_samples samples.
If float, then drawmax_samples*X.shape[0] samples. Thus,max_samples should be in the interval(0.0,1.0].

Added in version 0.22.

monotonic_cstarray-like of int of shape (n_features), default=None

Indicates the monotonicity constraint to enforce on each feature.

1: monotonically increasing
0: no constraint
-1: monotonically decreasing

If monotonic_cst is None, no constraints are applied.

Monotonicity constraints are not supported for:

multioutput regressions (i.e. whenn_outputs_>1),
regressions trained on data with missing values.

See also

ExtraTreesClassifier: An extra-trees classifier with random splits.
RandomForestClassifier: A random forest classifier with optimal splits.
RandomForestRegressor: Ensemble regressor using trees with optimal splits.

Notes

The default values for the parameters controlling the size of the trees(e.g.max_depth,min_samples_leaf, etc.) lead to fully grown andunpruned trees which can potentially be very large on some data sets. Toreduce memory consumption, the complexity and size of the trees should becontrolled by setting those parameter values.

References

[1]

P. Geurts, D. Ernst., and L. Wehenkel, “Extremely randomized trees”,Machine Learning, 63(1), 3-42, 2006.

Examples

>>>fromsklearn.datasetsimportload_diabetes>>>fromsklearn.model_selectionimporttrain_test_split>>>fromsklearn.ensembleimportExtraTreesRegressor>>>X,y=load_diabetes(return_X_y=True)>>>X_train,X_test,y_train,y_test=train_test_split(...X,y,random_state=0)>>>reg=ExtraTreesRegressor(n_estimators=100,random_state=0).fit(...X_train,y_train)>>>reg.score(X_test,y_test)0.2727...

apply(X)[source]#

Apply trees in the forest to X, return leaf indices.

Parameters:

X{array-like, sparse matrix} of shape (n_samples, n_features): The input samples. Internally, its dtype will be converted todtype=np.float32. If a sparse matrix is provided, it will beconverted into a sparsecsr_matrix.

Returns:

X_leavesndarray of shape (n_samples, n_estimators): For each datapoint x in X and for each tree in the forest,return the index of the leaf x ends up in.

decision_path(X)[source]#

Return the decision path in the forest.

Added in version 0.18.

Parameters:

X{array-like, sparse matrix} of shape (n_samples, n_features): The input samples. Internally, its dtype will be converted todtype=np.float32. If a sparse matrix is provided, it will beconverted into a sparsecsr_matrix.

Returns:

indicatorsparse matrix of shape (n_samples, n_nodes): Return a node indicator matrix where non zero elements indicatesthat the samples goes through the nodes. The matrix is of CSRformat.
n_nodes_ptrndarray of shape (n_estimators + 1,): The columns from indicator[n_nodes_ptr[i]:n_nodes_ptr[i+1]]gives the indicator value for the i-th estimator.

fit(X,y,sample_weight=None)[source]#

Build a forest of trees from the training set (X, y).

Parameters:

X{array-like, sparse matrix} of shape (n_samples, n_features): The training input samples. Internally, its dtype will be convertedtodtype=np.float32. If a sparse matrix is provided, it will beconverted into a sparsecsc_matrix.
yarray-like of shape (n_samples,) or (n_samples, n_outputs): The target values (class labels in classification, real numbers inregression).
sample_weightarray-like of shape (n_samples,), default=None: Sample weights. If None, then samples are equally weighted. Splitsthat would create child nodes with net zero or negative weight areignored while searching for a split in each node. In the case ofclassification, splits are also ignored if they would result in anysingle class carrying a negative weight in either child node.

Returns:

selfobject: Fitted estimator.

get_metadata_routing()[source]#

Get metadata routing of this object.

Please checkUser Guide on how the routingmechanism works.

Returns:

routingMetadataRequest: AMetadataRequest encapsulatingrouting information.

get_params(deep=True)[source]#

Get parameters for this estimator.

Parameters:

deepbool, default=True: If True, will return the parameters for this estimator andcontained subobjects that are estimators.

Returns:

paramsdict: Parameter names mapped to their values.

predict(X)[source]#

Predict regression target for X.

The predicted regression target of an input sample is computed as themean predicted regression targets of the trees in the forest.

Parameters:

X{array-like, sparse matrix} of shape (n_samples, n_features): The input samples. Internally, its dtype will be converted todtype=np.float32. If a sparse matrix is provided, it will beconverted into a sparsecsr_matrix.

Returns:

yndarray of shape (n_samples,) or (n_samples, n_outputs): The predicted values.

score(X,y,sample_weight=None)[source]#

Returncoefficient of determination on test data.

The coefficient of determination,$R^2$, is defined as$(1 - \frac{u}{v})$, where$u$ is the residualsum of squares((y_true-y_pred)**2).sum() and$v$is the total sum of squares((y_true-y_true.mean())**2).sum().The best possible score is 1.0 and it can be negative (because themodel can be arbitrarily worse). A constant model that always predictsthe expected value ofy, disregarding the input features, would geta$R^2$ score of 0.0.

Parameters:

Xarray-like of shape (n_samples, n_features): Test samples. For some estimators this may be a precomputedkernel matrix or a list of generic objects instead with shape(n_samples,n_samples_fitted), wheren_samples_fittedis the number of samples used in the fitting for the estimator.
yarray-like of shape (n_samples,) or (n_samples, n_outputs): True values forX.
sample_weightarray-like of shape (n_samples,), default=None: Sample weights.

Returns:

scorefloat: $R^2$ ofself.predict(X) w.r.t.y.

Notes

The$R^2$ score used when callingscore on a regressor usesmultioutput='uniform_average' from version 0.23 to keep consistentwith default value ofr2_score.This influences thescore method of all the multioutputregressors (except forMultiOutputRegressor).

set_fit_request(*,sample_weight:bool|None|str='$UNCHANGED$')→ExtraTreesRegressor[source]#

Configure whether metadata should be requested to be passed to thefit method.

Note that this method is only relevant when this estimator is used as asub-estimator within ameta-estimator and metadata routing is enabledwithenable_metadata_routing=True (seesklearn.set_config).Please check theUser Guide on how the routingmechanism works.

The options for each parameter are:

True: metadata is requested, and passed tofit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it tofit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains theexisting request. This allows you to change the request for someparameters and not others.

Added in version 1.3.

Parameters:

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing forsample_weight parameter infit.

Returns:

selfobject: The updated object.

set_params(**params)[source]#

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects(such asPipeline). The latter haveparameters of the form<component>__<parameter> so that it’spossible to update each component of a nested object.

Parameters:

**paramsdict: Estimator parameters.

Returns:

selfestimator instance: Estimator instance.

set_score_request(*,sample_weight:bool|None|str='$UNCHANGED$')→ExtraTreesRegressor[source]#

Configure whether metadata should be requested to be passed to thescore method.

The options for each parameter are:

True: metadata is requested, and passed toscore if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it toscore.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains theexisting request. This allows you to change the request for someparameters and not others.

Added in version 1.3.

Parameters:

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing forsample_weight parameter inscore.

Returns:

selfobject: The updated object.

Gallery examples#

Face completion with a multi-output estimators

On this page

This Page

Show Source

Movatterモバイル変換

ExtraTreesRegressor#

Gallery examples#

This Page

ExtraTreesRegressor #