RandomForestClassifier #

classsklearn.ensemble.RandomForestClassifier(n_estimators=100,*,criterion='gini',max_depth=None,min_samples_split=2,min_samples_leaf=1,min_weight_fraction_leaf=0.0,max_features='sqrt',max_leaf_nodes=None,min_impurity_decrease=0.0,bootstrap=True,oob_score=False,n_jobs=None,random_state=None,verbose=0,warm_start=False,class_weight=None,ccp_alpha=0.0,max_samples=None,monotonic_cst=None)[source]#

A random forest classifier.

A random forest is a meta estimator that fits a number of decision treeclassifiers on various sub-samples of the dataset and uses averaging toimprove the predictive accuracy and control over-fitting.Trees in the forest use the best split strategy, i.e. equivalent to passingsplitter="best" to the underlyingDecisionTreeClassifier.The sub-sample size is controlled with themax_samples parameter ifbootstrap=True (default), otherwise the whole dataset is used to buildeach tree.

For a comparison between tree-based ensemble models see the exampleComparing Random Forests and Histogram Gradient Boosting models.

This estimator has native support for missing values (NaNs). During training,the tree grower learns at each split point whether samples with missing valuesshould go to the left or right child, based on the potential gain. When predicting,samples with missing values are assigned to the left or right child consequently.If no missing values were encountered for a given feature during training, thensamples with missing values are mapped to whichever child has the most samples.

Read more in theUser Guide.

Parameters:

n_estimatorsint, default=100

The number of trees in the forest.

Changed in version 0.22:The default value ofn_estimators changed from 10 to 100in 0.22.

criterion{“gini”, “entropy”, “log_loss”}, default=”gini”

The function to measure the quality of a split. Supported criteria are“gini” for the Gini impurity and “log_loss” and “entropy” both for theShannon information gain, seeMathematical formulation.Note: This parameter is tree-specific.

max_depthint, default=None

The maximum depth of the tree. If None, then nodes are expanded untilall leaves are pure or until all leaves contain less thanmin_samples_split samples.

min_samples_splitint or float, default=2

The minimum number of samples required to split an internal node:

If int, then considermin_samples_split as the minimum number.
If float, thenmin_samples_split is a fraction andceil(min_samples_split*n_samples) are the minimumnumber of samples for each split.

Changed in version 0.18:Added float values for fractions.

min_samples_leafint or float, default=1

The minimum number of samples required to be at a leaf node.A split point at any depth will only be considered if it leaves atleastmin_samples_leaf training samples in each of the left andright branches. This may have the effect of smoothing the model,especially in regression.

If int, then considermin_samples_leaf as the minimum number.
If float, thenmin_samples_leaf is a fraction andceil(min_samples_leaf*n_samples) are the minimumnumber of samples for each node.

Changed in version 0.18:Added float values for fractions.

min_weight_fraction_leaffloat, default=0.0

The minimum weighted fraction of the sum total of weights (of allthe input samples) required to be at a leaf node. Samples haveequal weight when sample_weight is not provided.

max_features{“sqrt”, “log2”, None}, int or float, default=”sqrt”

The number of features to consider when looking for the best split:

If int, then considermax_features features at each split.
If float, thenmax_features is a fraction andmax(1,int(max_features*n_features_in_)) features are considered at eachsplit.
If “sqrt”, thenmax_features=sqrt(n_features).
If “log2”, thenmax_features=log2(n_features).
If None, thenmax_features=n_features.

Changed in version 1.1:The default ofmax_features changed from"auto" to"sqrt".

Note: the search for a split does not stop until at least onevalid partition of the node samples is found, even if it requires toeffectively inspect more thanmax_features features.

max_leaf_nodesint, default=None

Grow trees withmax_leaf_nodes in best-first fashion.Best nodes are defined as relative reduction in impurity.If None then unlimited number of leaf nodes.

min_impurity_decreasefloat, default=0.0

A node will be split if this split induces a decrease of the impuritygreater than or equal to this value.

The weighted impurity decrease equation is the following:

N_t/N*(impurity-N_t_R/N_t*right_impurity-N_t_L/N_t*left_impurity)

whereN is the total number of samples,N_t is the number ofsamples at the current node,N_t_L is the number of samples in theleft child, andN_t_R is the number of samples in the right child.

N,N_t,N_t_R andN_t_L all refer to the weighted sum,ifsample_weight is passed.

Added in version 0.19.

bootstrapbool, default=True

Whether bootstrap samples are used when building trees. If False, thewhole dataset is used to build each tree.

oob_scorebool or callable, default=False

Whether to use out-of-bag samples to estimate the generalization score.By default,accuracy_score is used.Provide a callable with signaturemetric(y_true,y_pred) to use acustom metric. Only available ifbootstrap=True.

For an illustration of out-of-bag (OOB) error estimation, see the exampleOOB Errors for Random Forests.

n_jobsint, default=None

The number of jobs to run in parallel.fit,predict,decision_path andapply are all parallelized over thetrees.None means 1 unless in ajoblib.parallel_backendcontext.-1 means using all processors. SeeGlossary for more details.

random_stateint, RandomState instance or None, default=None

Controls both the randomness of the bootstrapping of the samples usedwhen building trees (ifbootstrap=True) and the sampling of thefeatures to consider when looking for the best split at each node(ifmax_features<n_features).SeeGlossary for details.

verboseint, default=0

Controls the verbosity when fitting and predicting.

warm_startbool, default=False

When set toTrue, reuse the solution of the previous call to fitand add more estimators to the ensemble, otherwise, just fit a wholenew forest. SeeGlossary andFitting additional trees for details.

class_weight{“balanced”, “balanced_subsample”}, dict or list of dicts, default=None

Weights associated with classes in the form{class_label:weight}.If not given, all classes are supposed to have weight one. Formulti-output problems, a list of dicts can be provided in the sameorder as the columns of y.

Note that for multioutput (including multilabel) weights should bedefined for each class of every column in its own dict. For example,for four-class multilabel classification weights should be[{0: 1, 1: 1}, {0: 1, 1: 5}, {0: 1, 1: 1}, {0: 1, 1: 1}] instead of[{1:1}, {2:5}, {3:1}, {4:1}].

The “balanced” mode uses the values of y to automatically adjustweights inversely proportional to class frequencies in the input dataasn_samples/(n_classes*np.bincount(y))

The “balanced_subsample” mode is the same as “balanced” except thatweights are computed based on the bootstrap sample for every treegrown.

For multi-output, the weights of each column of y will be multiplied.

Note that these weights will be multiplied with sample_weight (passedthrough the fit method) if sample_weight is specified.

ccp_alphanon-negative float, default=0.0

Complexity parameter used for Minimal Cost-Complexity Pruning. Thesubtree with the largest cost complexity that is smaller thanccp_alpha will be chosen. By default, no pruning is performed. SeeMinimal Cost-Complexity Pruning for details. SeePost pruning decision trees with cost complexity pruningfor an example of such pruning.

Added in version 0.22.

max_samplesint or float, default=None

If bootstrap is True, the number of samples to draw from Xto train each base estimator.

If None (default), then drawX.shape[0] samples.
If int, then drawmax_samples samples.
If float, then drawmax(round(n_samples*max_samples),1) samples. Thus,max_samples should be in the interval(0.0,1.0].

Added in version 0.22.

monotonic_cstarray-like of int of shape (n_features), default=None

Indicates the monotonicity constraint to enforce on each feature.

1: monotonic increase
0: no constraint
-1: monotonic decrease

If monotonic_cst is None, no constraints are applied.

Monotonicity constraints are not supported for:

multiclass classifications (i.e. whenn_classes>2),
multioutput classifications (i.e. whenn_outputs_>1),
classifications trained on data with missing values.

The constraints hold over the probability of the positive class.

See also

sklearn.tree.DecisionTreeClassifier: A decision tree classifier.
sklearn.ensemble.ExtraTreesClassifier: Ensemble of extremely randomized tree classifiers.
sklearn.ensemble.HistGradientBoostingClassifier: A Histogram-based Gradient Boosting Classification Tree, very fast for big datasets (n_samples >= 10_000).

Notes

The default values for the parameters controlling the size of the trees(e.g.max_depth,min_samples_leaf, etc.) lead to fully grown andunpruned trees which can potentially be very large on some data sets. Toreduce memory consumption, the complexity and size of the trees should becontrolled by setting those parameter values.

The features are always randomly permuted at each split. Therefore,the best found split may vary, even with the same training data,max_features=n_features andbootstrap=False, if the improvementof the criterion is identical for several splits enumerated during thesearch of the best split. To obtain a deterministic behaviour duringfitting,random_state has to be fixed.

References

[1]

Breiman, “Random Forests”, Machine Learning, 45(1), 5-32, 2001.

Examples

>>>fromsklearn.ensembleimportRandomForestClassifier>>>fromsklearn.datasetsimportmake_classification>>>X,y=make_classification(n_samples=1000,n_features=4,...n_informative=2,n_redundant=0,...random_state=0,shuffle=False)>>>clf=RandomForestClassifier(max_depth=2,random_state=0)>>>clf.fit(X,y)RandomForestClassifier(...)>>>print(clf.predict([[0,0,0,0]]))[1]

apply(X)[source]#

Apply trees in the forest to X, return leaf indices.

Parameters:

X{array-like, sparse matrix} of shape (n_samples, n_features): The input samples. Internally, its dtype will be converted todtype=np.float32. If a sparse matrix is provided, it will beconverted into a sparsecsr_matrix.

Returns:

X_leavesndarray of shape (n_samples, n_estimators): For each datapoint x in X and for each tree in the forest,return the index of the leaf x ends up in.

decision_path(X)[source]#

Return the decision path in the forest.

Added in version 0.18.

Parameters:

X{array-like, sparse matrix} of shape (n_samples, n_features): The input samples. Internally, its dtype will be converted todtype=np.float32. If a sparse matrix is provided, it will beconverted into a sparsecsr_matrix.

Returns:

indicatorsparse matrix of shape (n_samples, n_nodes): Return a node indicator matrix where non zero elements indicatesthat the samples goes through the nodes. The matrix is of CSRformat.
n_nodes_ptrndarray of shape (n_estimators + 1,): The columns from indicator[n_nodes_ptr[i]:n_nodes_ptr[i+1]]gives the indicator value for the i-th estimator.

fit(X,y,sample_weight=None)[source]#

Build a forest of trees from the training set (X, y).

Parameters:

X{array-like, sparse matrix} of shape (n_samples, n_features): The training input samples. Internally, its dtype will be convertedtodtype=np.float32. If a sparse matrix is provided, it will beconverted into a sparsecsc_matrix.
yarray-like of shape (n_samples,) or (n_samples, n_outputs): The target values (class labels in classification, real numbers inregression).
sample_weightarray-like of shape (n_samples,), default=None: Sample weights. If None, then samples are equally weighted. Splitsthat would create child nodes with net zero or negative weight areignored while searching for a split in each node. In the case ofclassification, splits are also ignored if they would result in anysingle class carrying a negative weight in either child node.

Returns:

selfobject: Fitted estimator.

get_metadata_routing()[source]#

Get metadata routing of this object.

Please checkUser Guide on how the routingmechanism works.

Returns:

routingMetadataRequest: AMetadataRequest encapsulatingrouting information.

get_params(deep=True)[source]#

Get parameters for this estimator.

Parameters:

deepbool, default=True: If True, will return the parameters for this estimator andcontained subobjects that are estimators.

Returns:

paramsdict: Parameter names mapped to their values.

predict(X)[source]#

Predict class for X.

The predicted class of an input sample is a vote by the trees inthe forest, weighted by their probability estimates. That is,the predicted class is the one with highest mean probabilityestimate across the trees.

Parameters:

X{array-like, sparse matrix} of shape (n_samples, n_features): The input samples. Internally, its dtype will be converted todtype=np.float32. If a sparse matrix is provided, it will beconverted into a sparsecsr_matrix.

Returns:

yndarray of shape (n_samples,) or (n_samples, n_outputs): The predicted classes.

predict_log_proba(X)[source]#

Predict class log-probabilities for X.

The predicted class log-probabilities of an input sample is computed asthe log of the mean predicted class probabilities of the trees in theforest.

Parameters:

X{array-like, sparse matrix} of shape (n_samples, n_features): The input samples. Internally, its dtype will be converted todtype=np.float32. If a sparse matrix is provided, it will beconverted into a sparsecsr_matrix.

Returns:

pndarray of shape (n_samples, n_classes), or a list of such arrays: The class probabilities of the input samples. The order of theclasses corresponds to that in the attributeclasses_.

predict_proba(X)[source]#

Predict class probabilities for X.

The predicted class probabilities of an input sample are computed asthe mean predicted class probabilities of the trees in the forest.The class probability of a single tree is the fraction of samples ofthe same class in a leaf.

Parameters:

X{array-like, sparse matrix} of shape (n_samples, n_features): The input samples. Internally, its dtype will be converted todtype=np.float32. If a sparse matrix is provided, it will beconverted into a sparsecsr_matrix.

Returns:

pndarray of shape (n_samples, n_classes), or a list of such arrays: The class probabilities of the input samples. The order of theclasses corresponds to that in the attributeclasses_.

score(X,y,sample_weight=None)[source]#

Returnaccuracy on provided data and labels.

In multi-label classification, this is the subset accuracywhich is a harsh metric since you require for each sample thateach label set be correctly predicted.

Parameters:

Xarray-like of shape (n_samples, n_features): Test samples.
yarray-like of shape (n_samples,) or (n_samples, n_outputs): True labels forX.
sample_weightarray-like of shape (n_samples,), default=None: Sample weights.

Returns:

scorefloat: Mean accuracy ofself.predict(X) w.r.t.y.

set_fit_request(*,sample_weight:bool|None|str='$UNCHANGED$')→RandomForestClassifier[source]#

Configure whether metadata should be requested to be passed to thefit method.

Note that this method is only relevant when this estimator is used as asub-estimator within ameta-estimator and metadata routing is enabledwithenable_metadata_routing=True (seesklearn.set_config).Please check theUser Guide on how the routingmechanism works.

The options for each parameter are:

True: metadata is requested, and passed tofit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it tofit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains theexisting request. This allows you to change the request for someparameters and not others.

Added in version 1.3.

Parameters:

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing forsample_weight parameter infit.

Returns:

selfobject: The updated object.

set_params(**params)[source]#

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects(such asPipeline). The latter haveparameters of the form<component>__<parameter> so that it’spossible to update each component of a nested object.

Parameters:

**paramsdict: Estimator parameters.

Returns:

selfestimator instance: Estimator instance.

set_score_request(*,sample_weight:bool|None|str='$UNCHANGED$')→RandomForestClassifier[source]#

Configure whether metadata should be requested to be passed to thescore method.

The options for each parameter are:

True: metadata is requested, and passed toscore if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it toscore.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains theexisting request. This allows you to change the request for someparameters and not others.

Added in version 1.3.

Parameters:

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing forsample_weight parameter inscore.

Returns: