Different objects#

The main objects in scikit-learn are (one class can implement multiple interfaces):

Estimator:

The base object, implements afit method to learn from data, either:

estimator=estimator.fit(data,targets)

or:

estimator=estimator.fit(data)

Predictor:

For supervised learning, or some unsupervised problems, implements:

prediction=predictor.predict(data)

Classification algorithms usually also offer a way to quantify certaintyof a prediction, either usingdecision_function orpredict_proba:

probability=predictor.predict_proba(data)

Transformer:

For modifying the data in a supervised or unsupervised way (e.g. by adding, changing,or removing columns, but not by adding or removing rows). Implements:

new_data=transformer.transform(data)

When fitting and transforming can be performed much more efficientlytogether than separately, implements:

new_data=transformer.fit_transform(data)

Model:

A model that can give agoodness of fit measure or a likelihood ofunseen data, implements (higher is better):

score=model.score(data)

Estimators#

The API has one predominant object: the estimator. An estimator is anobject that fits a model based on some training data and is capable ofinferring some properties on new data. It can be, for instance, aclassifier or a regressor. All estimators implement the fit method:

estimator.fit(X,y)

Out of all the methods that an estimator implements,fit is usually the one youwant to implement yourself. Other methods such asset_params,get_params, etc.are implemented inBaseEstimator, which you should inherit from.You might need to inherit from more mixins, which we will explain later.

clf2=SGDClassifier(alpha=2.3)clf3=SGDClassifier([[1,2],[2,3]],[-1,1])# WRONG!

def__init__(self,param1=1,param2=2):self.param1=param1self.param2=param2

def__init__(self,param1=1,param2=2,param3=3):# WRONG: parameters should not be modifiedifparam1>1:param2+=1self.param1=param1# WRONG: the object's attributes should have exactly the name of# the argument in the constructorself.param3=param2

y_predicted=SGDClassifier(alpha=10).fit(X_train,y_train).predict(X_test)

get_params and set_params#

All scikit-learn estimators haveget_params andset_params functions.

Theget_params function takes no arguments and returns a dict of the__init__ parameters of the estimator, together with their values.

It takes one keyword argument,deep, which receives a boolean value that determineswhether the method should return the parameters of sub-estimators (only relevant formeta-estimators). The default value fordeep isTrue. For instance consideringthe following estimator:

>>>fromsklearn.baseimportBaseEstimator>>>fromsklearn.linear_modelimportLogisticRegression>>>classMyEstimator(BaseEstimator):...def__init__(self,subestimator=None,my_extra_param="random"):...self.subestimator=subestimator...self.my_extra_param=my_extra_param

The parameterdeep controls whether or not the parameters of thesubestimator should be reported. Thus whendeep=True, the output will be:

>>>my_estimator=MyEstimator(subestimator=LogisticRegression())>>>forparam,valueinmy_estimator.get_params(deep=True).items():...print(f"{param} ->{value}")my_extra_param -> randomsubestimator__C -> 1.0subestimator__class_weight -> Nonesubestimator__dual -> Falsesubestimator__fit_intercept -> Truesubestimator__intercept_scaling -> 1subestimator__l1_ratio -> Nonesubestimator__max_iter -> 100subestimator__multi_class -> deprecatedsubestimator__n_jobs -> Nonesubestimator__penalty -> l2subestimator__random_state -> Nonesubestimator__solver -> lbfgssubestimator__tol -> 0.0001subestimator__verbose -> 0subestimator__warm_start -> Falsesubestimator -> LogisticRegression()

If the meta-estimator takes multiple sub-estimators, often, those sub-estimators havenames (as e.g. named steps in aPipeline object), in which case thekey should become<name>__C,<name>__class_weight, etc.

Whendeep=False, the output will be:

>>>forparam,valueinmy_estimator.get_params(deep=False).items():...print(f"{param} ->{value}")my_extra_param -> randomsubestimator -> LogisticRegression()

On the other hand,set_params takes the parameters of__init__ as keywordarguments, unpacks them into a dict of the form'parameter':value and sets theparameters of the estimator using this dict. It returns the estimator itself.

Theset_params function is used to set parameters duringgrid search for instance.

Cloning#

As already mentioned that when constructor arguments are mutable, they should becopied before modifying them. This also applies to constructor arguments which areestimators. That’s why meta-estimators such asGridSearchCVcreate a copy of the given estimator before modifying it.

However, in scikit-learn, when we copy an estimator, we get an unfitted estimatorwhere only the constructor arguments are copied (with some exceptions, e.g. attributesrelated to certain internal machinery such as metadata routing).

The function responsible for this behavior isclone.

Estimators can customize the behavior ofbase.clone by overriding thebase.BaseEstimator.__sklearn_clone__ method.__sklearn_clone__ must return aninstance of the estimator.__sklearn_clone__ is useful when an estimator needs to holdon to some state whenbase.clone is called on the estimator. For example,FrozenEstimator makes use of this.

Estimator types#

Among simple estimators (as opposed to meta-estimators), the most common types aretransformers, classifiers, regressors, and clustering algorithms.

Transformers inherit fromTransformerMixin, and implement atransformmethod. These are estimators which take the input, and transform it in some way. Notethat they should never change the number of input samples, and the output oftransformshould correspond to its input samples in the same given order.

Regressors inherit fromRegressorMixin, and implement apredict method.They should accept numericaly in theirfit method. Regressors user2_score by default in theirscore method.

Classifiers inherit fromClassifierMixin. If it applies, classifiers canimplementdecision_function to return raw decision values, based on whichpredict can make its decision. If calculating probabilities is supported,classifiers can also implementpredict_proba andpredict_log_proba.

Classifiers should accepty (target) arguments tofit that are sequences (lists,arrays) of either strings or integers. They should not assume that the class labels area contiguous range of integers; instead, they should store a list of classes in aclasses_ attribute or property. The order of class labels in this attribute shouldmatch the order in whichpredict_proba,predict_log_proba anddecision_function return their values. The easiest way to achieve this is to put:

self.classes_,y=np.unique(y,return_inverse=True)

infit. This returns a newy that contains class indexes, rather than labels,in the range [0,n_classes).

A classifier’spredict method should return arrays containing class labels fromclasses_. In a classifier that implementsdecision_function, this can beachieved with:

defpredict(self,X):D=self.decision_function(X)returnself.classes_[np.argmax(D,axis=1)]

Themulticlass module contains useful functions for working withmulticlass and multilabel problems.

Clustering algorithms inherit fromClusterMixin. Ideally, they shouldaccept ay parameter in theirfit method, but it should be ignored. Clusteringalgorithms should set alabels_ attribute, storing the labels assigned to eachsample. If applicable, they can also implement apredict method, returning thelabels assigned to newly given samples.

If one needs to check the type of a given estimator, e.g. in a meta-estimator, one cancheck if the given object implements atransform method for transformers, andotherwise use helper functions such asis_classifier oris_regressor.

Estimator Tags#

The estimator tags are annotations of estimators that allow programmatic inspection oftheir capabilities, such as sparse matrix support, supported output types and supportedmethods. The estimator tags are an instance ofTags returned bythe method__sklearn_tags__. These tags are usedin different places, such asis_regressor or the common checks run bycheck_estimator andparametrize_with_checks, where tags determinewhich checks to run and what input data is appropriate. Tags can depend on estimatorparameters or even system architecture and can in general only be determined at runtimeand are therefore instance attributes rather than class attributes. SeeTags for more information about individual tags.

It is unlikely that the default values for each tag will suit the needs of your specificestimator. You can change the default values by defining a__sklearn_tags__() methodwhich returns the new values for your estimator’s tags. For example:

classMyMultiOutputEstimator(BaseEstimator):def__sklearn_tags__(self):tags=super().__sklearn_tags__()tags.target_tags.single_output=Falsetags.non_deterministic=Truereturntags

You can create a new subclass ofTags if you wish to add newtags to the existing set. Note that all attributes that you add in a child class needto have a default value. It can be of the form:

fromdataclassesimportdataclass,asdict@dataclassclassMyTags(Tags):my_tag:bool=TrueclassMyEstimator(BaseEstimator):def__sklearn_tags__(self):tags_orig=super().__sklearn_tags__()as_dict={field.name:getattr(tags_orig,field.name)forfieldinfields(tags_orig)}tags=MyTags(**as_dict)tags.my_tag=Truereturntags

Input validation#

The modulesklearn.utils contains various functions for doing inputvalidation and conversion. Sometimes,np.asarray suffices for validation;donot usenp.asanyarray ornp.atleast_2d, since those let NumPy’snp.matrix through, which has a different API(e.g.,* means dot product onnp.matrix,but Hadamard product onnp.ndarray).

In other cases, be sure to callcheck_array on any array-like argumentpassed to a scikit-learn API function. The exact parameters to use dependsmainly on whether and whichscipy.sparse matrices must be accepted.

For more information, refer to theUtilities for Developers page.

Random Numbers#

If your code depends on a random number generator, do not usenumpy.random.random() or similar routines. To ensurerepeatability in error checking, the routine should accept a keywordrandom_state and use this to construct anumpy.random.RandomState object.Seesklearn.utils.check_random_state inUtilities for Developers.

Here’s a simple example of code using some of the above guidelines:

fromsklearn.utilsimportcheck_array,check_random_statedefchoose_random_sample(X,random_state=0):"""Choose a random point from X.    Parameters    ----------    X : array-like of shape (n_samples, n_features)        An array representing the data.    random_state : int or RandomState instance, default=0        The seed of the pseudo random number generator that selects a        random sample. Pass an int for reproducible output across multiple        function calls.        See :term:`Glossary <random_state>`.    Returns    -------    x : ndarray of shape (n_features,)        A random point selected from X.    """X=check_array(X)random_state=check_random_state(random_state)i=random_state.randint(X.shape[0])returnX[i]

If you use randomness in an estimator instead of a freestanding function,some additional guidelines apply.

First off, the estimator should take arandom_state argument to its__init__ with a default value ofNone.It should store that argument’s value,unmodified,in an attributerandom_state.fit can callcheck_random_state on that attributeto get an actual random number generator.If, for some reason, randomness is needed afterfit,the RNG should be stored in an attributerandom_state_.The following example should make this clear:

classGaussianNoise(BaseEstimator,TransformerMixin):"""This estimator ignores its input and returns random Gaussian noise.    It also does not adhere to all scikit-learn conventions,    but showcases how to handle randomness.    """def__init__(self,n_components=100,random_state=None):self.random_state=random_stateself.n_components=n_components# the arguments are ignored anyway, so we make them optionaldeffit(self,X=None,y=None):self.random_state_=check_random_state(self.random_state)deftransform(self,X):n_samples=X.shape[0]returnself.random_state_.randn(n_samples,self.n_components)

The reason for this setup is reproducibility:when an estimator isfit twice to the same data,it should produce an identical model both times,hence the validation infit, not__init__.

Numerical assertions in tests#

When asserting the quasi-equality of arrays of continuous values,do usesklearn.utils._testing.assert_allclose.

The relative tolerance is automatically inferred from the provided arraysdtypes (for float32 and float64 dtypes in particular) but you can overrideviartol.

When comparing arrays of zero-elements, please do provide a non-zero value forthe absolute tolerance viaatol.

For more information, please refer to the docstring ofsklearn.utils._testing.assert_allclose.