adapt.instance_based.TrAdaBoost

classadapt.instance_based.TrAdaBoost(estimator=None,Xt=None,yt=None,n_estimators=10,lr=1.0,copy=True,verbose=1,random_state=None,**params)[source]

Transfer AdaBoost for Classification

TrAdaBoost algorithm is asupervised instances-based domainadaptation method suited forclassification tasks.

The method is based on a “reverse boosting” principle where theweights of source instances poorly predicted decrease at eachboosting iteration whereas the ones of target instances increase.

The algorithm performs the following steps:

  • 1. Normalize weights:\(\sum w_S + \sum w_T = 1\).

  • 2. Fit an estimator\(f\) on source and target labeled data\((X_S, y_S), (X_T, y_T)\) with the respective importancesweights:\(w_S, w_T\).

  • 3. Compute error vectors of training instances:

    • \(\epsilon_S = L_{01}(f(X_S), y_S)\).

    • \(\epsilon_T = L_{01}(f(X_T), y_T)\).

  • 4. Compute total weighted error of target instances:\(E_T = \frac{1}{n_T} w_T^T \epsilon_T\).

  • 5. Update source and target weights:

    • \(w_S = w_S \beta^{\epsilon_S}\).

    • \(w_T = w_T \beta_T^{-\epsilon_T}\).

    Where:

    • \(\beta = 1 \setminus (1 + \sqrt{2 \text{ln} n_S \setminus N})\).

    • \(\beta_T = E_T \setminus (1 - E_T)\).

  • 6. Return to step1 and loop until the number\(N\)of boosting iteration is reached.

The prediction are then given by the vote of the\(N \setminus 2\)last computed estimators weighted by their respective parameter\(\beta_T\).

Parameters
estimatorsklearn estimator or tensorflow Model (default=None)

Estimator used to learn the task. If estimator isNone, aLinearRegressioninstance is used as estimator.

Xtnumpy array (default=None)

Target input data.

ytnumpy array (default=None)

Target output data.

n_estimatorsint (default=10)

Number of boosting iteration.

lrfloat (default=1.)

Learning rate. For higherlr, the sampleweights are updating faster.

copyboolean (default=True)

Whether to make a copy ofestimator or not.

verboseint (default=1)

Verbosity level.

random_stateint (default=None)

Seed of random generator.

paramskey, value arguments

Arguments given at the different level of the adapt object.It can be, for instance, compile or fit parameters of theestimator or kernel parameters etc…Accepted parameters can be found by calling the method_get_legal_params(params).

References

1

[1] Dai W., Yang Q., Xue G., and Yu Y. “Boosting for transfer learning”. In ICML, 2007.

Examples

>>>fromsklearn.linear_modelimportRidgeClassifier>>>fromadapt.utilsimportmake_classification_da>>>fromadapt.instance_basedimportTrAdaBoost>>>Xs,ys,Xt,yt=make_classification_da()>>>model=TrAdaBoost(RidgeClassifier(),n_estimators=10,Xt=Xt[:10],yt=yt[:10],random_state=0)>>>model.fit(Xs,ys)Iteration 0 - Error: 0.2550Iteration 1 - Error: 0.2820Iteration 2 - Error: 0.3011Iteration 3 - Error: 0.3087Iteration 4 - Error: 0.3046Iteration 5 - Error: 0.2933Iteration 6 - Error: 0.2819Iteration 7 - Error: 0.2747Iteration 8 - Error: 0.2712Iteration 9 - Error: 0.2698>>>model.score(Xt,yt)0.66
Attributes
estimators_list of object

List of fitted estimators

estimator_errors_1D array of float

Array of weighted estimator errors computed onlabeled target data.

estimator_weights_1D array of float

Array of estimator importance weights.

sample_weights_src_list of numpy arrays

List of source sample weight for each iteration.

sample_weights_tgt_list of numpy arrays

List of target sample weight for each iteration.

Methods

__init__([estimator, Xt, yt, n_estimators, ...])

fit(X, y[, Xt, yt, sample_weight_src, ...])

Fit TrAdaBoost

fit_estimator(X, y[, sample_weight, ...])

Fit estimator on X, y.

get_metadata_routing()

Get metadata routing of this object.

get_params([deep])

Get parameters for this estimator.

predict(X)

Return weighted vote of estimators.

predict_estimator(X, **predict_params)

Return estimator predictions for X.

predict_weights([domain])

Return sample weights.

score(X, y)

Return the TrAdaboost score on X, y.

set_fit_request(*[, sample_weight_src, ...])

Request metadata passed to thefit method.

set_params(**params)

Set the parameters of this estimator.

set_predict_request(*[, domain])

Request metadata passed to thepredict method.

set_score_request(*[, domain, sample_weight])

Request metadata passed to thescore method.

unsupervised_score(Xs, Xt)

Return unsupervised score.

__init__(estimator=None,Xt=None,yt=None,n_estimators=10,lr=1.0,copy=True,verbose=1,random_state=None,**params)[source]
fit(X,y,Xt=None,yt=None,sample_weight_src=None,sample_weight_tgt=None,**fit_params)[source]

Fit TrAdaBoost

Parameters
Xnumpy array

Source input data.

ynumpy array

Source output data.

Xtarray (default=None)

Target input data. If None, theXt argumentgiven ininit is used.

ytarray (default=None)

Target input data. If None, theXt argumentgiven ininit is used.

sample_weight_srcnumpy array, (default=None)

Initial sample weight of source data

sample_weight_tgtnumpy array, (default=None)

Initial sample weight of target data

fit_paramskey, value arguments

Arguments given to the fit method of theestimator.

Returns
selfreturns an instance of self
fit_estimator(X,y,sample_weight=None,random_state=None,warm_start=True,**fit_params)[source]

Fit estimator on X, y.

Parameters
Xarray

Input data.

yarray

Output data.

sample_weightarray

Importance weighting.

random_stateint (default=None)

Seed of the random generator

warm_startbool (default=True)

If True, continue to fitestimator_,else, a new estimator is fitted based ona copy ofestimator. (Be sure to setcopy=True to usewarm_start=False)

fit_paramskey, value arguments

Arguments given to the fit method ofthe estimator and to the compile methodfor tensorflow estimator.

Returns
estimator_fitted estimator
get_metadata_routing()[source]

Get metadata routing of this object.

Please checkUser Guide on how the routingmechanism works.

Returns
routingMetadataRequest

AMetadataRequest encapsulatingrouting information.

get_params(deep=True)[source]

Get parameters for this estimator.

Parameters
deepbool, default=True

Not used, here for scikit-learn compatibility.

Returns
paramsdict

Parameter names mapped to their values.

predict(X)[source]

Return weighted vote of estimators.

Parameters
Xarray

Input data.

Returns
y_predarray

Vote results.

predict_estimator(X,**predict_params)[source]

Return estimator predictions for X.

Parameters
Xarray

input data

Returns
y_predarray

prediction of estimator.

predict_weights(domain='src')[source]

Return sample weights.

Return the final importance weighting.

You can secify between “source” and “target” weightswith the domain parameter.

Parameters
domainstr (default=”tgt”)

Choose between"source","src" and"target","tgt".

Returns
weightssource sample weights
score(X,y)[source]

Return the TrAdaboost score on X, y.

Parameters
Xarray

input data

yarray

output data

Returns
scorefloat

estimator score.

set_fit_request(*,sample_weight_src:Union[bool,None,str]='$UNCHANGED$',sample_weight_tgt:Union[bool,None,str]='$UNCHANGED$')→adapt.instance_based._tradaboost.TrAdaBoost[source]

Request metadata passed to thefit method.

Note that this method is only relevant ifenable_metadata_routing=True (seesklearn.set_config()).Please seeUser Guide on how the routingmechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed tofit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it tofit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains theexisting request. This allows you to change the request for someparameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as asub-estimator of a meta-estimator, e.g. used inside aPipeline. Otherwise it has no effect.

Parameters
sample_weight_srcstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing forsample_weight_src parameter infit.

sample_weight_tgtstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing forsample_weight_tgt parameter infit.

Returns
selfobject

The updated object.

set_params(**params)[source]

Set the parameters of this estimator.

Parameters
**paramsdict

Estimator parameters.

Returns
selfestimator instance

Estimator instance.

set_predict_request(*,domain:Union[bool,None,str]='$UNCHANGED$')→adapt.instance_based._tradaboost.TrAdaBoost[source]

Request metadata passed to thepredict method.

Note that this method is only relevant ifenable_metadata_routing=True (seesklearn.set_config()).Please seeUser Guide on how the routingmechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed topredict if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it topredict.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains theexisting request. This allows you to change the request for someparameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as asub-estimator of a meta-estimator, e.g. used inside aPipeline. Otherwise it has no effect.

Parameters
domainstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing fordomain parameter inpredict.

Returns
selfobject

The updated object.

set_score_request(*,domain:Union[bool,None,str]='$UNCHANGED$',sample_weight:Union[bool,None,str]='$UNCHANGED$')→adapt.instance_based._tradaboost.TrAdaBoost[source]

Request metadata passed to thescore method.

Note that this method is only relevant ifenable_metadata_routing=True (seesklearn.set_config()).Please seeUser Guide on how the routingmechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed toscore if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it toscore.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains theexisting request. This allows you to change the request for someparameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as asub-estimator of a meta-estimator, e.g. used inside aPipeline. Otherwise it has no effect.

Parameters
domainstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing fordomain parameter inscore.

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing forsample_weight parameter inscore.

Returns
selfobject

The updated object.

unsupervised_score(Xs,Xt)[source]

Return unsupervised score.

The normalized discrepancy distance is computedbetween the reweighted/transformed source inputdata and the target input data.

Parameters
Xsarray

Source input data.

Xtarray

Source input data.

Returns
scorefloat

Unsupervised score.

Examples