Movatterモバイル変換

[0]ホーム
URL:

SALE!Use codeBF40 for 40% off everything!
Hurry, sale ends soon!Click to see the full catalog.

Navigation

MachineLearningMastery.com

Making developers awesome at machine learning

Making developers awesome at machine learning

6 Dimensionality Reduction Algorithms With Python

Dimensionality reduction is an unsupervised learning technique.

Nevertheless, it can be used as a data transform pre-processing step for machine learning algorithms on classification and regression predictive modeling datasets with supervised learning algorithms.

There are many dimensionality reduction algorithms to choose from and no single best algorithm for all cases. Instead, it is a good idea to explore a range of dimensionality reduction algorithms and different configurations for each algorithm.

In this tutorial, you will discover how to fit and evaluate top dimensionality reduction algorithms in Python.

After completing this tutorial, you will know:

  • Dimensionality reduction seeks a lower-dimensional representation of numerical input data that preserves the salient relationships in the data.
  • There are many different dimensionality reduction algorithms and no single best method for all datasets.
  • How to implement, fit, and evaluate top dimensionality reduction in Python with the scikit-learn machine learning library.

Kick-start your project with my new bookData Preparation for Machine Learning, includingstep-by-step tutorials and thePython source code files for all examples.

Let’s get started.

Dimensionality Reduction Algorithms With Python

Dimensionality Reduction Algorithms With Python
Photo byBernard Spragg. NZ, some rights reserved.

Tutorial Overview

This tutorial is divided into three parts; they are:

  1. Dimensionality Reduction
  2. Dimensionality Reduction Algorithms
  3. Examples of Dimensionality Reduction
    1. Scikit-Learn Library Installation
    2. Classification Dataset
    3. Principal Component Analysis
    4. Singular Value Decomposition
    5. Linear Discriminant Analysis
    6. Isomap Embedding
    7. Locally Linear Embedding
    8. Modified Locally Linear Embedding

Dimensionality Reduction

Dimensionality reduction refers to techniques for reducing the number of input variables in training data.

When dealing with high dimensional data, it is often useful to reduce the dimensionality by projecting the data to a lower dimensional subspace which captures the “essence” of the data. This is called dimensionality reduction.

— Page 11,Machine Learning: A Probabilistic Perspective, 2012.

High-dimensionality might mean hundreds, thousands, or even millions of input variables.

Fewer input dimensions often means correspondingly fewer parameters or a simpler structure in the machine learning model, referred to asdegrees of freedom. A model with too many degrees of freedom is likely to overfit the training dataset and may not perform well on new data.

It is desirable to have simple models that generalize well, and in turn, input data with few input variables. This is particularly true for linear models where the number of inputs and the degrees of freedom of the model are often closely related.

Dimensionality reduction is a data preparation technique performed on data prior to modeling. It might be performed after data cleaning and data scaling and before training a predictive model.

… dimensionality reduction yields a more compact, more easily interpretable representation of the target concept, focusing the user’s attention on the most relevant variables.

— Page 289,Data Mining: Practical Machine Learning Tools and Techniques, 4th edition, 2016.

As such, any dimensionality reduction performed on training data must also be performed on new data, such as a test dataset, validation dataset, and data when making a prediction with thefinal model.

Want to Get Started With Data Preparation?

Take my free 7-day email crash course now (with sample code).

Click to sign-up and also get a free PDF Ebook version of the course.

Dimensionality Reduction Algorithms

There are many algorithms that can be used for dimensionality reduction.

Two main classes of methods are those drawn from linear algebra and those drawn from manifold learning.

Linear Algebra Methods

Matrix factorization methods drawn from the field of linear algebra can be used for dimensionality.

For more on matrix factorization, see the tutorial:

Some of the more popular methods include:

  • Principal Components Analysis
  • Singular Value Decomposition
  • Non-Negative Matrix Factorization

Manifold Learning Methods

Manifold learning methods seek a lower-dimensional projection of high dimensional input that captures the salient properties of the input data.

Some of the more popular methods include:

  • Isomap Embedding
  • Locally Linear Embedding
  • Multidimensional Scaling
  • Spectral Embedding
  • t-distributed Stochastic Neighbor Embedding

Each algorithm offers a different approach to the challenge of discovering natural relationships in data at lower dimensions.

There is no best dimensionality reduction algorithm, and no easy way to find the best algorithm for your data without using controlled experiments.

In this tutorial, we will review how to use each subset of these popular dimensionality reduction algorithms from the scikit-learn library.

The examples will provide the basis for you to copy-paste the examples and test the methods on your own data.

We will not dive into the theory behind how the algorithms work or compare them directly. For a good starting point on this topic, see:

Let’s dive in.

Examples of Dimensionality Reduction

In this section, we will review how to use popular dimensionality reduction algorithms in scikit-learn.

This includes an example of using the dimensionality reduction technique as a data transform in a modeling pipeline and evaluating a model fit on the data.

The examples are designed for you to copy-paste into your own project and apply the methods to your own data. There are some algorithms available in the scikit-learn library that are not demonstrated because they cannot be used as a data transform directly given the nature of the algorithm.

As such, we will use a synthetic classification dataset in each example.

Scikit-Learn Library Installation

First, let’s install the library.

Don’t skip this step as you will need to ensure you have the latest version installed.

You can install the scikit-learn library using the pip Python installer, as follows:

1
sudo pip install scikit-learn

For additional installation instructions specific to your platform, see:

Next, let’s confirm that the library is installed and you are using a modern version.

Run the following script to print the library version number.

1
2
3
# check scikit-learn version
importsklearn
print(sklearn.__version__)

Running the example, you should see the following version number or higher.

1
0.23.0

Classification Dataset

We will use themake_classification() function to create a test binary classification dataset.

The dataset will have 1,000 examples with 20 input features, 10 of which are informative and 10 of which are redundant. This provides an opportunity for each technique to identify and remove redundant input features.

The fixed random seed for the pseudorandom number generator ensures we generate the same synthetic dataset each time the code runs.

An example of creating and summarizing the synthetic classification dataset is listed below.

1
2
3
4
5
6
# synthetic classification dataset
fromsklearn.datasetsimportmake_classification
# define dataset
X,y=make_classification(n_samples=1000,n_features=20,n_informative=10,n_redundant=10,random_state=7)
# summarize the dataset
print(X.shape,y.shape)

Running the example creates the dataset and reports the number of rows and columns matching our expectations.

1
(1000, 20) (1000,)

It is a binary classification task and we will evaluate aLogisticRegression model after each dimensionality reduction transform.

The model will be evaluated using the gold standard ofrepeated stratified 10-fold cross-validation. The mean and standard deviation classification accuracy across all folds and repeats will be reported.

The example below evaluates the model on the raw dataset as a point of comparison.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# evaluate logistic regression model on raw data
fromnumpyimportmean
fromnumpyimportstd
fromsklearn.datasetsimportmake_classification
fromsklearn.model_selectionimportcross_val_score
fromsklearn.model_selectionimportRepeatedStratifiedKFold
fromsklearn.linear_modelimportLogisticRegression
# define dataset
X,y=make_classification(n_samples=1000,n_features=20,n_informative=10,n_redundant=10,random_state=7)
# define the model
model=LogisticRegression()
# evaluate model
cv=RepeatedStratifiedKFold(n_splits=10,n_repeats=3,random_state=1)
n_scores=cross_val_score(model,X,y,scoring='accuracy',cv=cv,n_jobs=-1)
# report performance
print('Accuracy: %.3f (%.3f)'%(mean(n_scores),std(n_scores)))

Running the example evaluates the logistic regression on the raw dataset with all 20 columns, achieving a classification accuracy of about 82.4 percent.

A successful dimensionality reduction transform on this data should result in a model that has better accuracy than this baseline, although this may not be possible with all techniques.

Note: we are not trying to “solve” this dataset, just provide working examples that you can use as a starting point.

1
Accuracy: 0.824 (0.034)

Next, we can start looking at examples of dimensionality reduction algorithms applied to this dataset.

I have made some minimal attempts to tune each method to the dataset. Each dimensionality reduction method will be configured to reduce the 20 input columns to 10 where possible.

We will use aPipeline to combine the data transform and model into an atomic unit that can be evaluated using the cross-validation procedure; for example:

1
2
3
4
...
# define the pipeline
steps=[('pca',PCA(n_components=10)),('m',LogisticRegression())]
model=Pipeline(steps=steps)

Let’s get started.

Can you get a better result for one of the algorithms?
Let me know in the comments below.

Principal Component Analysis

Principal Component Analysis, or PCA, might be the most popular technique for dimensionality reduction with dense data (few zero values).

For more on how PCA works, see the tutorial:

The scikit-learn library provides thePCA class implementation of Principal Component Analysis that can be used as a dimensionality reduction data transform. The “n_components” argument can be set to configure the number of desired dimensions in the output of the transform.

The complete example of evaluating a model with PCA dimensionality reduction is listed below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# evaluate pca with logistic regression algorithm for classification
fromnumpyimportmean
fromnumpyimportstd
fromsklearn.datasetsimportmake_classification
fromsklearn.model_selectionimportcross_val_score
fromsklearn.model_selectionimportRepeatedStratifiedKFold
fromsklearn.pipelineimportPipeline
fromsklearn.decompositionimportPCA
fromsklearn.linear_modelimportLogisticRegression
# define dataset
X,y=make_classification(n_samples=1000,n_features=20,n_informative=10,n_redundant=10,random_state=7)
# define the pipeline
steps=[('pca',PCA(n_components=10)),('m',LogisticRegression())]
model=Pipeline(steps=steps)
# evaluate model
cv=RepeatedStratifiedKFold(n_splits=10,n_repeats=3,random_state=1)
n_scores=cross_val_score(model,X,y,scoring='accuracy',cv=cv,n_jobs=-1)
# report performance
print('Accuracy: %.3f (%.3f)'%(mean(n_scores),std(n_scores)))

Running the example evaluates the modeling pipeline with dimensionality reduction and a logistic regression predictive model.

Note: Yourresults may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

In this case, we don’t see any lift in model performance in using the PCA transform.

1
Accuracy: 0.824 (0.034)

Singular Value Decomposition

Singular Value Decomposition, or SVD, is one of the most popular techniques for dimensionality reduction forsparse data (data with many zero values).

For more on how SVD works, see the tutorial:

The scikit-learn library provides theTruncatedSVD class implementation of Singular Value Decomposition that can be used as a dimensionality reduction data transform. The “n_components” argument can be set to configure the number of desired dimensions in the output of the transform.

The complete example of evaluating a model with SVD dimensionality reduction is listed below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# evaluate svd with logistic regression algorithm for classification
fromnumpyimportmean
fromnumpyimportstd
fromsklearn.datasetsimportmake_classification
fromsklearn.model_selectionimportcross_val_score
fromsklearn.model_selectionimportRepeatedStratifiedKFold
fromsklearn.pipelineimportPipeline
fromsklearn.decompositionimportTruncatedSVD
fromsklearn.linear_modelimportLogisticRegression
# define dataset
X,y=make_classification(n_samples=1000,n_features=20,n_informative=10,n_redundant=10,random_state=7)
# define the pipeline
steps=[('svd',TruncatedSVD(n_components=10)),('m',LogisticRegression())]
model=Pipeline(steps=steps)
# evaluate model
cv=RepeatedStratifiedKFold(n_splits=10,n_repeats=3,random_state=1)
n_scores=cross_val_score(model,X,y,scoring='accuracy',cv=cv,n_jobs=-1)
# report performance
print('Accuracy: %.3f (%.3f)'%(mean(n_scores),std(n_scores)))

Running the example evaluates the modeling pipeline with dimensionality reduction and a logistic regression predictive model.

Note: Yourresults may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

In this case, we don’t see any lift in model performance in using the SVD transform.

1
Accuracy: 0.824 (0.034)

Linear Discriminant Analysis

Linear Discriminant Analysis, or LDA, is a multi-class classification algorithm that can be used for dimensionality reduction.

The number of dimensions for the projection is limited to 1 and C-1, where C is the number of classes. In this case, our dataset is a binary classification problem (two classes), limiting the number of dimensions to 1.

For more on LDA for dimensionality reduction, see the tutorial:

The scikit-learn library provides theLinearDiscriminantAnalysis class implementation of Linear Discriminant Analysis that can be used as a dimensionality reduction data transform. The “n_components” argument can be set to configure the number of desired dimensions in the output of the transform.

The complete example of evaluating a model with LDA dimensionality reduction is listed below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# evaluate lda with logistic regression algorithm for classification
fromnumpyimportmean
fromnumpyimportstd
fromsklearn.datasetsimportmake_classification
fromsklearn.model_selectionimportcross_val_score
fromsklearn.model_selectionimportRepeatedStratifiedKFold
fromsklearn.pipelineimportPipeline
fromsklearn.discriminant_analysisimportLinearDiscriminantAnalysis
fromsklearn.linear_modelimportLogisticRegression
# define dataset
X,y=make_classification(n_samples=1000,n_features=20,n_informative=10,n_redundant=10,random_state=7)
# define the pipeline
steps=[('lda',LinearDiscriminantAnalysis(n_components=1)),('m',LogisticRegression())]
model=Pipeline(steps=steps)
# evaluate model
cv=RepeatedStratifiedKFold(n_splits=10,n_repeats=3,random_state=1)
n_scores=cross_val_score(model,X,y,scoring='accuracy',cv=cv,n_jobs=-1)
# report performance
print('Accuracy: %.3f (%.3f)'%(mean(n_scores),std(n_scores)))

Running the example evaluates the modeling pipeline with dimensionality reduction and a logistic regression predictive model.

Note: Yourresults may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

In this case, we can see a slight lift in performance as compared to the baseline fit on the raw data.

1
Accuracy: 0.825 (0.034)

Isomap Embedding

Isomap Embedding, or Isomap, creates an embedding of the dataset and attempts to preserve the relationships in the dataset.

The scikit-learn library provides theIsomap class implementation of Isomap Embedding that can be used as a dimensionality reduction data transform. The “n_components” argument can be set to configure the number of desired dimensions in the output of the transform.

The complete example of evaluating a model with SVD dimensionality reduction is listed below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# evaluate isomap with logistic regression algorithm for classification
fromnumpyimportmean
fromnumpyimportstd
fromsklearn.datasetsimportmake_classification
fromsklearn.model_selectionimportcross_val_score
fromsklearn.model_selectionimportRepeatedStratifiedKFold
fromsklearn.pipelineimportPipeline
fromsklearn.manifoldimportIsomap
fromsklearn.linear_modelimportLogisticRegression
# define dataset
X,y=make_classification(n_samples=1000,n_features=20,n_informative=10,n_redundant=10,random_state=7)
# define the pipeline
steps=[('iso',Isomap(n_components=10)),('m',LogisticRegression())]
model=Pipeline(steps=steps)
# evaluate model
cv=RepeatedStratifiedKFold(n_splits=10,n_repeats=3,random_state=1)
n_scores=cross_val_score(model,X,y,scoring='accuracy',cv=cv,n_jobs=-1)
# report performance
print('Accuracy: %.3f (%.3f)'%(mean(n_scores),std(n_scores)))

Running the example evaluates the modeling pipeline with dimensionality reduction and a logistic regression predictive model.

Note: Yourresults may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

In this case, we can see a lift in performance with the Isomap data transform as compared to the baseline fit on the raw data.

1
Accuracy: 0.888 (0.029)

Locally Linear Embedding

Locally Linear Embedding, or LLE, creates an embedding of the dataset and attempts to preserve the relationships between neighborhoods in the dataset.

The scikit-learn library provides theLocallyLinearEmbedding class implementation of Locally Linear Embedding that can be used as a dimensionality reduction data transform. The “n_components” argument can be set to configure the number of desired dimensions in the output of the transform

The complete example of evaluating a model with LLE dimensionality reduction is listed below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# evaluate lle and logistic regression for classification
fromnumpyimportmean
fromnumpyimportstd
fromsklearn.datasetsimportmake_classification
fromsklearn.model_selectionimportcross_val_score
fromsklearn.model_selectionimportRepeatedStratifiedKFold
fromsklearn.pipelineimportPipeline
fromsklearn.manifoldimportLocallyLinearEmbedding
fromsklearn.linear_modelimportLogisticRegression
# define dataset
X,y=make_classification(n_samples=1000,n_features=20,n_informative=10,n_redundant=10,random_state=7)
# define the pipeline
steps=[('lle',LocallyLinearEmbedding(n_components=10)),('m',LogisticRegression())]
model=Pipeline(steps=steps)
# evaluate model
cv=RepeatedStratifiedKFold(n_splits=10,n_repeats=3,random_state=1)
n_scores=cross_val_score(model,X,y,scoring='accuracy',cv=cv,n_jobs=-1)
# report performance
print('Accuracy: %.3f (%.3f)'%(mean(n_scores),std(n_scores)))

Running the example evaluates the modeling pipeline with dimensionality reduction and a logistic regression predictive model.

Note: Yourresults may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

In this case, we can see a lift in performance with the LLE data transform as compared to the baseline fit on the raw data.

1
Accuracy: 0.886 (0.028)

Modified Locally Linear Embedding

Modified Locally Linear Embedding, or Modified LLE, is an extension of Locally Linear Embedding that creates multiple weighting vectors for each neighborhood.

The scikit-learn library provides theLocallyLinearEmbedding class implementation of Modified Locally Linear Embedding that can be used as a dimensionality reduction data transform. The “method” argument must be set to ‘modified’ and the “n_components” argument can be set to configure the number of desired dimensions in the output of the transform which must be less than the “n_neighbors” argument.

The complete example of evaluating a model with Modified LLE dimensionality reduction is listed below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# evaluate modified lle and logistic regression for classification
fromnumpyimportmean
fromnumpyimportstd
fromsklearn.datasetsimportmake_classification
fromsklearn.model_selectionimportcross_val_score
fromsklearn.model_selectionimportRepeatedStratifiedKFold
fromsklearn.pipelineimportPipeline
fromsklearn.manifoldimportLocallyLinearEmbedding
fromsklearn.linear_modelimportLogisticRegression
# define dataset
X,y=make_classification(n_samples=1000,n_features=20,n_informative=10,n_redundant=10,random_state=7)
# define the pipeline
steps=[('lle',LocallyLinearEmbedding(n_components=5,method='modified',n_neighbors=10)),('m',LogisticRegression())]
model=Pipeline(steps=steps)
# evaluate model
cv=RepeatedStratifiedKFold(n_splits=10,n_repeats=3,random_state=1)
n_scores=cross_val_score(model,X,y,scoring='accuracy',cv=cv,n_jobs=-1)
# report performance
print('Accuracy: %.3f (%.3f)'%(mean(n_scores),std(n_scores)))

Running the example evaluates the modeling pipeline with dimensionality reduction and a logistic regression predictive model.

Note: Yourresults may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

In this case, we can see a lift in performance with the modified LLE data transform as compared to the baseline fit on the raw data.

1
Accuracy: 0.846 (0.036)

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Tutorials

APIs

Summary

In this tutorial, you discovered how to fit and evaluate top dimensionality reduction algorithms in Python.

Specifically, you learned:

  • Dimensionality reduction seeks a lower-dimensional representation of numerical input data that preserves the salient relationships in the data.
  • There are many different dimensionality reduction algorithms and no single best method for all datasets.
  • How to implement, fit, and evaluate top dimensionality reduction in Python with the scikit-learn machine learning library.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Get a Handle on Modern Data Preparation!

Data Preparation for Machine Learning

Prepare Your Machine Learning Data in Minutes

...with just a few lines of python code

Discover how in my new Ebook:
Data Preparation for Machine Learning

It providesself-study tutorials withfull working code on:
Feature Selection,RFE,Data Cleaning,Data Transforms,Scaling,Dimensionality Reduction, and much more...

Bring Modern Data Preparation Techniques to
Your Machine Learning Projects


See What's Inside

20 Responses to6 Dimensionality Reduction Algorithms With Python

  1. AbcJuly 10, 2020 at 9:40 am#

    How to select the number of dimensions

    • Jason BrownleeJuly 10, 2020 at 1:48 pm#

      Some methods provide insight about the number of components/dimensions are useful, e.g. PCA.

      Generally, treat it as a hyperparameter for your modeling pipeline.

  2. AlanzinhoJuly 11, 2020 at 9:24 pm#

    It is not mentioned which inputs were dropped and which may be considered “strong”. In a real scenario, it is necessary know these informations to justify the dimensionality reduction.

    • Jason BrownleeJuly 12, 2020 at 5:51 am#

      I use results to guide me. E.g. use the number of dimensions that result in the best performing model.

  3. SangramAugust 4, 2020 at 10:16 pm#

    UMAP is also a great dimensionality reduction algorithm.

    • Jason BrownleeAugust 5, 2020 at 6:12 am#

      Thanks!

    • AlexandraAugust 10, 2023 at 3:09 am#

      I have tried UMAP for visualizing my data. Could you please give some insight on how to use it as a dimensionality reduction algorithm?

  4. SamuelAugust 13, 2020 at 6:06 am#

    Hello Jason. Can any of the manifold methods be used for time series data?

    • Jason BrownleeAugust 13, 2020 at 6:24 am#

      Perhaps, I am not up to speed on clustering for time series sorry.

  5. Ancy Sherin JoseAugust 15, 2021 at 12:06 am#

    Hi Jason,

    Is there any thing wrong if I perform Linear Discriminant Analysis dimensionality reduction technique to reduce the number of dimensions before KMeans clustering.

    I performed PCA + KMeans and LDA + KMeans. I got good results for LDA + KMeans. But I couldn’t find any work that uses LDA + KMeans. As I am working on a labelled dataset, I can do LDA technique, but in the real situations, there maynot be labelled dataset. So my question is, if I use LDA + KMeans does that make sense. I didnt find any works that use this technique for clustering.

    Thankyou

  6. collinsAugust 21, 2021 at 5:58 am#

    Jason am lost here ,with dimensionality reduction, do we used eg PCA +logistic regression to find the accuracy of the reduction before using another model to train and test the data?

    • Adrian Tam
      Adrian TamAugust 23, 2021 at 4:42 am#

      PCA is a technique of dimensionality reduction. But any reduction will lost some information, only a matter of whether it is material. Applying logistic regression, for example, can help you find a way to measure whether the reduction can preserve the information you needed for a specific problem.

  7. Ancy Sherin JoseAugust 28, 2021 at 2:33 pm#

    Thankyou.

  8. MedJuly 21, 2022 at 9:07 am#

    Hi all,

    Is there any techniques related to PLA (Piecewise Linear Approximation) and which can be used for dimensionalilty reduction. I have the following problem: I have a dataset as a matrix consisting of time series and each row corresponds to a time series. I need an algorithm that uses PLA to reduce the number of columns.

    Thanks and best regards

    Med

  9. SarkaOctober 22, 2022 at 10:58 pm#

    Hello, I tried to use spectral embedding to see the cross val score, but it returned nan values. Can you help where is the mistake ? Thank you, Sarka

    # Spectral embedding
    from sklearn import metrics
    from sklearn.datasets import load_digits
    from sklearn.manifold import SpectralEmbedding
    from sklearn.metrics import accuracy_score

    # define dataset
    X, y = make_classification(n_samples=1000, n_features=20, n_informative=10, n_redundant=10, random_state=7)

    # define the pipeline
    model = SpectralEmbedding(n_components=2)
    X_transformed = model.fit_transform(X)

    # evaluate model
    cv = RepeatedStratifiedKFold(n_splits=10, n_repeats=3, random_state=1)
    n_scores = cross_val_score(model, X, y, scoring=’accuracy’, cv=cv, n_jobs=-1)

    # report performance
    print(‘Accuracy: %.3f (%.3f)’ % (mean(n_scores), std(n_scores)))

    • James CarmichaelOctober 23, 2022 at 8:21 am#

      Hi Sarka…I do not see where you defined ‘make_classification’.

  10. Bryam BlasAugust 19, 2023 at 5:17 am#

    I have a query, if I have categorical and numeric variables in my data set, what technique do you recommend I use or what should I do in those cases. Thank you.

Leave a ReplyClick here to cancel reply.

Never miss a tutorial:


LinkedIn   Twitter   Facebook   Email Newsletter   RSS Feed

Loving the Tutorials?

TheData Preparation EBook is
where you'll find theReally Good stuff.

>> See What's Inside

[8]ページ先頭
©2009-2025 Movatter.jp