pygad.kerasga Module

This section of the PyGAD’s library documentation discusses thepygad.kerasgamodule.

Thepygad.kerarsga module has helper a class and 2 functions totrain Keras models using the genetic algorithm (PyGAD). The Keras modelcan be built either using theSequentialModel or theFunctionalAPI.

The contents of this module are:

  1. KerasGA: A class for creating an initial population of allparameters in the Keras model.

  2. model_weights_as_vector(): A function to reshape the Keras modelweights to a single vector.

  3. model_weights_as_matrix(): A function to restore the Keras modelweights from a vector.

  4. predict(): A function to make predictions based on the Kerasmodel and a solution.

More details are given in the next sections.

Steps Summary

The summary of the steps used to train a Keras model using PyGAD is asfollows:

  1. Create a Keras model.

  2. Create an instance of thepygad.kerasga.KerasGA class.

  3. Prepare the training data.

  4. Build the fitness function.

  5. Create an instance of thepygad.GA class.

  6. Run the genetic algorithm.

Create Keras Model

Before discussing training a Keras model using PyGAD, the first thing todo is to create the Keras model.

According to theKeras librarydocumentation, there are 3 ways tobuild a Keras model:

  1. Sequential Model

  2. Functional API

  3. Model Subclassing

PyGAD supports training the models created either using the SequentialModel or the Functional API.

Here is an example of a model created using the Sequential Model.

importtensorflow.kerasinput_layer=tensorflow.keras.layers.Input(3)dense_layer1=tensorflow.keras.layers.Dense(5,activation="relu")output_layer=tensorflow.keras.layers.Dense(1,activation="linear")model=tensorflow.keras.Sequential()model.add(input_layer)model.add(dense_layer1)model.add(output_layer)

This is the same model created using the Functional API.

input_layer=tensorflow.keras.layers.Input(3)dense_layer1=tensorflow.keras.layers.Dense(5,activation="relu")(input_layer)output_layer=tensorflow.keras.layers.Dense(1,activation="linear")(dense_layer1)model=tensorflow.keras.Model(inputs=input_layer,outputs=output_layer)

Feel free to add the layers of your choice.

pygad.kerasga.KerasGA Class

Thepygad.kerasga module has a class namedKerasGA for creatingan initial population for the genetic algorithm based on a Keras model.The constructor, methods, and attributes within the class are discussedin this section.

__init__()

Thepygad.kerasga.KerasGA class constructor accepts the followingparameters:

  • model: An instance of the Keras model.

  • num_solutions: Number of solutions in the population. Eachsolution has different parameters of the model.

Instance Attributes

All parameters in thepygad.kerasga.KerasGA class constructor areused as instance attributes in addition to adding a new attribute calledpopulation_weights.

Here is a list of all instance attributes:

  • model

  • num_solutions

  • population_weights: A nested list holding the weights of allsolutions in the population.

Methods in theKerasGA Class

This section discusses the methods available for instances of thepygad.kerasga.KerasGA class.

create_population()

Thecreate_population() method creates the initial population of thegenetic algorithm as a list of solutions where each solution representsdifferent model parameters. The list of networks is assigned to thepopulation_weights attribute of the instance.

Functions in thepygad.kerasga Module

This section discusses the functions in thepygad.kerasga module.

pygad.kerasga.model_weights_as_vector()

Themodel_weights_as_vector() function accepts a single parameternamedmodel representing the Keras model. It returns a vectorholding all model weights. The reason for representing the model weightsas a vector is that the genetic algorithm expects all parameters of anysolution to be in a 1D vector form.

This function filters the layers based on thetrainable attribute tosee whether the layer weights are trained or not. For each layer, if itstrainable=False, then its weights will not be evolved using thegenetic algorithm. Otherwise, it will be represented in the chromosomeand evolved.

The function accepts the following parameters:

  • model: The Keras model.

It returns a 1D vector holding the model weights.

pygad.kerasga.model_weights_as_matrix()

Themodel_weights_as_matrix() function accepts the followingparameters:

  1. model: The Keras model.

  2. weights_vector: The model parameters as a vector.

It returns the restored model weights after reshaping the vector.

pygad.kerasga.predict()

Thepredict() function makes a prediction based on a solution. Itaccepts the following parameters:

  1. model: The Keras model.

  2. solution: The solution evolved.

  3. data: The test data inputs.

  4. batch_size=None: The batch size (i.e. number of samples per stepor batch).

  5. verbose=None: Verbosity mode.

  6. steps=None: The total number of steps (batches of samples).

Check documentation of theKerasModel.predict()method for more information about thebatch_size,verbose, andsteps parameters.

It returns the predictions of the data samples.

Examples

This section gives the complete code of some examples that build andtrain a Keras model using PyGAD. Each subsection builds a differentnetwork.

Example 1: Regression Example

The next code builds a simple Keras model for regression. The nextsubsections discuss each part in the code.

importtensorflow.kerasimportpygad.kerasgaimportnumpyimportpygaddeffitness_func(ga_instance,solution,sol_idx):globaldata_inputs,data_outputs,keras_ga,modelpredictions=pygad.kerasga.predict(model=model,solution=solution,data=data_inputs)mae=tensorflow.keras.losses.MeanAbsoluteError()abs_error=mae(data_outputs,predictions).numpy()+0.00000001solution_fitness=1.0/abs_errorreturnsolution_fitnessdefon_generation(ga_instance):print(f"Generation ={ga_instance.generations_completed}")print(f"Fitness    ={ga_instance.best_solution()[1]}")input_layer=tensorflow.keras.layers.Input(3)dense_layer1=tensorflow.keras.layers.Dense(5,activation="relu")(input_layer)output_layer=tensorflow.keras.layers.Dense(1,activation="linear")(dense_layer1)model=tensorflow.keras.Model(inputs=input_layer,outputs=output_layer)keras_ga=pygad.kerasga.KerasGA(model=model,num_solutions=10)# Data inputsdata_inputs=numpy.array([[0.02,0.1,0.15],[0.7,0.6,0.8],[1.5,1.2,1.7],[3.2,2.9,3.1]])# Data outputsdata_outputs=numpy.array([[0.1],[0.6],[1.3],[2.5]])# Prepare the PyGAD parameters. Check the documentation for more information: https://pygad.readthedocs.io/en/latest/pygad.html#pygad-ga-classnum_generations=250# Number of generations.num_parents_mating=5# Number of solutions to be selected as parents in the mating pool.initial_population=keras_ga.population_weights# Initial population of network weightsga_instance=pygad.GA(num_generations=num_generations,num_parents_mating=num_parents_mating,initial_population=initial_population,fitness_func=fitness_func,on_generation=on_generation)ga_instance.run()# After the generations complete, some plots are showed that summarize how the outputs/fitness values evolve over generations.ga_instance.plot_fitness(title="PyGAD & Keras - Iteration vs. Fitness",linewidth=4)# Returning the details of the best solution.solution,solution_fitness,solution_idx=ga_instance.best_solution()print(f"Fitness value of the best solution ={solution_fitness}")print(f"Index of the best solution :{solution_idx}")# Make prediction based on the best solution.predictions=pygad.kerasga.predict(model=model,solution=solution,data=data_inputs)print(f"Predictions :\n{predictions}")mae=tensorflow.keras.losses.MeanAbsoluteError()abs_error=mae(data_outputs,predictions).numpy()print(f"Absolute Error :{abs_error}")

Create a Keras Model

According to the steps mentioned previously, the first step is to createa Keras model. Here is the code that builds the model using theFunctional API.

importtensorflow.kerasinput_layer=tensorflow.keras.layers.Input(3)dense_layer1=tensorflow.keras.layers.Dense(5,activation="relu")(input_layer)output_layer=tensorflow.keras.layers.Dense(1,activation="linear")(dense_layer1)model=tensorflow.keras.Model(inputs=input_layer,outputs=output_layer)

The model can also be build using the Keras Sequential Model API.

input_layer=tensorflow.keras.layers.Input(3)dense_layer1=tensorflow.keras.layers.Dense(5,activation="relu")output_layer=tensorflow.keras.layers.Dense(1,activation="linear")model=tensorflow.keras.Sequential()model.add(input_layer)model.add(dense_layer1)model.add(output_layer)

Create an Instance of thepygad.kerasga.KerasGA Class

The second step is to create an instance of thepygad.kerasga.KerasGA class. There are 10 solutions per population.Change this number according to your needs.

importpygad.kerasgakeras_ga=pygad.kerasga.KerasGA(model=model,num_solutions=10)

Prepare the Training Data

The third step is to prepare the training data inputs and outputs. Hereis an example where there are 4 samples. Each sample has 3 inputs and 1output.

importnumpy# Data inputsdata_inputs=numpy.array([[0.02,0.1,0.15],[0.7,0.6,0.8],[1.5,1.2,1.7],[3.2,2.9,3.1]])# Data outputsdata_outputs=numpy.array([[0.1],[0.6],[1.3],[2.5]])

Build the Fitness Function

The fourth step is to build the fitness function. This function mustaccept 2 parameters representing the solution and its index within thepopulation.

The next fitness function returns the model predictions based on thecurrent solution using thepredict() function. Then, it calculatesthe mean absolute error (MAE) of the Keras model based on the parametersin the solution. The reciprocal of the MAE is used as the fitness value.Feel free to use any other loss function to calculate the fitness value.

deffitness_func(ga_instance,solution,sol_idx):globaldata_inputs,data_outputs,keras_ga,modelpredictions=pygad.kerasga.predict(model=model,solution=solution,data=data_inputs)mae=tensorflow.keras.losses.MeanAbsoluteError()abs_error=mae(data_outputs,predictions).numpy()+0.00000001solution_fitness=1.0/abs_errorreturnsolution_fitness

Create an Instance of thepygad.GA Class

The fifth step is to instantiate thepygad.GA class. Note how theinitial_population parameter is assigned to the initial weights ofthe Keras models.

For more information, please check theparameters this classaccepts.

# Prepare the PyGAD parameters. Check the documentation for more information: https://pygad.readthedocs.io/en/latest/pygad.html#pygad-ga-classnum_generations=250# Number of generations.num_parents_mating=5# Number of solutions to be selected as parents in the mating pool.initial_population=keras_ga.population_weights# Initial population of network weightsga_instance=pygad.GA(num_generations=num_generations,num_parents_mating=num_parents_mating,initial_population=initial_population,fitness_func=fitness_func,on_generation=on_generation)

Run the Genetic Algorithm

The sixth and last step is to run the genetic algorithm by calling therun() method.

ga_instance.run()

After the PyGAD completes its execution, then there is a figure thatshows how the fitness value changes by generation. Call theplot_fitness() method to show the figure.

ga_instance.plot_fitness(title="PyGAD & Keras - Iteration vs. Fitness",linewidth=4)

Here is the figure.

To get information about the best solution found by PyGAD, use thebest_solution() method.

# Returning the details of the best solution.solution,solution_fitness,solution_idx=ga_instance.best_solution()print(f"Fitness value of the best solution ={solution_fitness}")print(f"Index of the best solution :{solution_idx}")
Fitnessvalueofthebestsolution=72.77768757825352Indexofthebestsolution:0

The next code makes prediction using thepredict() function toreturn the model predictions based on the best solution.

# Fetch the parameters of the best solution.predictions=pygad.kerasga.predict(model=model,solution=solution,data=data_inputs)print(f"Predictions :\n{predictions}")
Predictions:[[0.09935353][0.63082725][1.2765523][2.4999595]]

The next code measures the trained model error.

mae=tensorflow.keras.losses.MeanAbsoluteError()abs_error=mae(data_outputs,predictions).numpy()print(f"Absolute Error :{abs_error}")
AbsoluteError:0.013740465

Example 2: XOR Binary Classification

The next code creates a Keras model to build the XOR binaryclassification problem. Let’s highlight the changes compared to theprevious example.

importtensorflow.kerasimportpygad.kerasgaimportnumpyimportpygaddeffitness_func(ga_instance,solution,sol_idx):globaldata_inputs,data_outputs,keras_ga,modelpredictions=pygad.kerasga.predict(model=model,solution=solution,data=data_inputs)bce=tensorflow.keras.losses.BinaryCrossentropy()solution_fitness=1.0/(bce(data_outputs,predictions).numpy()+0.00000001)returnsolution_fitnessdefon_generation(ga_instance):print(f"Generation ={ga_instance.generations_completed}")print(f"Fitness    ={ga_instance.best_solution()[1]}")# Build the keras model using the functional API.input_layer=tensorflow.keras.layers.Input(2)dense_layer=tensorflow.keras.layers.Dense(4,activation="relu")(input_layer)output_layer=tensorflow.keras.layers.Dense(2,activation="softmax")(dense_layer)model=tensorflow.keras.Model(inputs=input_layer,outputs=output_layer)# Create an instance of the pygad.kerasga.KerasGA class to build the initial population.keras_ga=pygad.kerasga.KerasGA(model=model,num_solutions=10)# XOR problem inputsdata_inputs=numpy.array([[0,0],[0,1],[1,0],[1,1]])# XOR problem outputsdata_outputs=numpy.array([[1,0],[0,1],[0,1],[1,0]])# Prepare the PyGAD parameters. Check the documentation for more information: https://pygad.readthedocs.io/en/latest/pygad.html#pygad-ga-classnum_generations=250# Number of generations.num_parents_mating=5# Number of solutions to be selected as parents in the mating pool.initial_population=keras_ga.population_weights# Initial population of network weights.# Create an instance of the pygad.GA classga_instance=pygad.GA(num_generations=num_generations,num_parents_mating=num_parents_mating,initial_population=initial_population,fitness_func=fitness_func,on_generation=on_generation)# Start the genetic algorithm evolution.ga_instance.run()# After the generations complete, some plots are showed that summarize how the outputs/fitness values evolve over generations.ga_instance.plot_fitness(title="PyGAD & Keras - Iteration vs. Fitness",linewidth=4)# Returning the details of the best solution.solution,solution_fitness,solution_idx=ga_instance.best_solution()print(f"Fitness value of the best solution ={solution_fitness}")print(f"Index of the best solution :{solution_idx}")# Make predictions based on the best solution.predictions=pygad.kerasga.predict(model=model,solution=solution,data=data_inputs)print(f"Predictions :\n{predictions}")# Calculate the binary crossentropy for the trained model.bce=tensorflow.keras.losses.BinaryCrossentropy()print("Binary Crossentropy : ",bce(data_outputs,predictions).numpy())# Calculate the classification accuracy for the trained model.ba=tensorflow.keras.metrics.BinaryAccuracy()ba.update_state(data_outputs,predictions)accuracy=ba.result().numpy()print(f"Accuracy :{accuracy}")

Compared to the previous regression example, here are the changes:

  • The Keras model is changed according to the nature of the problem.Now, it has 2 inputs and 2 outputs with an in-between hidden layer of4 neurons.

# Build the keras model using the functional API.input_layer=tensorflow.keras.layers.Input(2)dense_layer=tensorflow.keras.layers.Dense(4,activation="relu")(input_layer)output_layer=tensorflow.keras.layers.Dense(2,activation="softmax")(dense_layer)model=tensorflow.keras.Model(inputs=input_layer,outputs=output_layer)

  • The train data is changed. Note that the output of each sample is a1D vector of 2 values, 1 for each class.

# XOR problem inputsdata_inputs=numpy.array([[0,0],[0,1],[1,0],[1,1]])# XOR problem outputsdata_outputs=numpy.array([[1,0],[0,1],[0,1],[1,0]])

  • The fitness value is calculated based on the binary cross entropy.

bce=tensorflow.keras.losses.BinaryCrossentropy()solution_fitness=1.0/(bce(data_outputs,predictions).numpy()+0.00000001)

After the previous code completes, the next figure shows how the fitnessvalue change by generation.

Here is some information about the trained model. Its fitness value is739.24, loss is0.0013527311 and accuracy is 100%.

Fitnessvalueofthebestsolution=739.2397344644013Indexofthebestsolution:7Predictions:[[9.9694413e-013.0558957e-03][5.0176249e-049.9949825e-01][1.8470541e-039.9815291e-01][9.9999976e-012.0538971e-07]]BinaryCrossentropy:0.0013527311Accuracy:1.0

Example 3: Image Multi-Class Classification (Dense Layers)

Here is the code.

importtensorflow.kerasimportpygad.kerasgaimportnumpyimportpygaddeffitness_func(ga_instance,solution,sol_idx):globaldata_inputs,data_outputs,keras_ga,modelpredictions=pygad.kerasga.predict(model=model,solution=solution,data=data_inputs)cce=tensorflow.keras.losses.CategoricalCrossentropy()solution_fitness=1.0/(cce(data_outputs,predictions).numpy()+0.00000001)returnsolution_fitnessdefon_generation(ga_instance):print(f"Generation ={ga_instance.generations_completed}")print(f"Fitness    ={ga_instance.best_solution()[1]}")# Build the keras model using the functional API.input_layer=tensorflow.keras.layers.Input(360)dense_layer=tensorflow.keras.layers.Dense(50,activation="relu")(input_layer)output_layer=tensorflow.keras.layers.Dense(4,activation="softmax")(dense_layer)model=tensorflow.keras.Model(inputs=input_layer,outputs=output_layer)# Create an instance of the pygad.kerasga.KerasGA class to build the initial population.keras_ga=pygad.kerasga.KerasGA(model=model,num_solutions=10)# Data inputsdata_inputs=numpy.load("../data/dataset_features.npy")# Data outputsdata_outputs=numpy.load("../data/outputs.npy")data_outputs=tensorflow.keras.utils.to_categorical(data_outputs)# Prepare the PyGAD parameters. Check the documentation for more information: https://pygad.readthedocs.io/en/latest/pygad.html#pygad-ga-classnum_generations=100# Number of generations.num_parents_mating=5# Number of solutions to be selected as parents in the mating pool.initial_population=keras_ga.population_weights# Initial population of network weights.# Create an instance of the pygad.GA classga_instance=pygad.GA(num_generations=num_generations,num_parents_mating=num_parents_mating,initial_population=initial_population,fitness_func=fitness_func,on_generation=on_generation)# Start the genetic algorithm evolution.ga_instance.run()# After the generations complete, some plots are showed that summarize how the outputs/fitness values evolve over generations.ga_instance.plot_fitness(title="PyGAD & Keras - Iteration vs. Fitness",linewidth=4)# Returning the details of the best solution.solution,solution_fitness,solution_idx=ga_instance.best_solution()print(f"Fitness value of the best solution ={solution_fitness}")print(f"Index of the best solution :{solution_idx}")# Make predictions based on the best solution.predictions=pygad.kerasga.predict(model=model,solution=solution,data=data_inputs)# print(f"Predictions : \n{predictions}")# Calculate the categorical crossentropy for the trained model.cce=tensorflow.keras.losses.CategoricalCrossentropy()print(f"Categorical Crossentropy :{cce(data_outputs,predictions).numpy()}")# Calculate the classification accuracy for the trained model.ca=tensorflow.keras.metrics.CategoricalAccuracy()ca.update_state(data_outputs,predictions)accuracy=ca.result().numpy()print(f"Accuracy :{accuracy}")

Compared to the previous binary classification example, this example hasmultiple classes (4) and thus the loss is measured using categoricalcross entropy.

cce=tensorflow.keras.losses.CategoricalCrossentropy()solution_fitness=1.0/(cce(data_outputs,predictions).numpy()+0.00000001)

Prepare the Training Data

Before building and training neural networks, the training data (inputand output) needs to be prepared. The inputs and the outputs of thetraining data are NumPy arrays.

The data used in this example is available as 2 files:

  1. dataset_features.npy:Data inputs.https://github.com/ahmedfgad/NumPyANN/blob/master/dataset_features.npy

  2. outputs.npy:Class labels.https://github.com/ahmedfgad/NumPyANN/blob/master/outputs.npy

The data consists of 4 classes of images. The image shape is(100,100,3). The number of training samples is 1962. The featurevector extracted from each image has a length 360.

Simply download these 2 files and read them according to the next code.Note that the class labels are one-hot encoded using thetensorflow.keras.utils.to_categorical() function.

importnumpydata_inputs=numpy.load("../data/dataset_features.npy")data_outputs=numpy.load("../data/outputs.npy")data_outputs=tensorflow.keras.utils.to_categorical(data_outputs)

The next figure shows how the fitness value changes.

Here are some statistics about the trained model.

Fitnessvalueofthebestsolution=4.197464252185969Indexofthebestsolution:0CategoricalCrossentropy:0.23823906Accuracy:0.9852192

Example 4: Image Multi-Class Classification (Conv Layers)

Compared to the previous example that uses only dense layers, thisexample uses convolutional layers to classify the same dataset.

Here is the complete code.

importtensorflow.kerasimportpygad.kerasgaimportnumpyimportpygaddeffitness_func(ga_instance,solution,sol_idx):globaldata_inputs,data_outputs,keras_ga,modelpredictions=pygad.kerasga.predict(model=model,solution=solution,data=data_inputs)cce=tensorflow.keras.losses.CategoricalCrossentropy()solution_fitness=1.0/(cce(data_outputs,predictions).numpy()+0.00000001)returnsolution_fitnessdefon_generation(ga_instance):print(f"Generation ={ga_instance.generations_completed}")print(f"Fitness    ={ga_instance.best_solution()[1]}")# Build the keras model using the functional API.input_layer=tensorflow.keras.layers.Input(shape=(100,100,3))conv_layer1=tensorflow.keras.layers.Conv2D(filters=5,kernel_size=7,activation="relu")(input_layer)max_pool1=tensorflow.keras.layers.MaxPooling2D(pool_size=(5,5),strides=5)(conv_layer1)conv_layer2=tensorflow.keras.layers.Conv2D(filters=3,kernel_size=3,activation="relu")(max_pool1)flatten_layer=tensorflow.keras.layers.Flatten()(conv_layer2)dense_layer=tensorflow.keras.layers.Dense(15,activation="relu")(flatten_layer)output_layer=tensorflow.keras.layers.Dense(4,activation="softmax")(dense_layer)model=tensorflow.keras.Model(inputs=input_layer,outputs=output_layer)# Create an instance of the pygad.kerasga.KerasGA class to build the initial population.keras_ga=pygad.kerasga.KerasGA(model=model,num_solutions=10)# Data inputsdata_inputs=numpy.load("../data/dataset_inputs.npy")# Data outputsdata_outputs=numpy.load("../data/dataset_outputs.npy")data_outputs=tensorflow.keras.utils.to_categorical(data_outputs)# Prepare the PyGAD parameters. Check the documentation for more information: https://pygad.readthedocs.io/en/latest/pygad.html#pygad-ga-classnum_generations=200# Number of generations.num_parents_mating=5# Number of solutions to be selected as parents in the mating pool.initial_population=keras_ga.population_weights# Initial population of network weights.# Create an instance of the pygad.GA classga_instance=pygad.GA(num_generations=num_generations,num_parents_mating=num_parents_mating,initial_population=initial_population,fitness_func=fitness_func,on_generation=on_generation)# Start the genetic algorithm evolution.ga_instance.run()# After the generations complete, some plots are showed that summarize how the outputs/fitness values evolve over generations.ga_instance.plot_fitness(title="PyGAD & Keras - Iteration vs. Fitness",linewidth=4)# Returning the details of the best solution.solution,solution_fitness,solution_idx=ga_instance.best_solution()print(f"Fitness value of the best solution ={solution_fitness}")print(f"Index of the best solution :{solution_idx}")# Make predictions based on the best solution.predictions=pygad.kerasga.predict(model=model,solution=solution,data=data_inputs)# print(f"Predictions : \n{predictions}")# Calculate the categorical crossentropy for the trained model.cce=tensorflow.keras.losses.CategoricalCrossentropy()print(f"Categorical Crossentropy :{cce(data_outputs,predictions).numpy()}")# Calculate the classification accuracy for the trained model.ca=tensorflow.keras.metrics.CategoricalAccuracy()ca.update_state(data_outputs,predictions)accuracy=ca.result().numpy()print(f"Accuracy :{accuracy}")

Compared to the previous example, the only change is that thearchitecture uses convolutional and max-pooling layers. The shape ofeach input sample is 100x100x3.

# Build the keras model using the functional API.input_layer=tensorflow.keras.layers.Input(shape=(100,100,3))conv_layer1=tensorflow.keras.layers.Conv2D(filters=5,kernel_size=7,activation="relu")(input_layer)max_pool1=tensorflow.keras.layers.MaxPooling2D(pool_size=(5,5),strides=5)(conv_layer1)conv_layer2=tensorflow.keras.layers.Conv2D(filters=3,kernel_size=3,activation="relu")(max_pool1)flatten_layer=tensorflow.keras.layers.Flatten()(conv_layer2)dense_layer=tensorflow.keras.layers.Dense(15,activation="relu")(flatten_layer)output_layer=tensorflow.keras.layers.Dense(4,activation="softmax")(dense_layer)model=tensorflow.keras.Model(inputs=input_layer,outputs=output_layer)

Prepare the Training Data

The data used in this example is available as 2 files:

  1. dataset_inputs.npy:Data inputs.https://github.com/ahmedfgad/NumPyCNN/blob/master/dataset_inputs.npy

  2. dataset_outputs.npy:Class labels.https://github.com/ahmedfgad/NumPyCNN/blob/master/dataset_outputs.npy

The data consists of 4 classes of images. The image shape is(100,100,3) and there are 20 images per class for a total of 80training samples. For more information about the dataset, check theReading theDatasection of thepygad.cnn module.

Simply download these 2 files and read them according to the next code.Note that the class labels are one-hot encoded using thetensorflow.keras.utils.to_categorical() function.

importnumpydata_inputs=numpy.load("../data/dataset_inputs.npy")data_outputs=numpy.load("../data/dataset_outputs.npy")data_outputs=tensorflow.keras.utils.to_categorical(data_outputs)

The next figure shows how the fitness value changes.

Here are some statistics about the trained model. The model accuracy is75% after the 200 generations. Note that just running the code again maygive different results.

Fitnessvalueofthebestsolution=2.7462310258668805Indexofthebestsolution:0CategoricalCrossentropy:0.3641354Accuracy:0.75

To improve the model performance, you can do the following:

  • Add more layers

  • Modify the existing layers.

  • Use different parameters for the layers.

  • Use different parameters for the genetic algorithm (e.g. number ofsolution, number of generations, etc)

Example 5: Image Classification using Data Generator

This example uses the image data generatortensorflow.keras.preprocessing.image.ImageDataGenerator to feed datato the model. Instead of reading all the data in the memory, the datagenerator generates the data needed by the model and only save it in thememory instead of saving all the data. This frees the memory but addsmore computational time.

importtensorflowastfimporttensorflow.kerasimportpygad.kerasgaimportpygaddeffitness_func(ga_instanse,solution,sol_idx):globaltrain_generator,data_outputs,keras_ga,modelpredictions=pygad.kerasga.predict(model=model,solution=solution,data=train_generator)cce=tensorflow.keras.losses.CategoricalCrossentropy()solution_fitness=1.0/(cce(data_outputs,predictions).numpy()+0.00000001)returnsolution_fitnessdefon_generation(ga_instance):print("Generation ={ga_instance.generations_completed}")print("Fitness    = {ga_instance.best_solution(ga_instance.last_generation_fitness)[1]}")# The dataset path.dataset_path=r'../data/Skin_Cancer_Dataset'num_classes=2img_size=224# Create a simple CNN. This does not gurantee high classification accuracy.model=tf.keras.models.Sequential()model.add(tf.keras.layers.Input(shape=(img_size,img_size,3)))model.add(tf.keras.layers.Conv2D(32,(3,3),activation="relu",padding="same"))model.add(tf.keras.layers.MaxPooling2D((2,2)))model.add(tf.keras.layers.Flatten())model.add(tf.keras.layers.Dropout(rate=0.2))model.add(tf.keras.layers.Dense(num_classes,activation="softmax"))# Create an instance of the pygad.kerasga.KerasGA class to build the initial population.keras_ga=pygad.kerasga.KerasGA(model=model,num_solutions=10)data_generator=tf.keras.preprocessing.image.ImageDataGenerator()train_generator=data_generator.flow_from_directory(dataset_path,class_mode='categorical',target_size=(224,224),batch_size=32,shuffle=False)# train_generator.class_indicesdata_outputs=tf.keras.utils.to_categorical(train_generator.labels)# Check the documentation for more information about the parameters: https://pygad.readthedocs.io/en/latest/pygad.html#pygad-ga-classinitial_population=keras_ga.population_weights# Initial population of network weights.# Create an instance of the pygad.GA classga_instance=pygad.GA(num_generations=10,num_parents_mating=5,initial_population=initial_population,fitness_func=fitness_func,on_generation=on_generation)# Start the genetic algorithm evolution.ga_instance.run()# After the generations complete, some plots are showed that summarize how the outputs/fitness values evolve over generations.ga_instance.plot_fitness(title="PyGAD & Keras - Iteration vs. Fitness",linewidth=4)# Returning the details of the best solution.solution,solution_fitness,solution_idx=ga_instance.best_solution(ga_instance.last_generation_fitness)print(f"Fitness value of the best solution ={solution_fitness}")print(f"Index of the best solution :{solution_idx}")predictions=pygad.kerasga.predict(model=model,solution=solution,data=train_generator)# print(f"Predictions : \n{predictions}")# Calculate the categorical crossentropy for the trained model.cce=tensorflow.keras.losses.CategoricalCrossentropy()print(f"Categorical Crossentropy :{cce(data_outputs,predictions).numpy()}")# Calculate the classification accuracy for the trained model.ca=tensorflow.keras.metrics.CategoricalAccuracy()ca.update_state(data_outputs,predictions)accuracy=ca.result().numpy()print(f"Accuracy :{accuracy}")