pygad.cnn.Input2D Class¶

Thepygad.cnn.Input2D class creates the input layer for theconvolutional neural network. For each network, there is only a singleinput layer. The network architecture must start with an input layer.

This class has no methods or class attributes. All it has is aconstructor that accepts a parameter namedinput_shape representingthe shape of the input.

The instances from theInput2D class has the following attributes:

1. input_shape: The shape of the input to the pygad.cnn.

2. layer_output_size

Here is an example of building an input layer with shape(50,50,3).

input_layer=pygad.cnn.Input2D(input_shape=(50,50,3))

Here is how to access the attributes within the instance of thepygad.cnn.Input2D class.

input_shape=input_layer.input_shapelayer_output_size=input_layer.layer_output_sizeprint("Input2D Input shape =",input_shape)print("Input2D Output shape =",layer_output_size)

This is everything about the input layer.

pygad.cnn.Conv2D Class¶

Using thepygad.cnn.Conv2D class, convolution (conv) layers can becreated. To create a convolution layer, just create a new instance ofthe class. The constructor accepts the following parameters:

• num_filters: Number of filters.

• kernel_size: Filter kernel size.

• previous_layer: A reference to the previous layer. Using theprevious_layer attribute, a linked list is created that connectsall network layers. For more information about this attribute, pleasecheck theprevious_layer attribute section of thepygad.nnmodule documentation.

• activation_function=None: A string representing the activationfunction to be used in this layer. Defaults toNone which means noactivation function is applied while applying the convolution layer.An activation layer can be added separately in this case. Thesupported activation functions in the conv layer arerelu andsigmoid.

Within the constructor, the accepted parameters are used as instanceattributes. Besides the parameters, some new instance attributes arecreated which are:

• filter_bank_size: Size of the filter bank in this layer.

• initial_weights: The initial weights for the conv layer.

• trained_weights: The trained weights of the conv layer. Thisattribute is initialized by the value in theinitial_weightsattribute.

• layer_input_size

• layer_output_size

• layer_output

Here is an example for creating a conv layer with 2 filters and a kernelsize of 3. Note that theprevious_layer parameter is assigned to theinput layerinput_layer.

conv_layer=pygad.cnn.Conv2D(num_filters=2,kernel_size=3,previous_layer=input_layer,activation_function=None)

Here is how to access some attributes in the dense layer:

filter_bank_size=conv_layer.filter_bank_sizeconv_initail_weights=conv_layer.initial_weightsprint("Filter bank size attributes =",filter_bank_size)print("Initial weights of the conv layer :",conv_initail_weights)

Becauseconv_layer holds a reference to the input layer, then thenumber of input neurons can be accessed.

input_layer=conv_layer.previous_layerinput_shape=input_layer.num_neuronsprint("Input shape =",input_shape)

Here is another conv layer where itsprevious_layer attribute pointsto the previously created conv layer and it uses theReLU activationfunction.

conv_layer2=pygad.cnn.Conv2D(num_filters=2,kernel_size=3,previous_layer=conv_layer,activation_function="relu")

Becauseconv_layer2 holds a reference toconv_layer in itsprevious_layer attribute, then the attributes inconv_layer canbe accessed.

conv_layer=conv_layer2.previous_layerfilter_bank_size=conv_layer.filter_bank_sizeprint("Filter bank size attributes =",filter_bank_size)

After getting the reference toconv_layer, we can use it to accessthe number of input neurons.

conv_layer=conv_layer2.previous_layerinput_layer=conv_layer.previous_layerinput_shape=input_layer.num_neuronsprint("Input shape =",input_shape)

pygad.cnn.MaxPooling2D Class¶

Thepygad.cnn.MaxPooling2D class builds a max pooling layer for theCNN architecture. The constructor of this class accepts the followingparameter:

• pool_size: Size of the window.

• previous_layer: A reference to the previous layer in the CNNarchitecture.

• stride=2: A stride that default to 2.

Within the constructor, the accepted parameters are used as instanceattributes. Besides the parameters, some new instance attributes arecreated which are:

• layer_input_size

• layer_output_size

• layer_output

pygad.cnn.AveragePooling2D Class¶

Thepygad.cnn.AveragePooling2D class is similar to thepygad.cnn.MaxPooling2D class except that it applies the max poolingoperation rather than average pooling.

pygad.cnn.Flatten Class¶

Thepygad.cnn.Flatten class implements the flatten layer whichconverts the output of the previous layer into a 1D vector. Theconstructor accepts only theprevious_layer parameter.

The following instance attributes exist:

• previous_layer

• layer_input_size

• layer_output_size

• layer_output

pygad.cnn.ReLU Class¶

Thepygad.cnn.ReLU class implements the ReLU layer which applies theReLU activation function to the output of the previous layer.

The constructor accepts only theprevious_layer parameter.

The following instance attributes exist:

• previous_layer

• layer_input_size

• layer_output_size

• layer_output

pygad.cnn.Sigmoid Class¶

Thepygad.cnn.Sigmoid class is similar to thepygad.cnn.ReLUclass except that it applies the sigmoid function rather than the ReLUfunction.

pygad.cnn.Dense Class¶

Thepygad.cnn.Dense class implement the dense layer. Its constructoraccepts the following parameters:

• num_neurons: Number of neurons in the dense layer.

• previous_layer: A reference to the previous layer.

• activation_function: A string representing the activation functionto be used in this layer. Defaults to"sigmoid". Currently, thesupported activation functions in the dense layer are"sigmoid","relu", andsoftmax.

Within the constructor, the accepted parameters are used as instanceattributes. Besides the parameters, some new instance attributes arecreated which are:

• initial_weights: The initial weights for the dense layer.

• trained_weights: The trained weights of the dense layer. Thisattribute is initialized by the value in theinitial_weightsattribute.

• layer_input_size

• layer_output_size

• layer_output

get_layers()¶

Creates a list of all layers in the CNN model. It accepts no parameters.

train()¶

Trains the CNN model.

Accepts the following parameters:

• train_inputs: Training data inputs.

• train_outputs: Training data outputs.

This method trains the CNN model according to the number of epochsspecified in the constructor of thepygad.cnn.Model class.

It is important to note that no learning algorithm is used for trainingthe pygad.cnn. Just the learning rate is used for making some changeswhich is better than leaving the weights unchanged.

feed_sample()¶

Feeds a sample in the CNN layers and returns results of the last layerin the pygad.cnn.

update_weights()¶

Updates the CNN weights using the learning rate. It is important to notethat no learning algorithm is used for training the pygad.cnn. Just thelearning rate is used for making some changes which is better thanleaving the weights unchanged.

predict()¶

Uses the trained CNN for making predictions.

Accepts the following parameter:

• data_inputs: The inputs to predict their label.

It returns a list holding the samples predictions.

summary()¶

Prints a summary of the CNN architecture.

Reading the Data¶

Before building the network architecture, the first thing to do is toprepare the data that will be used for training the network.

In this example, 4 classes of theFruits360 dataset are used forpreparing the training data. The 4 classes are:

1. AppleBraeburn:This class’s data is available athttps://github.com/ahmedfgad/NumPyANN/tree/master/apple

2. LemonMeyer:This class’s data is available athttps://github.com/ahmedfgad/NumPyANN/tree/master/lemon

3. Mango:This class’s data is available athttps://github.com/ahmedfgad/NumPyANN/tree/master/mango

4. Raspberry:This class’s data is available athttps://github.com/ahmedfgad/NumPyANN/tree/master/raspberry

Just 20 samples from each of the 4 classes are saved into a NumPy arrayavailable in thedataset_inputs.npyfile:https://github.com/ahmedfgad/NumPyCNN/blob/master/dataset_inputs.npy

The shape of this array is(80,100,100,3) where the shape of thesingle image is(100,100,3).

Thedataset_outputs.npyfile(https://github.com/ahmedfgad/NumPyCNN/blob/master/dataset_outputs.npy)has the class labels for the 4 classes:

1. AppleBraeburn:Class label is0

2. LemonMeyer:Class label is1

3. Mango:Class label is2

4. Raspberry:Class label is3

Simply, download and reach the 2 files to return the NumPy arraysaccording to the next 2 lines:

train_inputs=numpy.load("dataset_inputs.npy")train_outputs=numpy.load("dataset_outputs.npy")

After the data is prepared, next is to create the network architecture.

Building the Network Architecture¶

The input layer is created by instantiating thepygad.cnn.Input2Dclass according to the next code. A network can only have a single inputlayer.

importpygad.cnnsample_shape=train_inputs.shape[1:]input_layer=pygad.cnn.Input2D(input_shape=sample_shape)

After the input layer is created, next is to create a number of layerslayers according to the next code. Normally, the last dense layer isregarded as the output layer. Note that the output layer has a number ofneurons equal to the number of classes in the dataset which is 4.

conv_layer1=pygad.cnn.Conv2D(num_filters=2,kernel_size=3,previous_layer=input_layer,activation_function=None)relu_layer1=pygad.cnn.Sigmoid(previous_layer=conv_layer1)average_pooling_layer=pygad.cnn.AveragePooling2D(pool_size=2,previous_layer=relu_layer1,stride=2)conv_layer2=pygad.cnn.Conv2D(num_filters=3,kernel_size=3,previous_layer=average_pooling_layer,activation_function=None)relu_layer2=pygad.cnn.ReLU(previous_layer=conv_layer2)max_pooling_layer=pygad.cnn.MaxPooling2D(pool_size=2,previous_layer=relu_layer2,stride=2)conv_layer3=pygad.cnn.Conv2D(num_filters=1,kernel_size=3,previous_layer=max_pooling_layer,activation_function=None)relu_layer3=pygad.cnn.ReLU(previous_layer=conv_layer3)pooling_layer=pygad.cnn.AveragePooling2D(pool_size=2,previous_layer=relu_layer3,stride=2)flatten_layer=pygad.cnn.Flatten(previous_layer=pooling_layer)dense_layer1=pygad.cnn.Dense(num_neurons=100,previous_layer=flatten_layer,activation_function="relu")dense_layer2=pygad.cnn.Dense(num_neurons=4,previous_layer=dense_layer1,activation_function="softmax")

After the network architecture is prepared, the next step is to create aCNN model.

Building Model¶

The CNN model is created as an instance of thepygad.cnn.Modelclass. Here is an example.

model=pygad.cnn.Model(last_layer=dense_layer2,epochs=5,learning_rate=0.01)

After the model is created, a summary of the model architecture can beprinted.

Model Summary¶

Thesummary() method in thepygad.cnn.Model class prints asummary of the CNN model.

model.summary()

----------NetworkArchitecture----------<class'pygad.cnn.Conv2D'><class'pygad.cnn.Sigmoid'><class'pygad.cnn.AveragePooling2D'><class'pygad.cnn.Conv2D'><class'pygad.cnn.ReLU'><class'pygad.cnn.MaxPooling2D'><class'pygad.cnn.Conv2D'><class'pygad.cnn.ReLU'><class'pygad.cnn.AveragePooling2D'><class'pygad.cnn.Flatten'><class'pygad.cnn.Dense'><class'pygad.cnn.Dense'>----------------------------------------

Training the Network¶

After the model and the data are prepared, then the model can be trainedusing the thepygad.cnn.train() method.

model.train(train_inputs=train_inputs,train_outputs=train_outputs)

After training the network, the next step is to make predictions.

Making Predictions¶

Thepygad.cnn.predict() method uses the trained network for makingpredictions. Here is an example.

predictions=model.predict(data_inputs=train_inputs)

It is not expected to have high accuracy in the predictions because notraining algorithm is used.

Calculating Some Statistics¶

Based on the predictions the network made, some statistics can becalculated such as the number of correct and wrong predictions inaddition to the classification accuracy.

num_wrong=numpy.where(predictions!=train_outputs)[0]num_correct=train_outputs.size-num_wrong.sizeaccuracy=100*(num_correct/train_outputs.size)print(f"Number of correct classifications :{num_correct}.")print(f"Number of wrong classifications :{num_wrong.size}.")print(f"Classification accuracy :{accuracy}.")

It is very important to note that it is not expected that theclassification accuracy is high because no training algorithm is used.Please check the documentation of thepygad.gacnn module fortraining the CNN using the genetic algorithm.

Image Classification¶

This example is discussed in theSteps to Build a Convolutional NeuralNetwork section and its complete code is listed below.

Remember to either download or create thedataset_features.npyanddataset_outputs.npyfiles before running this code.

importnumpyimportpygad.cnn"""Convolutional neural network implementation using NumPyA tutorial that helps to get started (Building Convolutional Neural Network using NumPy from Scratch) available in these links:    https://www.linkedin.com/pulse/building-convolutional-neural-network-using-numpy-from-ahmed-gad    https://towardsdatascience.com/building-convolutional-neural-network-using-numpy-from-scratch-b30aac50e50a    https://www.kdnuggets.com/2018/04/building-convolutional-neural-network-numpy-scratch.htmlIt is also translated into Chinese: http://m.aliyun.com/yunqi/articles/585741"""train_inputs=numpy.load("dataset_inputs.npy")train_outputs=numpy.load("dataset_outputs.npy")sample_shape=train_inputs.shape[1:]num_classes=4input_layer=pygad.cnn.Input2D(input_shape=sample_shape)conv_layer1=pygad.cnn.Conv2D(num_filters=2,kernel_size=3,previous_layer=input_layer,activation_function=None)relu_layer1=pygad.cnn.Sigmoid(previous_layer=conv_layer1)average_pooling_layer=pygad.cnn.AveragePooling2D(pool_size=2,previous_layer=relu_layer1,stride=2)conv_layer2=pygad.cnn.Conv2D(num_filters=3,kernel_size=3,previous_layer=average_pooling_layer,activation_function=None)relu_layer2=pygad.cnn.ReLU(previous_layer=conv_layer2)max_pooling_layer=pygad.cnn.MaxPooling2D(pool_size=2,previous_layer=relu_layer2,stride=2)conv_layer3=pygad.cnn.Conv2D(num_filters=1,kernel_size=3,previous_layer=max_pooling_layer,activation_function=None)relu_layer3=pygad.cnn.ReLU(previous_layer=conv_layer3)pooling_layer=pygad.cnn.AveragePooling2D(pool_size=2,previous_layer=relu_layer3,stride=2)flatten_layer=pygad.cnn.Flatten(previous_layer=pooling_layer)dense_layer1=pygad.cnn.Dense(num_neurons=100,previous_layer=flatten_layer,activation_function="relu")dense_layer2=pygad.cnn.Dense(num_neurons=num_classes,previous_layer=dense_layer1,activation_function="softmax")model=pygad.cnn.Model(last_layer=dense_layer2,epochs=1,learning_rate=0.01)model.summary()model.train(train_inputs=train_inputs,train_outputs=train_outputs)predictions=model.predict(data_inputs=train_inputs)print(predictions)num_wrong=numpy.where(predictions!=train_outputs)[0]num_correct=train_outputs.size-num_wrong.sizeaccuracy=100*(num_correct/train_outputs.size)print(f"Number of correct classifications :{num_correct}.")print(f"Number of wrong classifications :{num_wrong.size}.")print(f"Classification accuracy :{accuracy}.")