Disclosure of Invention
The invention aims to provide an image identification method and system based on a brain-like computing platform, which reduces the energy consumption of image identification in the computing of a hardware platform.
In order to achieve the purpose, the invention provides the following scheme:
an image recognition method based on a brain-like computing platform comprises the following steps:
constructing a pulse neural network; the impulse neural network comprises a convolutional layer, an approximate maximum pooling layer and a classifier which are connected in sequence, wherein the approximate maximum pooling layer comprises a pre-synaptic neuron cluster and a post-synaptic neuron cluster, the pre-synaptic neuron cluster comprises a plurality of neuron groups which are arranged in sequence, the number of pre-synaptic neurons in each neuron group is the same, pre-synaptic neurons in the ith neuron group in the pre-synaptic neuron cluster are connected with the ith post-synaptic neurons in the post-synaptic neuron cluster, and when the accumulation of impulses input by the pre-synaptic neurons in the ith neuron group exceeds a set threshold value, the ith post-synaptic neurons emit one impulse;
training the image recognition of the impulse neural network to obtain a trained impulse neural network;
deploying the trained impulse neural network to a brain-like computing platform;
and image recognition is carried out by adopting a pulse neural network deployed on a brain-like computing platform.
Optionally, the input of the spiking neural network is a one-dimensional pulse sequence.
Optionally, there is a full connection between the approximate maximum pooling layer and the classifier.
Optionally, the training of image recognition on the impulse neural network to obtain the trained impulse neural network specifically includes:
and training synaptic connection weights between the neurons from the convolutional layer to the approximate maximum pooling layer and synaptic connection weights between the neurons from the approximate maximum pooling layer to the classifier in a back propagation mode, and storing the trained weights.
Optionally, the set threshold is 2.
The invention also discloses an image recognition system based on the brain-like computing platform, which comprises:
the impulse neural network construction module is used for constructing an impulse neural network; the impulse neural network comprises a convolutional layer, an approximate maximum pooling layer and a classifier which are connected in sequence, wherein the approximate maximum pooling layer comprises a pre-synaptic neuron cluster and a post-synaptic neuron cluster, the pre-synaptic neuron cluster comprises a plurality of neuron groups which are arranged in sequence, the number of pre-synaptic neurons in each neuron group is the same, pre-synaptic neurons in the ith neuron group in the pre-synaptic neuron cluster are connected with the ith post-synaptic neurons in the post-synaptic neuron cluster, and when the input impulse accumulation of the pre-synaptic neurons in the ith neuron group exceeds a set threshold value, the ith post-synaptic neurons emit one impulse;
the impulse neural network training module is used for training the impulse neural network for image recognition to obtain a trained impulse neural network;
the impulse neural network deployment module is used for deploying the trained impulse neural network to a brain-like computing platform;
and the image recognition module is used for performing image recognition by adopting a pulse neural network deployed on the brain-like computing platform.
Optionally, the input of the spiking neural network is a one-dimensional pulse sequence.
Optionally, there is a full connection between the approximate maximum pooling layer and the classifier.
Optionally, the spiking neural network training module specifically includes:
and the impulse neural network training unit is used for training the synaptic connection weight between the convolutional layer and the neuron of the approximate maximum pooling layer and the synaptic connection weight between the approximate maximum pooling layer and the neuron of the classifier in a back propagation mode, and storing the trained weights.
Optionally, the set threshold is 2.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the method realizes the maximum pooling on a brain-like computing platform by constructing the pulse neural network similar to the convolutional neural network, approximating the pre-synaptic neuron cluster and the post-synaptic neuron cluster in the maximum pooling layer and setting the threshold, reduces the deviation of an estimated mean value caused by parameter errors of the convolutional layer when extracting the characteristics, and simultaneously realizes the dimension reduction, reduces the operation amount and reduces the energy consumption of image recognition when a hardware platform calculates.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide an image identification method and system based on a brain-like computing platform, which reduces the energy consumption of image identification in the computing of a hardware platform.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a schematic flow chart of an image recognition method based on a brain-like computing platform according to the present invention, and as shown in fig. 1, an image recognition method based on a brain-like computing platform includes:
step 101: constructing a pulse neural network; the impulse neural network comprises a convolutional layer, an approximate maximum pooling layer and a classifier which are sequentially connected, wherein the approximate maximum pooling layer comprises a pre-synaptic neuron cluster and a post-synaptic neuron cluster, the pre-synaptic neuron cluster comprises a plurality of neuron groups which are sequentially arranged, the number of pre-synaptic neurons in each neuron group is the same, pre-synaptic neurons in the ith neuron group in the pre-synaptic neuron cluster are all connected with the ith post-synaptic neurons in the post-synaptic neuron cluster, and when the accumulated pulse input by the pre-synaptic neurons in the ith neuron group exceeds a set threshold, the ith post-synaptic neurons issue one pulse.
The presynaptic neuron clusters approximating the maximum pooling layer are connected with the convolution layer, and the postsynaptic neuron clusters approximating the maximum pooling layer are connected with the classifier.
The approximate maximum pooling layer is used for sampling the input by setting a one-dimensional sliding window, and areas sampled twice continuously do not overlap. The sliding step length of the one-dimensional sliding window is set to be the same as the number of presynaptic neurons in the neuron group.
The input of the impulse neural network is a one-dimensional pulse sequence. Specifically, a binary image of the image to be identified is obtained, and pulse coding is performed on the binary image. For example, the input image is a 28 × 28 binary map, i.e., the input image is a matrix of size 28 × 28. And transforming the matrix into a column of vectors with the length of 784 and 0 and 1, and finishing the encoding.
An encoded pulse sequence, such as the sequence of numbers in fig. 2, is input to a spiking neural network equivalent to a convolutional neural network.
The main calculation in the impulse neural network includes three parts of convolution calculation, approximate maximum pooling calculation and classification calculation, as shown in fig. 2, the impulse neural network performs convolution, approximate maximum pooling and classification operation through a convolution layer, an approximate maximum pooling layer and a classifier, respectively. And extracting the characteristics of the pulse sequence through convolution and pooling calculation, and then entering a classifier for printing letter identification. In the invention, the convolution layer and the approximate maximum pooling layer both adopt a many-to-one connection mode to achieve the purposes of convolution and pooling calculation, and the approximate maximum pooling layer and the classification layer (classifier) adopt full connection.
The threshold value is set to 2.
Step 102: and carrying out image recognition training on the impulse neural network to obtain the trained impulse neural network.
The pulse neural network performs weight training of synaptic connections between neurons from a convolutional layer to an approximate pooling layer and from the approximate pooling layer to a classification layer on a CPU in a supervised learning mode, namely a back propagation mode, and then takes out and stores the trained weights in a CPU file for later calling.
Wherein,step 102 specifically comprises:
and training synaptic connection weights between the convolutional layers and the neurons approximating the maximum pooling layer and synaptic connection weights between the neurons approximating the maximum pooling layer and the classifiers by a back propagation mode, and storing the trained weights.
Step 103: and deploying the trained spiking neural network to a brain-like computing platform.
Step 103 specifically comprises equivalently converting the convolutional layer into a form of a pulse neural network by using an ann2snn library in a SpikingJelly software framework, completely deploying the pulse neural network on a brain-like computing platform, taking out weights trained and stored in a CPU file, and loading the weights into the brain-like computing platform for testing.
When the impulse neural network is deployed on a hardware platform, the convolution is performed in a mode similar to a convolution neural network, and then approximate maximum pooling is adopted to replace the maximum pooling used in the original convolution neural network. The approximate maximum pooling implementation method is as follows:
1) first the sliding window size and the sliding step size are determined. Since maximum pooling is typically done in a general pooling manner, i.e., the two sampled regions of the sliding window do not overlap, if the sliding window size is 4 (i.e., four neurons are contained within the sliding window), the step size is also 4 (i.e., assumingneuron numbers 1, 2, … …, neurons numbered 1, 2, 3, 4 are taken for the first time, neurons numbered 5, 6, 7, 8 are taken for the second time, and so on). Unlike convolutional neural networks, where the two-dimensional sliding window of the pooling layer is used, in impulse neural networks, one-dimensional sliding windows, i.e., in the form of vectors, are typically used.
2) Creating a Population on the brain-like computing platform. Approximately maximal pooling requires the Population of two neurons to be achieved, Population of pre-synaptic neurons and Population of post-synaptic neurons, respectively. The manner of connection between two posts will be illustrated in step 3) by way of example.
3) A connection is established. As exemplified in step 1), when the sliding window size is 4 and the step size is also 4, the presynaptic neurons numbered 1, 2, 3, 4 are all connected to the postsynaptic neuron numbered 1; the pre-synaptic neurons numbered 5, 6, 7, 8 are connected to the post-synaptic neuron numbered 2, and so on. The synaptic weight for the connection is set to a fixed value of 1, as shown in FIG. 3.
4) A threshold value is set. When the cumulative number of impulses of the four pre-synaptic neurons exceeds a set Threshold on the post-synaptic neuron, the post-synaptic neuron sends an impulse once to realize the approximation of the maximal pooling on the spiking neural network, as shown in fig. 3, in which thenumbers 1, 2, 3, 4, 5, 6, 7, 8 in the first column are pre-synaptic neurons, thenumbers 1, 2 in the second column are post-synaptic neurons, and Threshold represents a set Threshold.
And the output value of the approximate maximum pooling layer is the input of the classification layer, and the pulse sequence generated by the approximate maximum pooling layer is input into the classification layer so as to identify the printing letters. The approximate maximum pooling layer and the classification layer adopt a full connection mode, and the synaptic weight is a weight which is trained and stored in the CPU in advance.
Step 104: and image recognition is carried out by adopting a pulse neural network deployed on a brain-like computing platform.
There is a full connection between the approximate maximum pooling layer and the classifier.
The invention deploys the pulse neural network converted from the convolutional neural network on a brain-like computing platform, approximates the maximum pooling, and regulates and controls whether the postsynaptic neuron sends pulses or not by utilizing the threshold value, thereby realizing the operation of the pulse convolutional neural network on a hardware platform. In the pulse convolution neural network, the maximum pooling is an important ring, the maximum pooling can finish extraction and processing of the most significant information features of data, meanwhile, the dimensionality reduction is realized, the calculation amount of a hardware platform is reduced, and therefore, the energy consumption is reduced, and the recognition task of printing letters can be conveniently finished on a brain-like computing platform.
The invention replaces the convolutional neural network with the corresponding impulse neural network and deploys the convolutional neural network on a brain-like computing platform to realize the recognition task of printing letters. Compared with the task completed by a CPU, the invention utilizes a brain-like computing platform, has lower power consumption and can still keep the identification accuracy.
The approximate maximum pooling mode adopted in the invention is the approximate maximum pooling of the impulse neural network realized on the brain-like computing platform. The maximum pooling is widely applied to convolutional neural networks and is used for reducing the deviation of an estimated mean value caused by parameter errors of convolutional layers when characteristics are extracted. Meanwhile, the dimension reduction can be realized, the calculation amount is reduced, and the energy consumption of the hardware platform during calculation is reduced.
In a spiking neural network, maximal pooling plays a similar role. However, because of the particularity of the spiking neural network, that is, the connections of neurons must be determined in advance, and the maximal pooling realized in the convolutional neural network is to establish the connections after the maximum value is selected, the spiking neural network cannot be directly applied; and brain-like computing platforms cannot be counter-propagated. The above two points determine that only few ways of achieving the maximum pooling in the impulse neural network can be implemented on a hardware platform.
The invention realizes approximate maximum pooling on a hardware platform, and has the following main advantages:
the invention only needs to establish the Populations of two neurons, and is different from other schemes for realizing the maximum pooling in the impulse neural network, and often needs to establish three Populations. The method effectively saves the computing space of the brain-like computing platform and is beneficial to realizing the pulse neural network for real-time computing.
The pulse passing through the threshold value is adjusted, so that the method is more consistent with biological characteristics, and the method can be effectively realized on a brain-like computing platform. Generally, a maximum pooling method used in a spiking neural network usually needs an intermediate layer, where the intermediate layer is used to record the number of spiking pulses or the accumulated voltage, and then connects the neuron with the maximum number of spiking pulses or the maximum accumulated voltage to the maximum pooled output layer, but the current brain-like computing platforms create connections in advance, so that the implementation of such a manner on a hardware platform is difficult. Furthermore, using the near-maximal pooling employed in the method of the present invention, more information can be retained than the maximal pooling method typically used in spiking neural networks.
Through the maximum pooling, the data volume of the lower layer can be effectively reduced, so that the calculation amount is reduced, and the calculation efficiency is improved.
Meanwhile, the maximal pooling is difficult to realize when the impulse neural network is deployed on a brain-like computing platform, so that the work of converting the conventional convolutional neural network in the image recognition field into the impulse neural network cannot be smoothly developed in an end-to-end mode all the time. And the brain-like computing platform has the advantage of low power consumption, and can complete the computation of large computation amount with low cost. The present method provides a practical solution for this.
Fig. 4 is a schematic structural diagram of an image recognition system based on a brain-like computing platform according to the present invention, and as shown in fig. 4, an image recognition system based on a brain-like computing platform includes:
a spiking neuralnetwork constructing module 201, configured to construct a spiking neural network; the impulse neural network comprises a convolutional layer, an approximate maximum pooling layer and a classifier which are connected in sequence, wherein the approximate maximum pooling layer comprises a pre-synaptic neuron cluster and a post-synaptic neuron cluster, the pre-synaptic neuron cluster comprises a plurality of neuron groups which are arranged in sequence, the number of pre-synaptic neurons in each neuron group is the same, pre-synaptic neurons in the ith neuron group in the pre-synaptic neuron cluster are all connected with the ith post-synaptic neurons in the post-synaptic neuron cluster, when the accumulated number of pulses input by the pre-synaptic neurons in the ith neuron group exceeds a set threshold value, the ith post-synaptic neurons issue one-time pulses, the pre-synaptic neurons in the pre-synaptic neuron cluster are connected with the convolutional layer, and the post-synaptic neurons are connected with the classifier.
And the impulse neuralnetwork training module 202 is configured to train the impulse neural network for image recognition, so as to obtain a trained impulse neural network.
And the spiking neuralnetwork deployment module 203 is used for deploying the trained spiking neural network to the brain-like computing platform.
And theimage recognition module 204 is used for performing image recognition by adopting a pulse neural network deployed on a brain-like computing platform.
The input of the impulse neural network is a one-dimensional pulse sequence.
There is a full connection between the approximate maximum pooling layer and the classifier.
The spiking neuralnetwork training module 202 specifically includes:
and the impulse neural network training unit is used for training the synaptic connection weight between the convolutional layer and the neuron approximating the maximum pooling layer and the synaptic connection weight between the neuron approximating the maximum pooling layer and the neuron of the classifier in a back propagation mode, and storing each trained weight.
The threshold value is set to 2.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.