Background
As is known, the performance of deep learning based on a convolutional neural network in the field of artificial intelligence exceeds that of the traditional machine learning method, the deep learning plays a leading role in the fields of image recognition, natural language processing, voice processing and the like, and particularly in the field of image classification, the performance of the most advanced model exceeds that of human beings.
However, szegdy et al found that the model became fragile when anti-turbulence was added to the image. At the same time, these perturbations are hardly distinguishable to humans. In the following years, a plurality of algorithms generated for the confrontation sample show that the confrontation image can obtain extremely high accuracy when attacking the normal model.
At present, most of attack algorithms aiming at deep learning models take an attack classification model under the condition of minimum disturbance as a target and limit l2Or l∞To ensure a range of variation for the perturbation addition. To achieve that the addition of the disturbance is not perceived by a human or does not affect the human's judgment of the image. However, few algorithms consider whether this perturbation-limiting approach is reasonable enough or effective enough.
In the Alitian pool game, by l∞To limit the criteria, a gradient-based algorithm is used to demonstrate the attack on the classification model. The maximum change value of each pixel point is 32 pixels. When a targeted classified attack is performed on images of snails, dung beetles and spiders, these images seem to become corn, electronic scales and dogs in human view.
Under the limitation of the 32-pixel change value, the original image may become recognized by human beings incorrectly after the image is modified, let alone by the classification model.
The main reason for this is that the magnitude of the modified pixel value does not represent the change of the modified image in human view, and the number of the disturbance pixel points has a larger influence on the human identification image than the value of the disturbance pixel points.
In conclusion, the method for the image classification neural network attack, which is effective and has small influence on human recognition, is found to have important research significance.
Disclosure of Invention
In order to overcome the defects of the method, the invention provides an image classification neural network attack method based on a Bezier curve, which can solve the problem that the modified image has a large influence on the human recognition effect and can effectively attack the image classification neural network. The specific method comprises the following steps:
step 1: building a plurality of image classification neural networks and loading pre-training parameters;
step 2: selecting one thousand image samples meeting the same distribution from the ImageNet data set, and carrying out parameter fine adjustment on the constructed neural network by using the image samples;
and step 3: randomly generating a Bezier curve on the basis of an original image;
and 4, step 4: searching an optimal curve in a large number of randomly generated Bezier curves by using a differential evolution algorithm;
and 5: adding the found optimal Bezier curve to the image sample, and misleading the image classification neural network to carry out misclassification on the image sample;
step 6: repeating the steps 3 to 5, and adding the optimal Bezier curve to all the images;
in the step 1, 3 classical image classification neural networks are respectively built: VGG16, ResNet50, densnet 201, and load the respective pre-training parameters.
Instep 2, a thousand images satisfying the same distribution are selected from the ImageNet data set, so as to better verify the effectiveness of the proposed method. And the samples are utilized to carry out parameter fine adjustment on the neural network, so that the accurate identification rate of each neural network to the samples is ensured to be 100%.
In the step 3, pixel point values of the randomly generated bezier curve are all the same randomized initial values, and the position and the direction of the curve are also randomly generated. The theoretical basis of the bezier curve formula used is the bernstein polynomial:
wherein, betaiIs the Bernstein coefficient, bi,n(t) is:
wherein, i is 0, 1, …, n,
while Bernstein seriesWhen the number is a fixed sequence of points lying on a two-dimensional plane (e.g. P)0,P1,P2) The bernstein polynomial becomes the bezier curve equation. By defining n ═ 2, the quadratic bezier curve equation can be obtained:
B(t)=(1-t)2P0+2t(1-t)P1+t2P2,t∈[0,1] (3)
a bezier curve can generate a complex smooth curve from only a few points. By means of a quadratic Bezier curve formula, a smooth Bezier curve can be drawn by giving three randomly generated coordinate points. And (4) randomly generating three key coordinate point values and RGB values of a large number of Bezier curves, and drawing different Bezier curves on a large number of copied images of the original image for the use of thestep 4.
In thestep 4, global optimization is performed on a large number of Bezier curves generated in the step 3 by using a differential evolution algorithm, and an optimal Bezier curve is searched. The basic idea of the differential evolution algorithm is as follows: starting from an initial population which is randomly generated, summing the vector difference of any two individuals in the population with the vector of a third individual to generate a new individual, then comparing the new individual with the corresponding individual in the current population, and if the adaptability of the new individual is stronger than that of the existing individual, replacing the old individual with the new individual in the next generation; through continuous evolution, good individuals are kept, bad individuals are eliminated, and the optimal solution is guided to be searched. The differential evolution algorithm does not require gradient information of the model, and therefore, it can be applied to a wider range of optimization problems than the gradient-based method. The differential evolution algorithm has the advantages of simple structure, rapid convergence and strong robustness, and is widely applied to the fields of data mining, pattern recognition, artificial neural networks and the like.
In the step 5, the found optimal Bezier curve is added to the image sample, and the neural network obtained in thestep 2 and subjected to parameter fine tuning is used for classifying the image sample, and the classification result is recorded.
Compared with the prior art, the invention has the advantages that:
first, it is proposed not to2Or l∞The distance is used as a standard for measuring the size of the change, so that the situation that the changed image cannot be identified or even is identified wrongly when viewed by human beings is avoided.
Secondly, a differential evolution algorithm is adopted as an optimization strategy, an optimal Bezier curve is heuristically searched, and the attack on the classification model is realized only by changing the pixel value of one curve in the image. Meanwhile, the curve does not influence the recognition of the image by human beings at all.
Finally, the differential evolution algorithm does not need gradient information of the model, so that the method can be applied to a wider optimization problem and has the characteristics of fast convergence, strong robustness and the like.
Detailed Description
To further illustrate the summary of the invention, the computing process, and the effectiveness of the method, reference is made to the following detailed description taken in conjunction with the accompanying drawings and the following examples:
the implementation flow chart is shown in figure 1.
And (3) building three neural network models according to the requirements of the step 1: ResNet50, VGG16 and DenseNet201, corresponding pre-training parameters are downloaded and loaded into the model, and the parameters in the program and the correctness of the model are ensured by testing the pre-training parameters once after the loading is finished. Meanwhile, in order to facilitate the calling of the subsequent steps, the input and output interfaces of the model are set.
And (3) selecting one thousand image samples meeting the same distribution from the ImageNet data set according to the requirements of the step (2), and then respectively carrying out parameter fine adjustment on the three models built in the step (1) by using the image samples, wherein the training times of the whole data set are required to be not more than 10, so that the over-fitting phenomenon is prevented, and the judgment accuracy of the image samples is improved to 100%.
Randomly generating Bezier curves on the basis of the original image according to the requirements of step 3, i.e. by randomly generating P0,P1,P2Three points are used as key points of the bezier curve by using the formula (3) b (t) ═ 1-t)2P0+2t(1-t)P1+t2P2And calculating the specific coordinate value of the Bezier curve, and randomly generating the RGB value of the Bezier curve. Here, it should be noted that: to ensure that the coordinate point position of the bezier curve does not exceed the maximum range of the image, the weight of the bezier curve needs to be set to 2, and the curve width needs to be a random value.
And (4) inputting a large number of Bezier curves generated in the step (3) into a differential algorithm according to the requirements of the step (4) to carry out iterative optimization. The specific settings of the differential evolution algorithm are as follows: population number 200,iteration number 10. The differential evolution algorithm obtains a new population at each iteration, and the new population has smaller confidence of the real label on the three image classification models than the old population, that is, the iterated images are more easily classified into other types by mistake during classification. And finally obtaining the optimal Bezier curve which makes the model classification wrong through iterative computation of a plurality of differential evolution algorithms.
And (5) adding the Bezier curve found in the step (4) to the image sample according to the requirement of the step (5), so that the three image classification neural networks can be wrongly classified.
And (5) repeating the steps 3-5 to add Bezier curves to all the images. After the Bezier curve of one thousand images is added, the results of the images on three image classification models are recorded as follows:
the error classification rate of the VGG16 model is 97.7%, the average iteration number of the differential evolution algorithm is 1.516, and the loss value is 0.0582.
The error classification rate of the ResNet50 model is 78%, the average iteration number of the differential evolution algorithm is 3.997, and the loss value is 0.2612.
The error classification rate of the DenseNet201 model is 59.3%, the average iteration number of the differential evolution algorithm is 5.692, and the loss value is 0.2859.
The method has the advantages that the attack effect on the classical image classification model is good, and the robustness is good because model gradient information is not needed. As can be seen from the iterative loss diagram of fig. 3, the differential evolution algorithm has a faster convergence rate. Fig. 4-6 are diagrams showing the number of hits of the predicted tag numbers when the present invention attacks ResNet50, and it can be seen that a large number of class tags with prediction errors are concentrated on a few numbers. This illustrates some of the shortcomings of the image classification neural network model, which is more prone to classification errors on these few digital labels.