Disclosure of Invention
In order to solve the above problems, the present invention provides an image tampering detection method, a terminal device, and a storage medium.
The specific scheme is as follows:
an image tampering detection method, comprising the steps of:
s1: acquiring images of two types, namely tampered images and images of non-tampered images to form a training set, and labeling tampered areas in the tampered images in the training set;
s2: constructing an image tampering detection model, and training the model through a training set;
a trunk part in a network structure of the image tampering detection model adopts a fast-RCNN network, after an input image is input into the fast-RCNN network, a candidate frame output by an RPN layer in the fast-RCNN network is input into an ROI posing layer in the fast-RCNN network on one hand, and a new ROI posing layer is input together with the input image on the other hand, an image output by the ROI posing layer is resampled to obtain a resampled spectrogram, the resampled spectrogram is input into an LSTM network to obtain a resampled characteristic diagram, the output of the resampled characteristic diagram after being subjected to an upsampling layer and a convolutional layer is input into a bilinear pooling layer together with the output of the fast-RCNN network to be fused, and the fused result is subjected to a full connection layer and a softmax layer to obtain a prediction result of whether the image is tampered or not;
s3: and detecting whether the image is tampered through the trained model.
Further, the resampling process comprises: and processing the candidate frame region in the image through an EM algorithm, and then performing Fourier transform to obtain a resample spectrogram.
Further, a loss function L of the image tampering detection modeltotalThe calculation formula of (2) is as follows:
Ltotal=LRPN+Ltamper+Lbbox
wherein L isRPNRepresenting the loss function, L, of the RPN layertamperRepresents the final classification loss function, LbboxA rectangular box regression loss function is represented.
Further, the loss function of the RPN layer includes a classification loss function LclsAnd a regression loss function LregThe formula for calculating the loss function of the RPN layer is:
wherein, g
iRepresenting the predicted classification probability of the candidate box,
true tag representing candidate box, f
iThe coordinates representing the prediction of the candidate box,
representing the real coordinates of the labeled frame corresponding to the candidate frame, i representing the serial number of the candidate frame, λ representing the weight coefficient, N
clsIndicating the size of the small batch, N
regIndicating the number of candidate boxes.
Further, the final classification loss function adopts a cross entropy loss function, and the rectangular box regression loss function adopts a SmoothL1 loss function.
An image tampering detection terminal device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to realize the steps of the method of the embodiment of the invention.
A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of the method as described above for an embodiment of the invention.
According to the technical scheme, the resampling characteristics are used in the LSTM, the relation between the tampered blocks and the non-tampered blocks is explored, and the problem that the characteristic extraction of the current deep learning network in the field of counterfeiting and tampering is insufficient is solved.
Detailed Description
To further illustrate the various embodiments, the invention provides the accompanying drawings. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the embodiments. Those skilled in the art will appreciate still other possible embodiments and advantages of the present invention with reference to these figures.
The invention will now be further described with reference to the accompanying drawings and detailed description.
The first embodiment is as follows:
an embodiment of the present invention provides an image tampering detection method, as shown in fig. 1, the method includes the following steps:
s1: and acquiring images of two types, namely tampered images and images of non-tampered images to form a training set, and labeling tampered areas in the tampered images in the training set.
S2: and constructing an image tampering detection model, and training the model through a training set.
As shown in fig. 2, the main part of the network structure of the image tampering detection model adopts the fast-RCNN network, and after the input image is input into the fast-RCNN network, the candidate frames output by the RPN layer in the fast-RCNN network are input into the ROI posing layer in the fast-RCNN network on one hand to complete the part of the original fast-RCNN network, that is, the coordinates of the candidate frame are mapped to the feature map through the ROI posing layer, on the other hand, a new ROI posing layer is input together with the input image, the coordinates of the candidate frame are corresponding to an input image, the image output by the ROI posing layer is subjected to resampling processing to obtain a resampled spectrogram, the resampled spectrogram is input into an LSTM network to obtain a resampled feature map, the output of the resampled feature map after passing through an upper sampling layer and a convolution layer and the output of a Faster-RCNN network are jointly input into a bilinear pooling layer to be fused, and the fused result passes through a full connection layer and a softmax layer to obtain a prediction result of whether the image is tampered.
The input image in this embodiment is an RGB image.
The resampling process in this embodiment is as follows: and processing the candidate frame region in the image through an EM algorithm, and then performing Fourier transform to obtain a resample spectrogram. The specific process is as follows:
1. first assume that each sample point isBelonging to two models, the first model is an M1 model, and the M1 model represents the currently investigated point yiThere is a linear correlation with its neighbors, that is, the generation of these points is according to the following model:
where n (i) is a gaussian distribution of the independent homographic zero mean unknown variance σ. The second model is the M2 model, the M2 model representing that the sample points under investigation do not have a linear correlation with their neighbors, i.e. they are generated by an independent process.
2. The EM algorithm is a two-step iterative algorithm: (1) e, step E: the probability of the model to which each sample point belongs is estimated first; (2) and M: a special form of correlation of the sample points is estimated. Sampling point yiThe probability of the M1 model can be obtained by a Bayesian formula:
in the above formula Pr { yi∈M1And Pr { y }i∈M2Is a priori probability, with an initial value of 1/2, and y, in additioniThe probability of belonging to the M1 model can be expressed as:
in step M, the estimate for the weight vector is calculated by the following formula:
(3) the numerator term in the equation can be expressed as the prediction residual error, i.e.:
after the calculation of the step E is finished, the weight vector in the step M is calculated
The calculation formula of (2) is as follows:
wherein the row vector of the Y matrix is 2 adjacent to each point in the original image matrix
NA vector of points, W is a diagonal matrix, and the values on the diagonal of the matrix are the values of ω (i), the weight vector calculated by this equation
And E and M steps are iterated repeatedly until convergence, and a final weight vector can be obtained at the moment.
3. And then carrying out Fourier transform on the posterior probability map to obtain a resampled spectrogram, wherein the purpose of Fourier transform is to take out high-frequency components with periodicity and remove low-frequency components through a high-pass filter. After the Fourier transformation, if the image contains relevant information of periodicity, the Fourier transformation image contains bright spots of periodicity.
Loss function L of image tampering detection modeltotalThe calculation formula of (2) is as follows:
Ltotal=LRPN+Ltamper+Lbbox
wherein L isRPNRepresenting the loss function, L, of the RPN layertamperRepresents the final classification loss function, LbboxA rectangular box regression loss function is represented.
The RPN layer has two tasks, which are to judge which candidate frames contain the picture main body and which only contain the background from a plurality of candidate frames, namely a classification task; for candidate frames containing a subject, regression obtains the true goal, i.e., regression task, and thus, in this embodiment, sets the loss packet of the RPN layerIncluding classification loss function LclsAnd a regression loss function LregThe formula for calculating the loss function of the RPN layer is:
wherein, g
iRepresenting the predicted classification probability of the candidate box (anchor),
true label, f, representing a candidate box (anchor)
iRepresenting the coordinates of the candidate box (anchor) prediction,
represents the real coordinate of the label box (ground channel) corresponding to the candidate box (anchor), i represents the number of the candidate box, lambda represents the weight coefficient, N
clsIndicates the size of the minibatch (mini-batch size), N
regRepresenting the number of candidate boxes.
The final classification loss function adopts a cross entropy loss function, and the rectangular box regression loss function adopts a SmoothL1 loss function.
S3: and detecting whether the image is tampered through the trained model.
And after the image to be detected is input into the trained image tampering detection model, the model outputs whether the image to be detected is a tampered image.
The method provided by the embodiment of the invention is beneficial to solving the problem that the conventional deep learning method is insufficient in extracting the forged features of the PS tampered image, and combines the RGB features of the forged region and the features of the resampling trace in the target detection network, thereby effectively improving the tampering detection and positioning capabilities of the PS image.
Example two:
the invention further provides an image tampering detection terminal device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to realize the steps of the method embodiment of the first embodiment of the invention.
Further, as an executable scheme, the image tampering detection terminal device may be a desktop computer, a notebook, a palm computer, a cloud server, and other computing devices. The image tampering detection terminal device can include, but is not limited to, a processor and a memory. It is understood by those skilled in the art that the above-mentioned constituent structure of the image tampering detection terminal device is only an example of the image tampering detection terminal device, and does not constitute a limitation to the image tampering detection terminal device, and may include more or less components than the above, or combine some components, or different components, for example, the image tampering detection terminal device may further include an input/output device, a network access device, a bus, and the like, which is not limited in this embodiment of the present invention.
Further, as an executable solution, the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, and the like. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, and the processor is a control center of the image tampering detection terminal device, and various interfaces and lines are used to connect various parts of the entire image tampering detection terminal device.
The memory may be configured to store the computer program and/or the module, and the processor may implement various functions of the image tamper detection terminal device by executing or executing the computer program and/or the module stored in the memory and calling data stored in the memory. The memory can mainly comprise a program storage area and a data storage area, wherein the program storage area can store an operating system and an application program required by at least one function; the storage data area may store data created according to the use of the mobile phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
The invention also provides a computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of the above-mentioned method of an embodiment of the invention.
The module/unit integrated with the image tampering detection terminal device may be stored in a computer-readable storage medium if it is implemented in the form of a software function unit and sold or used as a separate product. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments described above may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), software distribution medium, and the like.
While the invention has been particularly shown and described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.