CN114444565B - Image tampering detection method, terminal equipment and storage medium - Google Patents

Abstract

In an image tampering detection model, a network backbone part adopts a Faster-RCNN network, the output of an RPN layer and an input image are input into a new ROI pooling layer together, an image output by a ROI pooling layer is subjected to resampling treatment to obtain a resampling spectrogram, the resampling spectrogram is input into an LSTM network to obtain a resampling feature map, and the output of the resampling feature map after passing through an up-sampling layer and a convolution layer is input into a bilinear pooling layer together with the output of the Faster-RCNN network to be fused. The invention solves the problem that the feature extraction of the current deep learning network in the forging and falsifying field is insufficient.

Description

Image tampering detection method, terminal equipment and storage medium

Technical Field

The present invention relates to the field of image detection, and in particular, to an image tampering detection method, a terminal device, and a storage medium.

Background

With the development of technology and the popularization of software, PS has become increasingly simple and convenient to operate to forge pictures with high quality and reality. Many high quality P-maps not only can fool the human eye, but also can successfully fool some detection models, which poses a great hazard to the development of social security.

The image PS manipulation basic operations can be divided into four types, i.e., copy-move, split-splice (splicing), remove pixels and fill in with other pixels (remove), and generate characters (generated), respectively. In order to increase the fidelity of the forged image, the falsifier not only uses tools to carry out edge feathering, painting brush smearing, color change and other operations, but also carries out noise adding, blurring adding, compression and other post-processing operations. These operations make the image features of the counterfeit area indistinguishable from those of the non-tampered area, increasing the difficulty of detection to some extent.

The PS tampering identification and positioning method is mainly divided into a traditional identification method and a method based on deep learning. The traditional detection and positioning algorithm is an algorithm for detecting and positioning based on the statistical characteristics of images according to the artificial construction characteristics of the user, such as a DCT (discrete cosine transform) falsification detection algorithm, a SIFT (scale-invariant feature transform) detection algorithm, a resampling trace detection algorithm and the like, and the detection and positioning algorithm based on deep learning autonomously learns related characteristics so as to detect PS falsified image areas.

The detection positioning method based on deep learning has been developed for a period of time, but along with the development of image tampering technology, the existing tampering detection algorithm also has some defects, such as that most tampering detection positioning algorithms only extract features based on RGB format images, and the extracted features are limited, so that images of fake areas and non-tampered areas are difficult to distinguish.

Disclosure of Invention

In order to solve the problems, the invention provides an image tampering detection method, a terminal device and a storage medium.

The specific scheme is as follows:

an image tamper detection method comprising the steps of:

S1: acquiring tampered and untampered images to form a training set, and marking tampered areas in the tampered images in the training set;

S2: constructing an image tampering detection model, and training the model through a training set;

The method comprises the steps that a trunk part in a network structure of an image tampering detection model adopts a Faster-RCNN network, an input image is input into the Faster-RCNN network, a candidate frame output by an RPN layer in the Faster-RCNN network is input into a ROI pooling layer in the Faster-RCNN network on one hand, and is input into a new ROI pooling layer together with the input image on the other hand, an image output by the ROI pooling layer is subjected to resampling treatment to obtain a resampling spectrogram, the resampling spectrogram is input into the LSTM network to obtain a resampling feature map, the output of the resampling feature map after being subjected to an up-sampling layer and a convolution layer is input into a bilinear pooling layer together with the output of the Faster-RCNN network to be fused, and a prediction result of whether the image is tampered or not is obtained after the fused result passes through a full-connection layer and a softmax layer;

s3: and detecting whether the image is tampered or not through the trained model.

Further, the resampling process comprises the following steps: and processing the candidate frame region in the image through an EM algorithm, and then carrying out Fourier transformation to obtain a resampling spectrogram.

Further, the calculation formula of the loss function L_total of the image tampering detection model is as follows:

L_total＝L_RPN+L_tamper+L_bbox

where L_RPN represents the RPN layer loss function, L_tamper represents the final classification loss function, and L_bbox represents the rectangular regression loss function.

Further, the loss function of the RPN layer includes a classification loss function L_cls and a regression loss function L_reg, and the calculation formula of the loss function of the RPN layer is:

Where g_i denotes the prediction classification probability of the candidate box,Representing the true label of the candidate box, f_i representing the predicted coordinates of the candidate box,The true coordinates of the labeling frames corresponding to the candidate frames are represented, i represents the serial numbers of the candidate frames, lambda represents the weight coefficient, N_cls represents the size of the small batch, and N_reg represents the number of the candidate frames.

Further, the final classification loss function uses a cross entropy loss function, and the rectangular box regression loss function uses SmoothL loss functions.

An image tamper detection terminal device comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the steps of the method according to the embodiment of the invention are realized when the processor executes the computer program.

A computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of the method described above for embodiments of the present invention.

According to the technical scheme, the resampling characteristic is used in the LSTM, the connection between the tampered block and the untampered block is explored, and the problem that the characteristic extraction of the current deep learning network in the fake tampered field is insufficient is solved.

Drawings

Fig. 1 is a flowchart of a first embodiment of the present invention.

Fig. 2 is a diagram showing a network structure of an image falsification detection model in this embodiment.

Detailed Description

For further illustration of the various embodiments, the invention is provided with the accompanying drawings. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments and together with the description, serve to explain the principles of the embodiments. With reference to these matters, one of ordinary skill in the art will understand other possible embodiments and advantages of the present invention.

The invention will now be further described with reference to the drawings and detailed description.

Embodiment one:

the embodiment of the invention provides an image tampering detection method, as shown in fig. 1, comprising the following steps:

S1: and acquiring tampered and untampered images to form a training set, and marking tampered areas in the tampered images in the training set.

S2: and constructing an image tampering detection model, and training the model through a training set.

As shown in FIG. 2, in the network structure of the image tamper detection model, a fast-RCNN network is adopted in a trunk part, after an input image is input into the fast-RCNN network, a candidate frame output by an RPN layer in the fast-RCNN network is input into a ROI pooling layer in the fast-RCNN network on one hand so as to finish the part of the original fast-RCNN network, namely, the coordinates of the candidate frame are corresponding to a feature map through the ROI pooling layer, and a new ROI pooling layer is input together with the input image on the other hand so as to correspond the coordinates of the candidate frame to the input image, the image output by the ROI pooling layer is subjected to resampling processing to obtain a resampling spectrogram, the resampling spectrogram is input into the LSTM network to obtain a resampling feature map, the output of the resampling feature map after the upsampling spectrogram is subjected to upsampling layer and the convolution layer is input into a bilinear pooling layer together with the output of the fast-RCNN network, and the fusion result is subjected to a full-connection layer and a softmax layer so as to obtain a prediction result of whether the image is tampered.

The input image in this embodiment is an RGB image.

The resampling process in this embodiment is as follows: and processing the candidate frame region in the image through an EM algorithm, and then carrying out Fourier transformation to obtain a resampling spectrogram. The specific process is as follows:

1. First, it is assumed that each sampling point belongs to two models, the first model is an M1 model, and the M1 model represents that the point y_i currently under investigation has a linear correlation with its neighboring points, that is, the points are generated according to the following model:

Where n (i) is a gaussian distribution of independent co-distributed zero mean unknown variance σ. The second model is the M2 model, which represents that the sample points under investigation do not have a linear correlation with their neighbors, i.e. they are generated by a separate process.

2. The EM algorithm is a two-step iterative algorithm: (1) step E: the probability of the model to which each sampling point belongs is estimated first; (2) M steps: the particular form of correlation of the sampling points is again estimated. The probability of the M1 model to which the sampling point y_i belongs can be obtained by a Bayesian formula:

In the above formula, pr { y_i∈M₁ } and Pr { y_i∈M₂ } are prior probabilities, the initial value is set to 1/2, and the probability that y_i belongs to the M1 model can be expressed as:

In step M, the estimation of the weight vector is calculated by the following formula:

(3) The molecular term in the formula can be expressed as a predicted residual error, namely:

After the calculation of step E is completed, the weight vector in step M is calculatedThe calculation formula of (2) is as follows:

Wherein the row vector of the Y matrix is a vector composed of 2^N adjacent points of each point in the original image matrix, W is a diagonal matrix, the value on the diagonal of the matrix is the value of omega (i), and the weight vector is calculated by the formulaAnd E and M steps of iterative iteration are carried out until convergence, and a final weight vector can be obtained at the moment.

3. And then carrying out Fourier transform on the posterior probability map to obtain a resample spectrogram, wherein the Fourier transform aims at taking out high-frequency components with periodicity and removing low-frequency components through a high-pass filter. After the fourier transform, if the image contains periodic related information, the fourier transform map contains periodic bright spots.

The calculation formula of the loss function L_total of the image tampering detection model is as follows:

L_total＝L_RPN+L_tamper+L_bbox

where L_RPN represents the RPN layer loss function, L_tamper represents the final classification loss function, and L_bbox represents the rectangular regression loss function.

The RPN layer has two tasks, which candidate frames contain picture main bodies and which only contain backgrounds, namely classification tasks, are judged from a plurality of candidate frames; for the candidate box containing the subject, the regression obtains a real target, namely a regression task, so in this embodiment, the loss of the RPN layer is set to include a classification loss function L_cls and a regression loss function L_reg, and the calculation formula of the loss function of the RPN layer is as follows:

Where g_i denotes the prediction classification probability of the candidate box (anchor),A real label representing a candidate frame (anchor), f_i represents coordinates predicted by the candidate frame (anchor),The true coordinates of the label frame (ground truth) corresponding to the candidate frame (anchor) are represented, i represents the number of the candidate frame, λ represents the weight coefficient, N_cls represents the size of the small batch (mini-batch size), and N_reg represents the number of the candidate frames.

The final classification loss function adopts a cross entropy loss function, and the rectangular box regression loss function adopts SmoothL loss functions.

S3: and detecting whether the image is tampered or not through the trained model.

After the image to be detected is input into the trained image falsification detection model, the model outputs whether the image to be detected is a falsified image or not.

The method provided by the embodiment of the invention is beneficial to solving the problem that the existing deep learning method is insufficient in extraction of the falsified features of the PS falsified image, and the RGB features and the features of resampling traces of the falsified region are fused in the target detection network, so that the falsification detection and positioning capability of the PS image is effectively improved.

Embodiment two:

The invention also provides an image tampering detection terminal device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the steps in the method embodiment of the first embodiment of the invention are realized when the processor executes the computer program.

Further, as an executable scheme, the image tamper detection terminal device may be a computing device such as a desktop computer, a notebook computer, a palm computer, and a cloud server. The image tamper detection terminal device may include, but is not limited to, a processor, a memory. It will be appreciated by those skilled in the art that the above-described constituent structure of the image tampering detecting terminal device is merely an example of the image tampering detecting terminal device, and does not constitute limitation of the image tampering detecting terminal device, and may include more or less components than those described above, or may combine some components, or different components, for example, the image tampering detecting terminal device may further include an input/output device, a network access device, a bus, etc., which is not limited by the embodiment of the present invention.

Further, as an executable scheme, the Processor may be a central processing unit (Central Processing Unit, CPU), other general purpose Processor, digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like, which is a control center of the image tamper detection terminal device, and connects the respective parts of the entire image tamper detection terminal device using various interfaces and lines.

The memory may be used to store the computer program and/or the module, and the processor may implement various functions of the image tamper detection terminal device by running or executing the computer program and/or the module stored in the memory and calling the data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for a function; the storage data area may store data created according to the use of the cellular phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart memory card (SMART MEDIA CARD, SMC), secure Digital (SD) card, flash memory card (FLASH CARD), at least one disk storage device, flash memory device, or other volatile solid-state storage device.

The present invention also provides a computer readable storage medium storing a computer program which when executed by a processor implements the steps of the above-described method of an embodiment of the present invention.

The modules/units integrated in the image tamper detection terminal device may be stored in a computer readable storage medium if implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the present invention may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a software distribution medium, and so forth.

While the invention has been particularly shown and described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (6)

1. An image tamper detection method, comprising the steps of:

S1: acquiring tampered and untampered images to form a training set, and marking tampered areas in the tampered images in the training set;

S2: constructing an image tampering detection model, and training the model through a training set;

The method comprises the steps that a trunk part in a network structure of an image tampering detection model adopts a Faster-RCNN network, an input image is input into the Faster-RCNN network, a candidate frame output by an RPN layer in the Faster-RCNN network is input into a first ROI pooling layer in the Faster-RCNN network on one hand, the candidate frame and the input image are input into a second ROI pooling layer together, an image output by the second ROI pooling layer is subjected to resampling processing to obtain a resampling spectrogram, the resampling spectrogram is input into the LSTM network to obtain a resampling feature map, the output of the resampling feature map after being subjected to an up-sampling layer and a convolution layer is input into a bilinear pooling layer together with the output of the Faster-RCNN network to be fused, and a prediction result of whether the image is a tampered image is obtained after the fusion result passes through a full connection layer and a softmax layer; the resampling process comprises the following steps: processing the candidate frame region in the image through an EM algorithm, and then carrying out Fourier transformation to obtain a resampling spectrogram;

s3: and detecting whether the image is tampered or not through the trained model.

2. The image tampering detection method as defined in claim 1, wherein: the calculation formula of the loss function L_total of the image tampering detection model is as follows:

L_total＝L_RPN+L_tamper+L_bbox

where L_RPN represents the RPN layer loss function, L_tamper represents the final classification loss function, and L_bbox represents the rectangular regression loss function.

3. The image tampering detection method as defined in claim 2, wherein: the loss function of the RPN layer comprises a classification loss function L_cls and a regression loss function L_reg, and the calculation formula of the loss function of the RPN layer is as follows:

Where g_i denotes the prediction classification probability of the candidate box,Representing the true label of the candidate box, f_i representing the predicted coordinates of the candidate box,The true coordinates of the labeling frames corresponding to the candidate frames are represented, i represents the serial numbers of the candidate frames, lambda represents the weight coefficient, N_cls represents the size of the small batch, and N_reg represents the number of the candidate frames.

4. The image tampering detection method as defined in claim 2, wherein: the final classification loss function adopts a cross entropy loss function, and the rectangular box regression loss function adopts SmoothL loss functions.

5. An image tamper detection terminal device, characterized in that: comprising a processor, a memory and a computer program stored in the memory and running on the processor, which processor, when executing the computer program, carries out the steps of the method according to any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program, characterized in that: the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 4.