CN111105336A - Image watermarking removing method based on countermeasure network - Google Patents

Abstract

The invention particularly relates to a method for image watermarking based on a countermeasure network. The image watermarking removing method based on the countermeasure network comprises the steps of inputting a picture with a watermark into a generator of the countermeasure network, generating a picture with the watermark removed, sending the picture into a discriminator, distinguishing the generated picture from a real picture, and calculating loss; and gradually optimizing the confrontation network model through the confrontation process of the generator and the discriminator, inputting the picture with the watermark into the generator of the trained confrontation network, and generating the picture with the watermark eliminated. The image watermarking removing method based on the countermeasure network does not need to mark the position of the watermark, and reduces the cost of manually marking the position of the data set; the model is simple, and end-to-end training and conjecture can be realized; the watermark removing effect is greatly improved by adopting three loss functions, and the watermark removing method can adapt to watermarks with different scales; and the confrontation model can learn the potential prior knowledge of different types and different styles of watermarks, and the generalization performance is strong.

Description

Image watermarking removing method based on countermeasure network

Technical Field

The invention relates to the technical field of artificial intelligence and image watermark removal, in particular to an image watermark removal method based on a countermeasure network.

Background

With the popularization of the internet and the rapid development of communication tools, people tend to release photos and videos on the internet to store precious memories and record good moments in life. In order to protect the copyright of pictures or videos, many software adopts a mode of adding watermarks on pictures. At present, artificial intelligence is gradually popularized to various fields, the basis of the artificial intelligence is the learning of large-scale data, watermarks can interfere or destroy the internal information of the data to a certain degree, and the learning of a model is greatly influenced. Meanwhile, various watermarking technologies are different, and the robustness and the anti-interference performance of the watermarking technologies need to be tested. Based on the above two points, the watermark removal technology gradually becomes a big hotspot in the current computer vision field.

Due to the variety of watermark categories and styles, watermark removal technology is still a great challenge at present. Most of the existing methods are based on a target detection technology, a mature target detection framework is used for detecting the position of a watermark, then a watermark area is cut out, and image conversion is carried out by using priori knowledge. But most of this a priori knowledge is designed for a particular pattern watermark, e.g. only one type of mark can be removed. The technologies not only need to manually mark the position of the watermark in the picture, but also need to perform image conversion based on experience design prior knowledge, and are time-consuming and labor-consuming, and because most of the technologies are formed by splicing a plurality of stages, each stage needs to be trained and learned respectively, so that the efficiency is low.

With the development of artificial intelligence, deep learning is gradually applied to various tasks, but deep learning requires a large amount of data as a basis. The watermarked data collected on the network has a great influence on model learning, so that efficient and accurate watermark removal technology is required to clean the data. On the other hand, in order to test the anti-interference performance of different watermarking technologies, the watermarking removal technology provides a powerful test condition. However, the prior art detection-based techniques require a lot of manpower to label the raw data, and are very time-consuming to train due to the multi-stage nature of the techniques. With the advent of anti-neural networks, image transformation techniques have been developed. The anti-neural network can automatically learn the hidden prior knowledge in the image only by inputting the original image and the target image, so that not only is the manpower saved, but also more distinctive information can be learned, and the image conversion effect is greatly improved.

In view of the above situation, the present invention provides a method for image watermarking based on a countermeasure network.

Disclosure of Invention

In order to make up for the defects of the prior art, the invention provides a simple and efficient image watermarking removing method based on a countermeasure network.

The invention is realized by the following technical scheme:

a method for image watermarking based on a countermeasure network is characterized by comprising the following steps:

firstly, a data set containing a picture with a watermark and a corresponding real picture is given, the picture with the watermark is input into a generator of a countermeasure network, and a picture with the watermark eliminated is generated;

secondly, sending the generated image with the watermark removed into a discriminator of the countermeasure network, distinguishing the generated image from a real image through the discriminator, evaluating the difference between the image with the watermark removed and the real image generated by the generator on the image level, and calculating loss;

thirdly, gradually optimizing a countermeasure network model through the countermeasure process of the generator and the discriminator so that the countermeasure network model can generate a high-level watermark-removed picture;

and fourthly, inputting the picture with the watermark into a generator of the trained countermeasure network to generate the picture without the watermark.

The generator aims at generating a watermark-removed picture which is judged as a real picture by a discriminator as much as possible; the goal of the discriminator is to distinguish the actual picture from the generated picture as correctly as possible.

In the first step, the data set is divided into a training set and a testing set, and the watermarked picture and the corresponding real picture are labeled.

In the first step and the fourth step, the RGB three-channel watermark-carrying picture x is input into a generator, and after convolution, activation, regularization and up-sampling operations, an RGB three-channel watermark-removing picture G (x) with the same size as the input picture is generated.

The method is characterized in that a deep learning segmentation network U-net (relational Networks for biological Image segmentation) commonly used in the medical field is used as a generator of the countermeasure network, and the deep learning segmentation network U-net is connected with low-level features and high-level features of the network through a skip structure, so that the network can simultaneously utilize structural information of the low-level features and semantic information of the high-level features.

In the second step, the watermark-removed picture generated by the generator is marked as a negative sample and input into the discriminator, the real picture is marked as a positive sample and input into the discriminator, and the discriminator is gradually trained.

In the second step, the discriminator is used for distinguishing whether the input picture is a real picture or a generated picture, and belongs to the classification problem, so that a pre-training model Resnet-101 is selected as the discriminator, and a feature pyramid structure FPN is added on the basis of the pre-training model Resnet-101 to enhance the network information expression capacity.

In the third step, a picture-level loss function L _1, a conventional cross entropy objective function L _2 and a perceptual loss function L _3 are used, and the three loss functions are used for guiding the training of the confrontation model.

The picture-level loss function L _1 is used for evaluating the difference between the generated picture G (x) and the real picture Y, and the picture-level loss function L _1 adopts the square error loss at the pixel level;

the discriminator objective is to minimize the objective function, the generator objective is to maximize the objective function, and in order to unify the form of three loss functions, the conventional cross-entropy objective function L _2 is inverted, and the countermeasure objective becomes arg (min) _ G (max) _ D ((-L) _ 2);

when the watermark in the image is particularly small, the receptive field of the high-level features of the pre-training model Resnet-101 is too large, and the small watermark features are easy to ignore; in order to avoid neglecting small watermark features in the image, intercepting the final layer of conv5 of the pre-training model Resnet-101, and calculating the perception loss by using the feature map of the generated picture and the feature map of the real picture in the layer, wherein the perception loss function L _3 adopts the pixel-level squared error loss.

In the third step, optimizing the countermeasure network model includes the steps of:

(1) fixing parameters of a discriminator, inputting the picture with the watermark into a generator, and training a k wheel;

(2) fixing generator parameters, inputting the real picture and the generated watermark-removed picture into a discriminator, and training a k wheel;

(3) and (3) alternately performing the step (1) and the step (2), training the generator and the discriminator at the same time, and gradually optimizing the model on the whole.

The invention has the beneficial effects that: the image watermarking removing method based on the countermeasure network does not need to mark the position of the watermark, and reduces the cost of manually marking the position of the data set; the model is simple, and end-to-end training and conjecture can be realized; the watermark removing effect is greatly improved by adopting three loss functions, and the watermark removing method can adapt to watermarks with different scales; and the confrontation model can learn the potential prior knowledge of different types and different styles of watermarks, and the generalization performance is strong.

Drawings

FIG. 1 is a schematic diagram of an image watermarking method based on a countermeasure network.

Detailed Description

In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the embodiment of the present invention. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The image watermarking method based on the countermeasure network comprises the following steps:

firstly, a data set containing a picture with a watermark and a corresponding real picture is given, the picture with the watermark is input into a generator of a countermeasure network, and a picture with the watermark eliminated is generated;

secondly, sending the generated image with the watermark removed into a discriminator of the countermeasure network, distinguishing the generated image from a real image through the discriminator, evaluating the difference between the image with the watermark removed and the real image generated by the generator on the image level, and calculating loss;

thirdly, gradually optimizing a countermeasure network model through the countermeasure process of the generator and the discriminator so that the countermeasure network model can generate a high-level watermark-removed picture;

and fourthly, inputting the picture with the watermark into a generator of the trained countermeasure network to generate the picture without the watermark.

The generator aims at generating a watermark-removed picture which is judged as a real picture by a discriminator as much as possible; the goal of the discriminator is to distinguish the actual picture from the generated picture as correctly as possible.

In the first step, the data set is divided into a training set and a testing set, and the watermarked picture and the corresponding real picture are labeled.

In the first step and the fourth step, the RGB three-channel watermark-carrying picture x is input into a generator, and after convolution, activation, regularization and up-sampling operations, an RGB three-channel watermark-removing picture G (x) with the same size as the input picture is generated.

The method is characterized in that a deep learning segmentation network U-net (relational Networks for biological Image segmentation) commonly used in the medical field is used as a generator of the countermeasure network, and the deep learning segmentation network U-net is connected with low-level features and high-level features of the network through a skip structure, so that the network can simultaneously utilize structural information of the low-level features and semantic information of the high-level features.

In the second step, the watermark-removed picture generated by the generator is marked as a negative sample and input into the discriminator, the real picture is marked as a positive sample and input into the discriminator, and the discriminator is gradually trained.

In the second step, the discriminator is used for distinguishing whether the input picture is a real picture or a generated picture, and belongs to the classification problem, so that a pre-training model Resnet-101 is selected as the discriminator, and a feature pyramid structure FPN is added on the basis of the pre-training model Resnet-101 to enhance the network information expression capacity.

In the third step, a picture-level loss function L _1, a conventional cross entropy objective function L _2 and a perceptual loss function L _3 are used, and the three loss functions are used for guiding the training of the confrontation model.

The picture-level loss function L _1 is used for evaluating the difference between the generated picture G (x) and the real picture Y, and the picture-level loss function L _1 adopts the square error loss at the pixel level;

the discriminator objective is to minimize the objective function, the generator objective is to maximize the objective function, and in order to unify the form of three loss functions, the conventional cross-entropy objective function L _2 is inverted, and the countermeasure objective becomes arg (min) _ G (max) _ D ((-L) _ 2);

when the watermark in the image is particularly small, the receptive field of the high-level features of the pre-training model Resnet-101 is too large, and the small watermark features are easy to ignore; in order to avoid neglecting small watermark features in the image, intercepting the final layer of conv5 of the pre-training model Resnet-101, and calculating the perception loss by using the feature map of the generated picture and the feature map of the real picture in the layer, wherein the perception loss function L _3 adopts the pixel-level squared error loss.

In the third step, optimizing the countermeasure network model includes the steps of:

(1) fixing parameters of a discriminator, inputting the picture with the watermark into a generator, and training a k wheel;

(2) fixing generator parameters, inputting the real picture and the generated watermark-removed picture into a discriminator, and training a k wheel;

(3) and (3) alternately performing the step (1) and the step (2), training the generator and the discriminator at the same time, and gradually optimizing the model on the whole.

Compared with the prior art, the image watermarking removing method based on the countermeasure network has the following characteristics:

firstly, the data set of the used training model only needs to mark a watermark picture and a real picture, and does not need to mark a watermark position, so that the cost of manually marking the position of the data set is reduced;

secondly, the model is simple, and end-to-end training and conjecture can be realized without complex multi-stage respective training process;

thirdly, the watermark removing effect is greatly improved by adopting three loss functions, and the watermark removing method can adapt to watermarks with different scales;

fourthly, the countermeasure model can learn the potential priori knowledge of different types and different styles of watermarks, and the generalization performance is strong.

A method for image de-watermarking based on a countermeasure network in the embodiment of the present invention is described in detail above. While the present invention has been described with reference to specific examples, which are provided to assist in understanding the core concepts of the present invention, it is intended that all other embodiments that can be obtained by those skilled in the art without departing from the spirit of the present invention shall fall within the scope of the present invention.

Claims (9)

Translated fromChinese

1.一种基于对抗网络的图像去水印的方法，其特征在于，包括以下步骤：1. a method for dewatering an image based on adversarial network, is characterized in that, comprises the following steps:第一步，给定一个包含带水印图片和相应真实图片的数据集，将带水印图片输入到对抗网络的生成器中，生成消除水印的图片；In the first step, given a data set containing watermarked pictures and corresponding real pictures, input the watermarked pictures into the generator of the adversarial network to generate pictures with watermarks removed;第二步，接着将生成的消除水印的图片送入到对抗网络的判别器中，通过判别器区分是生成的图片还是真实图片，并在图片水平上评估生成器生成的去水印图片和真实图片的差距，计算损失；In the second step, the generated watermark-removed image is sent to the discriminator of the adversarial network, and the discriminator is used to distinguish whether it is the generated image or the real image, and evaluate the de-watermarked image and the real image generated by the generator at the image level. the gap, calculate the loss;第三步，通过生成器和判别器的对抗过程逐渐优化对抗网络模型，使得对抗网络模型能够生成高水平的去水印图片；The third step is to gradually optimize the adversarial network model through the confrontation process of the generator and the discriminator, so that the adversarial network model can generate high-level de-watermarked images;第四步，将带水印图片输入到训练好的对抗网络的生成器中，生成消除水印的图片即可。The fourth step is to input the watermarked image into the generator of the trained adversarial network to generate a watermark-eliminated image.2.根据权利要求1所述的基于对抗网络的图像去水印的方法，其特征在于：所述第一步中，将数据集分为训练集和测试集，并标注带水印图片及其相应的真实图片。2. The method for dewatering an image based on adversarial network according to claim 1, it is characterized in that: in the described first step, the data set is divided into a training set and a test set, and the watermarked picture and its corresponding real picture.3.根据权利要求1所述的基于对抗网络的图像去水印的方法，其特征在于：所述第一步和第四步中，将RGB三通道带水印的图片x输入生成器，经过卷积、激活、正则化以及上采样操作后，生成与输入图片相同尺寸的RGB三通道去水印图片G(x)。3. the method based on the image de-watermarking of adversarial network according to claim 1, is characterized in that: in the described first step and the 4th step, the picture x of RGB three-channel watermarked input generator, through convolution , activation, regularization, and upsampling operations to generate an RGB three-channel de-watermarked image G(x) of the same size as the input image.4.根据权利要求1或3所述的基于对抗网络的图像去水印的方法，其特征在于：采用医学领域常用的深度学习分割网络U-net作为对抗网络的生成器，深度学习分割网络U-net通过跳越结构连接网络的低层次特征和高层次特征，使得网络可以同时利用低层次特征的结构信息和高层次特征的语义信息。4. The method for dewatering an image based on an adversarial network according to claim 1 or 3, wherein the deep learning segmentation network U-net commonly used in the medical field is used as the generator of the adversarial network, and the deep learning segmentation network U-net is used as the generator of the adversarial network. net connects the low-level features and high-level features of the network by skipping the structure, so that the network can utilize the structural information of low-level features and the semantic information of high-level features at the same time.5.根据权利要求1所述的基于对抗网络的图像去水印的方法，其特征在于：所述第二步中，将生成器生成的去水印图片标记为负样本输入判别器，将真实图片标记为正样本输入判别器，逐渐训练判别器。5. The method for dewatering an image based on an adversarial network according to claim 1, wherein in the second step, the de-watermarked picture generated by the generator is marked as a negative sample input discriminator, and the real picture is marked with Feed the discriminator for positive samples and gradually train the discriminator.6.根据权利要求1或5所述的基于对抗网络的图像去水印的方法，其特征在于：所述第二步中，判别器用于区分输入图片是真实图片还是生成的图片，属于二分类问题，因而选用预训练模型Resnet-101作为判别器，同时在预训练模型Resnet-101的基础上加入特征金字塔结构FPN，以增强网络信息表达能力。6. The method for dewatering an image based on an adversarial network according to claim 1 or 5, wherein in the second step, the discriminator is used to distinguish whether the input picture is a real picture or a generated picture, which belongs to a two-class problem Therefore, the pre-training model Resnet-101 is selected as the discriminator, and the feature pyramid structure FPN is added on the basis of the pre-training model Resnet-101 to enhance the ability of network information expression.7.根据权利要求1所述的基于对抗网络的图像去水印的方法，其特征在于：所述第三步中，使用图片级损失函数L_1，常规的交叉熵目标函数L_2和感知损失函数L_3，三种损失函数用于指导对抗模型训练。7. The method for de-watermarking based on adversarial network according to claim 1, it is characterized in that: in the described 3rd step, use picture level loss function L_1, conventional cross entropy objective function L_2 and perceptual loss function L_3, Three loss functions are used to guide adversarial model training.8.根据权利要求7所述的基于对抗网络的图像去水印的方法，其特征在于：8. The method for image dewatering based on adversarial network according to claim 7, is characterized in that:所述图片级损失函数L_1用于评估生成图片G(x)和真实图片Y之间的差距，图片级损失函数L_1采用像素级别的平方差损失；The picture-level loss function L_1 is used to evaluate the gap between the generated picture G(x) and the real picture Y, and the picture-level loss function L_1 adopts the squared difference loss at the pixel level;判别器目标是最小化该目标函数，生成器目标是最大化该目标函数，为了统一三个损失函数的形式，将所述常规的交叉熵目标函数L_2取反，对抗的目标变为arg(min)_G(max)_D((-L)_2)；The objective of the discriminator is to minimize the objective function, and the objective of the generator is to maximize the objective function. In order to unify the form of the three loss functions, the conventional cross-entropy objective function L_2 is inverted, and the confrontation objective becomes arg( min)_G(max)_D((-L)_2);为了避免图像中小水印特征被忽略，截取预训练模型Resnet-101的conv5最后一层，利用生成图片和真实图片在该层的特征图feature map，所述感知损失函数L_3采用像素级别的平方差损失，计算感知损失。In order to avoid the small watermark feature in the image from being ignored, the last layer of conv5 of the pre-training model Resnet-101 is intercepted, and the feature map of the generated image and the real image in this layer is used. The perceptual loss function L_3 adopts the pixel-level squared difference loss. , computes the perceptual loss.9.根据权利要求7或8所述的基于对抗网络的图像去水印的方法，其特征在于：所述第三步中，优化对抗网络模型，包括以下步骤：9. The method for dewatering an image based on an adversarial network according to claim 7 or 8, wherein in the third step, optimizing the adversarial network model comprises the following steps:(1)固定判别器参数，输入带水印图片到生成器中，训练k轮；(1) Fix the parameters of the discriminator, input the watermarked image into the generator, and train for k rounds;(2)固定生成器参数，真实图片和生成的去水印图片输入到判别器中，训练k轮；(2) The generator parameters are fixed, the real image and the generated de-watermarked image are input into the discriminator, and k rounds are trained;(3)交替进行步骤(1)和步骤(2)，同时训练生成器和判别器，从整体上逐渐优化模型。(3) Perform steps (1) and (2) alternately, train the generator and the discriminator at the same time, and gradually optimize the model as a whole.

Images