CN113947409A - Sample generation method, payment method, sample generation system and related equipment - Google Patents
Sample generation method, payment method, sample generation system and related equipmentDownload PDFInfo
- Publication number
- CN113947409A CN113947409ACN202111184870.5ACN202111184870ACN113947409ACN 113947409 ACN113947409 ACN 113947409ACN 202111184870 ACN202111184870 ACN 202111184870ACN 113947409 ACN113947409 ACN 113947409A
- Authority
- CN
- China
- Prior art keywords
- image
- user
- training
- face recognition
- superimposed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q20/00—Payment architectures, schemes or protocols
- G06Q20/38—Payment protocols; Details thereof
- G06Q20/40—Authorisation, e.g. identification of payer or payee, verification of customer or shop credentials; Review and approval of payers, e.g. check credit lines or negative lists
- G06Q20/401—Transaction verification
- G06Q20/4014—Identity check for transactions
- G06Q20/40145—Biometric identity checks
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Finance (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
- Collating Specific Patterns (AREA)
- Image Analysis (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本申请实施例涉及人工智能领域，尤其涉及一种样本生成方法、支付方法、样本生成系统及相关设备。The embodiments of the present application relate to the field of artificial intelligence, and in particular, to a sample generation method, a payment method, a sample generation system, and related equipment.
背景技术Background technique
随着人工智能技术的不断发展，人脸识别技术在人们的日常生活中应用的越来越广泛，例如，刷脸支付。然而，在实际应用中，会存在不法分子进行对抗攻击，例如，带着面具冒充其他合法用户进行盗刷。With the continuous development of artificial intelligence technology, face recognition technology is more and more widely used in people's daily life, for example, face payment. However, in practical applications, there will be criminals conducting adversarial attacks, for example, posing as other legitimate users with masks for stealing.
为了能识别出上述盗刷行为，通常会对刷脸支付设备进行训练，用于防御上述这种对抗攻击。在上述训练过程中，需要大量的攻击样本进行训练。然而目前的技术很难产生高质量的攻击样本，而低质量攻击样本的训练难以提高刷脸支付设备识别对抗攻击的能力，因此，亟需一种方法能生成高质量的攻击样本。In order to identify the above-mentioned fraudulent brushing behaviors, face-swiping payment devices are usually trained to defend against the above-mentioned adversarial attacks. In the above training process, a large number of attack samples are required for training. However, the current technology is difficult to generate high-quality attack samples, and the training of low-quality attack samples is difficult to improve the ability of face-swiping payment devices to identify adversarial attacks. Therefore, there is an urgent need for a method to generate high-quality attack samples.
发明内容SUMMARY OF THE INVENTION
本申请实施例提供了一种样本生成方法、支付方法、样本生成系统及相关设备，以提供一种通过模型训练生成样本的方式，可以生成高质量的攻击样本。The embodiments of the present application provide a sample generation method, payment method, sample generation system and related equipment, so as to provide a way of generating samples through model training, which can generate high-quality attack samples.
第一方面，本申请实施例提供了一种样本生成方法，包括：In a first aspect, an embodiment of the present application provides a sample generation method, including:
获取第一用户图像及第二用户图像；obtaining a first user image and a second user image;
对所述第一用户图像进行面具遮盖，得到第一用户遮盖图像；masking the first user image with a mask to obtain a first user masking image;
将所述第一用户遮盖图像与所述第二用户图像进行叠加，得到叠加图像；superimposing the first user cover image and the second user image to obtain an overlay image;
将所述叠加图像输入一个或多个预设人脸识别模型进行训练，得到攻击样本。Input the superimposed image into one or more preset face recognition models for training to obtain attack samples.
本申请实施例中，通过对第一用户图像及第二用户图像的合成图像的训练，可以生成高质量的攻击样本。In this embodiment of the present application, high-quality attack samples can be generated by training the composite image of the first user image and the second user image.
其中一种可能的实现方式中，所述将所述叠加图像输入一个或多个预设人脸识别模型进行训练，得到攻击样本包括：In one of the possible implementations, the superimposed image is input into one or more preset face recognition models for training, and the attack samples obtained include:
将所述叠加图像进行数据增强，得到增强图像；performing data enhancement on the superimposed image to obtain an enhanced image;
将所述增强图像输入一个或多个预设人脸识别模型进行训练，得到攻击样本。The enhanced image is input into one or more preset face recognition models for training to obtain attack samples.
其中一种可能的实现方式中，所述将所述叠加图像输入一个或多个预设人脸识别模型进行训练包括：In one possible implementation manner, the inputting the superimposed image into one or more preset face recognition models for training includes:
将所述叠加图像输入一个或多个预设人脸识别模型，得到识别结果；Inputting the superimposed image into one or more preset face recognition models to obtain a recognition result;
基于所述识别结果与所述第一用户图像，计算得到损失；Calculate the loss based on the recognition result and the first user image;
基于所述损失对所述第一用户遮盖图像进行梯度更新，得到第一用户遮盖更新图像，并将所述第一用户遮盖更新图像输入一个或多个预设人脸识别模型进行训练。Gradient update is performed on the first user cover image based on the loss to obtain a first user cover update image, and the first user cover update image is input into one or more preset face recognition models for training.
其中一种可能的实现方式中，所述第一用户遮盖图像包括可见部分及不可见部分，所述将所述第一用户遮盖图像与所述第二用户图像进行叠加，得到叠加图像包括：In one possible implementation manner, the first user covering image includes a visible part and an invisible part, and the superimposing the first user covering image and the second user image to obtain the superimposed image includes:
将所述第一用户遮盖图像中的可见部分覆盖在所述第二用户图像上，得到叠加图像。A superimposed image is obtained by overlaying the visible portion of the first user mask image on the second user image.
其中一种可能的实现方式中，所述多个预设人脸识别模型包括多个不同网络类型的预设人脸识别模型。In one possible implementation manner, the multiple preset face recognition models include multiple preset face recognition models of different network types.
本申请实施例还提供了一种支付方法，包括：The embodiment of the present application also provides a payment method, including:
响应于用户的操作，获取用户的人脸图像；In response to the user's operation, obtain the user's face image;
对所述人脸图像进行人脸检测；performing face detection on the face image;
使用预设的活体识别模型对检测到的人脸进行活体识别，得到第一识别结果；其中，所述第一识别结果用于表征人脸的分类，所述预设的活体识别模型由上述样本生成方法得到的样本进行训练得到；Using a preset living body recognition model to perform living body recognition on the detected face to obtain a first recognition result; wherein, the first recognition result is used to characterize the classification of the human face, and the preset living body recognition model is determined by the above-mentioned sample. The samples obtained by the generation method are obtained by training;
基于所述第一识别结果，使用预设的人脸识别模型对检测到的人脸进行人脸识别，得到第二识别结果；其中，所述第二识别结果用于表征识别到的目标对象，所述预设的活体识别模型由上述样本生成方法得到的样本进行训练得到；Based on the first recognition result, a preset face recognition model is used to perform face recognition on the detected face to obtain a second recognition result; wherein, the second recognition result is used to represent the recognized target object, The preset living body recognition model is obtained by training the samples obtained by the above-mentioned sample generation method;
基于所述第二识别结果完成支付。The payment is completed based on the second identification result.
第二方面，本申请实施例提供一种样本生成系统，包括：In a second aspect, an embodiment of the present application provides a sample generation system, including:
获取模块，用于获取第一用户图像及第二用户图像；an acquisition module for acquiring the first user image and the second user image;
遮盖模块，用于对所述第一用户图像进行面具遮盖，得到第一用户遮盖图像；a masking module for masking the first user image to obtain the first user masking image;
叠加模块，用于将所述第一用户遮盖图像与所述第二用户图像进行叠加，得到叠加图像；an overlay module, configured to overlay the first user mask image and the second user image to obtain an overlay image;
生成模块，用于将所述叠加图像输入一个或多个预设人脸识别模型进行训练，得到攻击样本。The generating module is used for inputting the superimposed image into one or more preset face recognition models for training to obtain attack samples.
其中一种可能的实现方式中，所述生成模块还用于In one of the possible implementation manners, the generating module is further used for
将所述叠加图像进行数据增强，得到增强图像；performing data enhancement on the superimposed image to obtain an enhanced image;
将所述增强图像输入一个或多个预设人脸识别模型进行训练，得到攻击样本。The enhanced image is input into one or more preset face recognition models for training to obtain attack samples.
其中一种可能的实现方式中，所述生成模块还用于In one of the possible implementation manners, the generating module is further used for
将所述叠加图像输入一个或多个预设人脸识别模型，得到识别结果；Inputting the superimposed image into one or more preset face recognition models to obtain a recognition result;
基于所述识别结果与所述第一用户图像，计算得到损失；Calculate the loss based on the recognition result and the first user image;
基于所述损失对所述第一用户遮盖图像进行梯度更新，得到第一用户遮盖更新图像，并将所述第一用户遮盖更新图像输入一个或多个预设人脸识别模型进行训练。Gradient update is performed on the first user cover image based on the loss to obtain a first user cover update image, and the first user cover update image is input into one or more preset face recognition models for training.
其中一种可能的实现方式中，所述第一用户遮盖图像包括可见部分及不可见部分，所述叠加模块还用于In one possible implementation manner, the first user masking image includes a visible part and an invisible part, and the overlay module is further configured to
将所述第一用户遮盖图像中的可见部分覆盖在所述第二用户图像上，得到叠加图像。A superimposed image is obtained by overlaying the visible portion of the first user mask image on the second user image.
其中一种可能的实现方式中，所述多个预设人脸识别模型包括多个不同网络类型的预设人脸识别模型。In one possible implementation manner, the multiple preset face recognition models include multiple preset face recognition models of different network types.
第三方面，本申请实施例提供一种电子设备，包括：In a third aspect, an embodiment of the present application provides an electronic device, including:
存储器，上述存储器用于存储计算机程序代码，上述计算机程序代码包括指令，当上述电子设备从上述存储器中读取上述指令，以使得上述电子设备执行以下步骤：Memory, the memory is used to store computer program code, and the computer program code includes instructions, when the electronic equipment reads the instructions from the memory, so that the electronic equipment performs the following steps:
响应于用户的操作，获取用户的人脸图像；In response to the user's operation, obtain the user's face image;
对所述人脸图像进行人脸检测；performing face detection on the face image;
使用预设的活体识别模型对检测到的人脸进行活体识别，得到第一识别结果；其中，所述第一识别结果用于表征人脸的分类，所述预设的活体识别模型由第一方面所述方法得到的样本进行训练得到；Use a preset living body recognition model to perform living body recognition on the detected face, and obtain a first recognition result; wherein, the first recognition result is used to characterize the classification of the face, and the preset living body recognition model is determined by the first recognition result. The samples obtained by the method described in the aspect are obtained by training;
基于所述第一识别结果，使用预设的人脸识别模型对检测到的人脸进行人脸识别，得到第二识别结果；其中，所述第二识别结果用于表征识别到的目标对象，所述预设的活体识别模型由第一方面所述方法得到的样本进行训练得到；Based on the first recognition result, a preset face recognition model is used to perform face recognition on the detected face to obtain a second recognition result; wherein, the second recognition result is used to represent the recognized target object, The preset living body recognition model is obtained by training the samples obtained by the method described in the first aspect;
基于所述第二识别结果完成支付。The payment is completed based on the second identification result.
第四方面，本申请实施例提供一种计算机可读存储介质，该计算机可读存储介质中存储有计算机程序，当其在计算机上运行时，使得计算机执行如第一方面所述的方法。In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when it runs on a computer, causes the computer to execute the method described in the first aspect.
第五方面，本申请实施例提供一种计算机程序，当上述计算机程序被计算机执行时，用于执行第一方面所述的方法。In a fifth aspect, an embodiment of the present application provides a computer program, which is used to execute the method described in the first aspect when the computer program is executed by a computer.
在一种可能的设计中，第五方面中的程序可以全部或者部分存储在与处理器封装在一起的存储介质上，也可以部分或者全部存储在不与处理器封装在一起的存储器上。In a possible design, the program in the fifth aspect may be stored in whole or in part on a storage medium packaged with the processor, and may also be stored in part or in part in a memory not packaged with the processor.
附图说明Description of drawings
图1为本申请实施例提供的样本生成方法的流程示意图；1 is a schematic flowchart of a sample generation method provided by an embodiment of the present application;
图2为本申请实施例提供的支付方法的流程示意图；2 is a schematic flowchart of a payment method provided by an embodiment of the present application;
图3为本申请实施例提供的样本生成系统的结构示意图；3 is a schematic structural diagram of a sample generation system provided by an embodiment of the present application;
图4为本申请实施例提供的电子设备的硬件结构示意图。FIG. 4 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application.
具体实施方式Detailed ways
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述。其中，在本申请实施例的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. Wherein, in the description of the embodiments of the present application, unless otherwise stated, “/” means or means, for example, A/B can mean A or B; “and/or” in this document is only a description of the associated object The association relationship of , indicates that there can be three kinds of relationships, for example, A and/or B, can indicate that A exists alone, A and B exist at the same time, and B exists alone.
以下，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本申请实施例的描述中，除非另有说明，“多个”的含义是两个或两个以上。Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may expressly or implicitly include one or more of that feature. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.
随着人工智能技术的不断发展，人脸识别技术在人们的日常生活中应用的越来越广泛，例如，刷脸支付。然而，在实际应用中，会存在不法分子进行对抗攻击，例如，带着面具冒充其他合法用户进行盗刷。With the continuous development of artificial intelligence technology, face recognition technology is more and more widely used in people's daily life, for example, face payment. However, in practical applications, there will be criminals conducting adversarial attacks, for example, posing as other legitimate users with masks for stealing.
为了能识别出上述盗刷行为，通常会对刷脸支付设备进行训练，用于防御上述这种对抗攻击。在上述训练过程中，需要大量的攻击样本进行训练。然而目前的技术很难产生高质量的攻击样本，而低质量攻击样本的训练难以提高刷脸支付设备识别对抗攻击的能力，因此，亟需一种方法能生成高质量的攻击样本。In order to identify the above-mentioned fraudulent brushing behaviors, face-swiping payment devices are usually trained to defend against the above-mentioned adversarial attacks. In the above training process, a large number of attack samples are required for training. However, the current technology is difficult to generate high-quality attack samples, and the training of low-quality attack samples is difficult to improve the ability of face-swiping payment devices to identify adversarial attacks. Therefore, there is an urgent need for a method to generate high-quality attack samples.
基于上述问题，本申请实施例提出了一种样本生成方法。Based on the above problem, an embodiment of the present application proposes a sample generation method.
现结合图1对本申请实施例提供的样本生成方法进行说明。The sample generation method provided by the embodiment of the present application will now be described with reference to FIG. 1 .
如图1所示为本申请实施例提供的样本生成方法一个实施例的流程示意图，包括：FIG. 1 is a schematic flowchart of an embodiment of the sample generation method provided by the embodiment of the present application, including:
步骤101，获取训练数据集合。Step 101, acquiring a training data set.
具体地，该训练数据集合可以包括第一训练数据集合及第二训练数据集合。其中，该第一训练数据集合可以包括多个训练数据，该训练数据可以是任意用户(为说明方便，下文将此处的用户称为“第一用户”)的图像(为说明方便，下文将该图像称为“第一用户图像”)，也就是说，一个训练数据可以是一个图像(例如，第一用户图像)，该第一用户图像可以是攻击图像，也就是仿冒的图像。其中，该第一用户图像可以通过互联网图库获取，可以从本地图库获取，也可以通过拍摄获得，本申请实施例对上述第一用户图像的来源不作特殊限定。可以理解的是，为了训练的有效性，通常需要多个训练数据，因此，需要多个第一用户图像进行训练。此外，上述第一用户可以是同一个用户，也就是说，上述多个第一用户图像可以是同一个用户的多张图像。Specifically, the training data set may include a first training data set and a second training data set. The first training data set may include a plurality of training data, and the training data may be images of any user (for the convenience of description, the user here is referred to as the "first user") (for the convenience of description, the following will be This image is referred to as a "first user image"), that is, a training data may be an image (eg, a first user image), which may be an attack image, ie a fake image. Wherein, the first user image may be obtained through an Internet gallery, may be obtained from a local gallery, or may be obtained by photographing, and the source of the above-mentioned first user image is not particularly limited in this embodiment of the present application. It can be understood that, for training effectiveness, multiple training data are usually required, and thus, multiple first user images are required for training. In addition, the above-mentioned first user may be the same user, that is, the above-mentioned multiple first user images may be multiple images of the same user.
优选地，为了提高训练的质量，避免训练过拟合，上述第一用户可以是多个不同的用户，也就是说，上述多个第一用户图像可以是不同用户的图像。Preferably, in order to improve the quality of training and avoid training overfitting, the first user may be multiple different users, that is, the multiple first user images may be images of different users.
该第二训练数据集合也可以包括多个训练数据，该训练数据可以是任意用户(为说明方便，下文将此处的用户称为“第二用户”)的图像(为说明方便，下文将该图像称为“第二用户图像”)，也就是说，一个训练数据可以是一个图像(例如，第二用户图像)，该第二用户图像可以是被识别的图像，也就是被仿冒的图像。同样地，该第二用户图像可以通过互联网图库获取，可以从本地图库获取，也可以通过拍摄获得，本申请实施例对上述第二用户图像的来源不作特殊限定。可以理解的是，为了训练的有效性，通常需要多个训练数据，因此，需要多个第二用户图像进行训练。此外，上述第二用户可以是同一个用户，也就是说，上述多个第二用户图像可以是同一个用户的多张图像。The second training data set may also include a plurality of training data, and the training data may be images of any user (for the convenience of description, the user here is referred to as the "second user") (for the convenience of description, the following The image is referred to as a "second user image"), that is, a training data may be an image (eg, a second user image) that may be an identified image, ie a faked image. Similarly, the second user image may be obtained through an Internet gallery, may be obtained from a local gallery, or may be obtained by shooting, and the embodiment of the present application does not specifically limit the source of the second user image. It can be understood that, for training effectiveness, multiple training data are usually required, and thus, multiple second user images are required for training. In addition, the above-mentioned second user may be the same user, that is, the above-mentioned multiple images of the second user may be multiple images of the same user.
优选地，为了提高训练的质量，避免训练过拟合，上述第二用户可以是多个不同的用户，也就是说，上述多个第二用户图像可以是不同用户的图像。Preferably, in order to improve the quality of training and avoid training overfitting, the second user may be multiple different users, that is, the multiple second user images may be images of different users.
需要说明的是，上述第二用户与第一用户不相同。It should be noted that the above-mentioned second user is different from the first user.
步骤102，将第一用户图像进行初始化处理。Step 102: Initialize the first user image.
具体地，当获取到训练数据集合(例如，多个第一用户图像)后，可以对上述训练数据集合中的任意一个第一用户图像进行初始化处理。其中，该初始化处理可以是对上述第一用户图像进行遮盖。示例性的，如图1所示，第一用户图像1000进行初始化遮盖之后，可以得到第一用户遮盖图像1010，该第一用户遮盖图像1010包括可见部分10101及不可见部分10102。该可见部分10101包括眉、眼、鼻、口等部分，该可见部分10101没有遮挡物，因此，该可见部分10101中的器官都可见。该不可见部分10102包括除上述眉、眼、鼻、口外的部分，该不可见部分10102由于被面具遮挡，因此，该不可见部分10102中的器官都不可见。Specifically, after the training data set (for example, a plurality of first user images) is acquired, initialization processing may be performed on any one of the first user images in the above-mentioned training data set. Wherein, the initialization process may be covering the above-mentioned first user image. Exemplarily, as shown in FIG. 1 , after initial masking of the
步骤103，将第一用户遮盖图像与第二用户图像进行叠加，得到叠加图像。Step 103 , superimposing the first user mask image and the second user image to obtain a superimposed image.
具体地，当获取到上述第一用户遮盖图像后，可以在上述第二训练数据中任意选取一个第二用户图像。接着，可以将上述第一用户遮盖图像与第二用户图像进行叠加，由此可以得到叠加图像，该叠加图像可以认为是一个带了面具的攻击图像，也就是仿冒图像。其中，上述叠加可以是将上述第一用户遮盖图像中的可见部分覆盖在第二用户图像上。示例性的，以图1为例，可以将第一用户遮盖图像1010中的可见部分10101覆盖在第二用户图像2000上，由此可以得到叠加图像2010。可以理解的是，上述叠加完成的仅是可见部分10101覆盖在第二用户图像2000上，不可见部分10102不覆盖在第二用户图像2000上。因此，叠加图像2010上眉、眼、鼻、口等部分呈现的是第一用户图像1000中的眉、眼、鼻、口，叠加图像2010上除眉、眼、鼻、口外的部分呈现的是第二用户图像2000上的内容。Specifically, after the above-mentioned first user masking image is obtained, a second user image may be arbitrarily selected from the above-mentioned second training data. Next, the above-mentioned first user masking image and the second user image may be superimposed, thereby obtaining a superimposed image, which may be regarded as an attack image with a mask, that is, a counterfeit image. Wherein, the above-mentioned superimposition may be to overlay the visible part of the above-mentioned first user-covered image on the second user image. Exemplarily, taking FIG. 1 as an example, the
步骤104，对叠加图像进行数据增强，得到增强图像。Step 104, performing data enhancement on the superimposed image to obtain an enhanced image.
可选地，当获取到叠加图像之后，可以对上述叠加图像进行数据增强。其中，该数据增强可以包括对图像的宽度进行变换、图像的旋转等操作。通过上述数据增强，可以提高叠加图像的纹理，有利于识别。Optionally, after the superimposed image is acquired, data enhancement may be performed on the superimposed image. The data enhancement may include operations such as transforming the width of the image, rotating the image, and the like. Through the above data enhancement, the texture of the superimposed image can be improved, which is beneficial to the recognition.
需要说明的是，本步骤104是可选步骤，也就是说，也可以不执行本步骤104，直接将叠加图像输入人脸识别模型进行识别(例如，执行步骤105)。It should be noted that this step 104 is an optional step, that is, this step 104 may not be performed, and the superimposed image is directly input into the face recognition model for recognition (for example, step 105 is performed).
步骤105，将增强图像输入预设的人脸识别模型进行识别。Step 105: Input the enhanced image into a preset face recognition model for recognition.
具体地，当获取到增强图像后，可以将上述增强图像输入至预设的人脸识别模型中进行识别，用于获得识别结果。该识别结果可以用于表征该增强图像与第二用户图像之间的相似度。Specifically, after the enhanced image is acquired, the aforementioned enhanced image can be input into a preset face recognition model for recognition, so as to obtain the recognition result. The recognition result can be used to characterize the similarity between the enhanced image and the second user image.
需要说明的是，本步骤105中如果获取到增强图像(例如，经过步骤104进行数据增强)，则可以直接将增强图像输入预设的人脸识别模型。若本步骤105中仅获取到叠加图像，则可以直接将叠加图像输入预设的人脸识别模型。为说明方便，下文通过对增强图像进行识别为例进行说明。It should be noted that, if the enhanced image is obtained in step 105 (for example, data enhancement is performed through step 104 ), the enhanced image may be directly input into the preset face recognition model. If only the superimposed image is acquired in this step 105, the superimposed image may be directly input into the preset face recognition model. For the convenience of description, the following description is given by taking the recognition of the enhanced image as an example.
其中，上述预设的人脸识别模型可以是神经网络模型，上述人脸识别模型也可以是其他类型的深度学习网络模型。本申请实施例对上述人脸识别模型的网络类型不作特殊限定。Wherein, the above-mentioned preset face recognition model may be a neural network model, and the above-mentioned face recognition model may also be other types of deep learning network models. This embodiment of the present application does not specifically limit the network type of the above face recognition model.
步骤106，基于识别结果计算损失，并基于损失对第一用户遮盖图像进行梯度更新，基于更新结果得到攻击样本。Step 106: Calculate the loss based on the recognition result, perform gradient update on the first user mask image based on the loss, and obtain an attack sample based on the update result.
具体地，当该人脸识别模型对本次的增强图像进行识别，并得到识别结果之后，可以基于上述识别结果与第一用户图像计算损失，例如，可以将上述第一用户图像作为标签值，上述识别结果作为预测值，接着，可以基于上述标签值及预测值计算损失，并可以由上述损失计算梯度值。当计算得到梯度值之后，可以基于该梯度值对第一用户遮盖图像进行梯度更新，由此可以得到第一用户遮盖更新图像。Specifically, when the face recognition model recognizes the enhanced image this time and obtains the recognition result, the loss can be calculated based on the recognition result and the first user image. For example, the first user image can be used as the label value, The above-mentioned recognition result is used as a predicted value, and then a loss can be calculated based on the above-mentioned label value and predicted value, and a gradient value can be calculated from the above-mentioned loss. After the gradient value is obtained by calculation, gradient update may be performed on the first user cover image based on the gradient value, thereby obtaining the first user cover update image.
接着，可以通过上述步骤103-步骤105的方式对上述第一用户遮盖更新图像进行处理，例如，可以将第一用户遮盖更新图像与第二用户图像进行叠加，将叠加后得到的图像进行增强，并将增强后得到的图像输入至预设的人脸识别模型进行识别，由此可以得到损失。当上述损失达到预设目标(例如，该预设目标可以是增强图像与第二用户图像之间的相似度阈值)后，可以结束对上述第一用户图像的训练，其中，最后一次得到的叠加图像可以作为最终的攻击样本。可选地，也可以将最后一次得到的增强图像作为最终的攻击样本。Next, the above-mentioned first user cover-up update image may be processed in the manner of the above-mentioned steps 103 to 105. For example, the first user cover-up update image and the second user image may be superimposed, and the image obtained after the superposition is enhanced, The enhanced image is input to the preset face recognition model for recognition, and the loss can be obtained. When the above-mentioned loss reaches a preset target (for example, the preset target may be the similarity threshold between the enhanced image and the second user image), the training on the above-mentioned first user image can be ended, wherein the last obtained superimposition Images can be used as final attack samples. Optionally, the last enhanced image obtained can also be used as the final attack sample.
举例来说，以图1所示的第一用户图像1000及第二用户图像2000为例，当第一次对上述第一用户图像1000进行初始化，并和第二用户图像2000进行叠加之后，得到叠加图像2010。接着，对上述叠加图像2010进行识别之后，根据计算得到的损失判断是否满足预设条件。若满足预设条件，则可以将上述叠加图像2010作为本次的攻击样本。若不满足预设条件，则可以基于上述损失对上述第一用户遮盖图像1010进行梯度更新，由此可以得到第一用户遮盖第一更新图像1011。将上述第一用户遮盖第一更新图像1011与第二用户图像2000进行叠加，可以得到叠加第一更新图像2011。接着，对上述叠加图像2011进行识别之后，根据计算得到的损失判断是否满足预设条件，若满足预设条件，则可以将上述叠加图像2011作为本次的攻击样本。若不满足预设条件，则可以基于上述损失进一步对上述第一用户遮盖第一更新图像1011进行梯度更新，由此可以得到第一用户遮盖第二更新图像1012，并可以进一步按照上述方式进行叠加和识别，直到损失满足预设条件为止。需要说明的是，上述仅以叠加图像作为攻击样本为例进行了示例性说明，但并不限定于此。在一些实施例中，也可以将上述增强图像作为攻击样本，具体将上述增强样本作为攻击样本的方式可以参考上述将叠加图像作为攻击样本的方式，在此不再赘述。For example, taking the
可以理解的是，当一个第一用户图像与一个第二用户图像经过训练得到对应的攻击样本后，可以对下一个第一用户图像及下一个第二用户图像进行训练，用于得到对应的攻击样本。可选地，也可以仅替换上述第一用户图像及上述第二用户图像中的任一个图像，例如，仅替换第一用户图像或仅替换第二用户图像。It can be understood that after a first user image and a second user image are trained to obtain corresponding attack samples, the next first user image and the next second user image can be trained to obtain the corresponding attack samples. sample. Optionally, only any one of the first user image and the second user image may be replaced, for example, only the first user image or only the second user image may be replaced.
当上述人脸识别模型对任意一个叠加图像或增强图像进行识别之后，可以获得识别结果。接着，可以根据上述识别结果与该识别结果对应的第一用户图像计算损失(LOSS)。由于本申请实施例的目的是要构建攻击样本，因此，上述识别结果是要使得叠加图像或增强图像与第一用户图像相似，由此才能构造高质量的攻击样本。也就是说，上述损失是基于叠加图像或增强图像与第一用户图像之间的相似度为目标，叠加图像或增强图像与第一用户图像之间的相似度越高，损失越小，叠加图像或增强图像与第一用户图像之间的相似度越低，损失越大。通过上述训练，根据梯度不断更新上述遮盖图像(例如，第一用户遮盖图像1010)，使得损失降低，由此可以达到叠加图像或增强图像与第一用户图像相似的预设条件。其中，该预设条件可以是一个相似度阈值，例如，当叠加图像或增强图像与第一用户图像之间的相似度达到80％，可以认为达到预设条件。需要说明的是，上述80％的数值仅为示例性说明，并不构成对本申请实施例的限定，在一些实施例中，也可以是其他数值。After the above face recognition model recognizes any one of the superimposed images or enhanced images, the recognition result can be obtained. Next, a loss (LOSS) may be calculated according to the above recognition result and the first user image corresponding to the recognition result. Since the purpose of the embodiment of the present application is to construct an attack sample, the above identification result is to make the superimposed image or the enhanced image similar to the first user image, so that a high-quality attack sample can be constructed. That is to say, the above loss is based on the similarity between the superimposed image or enhanced image and the first user image as the goal. The higher the similarity between the superimposed image or enhanced image and the first user image, the smaller the loss. Or the lower the similarity between the enhanced image and the first user image, the greater the loss. Through the above training, the above-mentioned masked image (eg, the first user masked image 1010 ) is continuously updated according to the gradient, so that the loss is reduced, so that the preset condition that the superimposed image or the enhanced image is similar to the first user image can be achieved. The preset condition may be a similarity threshold. For example, when the similarity between the superimposed image or the enhanced image and the first user image reaches 80%, it can be considered that the preset condition is reached. It should be noted that the above-mentioned numerical value of 80% is only illustrative, and does not constitute a limitation on the embodiments of the present application, and in some embodiments, other numerical values may also be used.
优选地，在上述训练过程中，也可以通过多个人脸识别模型(例如，模型1、模型2…模型n)对上述第一用户图像及第二用户图像进行训练。示例性的，可以将上述叠加图像或增强图像输入一个人脸识别模型中进行训练。当上述叠加图像或增强图像经过当前人脸识别模型的训练，满足预设条件后，可以通过下一个人脸识别模型对上述叠加图像或增强图像进行训练。其中，上述多个人脸识别模型可以是不同类型的网络模型，由此可以使得训练结果可以适应不同种类的网络模型，进而可以提高攻击样本的质量。当经过n个人脸识别模型对上述叠加图像或增强图像进行训练，并都满足预设条件之后，可以将最后一个人脸识别模型中训练得到的叠加图像或增强图像作为攻击样本。其中，n的值可以预先设置，也就是说，可以预先选取n个人脸识别模型。Preferably, in the above-mentioned training process, the above-mentioned first user image and second user image may also be trained by using multiple face recognition models (eg,
可选地，在使用上述n个人脸识别模型进行训练时，还可以通过分批训练的方式进行训练。示例性的，可以首先通过n-1个人脸识别模型进行监督训练。其中，上述n-1个人脸识别模型可以随机选取。当通过上述n-1个人脸识别模型训练完之后，可以获取梯度更新后的攻击样本。接着，可以通过剩余的1个人脸识别模型进行监督训练，由此可以得到最终的攻击样本。相对于同时使用n个的方案，分批训练的方式可以进一步提升攻击样本的攻击成功率和稳定性。Optionally, when the above n face recognition models are used for training, the training may also be performed in batches. Exemplarily, supervised training may be performed first through an n-1 face recognition model. Wherein, the above n-1 face recognition models can be randomly selected. After the above n-1 face recognition model is trained, the attack samples after the gradient update can be obtained. Then, supervised training can be performed through the remaining one face recognition model, and the final attack sample can be obtained. Compared with the scheme of using n at the same time, the batch training method can further improve the attack success rate and stability of the attack samples.
本申请实施例还提出了一种支付方法，用于防御对抗攻击刷脸支付。The embodiment of the present application also proposes a payment method, which is used for face-swiping payment to defend against adversarial attacks.
现结合图2对本申请实施例提供的支付方法进行说明。The payment method provided by the embodiment of the present application will now be described with reference to FIG. 2 .
如图2所示为本申请实施例提供的支付方法一个实施例的流程示意图，包括：FIG. 2 shows a schematic flowchart of an embodiment of the payment method provided by the embodiment of the present application, including:
步骤201，响应于用户的操作，获取用户人脸图像。
具体地，用户可以在电子设备上进行操作，示例性的，用户可以在电子设备的显示屏上点击刷脸的功能控件。其中，上述电子设备可以是刷脸支付设备，例如，自动售卖机。响应于用户的操作，电子设备可以打开摄像头，用于获取用户的人脸图像。Specifically, the user may perform operations on the electronic device. Exemplarily, the user may click on the function control for brushing the face on the display screen of the electronic device. Wherein, the above-mentioned electronic device may be a face-scanning payment device, for example, a vending machine. In response to the user's operation, the electronic device may turn on the camera for acquiring the user's face image.
步骤202，对人脸图像进行检测，确定人脸的位置。Step 202: Detect the face image to determine the position of the face.
具体地，当电子设备获取到用户的人脸图像后，可以对上述用户的人脸图像进行人脸检测，用于对用户的人脸进行定位，由此可以获得人脸在电子设备的预览界面中的位置。其中，上述定位人脸的位置可以使用现有的人脸检测算法，在此不再赘述。Specifically, after the electronic device obtains the user's face image, it can perform face detection on the above-mentioned user's face image to locate the user's face, thereby obtaining a preview interface of the face on the electronic device. in the location. Wherein, the above-mentioned position of locating the face may use an existing face detection algorithm, which will not be repeated here.
步骤203，对人脸进行活体识别。
具体地，当电子设备对用户的人脸图像中的人脸进行定位之后，可以对上述人脸进行活体识别。其中，上述活体识别用于识别活体或假体，也就是对上述人脸的类型进行分类，上述类别可以包括活体及假体。上述活体识别可以使用预设的活体识别模型进行识别。上述预设的活体识别模型可以通过训练获得。在上述训练过程中，可以使用上述样本生成方法生成的攻击样本进行训练，用于提高上述活体识别模型的分类的准确度。Specifically, after the electronic device locates the face in the face image of the user, it can perform living body recognition on the face. The above-mentioned living body recognition is used to identify living bodies or prostheses, that is, to classify the types of the above-mentioned human faces, and the above-mentioned categories may include living bodies and prostheses. The above living body recognition can be performed using a preset living body recognition model. The above preset living body recognition model can be obtained through training. In the above-mentioned training process, the attack samples generated by the above-mentioned sample generation method may be used for training, so as to improve the classification accuracy of the above-mentioned living body recognition model.
步骤204，对人脸进行人脸识别。
具体地，当通过上述步骤203对人脸进行活体识别后，若确定上述人脸是活体类型，也就是说，该人脸是真实的人脸后，可以进一步对上述人脸进行人脸识别，用于识别该人脸是哪个用户。若确定上述人脸是假体类型，也就是说，可能是非法用户戴面具冒充合法用户，此时，可以结束本次支付任务。Specifically, after the living body recognition is performed on the human face through the above-mentioned
其中，上述人脸识别可以使用预设的人脸识别模型进行识别。上述预设的人脸识别模型可以通过训练获得。在上述训练过程中，可以使用上述样本生成方法生成的攻击样本进行训练，用于提高上述人脸识别模型的识别的准确度。Wherein, the above-mentioned face recognition can be performed by using a preset face recognition model. The above preset face recognition model can be obtained through training. In the above-mentioned training process, the attack samples generated by the above-mentioned sample generation method may be used for training, so as to improve the recognition accuracy of the above-mentioned face recognition model.
步骤205，基于人脸识别结果完成支付。
具体地，当对用户的人脸进行人脸识别后，可以得到识别结果，若确定该识别结果与合法用户的人脸匹配，则可以认为认证成功，此时，可以完成支付，例如，可以调用与识别结果对应的人脸的用户的账户，并进行相应的扣款，以完成本次的刷脸支付任务。若匹配不到合法的用户，则可以认为认证失败，本次支付任务结束。Specifically, after performing face recognition on the user's face, the recognition result can be obtained. If it is determined that the recognition result matches the legitimate user's face, the authentication can be considered successful. At this time, the payment can be completed. For example, calling The account of the user with the face corresponding to the recognition result, and the corresponding deduction will be made to complete the face-swiping payment task. If no valid user is matched, it can be considered that the authentication fails, and the payment task ends.
图3为本申请样本生成系统一个实施例的结构示意图，如图3所示，上述样本生成系统30可以包括：获取模块31、遮盖模块32、叠加模块33及生成模块34；其中，FIG. 3 is a schematic structural diagram of an embodiment of the sample generation system of the present application. As shown in FIG. 3 , the above-mentioned
获取模块31，用于获取第一用户图像及第二用户图像；an
遮盖模块32，用于对所述第一用户图像进行面具遮盖，得到第一用户遮盖图像；a
叠加模块33，用于将所述第一用户遮盖图像与所述第二用户图像进行叠加，得到叠加图像；an
生成模块34，用于将所述叠加图像输入一个或多个预设人脸识别模型进行训练，得到攻击样本。The generating
其中一种可能的实现方式中，所述生成模块34还用于In one of the possible implementations, the generating
将所述叠加图像进行数据增强，得到增强图像；performing data enhancement on the superimposed image to obtain an enhanced image;
将所述增强图像输入一个或多个预设人脸识别模型进行训练，得到攻击样本。The enhanced image is input into one or more preset face recognition models for training to obtain attack samples.
其中一种可能的实现方式中，所述生成模块34还用于In one of the possible implementations, the generating
将所述叠加图像输入一个或多个预设人脸识别模型，得到识别结果；Inputting the superimposed image into one or more preset face recognition models to obtain a recognition result;
基于所述识别结果与所述第一用户图像，计算得到损失；Calculate the loss based on the recognition result and the first user image;
基于所述损失对所述第一用户遮盖图像进行梯度更新，得到第一用户遮盖更新图像，并将所述第一用户遮盖更新图像输入一个或多个预设人脸识别模型进行训练。Gradient update is performed on the first user cover image based on the loss to obtain a first user cover update image, and the first user cover update image is input into one or more preset face recognition models for training.
其中一种可能的实现方式中，所述第一用户遮盖图像包括可见部分及不可见部分，所述叠加模块33还用于In one possible implementation manner, the first user masking image includes a visible part and an invisible part, and the
将所述第一用户遮盖图像中的可见部分覆盖在所述第二用户图像上，得到叠加图像。A superimposed image is obtained by overlaying the visible portion of the first user mask image on the second user image.
其中一种可能的实现方式中，所述多个预设人脸识别模型包括多个不同网络类型的预设人脸识别模型。In one possible implementation manner, the multiple preset face recognition models include multiple preset face recognition models of different network types.
图4示例性的示出了本申请实施例所提供的电子设备100的结构示意图，如图4所示，电子设备100可以包括处理器110，外部存储器接口120，内部存储器121，天线1，天线2，移动通信模块150，无线通信模块160，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。FIG. 4 exemplarily shows a schematic structural diagram of an electronic device 100 provided by an embodiment of the present application. As shown in FIG. 4 , the electronic device 100 may include a
可以理解的是，本发明实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。It can be understood that, the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or less components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processingunit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。The
控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。A memory may also be provided in the
电子设备100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。The wireless communication function of the electronic device 100 may be implemented by the
天线1和天线2均可用于发射和接收电磁波信号。电子设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线，当然考虑到设计的简单和其他因素的影响，也可以给每种通信方式配备单独的天线。在另外一些实施例中，天线可以和调谐开关结合使用。Both
移动通信模块150可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。移动通信模块150可以由天线1接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大，经天线1转为电磁波辐射出去。在一些实施例中，移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中，移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。The
无线通信模块160可以提供应用在电子设备100上的包括无线局域网(wirelesslocal area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术(infrared，IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波，将电磁波信号调频以及滤波处理，将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。The wireless communication module 160 can provide wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), and global navigation satellite systems applied on the electronic device 100 . (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the
在一些实施例中，电子设备100的天线1和移动通信模块150耦合，天线2和无线通信模块160耦合，使得电子设备100可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯系统(global system for mobile communications，GSM)，通用分组无线服务(general packet radio service，GPRS)，码分多址接入(codedivision multiple access，CDMA)，宽带码分多址(wideband code division multipleaccess，WCDMA)，时分码分多址(time-division code division multiple access，TD-SCDMA)，长期演进(long term evolution，LTE)，BT，GNSS，WLAN，NFC，FM，和/或IR技术等。所述GNSS可以包括全球卫星定位系统(global positioning system，GPS)，全球导航卫星系统(global navigation satellite system，GLONASS)，北斗卫星导航系统(beidounavigation satellite system，BDS)，准天顶卫星系统(quasi-zenith satellitesystem，QZSS)和/或星基增强系统(satellite based augmentation systems，SBAS)。In some embodiments, the
电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。The electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering.
显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emittingdiode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrixorganic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emittingdiode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot lightemitting diodes，QLED)等。在一些实施例中，电子设备100可以包括1个或N个显示屏194，N为大于1的正整数。Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light-emitting diode). , AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diodes (quantum dot light emitting diodes, QLED) and so on. In some embodiments, the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one.
电子设备100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.
ISP用于处理摄像头193反馈的数据。例如，拍照时，打开快门，光线通过镜头被传递到摄像头感光元件上，光信号转换为电信号，摄像头感光元件将所述电信号传递给ISP处理，转化为肉眼可见的图像。ISP还可以对图像的噪点，亮度，肤色进行算法优化。ISP还可以对拍摄场景的曝光，色温等参数优化。在一些实施例中，ISP可以设置在摄像头193中。The ISP is used to process the data fed back by the camera 193 . For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin tone. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193 .
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。在一些实施例中，电子设备100可以包括1个或N个摄像头193，N为大于1的正整数。Camera 193 is used to capture still images or video. The object is projected through the lens to generate an optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.
数字信号处理器用于处理数字信号，除了可以处理数字图像信号，还可以处理其他数字信号。例如，当电子设备100在频点选择时，数字信号处理器用于对频点能量进行傅里叶变换等。A digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy and so on.
视频编解码器用于对数字视频压缩或解压缩。电子设备100可以支持一种或多种视频编解码器。这样，电子设备100可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, for example, moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.
NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现电子设备100的智能认知等应用，例如：图像识别，人脸识别，语音识别，文本理解等。The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process the input information, and can continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展电子设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。The
内部存储器121可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储电子设备100使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。处理器110通过运行存储在内部存储器121的指令，和/或存储在设置于处理器中的存储器的指令，执行电子设备100的各种功能应用以及数据处理。Internal memory 121 may be used to store computer executable program code, which includes instructions. The internal memory 121 may include a storage program area and a storage data area. The storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like. The storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 100 and the like. In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like. The
SIM卡接口195用于连接SIM卡。SIM卡可以通过插入SIM卡接口195，或从SIM卡接口195拔出，实现和电子设备100的接触和分离。电子设备100可以支持1个或N个SIM卡接口，N为大于1的正整数。SIM卡接口195可以支持Nano SIM卡，Micro SIM卡，SIM卡等。同一个SIM卡接口195可以同时插入多张卡。所述多张卡的类型可以相同，也可以不同。SIM卡接口195也可以兼容不同类型的SIM卡。SIM卡接口195也可以兼容外部存储卡。电子设备100通过SIM卡和网络交互，实现通话以及数据通信等功能。在一些实施例中，电子设备100采用eSIM，即：嵌入式SIM卡。eSIM卡可以嵌在电子设备100中，不能和电子设备100分离。The
可以理解的是，本申请实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备100的结构限定。在本申请另一些实施例中，电子设备100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 . In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.
可以理解的是，上述电子设备100等为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请实施例能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请实施例的范围。It can be understood that, in order to realize the above-mentioned functions, the above-mentioned electronic device 100 and the like include corresponding hardware structures and/or software modules for executing each function. Those skilled in the art should easily realize that, in conjunction with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in hardware or a combination of hardware and computer software. Whether a function is performed by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Experts may use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of the embodiments of the present application.
本申请实施例可以根据上述方法示例对上述电子设备100等进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。In this embodiment of the present application, the electronic device 100 and the like can be divided into functional modules according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. It should be noted that, the division of modules in the embodiments of the present application is schematic, and is only a logical function division, and there may be other division manners in actual implementation.
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。From the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above functional modules is used as an example for illustration. In practical applications, the above functions can be allocated as required. It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. For the specific working process of the system, apparatus and unit described above, reference may be made to the corresponding process in the foregoing method embodiments, and details are not described herein again.
在本申请实施例各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。Each functional unit in each of the embodiments of the embodiments of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：快闪存储器、移动硬盘、只读存储器、随机存取存储器、磁碟或者光盘等各种可以存储程序代码的介质。The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence, or the parts that make contributions to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage The medium includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: flash memory, removable hard disk, read-only memory, random access memory, magnetic disk or optical disk and other media that can store program codes.
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何在本申请揭露的技术范围内的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this, and any changes or substitutions within the technical scope disclosed in the present application should be covered within the protection scope of the present application. . Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims (13)
Translated fromChinesePriority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111184870.5ACN113947409B (en) | 2021-10-12 | 2021-10-12 | Sample generation method, payment method, sample generation system and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111184870.5ACN113947409B (en) | 2021-10-12 | 2021-10-12 | Sample generation method, payment method, sample generation system and related equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113947409Atrue CN113947409A (en) | 2022-01-18 |
| CN113947409B CN113947409B (en) | 2025-02-25 |
Family
ID=79330491
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111184870.5AActiveCN113947409B (en) | 2021-10-12 | 2021-10-12 | Sample generation method, payment method, sample generation system and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113947409B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024221440A1 (en)* | 2023-04-28 | 2024-10-31 | 京东方科技集团股份有限公司 | Image processing method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190028509A1 (en)* | 2017-07-20 | 2019-01-24 | Barracuda Networks, Inc. | System and method for ai-based real-time communication fraud detection and prevention |
| US20190034703A1 (en)* | 2017-07-26 | 2019-01-31 | Baidu Online Network Technology (Beijing) Co., Ltd. | Attack sample generating method and apparatus, device and storage medium |
| CN111914628A (en)* | 2020-06-19 | 2020-11-10 | 北京百度网讯科技有限公司 | Training method and device for face recognition model |
| WO2020233564A1 (en)* | 2019-05-21 | 2020-11-26 | 华为技术有限公司 | Method and electronic device for detecting adversarial example |
| CN113221767A (en)* | 2021-05-18 | 2021-08-06 | 北京百度网讯科技有限公司 | Method for training living body face recognition model and method for recognizing living body face and related device |
| CN113256298A (en)* | 2020-02-10 | 2021-08-13 | 深圳市光鉴科技有限公司 | Payment system with 3D face recognition and using method |
- 2021
- 2021-10-12CNCN202111184870.5Apatent/CN113947409B/enactiveActive
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190028509A1 (en)* | 2017-07-20 | 2019-01-24 | Barracuda Networks, Inc. | System and method for ai-based real-time communication fraud detection and prevention |
| US20190034703A1 (en)* | 2017-07-26 | 2019-01-31 | Baidu Online Network Technology (Beijing) Co., Ltd. | Attack sample generating method and apparatus, device and storage medium |
| WO2020233564A1 (en)* | 2019-05-21 | 2020-11-26 | 华为技术有限公司 | Method and electronic device for detecting adversarial example |
| CN113256298A (en)* | 2020-02-10 | 2021-08-13 | 深圳市光鉴科技有限公司 | Payment system with 3D face recognition and using method |
| CN111914628A (en)* | 2020-06-19 | 2020-11-10 | 北京百度网讯科技有限公司 | Training method and device for face recognition model |
| CN113221767A (en)* | 2021-05-18 | 2021-08-06 | 北京百度网讯科技有限公司 | Method for training living body face recognition model and method for recognizing living body face and related device |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024221440A1 (en)* | 2023-04-28 | 2024-10-31 | 京东方科技集团股份有限公司 | Image processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113947409B (en) | 2025-02-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113538273B (en) | Image processing method and image processing apparatus | |
| CN112712470B (en) | Image enhancement method and device | |
| US12254544B2 (en) | Image-text fusion method and apparatus, and electronic device | |
| CN113591517B (en) | A living body detection method and related equipment | |
| CN113066048B (en) | A method and device for determining confidence of segmentation image | |
| CN114242037B (en) | Method and device for generating virtual character | |
| WO2020140726A1 (en) | Photographing method and electronic device | |
| CN112446832A (en) | Image processing method and electronic equipment | |
| CN115565212B (en) | Image processing method, neural network model training method and device | |
| CN112308202A (en) | Method for determining decision factors of convolutional neural network and electronic equipment | |
| CN117274109B (en) | Image processing method, noise reduction model training method and electronic equipment | |
| CN116309710B (en) | Target tracking method and electronic equipment | |
| CN113947409A (en) | Sample generation method, payment method, sample generation system and related equipment | |
| CN112183208B (en) | Identity recognition method and device | |
| EP4538958A1 (en) | Image rendering method and apparatus | |
| CN116843583B (en) | Image processing method, device, electronic equipment and storage medium | |
| CN112603259B (en) | Skin roughness detection method and electronic equipment | |
| CN115546858B (en) | Face image processing method and electronic equipment | |
| CN116912602B (en) | Training method of image processing model, image processing method and electronic equipment | |
| CN117729445B (en) | Image processing method, electronic device and computer readable storage medium | |
| CN117714884B (en) | Target tracking method and electronic equipment | |
| CN115861042B (en) | Image processing method, electronic equipment and medium | |
| CN113793590B (en) | Speech synthesis method and device | |
| CN117156260A (en) | Photographing method and electronic equipment | |
| CN120786168A (en) | Image processing method and device, cloud server and terminal equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |