CN112198966B - Stroke identification method and system based on FMCW radar system - Google Patents
Stroke identification method and system based on FMCW radar systemDownload PDFInfo
- Publication number
- CN112198966B CN112198966BCN202011420741.7ACN202011420741ACN112198966BCN 112198966 BCN112198966 BCN 112198966BCN 202011420741 ACN202011420741 ACN 202011420741ACN 112198966 BCN112198966 BCN 112198966B
- Authority
- CN
- China
- Prior art keywords
- matrix
- stroke
- feature
- algorithm
- recognition model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
- G01S17/08—Systems determining position data of a target for measuring distance only
- G01S17/32—Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Electromagnetism (AREA)
- Human Computer Interaction (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computer Networks & Wireless Communication (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Character Discrimination (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本发明属于手势识别技术领域，具体是涉及到一种基于FMCW雷达系统的笔画识别方法以及系统。The invention belongs to the technical field of gesture recognition, and in particular relates to a stroke recognition method and system based on an FMCW radar system.
背景技术Background technique
目前常用的人机交互方法有键盘、鼠标、手写板、触屏输入等，这些方式都是接触式人机交互方式。在很多特殊应用场景下接触式交互设备受环境限制，比如手术室的无菌操作和视觉障碍用户的操作。但随着科技的发展，很多智能终端设备以及人机交互设备出现在市场上，一些智能手机上设置了无障碍模式，提供了语音反馈以便用户在使用设备时不用看着屏幕，这能简化视觉障碍用户的操作，但可能存在隐私问题。针对视觉障碍用户设计专用的人机交互设备，用户需要费大量时间去适应产品，其没有主动去适应用户需求，而需要用户做出变化。At present, the commonly used human-computer interaction methods include keyboard, mouse, handwriting pad, touch screen input, etc. These methods are all contact human-computer interaction methods. In many special application scenarios, contact-based interactive devices are limited by the environment, such as sterile operations in operating rooms and operations by visually impaired users. However, with the development of science and technology, many smart terminal devices and human-computer interaction devices have appeared on the market. Some smart phones are set up with barrier-free mode, which provides voice feedback so that users do not need to look at the screen when using the device, which can simplify the visual Obstructs the user's operation, but may have privacy issues. A special human-computer interaction device is designed for visually impaired users. Users need to spend a lot of time to adapt to the product. They do not take the initiative to adapt to user needs, but require users to make changes.
在非接触式人机交互设备中，基于视觉的人机交互设备有人提出了通过手机摄像头识别汉字的方法：一个是通过摄像头框取，扫描并识别汉字的方法，这种方法需要用户不断调整摄像头的上下距离和左右位置来框取，并点击确认来获取所要识别的汉字，其操作不好控制；另一个是通过先拍得一张包含所需汉字的图片，然后涂抹所要识别汉字，再将涂抹区域进行识别的方法，这种方法步骤过多且涂抹位置不好把握。还有人提出基于计算机视觉手势的文字输入方式：一个途径是人们正常使用纸笔书写然后通过摄像头采集纸上的书写墨迹，将其识别为文字。这种方法仍然受到了外部条件（如，纸张大小等）的限制；另一个途径是直接通过手指在桌面或空中书写虚拟文字，使用摄像头或运动传感器（如，Kinect等）实时采集手指的移动轨迹，之后将轨迹识别为文字，这种方法字与字之间很难切割，只能识别最简单的文字。Among the non-contact human-computer interaction devices, some people have proposed a method of recognizing Chinese characters through mobile phone cameras: one is to frame, scan and recognize Chinese characters through the camera. This method requires the user to constantly adjust the camera. The upper and lower distances and the left and right positions of the screen are framed, and click OK to obtain the Chinese characters to be recognized, which is not easy to control; the other is to take a picture containing the required Chinese characters, then smear the Chinese characters to be recognized, and then The method of identifying the smeared area, this method has too many steps and the smearing position is not easy to grasp. Others have proposed text input methods based on computer vision gestures: one way is that people normally use paper and pen to write and then use a camera to capture the writing ink on the paper and recognize it as text. This method is still limited by external conditions (such as paper size, etc.); another way is to directly write virtual text on the desktop or in the air through the finger, and use the camera or motion sensor (such as Kinect, etc.) to collect the movement trajectory of the finger in real time , and then recognize the trajectory as text. This method is difficult to cut between words, and only the simplest text can be recognized.
基于射频的人机交互设备也只是做到了简单的手势指令操作，从目前已发表的文献资料来看，还没有利用毫米波雷达完成手写汉字基本笔画识别的相关研究。如果能够准确的识别手写汉字基本笔画，后续能发展到非接触式手写汉字人机交互设备，此项研究不仅能运用在无障碍模式中，也能运用在游戏、医疗、军事等人机交互领域。The human-computer interaction equipment based on radio frequency only achieves simple gesture command operations. From the literature published so far, there is no relevant research on the recognition of basic strokes of handwritten Chinese characters using millimeter-wave radar. If the basic strokes of handwritten Chinese characters can be accurately recognized, the subsequent development of non-contact handwritten Chinese character human-computer interaction equipment can be used not only in barrier-free mode, but also in human-computer interaction fields such as games, medical care, and military. .
发明内容SUMMARY OF THE INVENTION
针对现有非接触式人机交互设备存在的上述问题，本发明提供一种基于FMCW雷达系统的笔画识别方法以及系统。In view of the above problems existing in the existing non-contact human-computer interaction equipment, the present invention provides a stroke recognition method and system based on an FMCW radar system.
本发明的技术方案如下：The technical scheme of the present invention is as follows:
第一方面，一种基于FMCW雷达系统的笔画识别方法，包括：A first aspect, a stroke recognition method based on an FMCW radar system, comprising:
获取基于FMCW雷达系统手写汉字包含的至少一个待识别笔画的中频信号数据；Obtain the intermediate frequency signal data of at least one stroke to be recognized contained in the handwritten Chinese character based on the FMCW radar system;
对各所述待识别笔画的所述中频信号数据进行预处理，获取各所述待识别笔画的特征图集合；该预处理包括特征提取和特征增强；Preprocessing the intermediate frequency signal data of each of the strokes to be recognized to obtain a feature map set of each of the strokes to be recognized; the preprocessing includes feature extraction and feature enhancement;
获取训练完成的汉字基本笔画识别模型；所述汉字基本笔画识别模型是指以所述特征图集合为输入参数，以基本笔画类别为输出参数的卷积神经网络模型；Obtain the basic stroke recognition model of Chinese characters that has been trained; the basic Chinese stroke recognition model refers to a convolutional neural network model that takes the feature map set as an input parameter and a basic stroke category as an output parameter;
将各所述待识别笔画的所述特征图集合输入至所述汉字基本笔画识别模型中，并获取所述汉字基本笔画识别模型输出的与每个所述待识别笔画匹配的基本笔画类别。The feature map sets of the strokes to be recognized are input into the basic stroke recognition model for Chinese characters, and the basic stroke categories output by the basic stroke recognition model for Chinese characters that match each of the strokes to be recognized are acquired.
优选地，所述对各所述待识别笔画的所述中频信号数据进行预处理，获取各所述待识别笔画的特征图集合，包括：Preferably, the preprocessing of the intermediate frequency signal data of each of the strokes to be recognized, and the acquisition of a feature map set of each of the strokes to be recognized, includes:
通过第一算法对各所述待识别笔画的所述中频信号数据进行特征提取，获得对应的特征矩阵集合，所述特征矩阵集合包含距离-时间矩阵和角度-时间矩阵；Feature extraction is performed on the intermediate frequency signal data of each of the strokes to be recognized by the first algorithm, and a corresponding feature matrix set is obtained, and the feature matrix set includes a distance-time matrix and an angle-time matrix;
通过第二算法对各所述待识别笔画的所述特征矩阵集合进行特征增强，获得对应的所述特征图集合。The feature enhancement is performed on the feature matrix set of each of the strokes to be recognized by the second algorithm to obtain the corresponding feature map set.
优选地，所述通过第一算法对各所述待识别笔画的所述中频信号数据进行特征提取，获得对应的特征矩阵集合，所述特征矩阵集合包含距离-时间矩阵和角度-时间矩阵，包括：Preferably, the first algorithm is used to perform feature extraction on the intermediate frequency signal data of each of the strokes to be recognized, and a corresponding feature matrix set is obtained, and the feature matrix set includes a distance-time matrix and an angle-time matrix, including :
根据所述中频信号数据的格式和笔画挥动过程的趋势变化特征，通过第三算法获取所述中频信号数据的频率矩阵，并根据所述频率信息的变化获得距离随时间变化的距离-时间矩阵；According to the format of the intermediate frequency signal data and the trend change characteristics of the stroke waving process, the frequency matrix of the intermediate frequency signal data is obtained by the third algorithm, and the distance-time matrix of distance changes with time is obtained according to the change of the frequency information;
根据所述中频信号数据的格式和笔画挥动过程的趋势变化特征，通过第四算法获取所述距离矩阵的相位数据矩阵，并将所述相位数据矩阵转换为角度数据矩阵之后获得角度随时间变化的角度-时间矩阵。According to the format of the intermediate frequency signal data and the trend change characteristics of the stroke waving process, the phase data matrix of the distance matrix is obtained through the fourth algorithm, and the phase data matrix is converted into an angle data matrix to obtain the angle changing with time. Angle-time matrix.
优选地，所述根据所述中频信号数据的格式和笔画挥动过程的趋势变化特征，通过第四算法获取所述距离矩阵的相位数据矩阵，并将所述相位数据矩阵转换为角度数据矩阵之后获得角度随时间变化的角度-时间矩阵，包括：Preferably, according to the format of the intermediate frequency signal data and the trend change characteristics of the stroke waving process, the phase data matrix of the distance matrix is obtained through a fourth algorithm, and the phase data matrix is converted into an angle data matrix. The angle-time matrix of angle variation over time, including:
对所述中频信号数据进行快速傅氏变换，获得所述距离矩阵中每个矩阵元素的相位信息；performing fast Fourier transform on the intermediate frequency signal data to obtain phase information of each matrix element in the distance matrix;
对所述相位矩阵进行协方差运算获得协方差矩阵，并对预设的空间谱函数进行遍历获得所述角度矩阵；performing a covariance operation on the phase matrix to obtain a covariance matrix, and traversing a preset spatial spectral function to obtain the angle matrix;
将所述角度矩阵进行时间序列组合获得所述角度-时间矩阵。The angle-time matrix is obtained by performing time series combination of the angle matrix.
优选地，所述通过第二算法对各所述待识别笔画的所述特征矩阵集合进行特征增强，获得对应的所述特征图集合，包括：Preferably, the feature enhancement is performed on the feature matrix set of each of the strokes to be recognized by the second algorithm to obtain the corresponding feature map set, including:
根据用两种特征矩阵表示的图像获取所述图像中特征区域位置坐标，并根据预设的矩阵框提取图像中笔画挥动过程的趋势变化特征；Obtain the position coordinates of the feature area in the image according to the images represented by two feature matrices, and extract the trend change feature of the stroke waving process in the image according to the preset matrix frame;
通过第五算法获得最佳阈值，根据最佳阈值将两种所述特征矩阵二值化，获得对应的二值化图像；Obtain the optimal threshold through the fifth algorithm, and binarize the two feature matrices according to the optimal threshold to obtain the corresponding binarized image;
对两种所述特征矩阵对应的所述二值化图像进行开操作。An opening operation is performed on the binarized images corresponding to the two feature matrices.
优选地，所述获取训练好的汉字基本笔画识别模型之前，包括：Preferably, before obtaining the trained basic stroke recognition model of Chinese characters, it includes:
获取手写汉字预设基本笔画的第一样本数据和第二样本数据；所述第一样本数据和第二样本数据均包含预设数量的预设基本笔画对应的特征图集合；Obtain first sample data and second sample data of preset basic strokes of handwritten Chinese characters; the first sample data and the second sample data both include a preset number of preset basic strokes corresponding to feature map sets;
搭建待训练的汉字基本笔画识别模型；Build the basic stroke recognition model of Chinese characters to be trained;
设置模型训练参数以及损失函数；Set model training parameters and loss function;
将所述第一样本数据输入至所述汉字基本笔画识别模型进行迭代训练，根据所述损失函数计算所述汉字基本笔画识别模型输出的预测类别结果与实际类别结果之间的损失值；Inputting the first sample data into the basic Chinese character stroke recognition model for iterative training, and calculating the loss value between the predicted category result output by the Chinese basic stroke recognition model and the actual category result according to the loss function;
在每次迭代完成后，将所述第二样本数据输入至所述汉字基本笔画识别模型进行测试，获取所述汉字基本笔画识别模型的预测准确率；After each iteration is completed, the second sample data is input into the basic Chinese stroke recognition model for testing, and the prediction accuracy of the basic Chinese stroke recognition model is obtained;
每次迭代后判断损失值下降程度，以更新模型训练参数或者提前终止学习。After each iteration, the degree of decrease of the loss value is judged to update the model training parameters or terminate the learning in advance.
优选地，所述汉字基本笔画识别模型包括多个卷积层和多个池化层的卷积神经网路结构和三层全连接神经网路结构。Preferably, the basic stroke recognition model for Chinese characters includes a convolutional neural network structure with multiple convolution layers and multiple pooling layers and a three-layer fully connected neural network structure.
优选地，所述所述基本笔画类别为：Preferably, the basic stroke categories are:
其中，
第二方面，一种基于FMCW雷达系统的笔画识别系统，包括：A second aspect, a stroke recognition system based on an FMCW radar system, comprising:
数据获取模块，用于获取基于FMCW雷达系统手写汉字包含的至少一个待识别笔画的中频信号数据；A data acquisition module for acquiring the intermediate frequency signal data of at least one stroke to be recognized based on the handwritten Chinese characters of the FMCW radar system;
处理模块，用于对各所述待识别笔画的所述中频信号数据进行预处理，获取各所述待识别笔画的特征图集合；该预处理包括特征提取和特征增强；a processing module, configured to preprocess the intermediate frequency signal data of each of the strokes to be recognized, and obtain a feature map set of each of the strokes to be recognized; the preprocessing includes feature extraction and feature enhancement;
模型获取模块，用于获取训练完成的汉字基本笔画识别模型；所述汉字基本笔画识别模型是指以所述特征图集合为输入参数，以基本笔画类别为输出参数的卷积神经网络模型；The model acquisition module is used to obtain the basic stroke recognition model of Chinese characters that has been trained; the basic Chinese stroke recognition model refers to a convolutional neural network model that takes the feature map set as an input parameter and the basic stroke category as an output parameter;
识别模块，用于将各所述待识别笔画的所述特征图集合输入至所述汉字基本笔画识别模型中，并获取所述汉字基本笔画识别模型输出的与每个所述待识别笔画匹配的基本笔画类别。The recognition module is used to input the feature map sets of the strokes to be recognized into the basic stroke recognition model of Chinese characters, and obtain the output of the basic stroke recognition model of Chinese characters that matches each of the strokes to be recognized. Basic stroke categories.
优选地，所述的基于FMCW雷达系统的笔画识别系统，还包括Preferably, the stroke recognition system based on the FMCW radar system further includes
样本获取模块，用于获取手写汉字预设基本笔画的第一样本数据和第二样本数据；所述第一样本数据和第二样本数据均包含预设数量的预设基本笔画对应的特征图集合；A sample acquisition module, configured to acquire first sample data and second sample data of preset basic strokes of handwritten Chinese characters; the first sample data and second sample data both include a preset number of features corresponding to preset basic strokes collection of graphs;
搭建模块，用于搭建待训练的汉字基本笔画识别模型，设置模型训练参数和损失函数；Build a module to build a basic stroke recognition model for Chinese characters to be trained, and set model training parameters and loss functions;
训练模块，用于将所述第一样本数据输入至所述汉字基本笔画识别模型进行迭代训练，根据所述损失函数计算所述汉字基本笔画识别模型输出的预测类别结果与实际类别结果之间的损失值；A training module, configured to input the first sample data into the basic Chinese character stroke recognition model for iterative training, and calculate the difference between the predicted category result output by the Chinese basic stroke recognition model and the actual category result according to the loss function loss value;
输出模块，用于根据所述损失值判断所述汉字基本笔画识别模型是否训练完成；an output module, for judging whether the basic stroke recognition model of Chinese characters is trained according to the loss value;
测试模块，用于将所述第二样本数据输入至训练完成的所述汉字基本笔画识别模型进行测试，获取所述汉字基本笔画识别模型的预测准确率。A testing module, configured to input the second sample data into the trained Chinese basic stroke recognition model for testing, and obtain the prediction accuracy of the Chinese basic stroke recognition model.
采用上述方案，本发明具有以下有益效果：Adopt the above scheme, the present invention has the following beneficial effects:
1）本发明通过对手写汉字包含的待识别笔画的中频信号数据进行特征提取、特征增强等预处理，得到两种特征图，降低了用于表征手势运动趋势的数据量，提高了特征提取效率，同时节约了模型训练时间；1) The present invention obtains two feature maps by performing feature extraction, feature enhancement and other preprocessing on the intermediate frequency signal data of the strokes to be recognized contained in the handwritten Chinese characters, which reduces the amount of data used to characterize the gesture movement trend and improves the feature extraction efficiency. , while saving model training time;
2）本发明通过汉字基本笔画网络模型对待识别笔画的两种特征图进行识别，得到基本笔画类别，能够精确地判别出基本笔画类别。此外，本发明有利于非接触式人机交互设备后续发展。2) The present invention identifies the two feature maps of the strokes to be recognized through the network model of the basic strokes of Chinese characters to obtain the basic stroke categories, which can accurately discriminate the basic stroke categories. In addition, the present invention is beneficial to the subsequent development of the non-contact human-computer interaction device.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.
图1为本发明一实施例中基于FMCW雷达系统的笔画识别方法的流程示意图；1 is a schematic flowchart of a stroke recognition method based on an FMCW radar system in an embodiment of the present invention;
图2为本发明一实施例中基于FMCW雷达系统的笔画识别方法的步骤S20的流程示意图；2 is a schematic flowchart of step S20 of a stroke recognition method based on an FMCW radar system in an embodiment of the present invention;
图3为本发明一实施例中基于FMCW雷达系统的笔画识别方法的步骤S201的流程示意图；3 is a schematic flowchart of step S201 of the stroke recognition method based on the FMCW radar system according to an embodiment of the present invention;
图4为本发明一实施例中基于FMCW雷达系统的笔画识别方法的步骤S202的流程示意图；4 is a schematic flowchart of step S202 of the stroke recognition method based on the FMCW radar system according to an embodiment of the present invention;
图5为本发明另一实施例中基于FMCW雷达系统的笔画识别方法的模型训练过程的流程示意图；5 is a schematic flowchart of a model training process of a stroke recognition method based on an FMCW radar system in another embodiment of the present invention;
图6为本发明一实施例中基于FMCW雷达系统的笔画识别系统的结构示意图；6 is a schematic structural diagram of a stroke recognition system based on an FMCW radar system according to an embodiment of the present invention;
图7为本发明另一实施例中基于FMCW雷达系统的笔画识别系统的结构示意图。FIG. 7 is a schematic structural diagram of a stroke recognition system based on an FMCW radar system in another embodiment of the present invention.
具体实施方式Detailed ways
为了使本技术领域的人员更好地理解本发明方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分的实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本发明保护的范围。In order to make those skilled in the art better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only Embodiments are part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
需要说明的是，本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本发明的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。It should be noted that the terms "first", "second" and the like in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.
在一个本发明的实施例中，如图1所示，本发明实施例提供了一种基于FMCW雷达系统的笔画识别方法，该方法包括：In an embodiment of the present invention, as shown in FIG. 1 , an embodiment of the present invention provides a stroke recognition method based on an FMCW radar system, the method comprising:
步骤S10，获取基于FMCW雷达系统预设手写汉字包含的至少一个待识别笔画的中频信号数据。Step S10, acquiring intermediate frequency signal data of at least one stroke to be recognized that is included in preset handwritten Chinese characters based on the FMCW radar system.
在步骤S10之前，本实施例需要对FMCW雷达（Frequency Modulated ContinuousWave Radar，调频连续波雷达）系统的硬件设备进行配置，设置手部运动区域的空间范围。Before step S10, in this embodiment, the hardware device of the FMCW radar (Frequency Modulated Continuous Wave Radar, frequency modulated continuous wave radar) system needs to be configured, and the spatial range of the hand motion area is set.
以手写汉字“八”字为例，使FMCW雷达系统发射电磁波的方向正对着手部，对手写汉“八”字的二个待识别笔画的挥动过程进行数据采集，得到各待识别笔画的中频信号数据。Taking the handwritten Chinese character "eight" as an example, the direction of the electromagnetic wave emitted by the FMCW radar system is facing the hand, and the data collection is carried out on the waving process of the two strokes to be recognized of the handwritten Chinese character "eight", and the intermediate frequency of each stroke to be recognized is obtained. signal data.
步骤S20 ，对各所述待识别笔画的所述中频信号数据进行预处理，获取各所述待识别笔画的特征图集合。Step S20: Preprocess the intermediate frequency signal data of each of the strokes to be recognized, and obtain a feature map set of each of the strokes to be recognized.
在本实施例中，对手写汉字基本笔画的中频信号数据进行预处理包括特征提取和特征增强，其中，特征提取包括用于生成距离-时间矩阵（RTM）的距离估计和用于生成角度-时间矩阵（ATM）的角度估计；特征增强包括特征区域框定、二值化和开操作。作为优选，如图2所示，步骤S20具体包括以下步骤：In this embodiment, preprocessing the intermediate frequency signal data of the basic strokes of handwritten Chinese characters includes feature extraction and feature enhancement, wherein the feature extraction includes distance estimation for generating a distance-time matrix (RTM) and angle-time generation for generating an angle-time matrix. Matrix (ATM) angle estimation; feature enhancement includes feature region framing, binarization, and opening operations. Preferably, as shown in Figure 2, step S20 specifically includes the following steps:
步骤S201，通过第一算法对各所述待识别笔画的所述中频信号数据进行特征提取，获得对应的特征矩阵集合，所述特征矩阵集合包含距离-时间矩阵和角度-时间矩阵。Step S201 , perform feature extraction on the intermediate frequency signal data of each of the strokes to be recognized by the first algorithm to obtain a corresponding feature matrix set, where the feature matrix set includes a distance-time matrix and an angle-time matrix.
其中，第一算法为距离特征序列提取（Distance Feature Sequence Extraction，DFSE）和角度特征序列提取（Azimuth Feature Sequence Extraction，AFSE）结合的特征提取算法（DA-FSE算法），用于通过距离-时间矩阵（RTM）和角度-时间矩阵（ATM）来表征笔画挥动过程的趋势变化特征。Among them, the first algorithm is a feature extraction algorithm (DA-FSE algorithm) that combines distance feature sequence extraction (Distance Feature Sequence Extraction, DFSE) and angle feature sequence extraction (Azimuth Feature Sequence Extraction, AFSE). (RTM) and angle-time matrix (ATM) to characterize the trend change characteristics of stroke waving process.
作为优选，如图3所示，步骤S201具体包括以下步骤：Preferably, as shown in FIG. 3 , step S201 specifically includes the following steps:
步骤S2011，距离估计，根据所述中频信号数据的格式和笔画挥动过程的趋势变化特征，通过第三算法获取所述中频信号数据的频率矩阵，并将所述频率矩阵转换为距离矩阵之后获得距离随时间变化的距离-时间矩阵。Step S2011, distance estimation, according to the format of the intermediate frequency signal data and the trend change characteristics of the stroke waving process, obtain the frequency matrix of the intermediate frequency signal data by the third algorithm, and convert the frequency matrix into the distance matrix and obtain the distance. Distance-time matrix over time.
其中，第三算法为短时傅里叶变换（STFT算法）；频率矩阵包含中频信号数据每一帧的频率信息；距离矩阵包含由频率信息经转换得到的距离信息，距离-时间矩阵是指由距离矩阵按照时间维度进行组合的二维矩阵。作为优选，步骤S2011具体包括以下步骤：Among them, the third algorithm is the short-time Fourier transform (STFT algorithm); the frequency matrix contains the frequency information of each frame of the intermediate frequency signal data; the distance matrix contains the distance information converted from the frequency information, and the distance-time matrix refers to the The distance matrix is a two-dimensional matrix that combines the time dimension. Preferably, step S2011 specifically includes the following steps:
首先，获取窗函数以及窗函数的参数；可选的，根据中频信号数据的特性和参数配置，通过多次对比实验选取窗函数并设置窗函数大小、冗余度。First, obtain the window function and the parameters of the window function; optionally, according to the characteristics and parameter configuration of the intermediate frequency signal data, select the window function and set the size and redundancy of the window function through multiple comparison experiments.
然后，将所述窗函数和所述频率矩阵输入至基于第三算法的矩阵转化模型中，获得所述矩阵转化模型输出的所述距离矩阵；可选的，基于第三算法的矩阵转化模型为：Then, the window function and the frequency matrix are input into the matrix transformation model based on the third algorithm to obtain the distance matrix output by the matrix transformation model; optionally, the matrix transformation model based on the third algorithm is :
公式（1）中，
最后，将所述距离矩阵进行时间序列组合得到距离-时间矩阵（RTM）。Finally, the distance matrix is combined in time series to obtain a distance-time matrix (RTM).
步骤S2012，角度估计，根据所述中频信号数据的格式和笔画挥动过程的趋势变化特征，通过第四算法获取所述中频信号数据的相位矩阵，并将所述相位矩阵转换为角度矩阵之后获得角度随时间变化的角度-时间矩阵。Step S2012, angle estimation, according to the format of the intermediate frequency signal data and the trend change characteristics of the stroke waving process, obtain the phase matrix of the intermediate frequency signal data through the fourth algorithm, and convert the phase matrix into an angle matrix to obtain an angle. Angle-time matrix over time.
其中，第四算法为傅里叶变换和空间谱函数结合的算法（Range-Capon算法）；相位矩阵包含中频信号数据每一帧的相位信息；角度矩阵包含相位信息经转换得到的角度信息；角度-时间矩阵是指角度矩阵按照时间维度进行组合的二维矩阵。作为优选，步骤S2012具体包括以下步骤：Among them, the fourth algorithm is an algorithm combining Fourier transform and spatial spectral function (Range-Capon algorithm); the phase matrix contains the phase information of each frame of the intermediate frequency signal data; the angle matrix contains the angle information obtained by converting the phase information; - The time matrix refers to a two-dimensional matrix in which the angle matrix is combined according to the time dimension. Preferably, step S2012 specifically includes the following steps:
首先，将所述中频信号数据进行一次快速傅里叶变换（FFT），获得包含相位信息的相位矩阵，可理解的，该相位矩阵中每一个矩阵元素表征中频信号数据每一帧的相位信息。First, perform a Fast Fourier Transform (FFT) on the intermediate frequency signal data to obtain a phase matrix containing phase information. It is understandable that each matrix element in the phase matrix represents the phase information of each frame of the intermediate frequency signal data.
然后，对所述相位矩阵进行协方差运算获得协方差矩阵，并对预设的空间谱函数进行一次遍历获得角度矩阵。其中，空间谱函数为：Then, a covariance operation is performed on the phase matrix to obtain a covariance matrix, and an angle matrix is obtained by traversing the preset spatial spectral function once. Among them, the spatial spectral function is:
公式（2）中，
最后，将所述角度矩阵进行时间序列组合得到角度-时间矩阵（ATM）。Finally, the angle matrix is time-series combined to obtain an angle-time matrix (ATM).
本实施例中，采用第一算法（DA-FSE算法）获得距离随时间变化的距离-时间矩阵（RTM）和角度随时间变化的角度-时间矩阵（ATM），进而通过距离-时间矩阵（RTM）和角度-时间矩阵（ATM）来表征笔画挥动过程的趋势变化特征，能够减少数据量，同时有利于节约模型训练的时间。In this embodiment, the first algorithm (DA-FSE algorithm) is used to obtain the distance-time matrix (RTM) of the distance changing with time and the angle-time matrix (ATM) of the angle changing with time, and then through the distance-time matrix (RTM) ) and angle-time matrix (ATM) to characterize the trend change characteristics of the stroke process, which can reduce the amount of data and save the time of model training.
步骤S202，通过第二算法对各所述待识别笔画的所述特征矩阵集合进行特征增强，获得对应的所述特征图集合。Step S202, performing feature enhancement on the feature matrix set of each of the strokes to be recognized by the second algorithm to obtain the corresponding feature map set.
其中，第二算法为特定区域框定、二值化和开操作结合的特征增强算法（FA-FBO算法），用于通过特定区域框定、二值化和开操作等一系列操作对用特征矩阵表示的图像进行特征增强。作为优选，如图4所示，步骤S202具体包括以下步骤：Among them, the second algorithm is a feature enhancement algorithm (FA-FBO algorithm) combining specific area framing, binarization and opening operation, which is used to represent a series of operations such as specific area framing, binarization and opening operation with a feature matrix. image for feature enhancement. Preferably, as shown in FIG. 4 , step S202 specifically includes the following steps:
步骤S2021，特征区域框定，根据用两种特征矩阵表示的图像获取所述图像中特征区域位置坐标，并根据预设的矩阵框提取图像中笔画挥动过程的趋势变化特征。其中，两种特征矩阵为距离-时间矩阵和角度-时间矩阵；预设的矩阵框设置为固定大小。Step S2021 , frame the feature area, obtain the position coordinates of the feature area in the image according to the images represented by two feature matrices, and extract the trend change feature of the stroke waving process in the image according to the preset matrix frame. Among them, the two feature matrices are distance-time matrix and angle-time matrix; the preset matrix frame is set to a fixed size.
步骤S2022，二值化，通过第五算法获得最佳阈值，根据最佳阈值将两种所述特征矩阵二值化，获得对应的二值化图像。其中，第五算法为最大类间方差法（Otsu算法）。Step S2022, binarization, obtaining an optimal threshold through the fifth algorithm, and binarizing the two feature matrices according to the optimal threshold to obtain a corresponding binarized image. Among them, the fifth algorithm is the maximum between-class variance method (Otsu algorithm).
也即，采用Otsu算法获得最佳阈值，当图像像素灰度值大于或等于最佳阈值时，将其置于第一数值，例如第一数值为255；反之，当图像像素灰度值小于最佳阈值时，将其置于第二数值，例如第二数值为0，进而分别将两种特征矩阵二值化得到二值化图像。作为优选，步骤S2022的二值化具体表示为：That is, the Otsu algorithm is used to obtain the optimal threshold. When the gray value of the image pixel is greater than or equal to the optimal threshold, it is set to the first value, for example, the first value is 255; on the contrary, when the gray value of the image pixel is less than the optimal threshold When the optimal threshold is set, it is set to a second value, for example, the second value is 0, and then the two feature matrices are binarized respectively to obtain a binarized image. Preferably, the binarization in step S2022 is specifically expressed as:
公式（3）中，
步骤S2023，开操作，对两种所述特征矩阵对应的所述二值化图像进行开操作，使两种特征矩阵对应的二值化图像中的趋势变化特征的轮廓变得光滑，并断开狭窄的间断和消除细的突出物。作为优选，步骤S2023的开操作具体表示为：Step S2023, open operation, perform an open operation on the binarized images corresponding to the two feature matrices, so that the contours of the trend change features in the binarized images corresponding to the two feature matrices become smooth and disconnected. Narrow discontinuities and eliminate thin protrusions. Preferably, the opening operation of step S2023 is specifically expressed as:
公式（4）中，C为结构元素，B为集合。由公式（4）可知，用结构元素C对集合B进行开操作是指用结构元素C对集合B腐蚀，再用结构元素C对结果进行膨胀。In formula (4),C is a structural element, andB is a set. It can be seen from formula (4) that using the structural elementC to open the setB means using the structural elementC to corrode the setB , and then using the structural elementC to expand the result.
本实施例中，采用第二算法（FA-FBO算法）对距离随时间变化的RTM和角度随时间变化的ATM进行特征增强，能够减少误差干扰并使特征突出。In this embodiment, the second algorithm (FA-FBO algorithm) is used to enhance the features of the RTM whose distance varies with time and the ATM whose angle varies with time, which can reduce error interference and make features stand out.
步骤S30，获取训练完成的汉字基本笔画识别模型；所述汉字基本笔画识别模型是指以所述特征图集合为输入参数，以基本笔画类别为输出参数的卷积神经网络模型。Step S30: Obtain a trained Chinese basic stroke recognition model; the Chinese basic stroke recognition model refers to a convolutional neural network model that takes the feature map set as an input parameter and a basic stroke category as an output parameter.
在本实施中，所述步骤S30中的汉字基本笔画识别模型是基于后续步骤S501至步骤S505训练获得的卷积神经网络模型。In this implementation, the basic stroke recognition model for Chinese characters in the step S30 is based on the convolutional neural network model obtained by training in the subsequent steps S501 to S505.
步骤S40，将各所述待识别笔画的所述特征图集合输入至所述汉字基本笔画识别模型中，并获取所述汉字基本笔画识别模型输出的与每个所述待识别笔画匹配的基本笔画类别。Step S40, inputting the feature map sets of the strokes to be recognized into the basic stroke recognition model for Chinese characters, and obtaining the basic strokes output by the basic stroke recognition model for Chinese characters that match each of the strokes to be recognized category.
将各待识别笔画的特征图集合输入至汉字基本笔画识别模型，获得汉字基本笔画识别模型输出的一维数组，该输出数组是长度为
公式（5）、（6）中，
综上可知，本实施例首先通过对手写汉字包含的待识别笔画的中频信号数据进行特征提取、特征增强等预处理，得到两种特征图，降低了用于表征手势运动趋势的数据量，提高了特征提取效率，同时节约了模型训练时间。然后通过汉字基本笔画网络模型对待识别笔画的两种特征图进行识别，得到基本笔画类别，能够精确地判别出基本笔画类别。此外，本发明有利于非接触式人机交互设备后续发展。To sum up, in this embodiment, two kinds of feature maps are obtained by preprocessing the intermediate frequency signal data of the strokes to be recognized contained in the handwritten Chinese characters, such as feature extraction and feature enhancement, which reduces the amount of data used to characterize the gesture movement trend and improves the performance of the handwritten Chinese characters. The feature extraction efficiency is improved, and the model training time is saved. Then, the two feature maps of the strokes to be recognized are identified through the basic stroke network model of Chinese characters, and the basic stroke categories are obtained, which can accurately identify the basic stroke categories. In addition, the present invention is beneficial to the subsequent development of the non-contact human-computer interaction device.
作为本发明的再一个实施例，如图5所示，所述基于FMCW雷达系统的笔画识别方法，还包括模型训练的过程，具体包括以下步骤：As yet another embodiment of the present invention, as shown in FIG. 5 , the stroke recognition method based on the FMCW radar system also includes a process of model training, which specifically includes the following steps:
步骤S501，获取手写汉字预设基本笔画的第一样本数据和第二样本数据；所述第一样本数据和第二样本数据均包含预设数量的预设基本笔画对应的特征图集合。Step S501: Obtain first sample data and second sample data of preset basic strokes of handwritten Chinese characters; the first sample data and second sample data both include a preset number of feature map sets corresponding to preset basic strokes.
在本实施例中，所述第一样本数据和第二数据样本的数量样本数量可以根据需求进行设置。以手写汉字“永”字为例，基于FMCW雷达系统对手写汉“永”字八个基本笔画横、点、提、撇、弯、捺、竖、钩的中频信号数据进行了采集。对每种基本笔画的中频信号数据采集100组样本数据，总共可以采集800组手写汉字基本笔画的样本数据，对于每一个样本数据进行预处理，包括：首先通过第一算法对各样本数据进行特征提取，以获得各数据样本的特征矩阵集合，然后通过第二算法对特征矩阵集合包含的两种特征矩阵进行特征增强，以得到对应的特征图集合，最后将特征图集合中包含的两种特征图进行归一化处理。可选的，可以将特征图规范化为125×189大小。In this embodiment, the quantity of the first sample data and the quantity of the second data sample The sample quantity can be set according to requirements. Taking the handwritten Chinese character "Yong" as an example, the intermediate frequency signal data of the eight basic strokes of the handwritten Chinese "Yong" character, horizontal, dot, lift, smear, bend, smash, vertical and hook were collected based on the FMCW radar system. 100 groups of sample data are collected for the intermediate frequency signal data of each basic stroke, and a total of 800 groups of sample data of the basic strokes of handwritten Chinese characters can be collected, and each sample data is preprocessed, including: first, the first algorithm is used to characterize each sample data. Extraction to obtain the feature matrix set of each data sample, and then perform feature enhancement on the two feature matrices contained in the feature matrix set through the second algorithm to obtain the corresponding feature map set, and finally combine the two features contained in the feature map set. The graph is normalized. Optionally, the feature map can be normalized to a size of 125×189.
进一步的，对预处理后得到的手写汉字基本笔画的特征图集合进行分组，可选的，将其中的500组特征图集合作为手写汉字预设基本笔画的的第一样本数据，另外的300组特征图集合作为手写汉字基本笔画的第二样本数据。其中，该第一样本数据用于训练汉字基本笔画模型的训练，该第二样本数据用于测试汉字基本笔画模型的预测准确率。Further, the feature map sets of the basic strokes of handwritten Chinese characters obtained after preprocessing are grouped, optionally, 500 groups of feature map sets are used as the first sample data of the preset basic strokes of handwritten Chinese characters, and the other 300 The set of group feature maps is used as the second sample data for the basic strokes of handwritten Chinese characters. The first sample data is used for training the basic stroke model of Chinese characters, and the second sample data is used to test the prediction accuracy of the basic stroke model of Chinese characters.
需要说明的是，在其他实施例中，可以先对样本数据进行分组，再对分组后的样本数据进行预处理。It should be noted that, in other embodiments, the sample data may be grouped first, and then the grouped sample data may be preprocessed.
步骤S502，搭建待训练的汉字基本笔画识别模型，设置模型训练参数以及损失函数。Step S502, build a basic stroke recognition model of Chinese characters to be trained, and set model training parameters and a loss function.
在本实施例中，该汉字基本笔画识别模型包括具有多个卷积层和多个池化层的卷积神经网路（CNN）结构和三层全连接神经网路结构。可选的，卷积神经网路（CNN）结构包含三层卷积层和三层池化层；CNN的三层卷积层的卷积核大小均为3×3，步长为1，每层的卷积核个数分别为4、4和8，卷积时采用全零填充操作；池化层核大小为2×2，步长为2。三层全连接神经网路结构中，隐藏层节点数为100，输出层节点数为
模型训练参数可以根据需求进行设置，模型训练参数包含但不限定于批量大小、学习率、迭代次数和方向传播算法等。可选的，可以分别将批量大小设置为10，学习率设置为0.00001，迭代次数设置为100，方向传播算法设置为随机梯度下降算法。Model training parameters can be set according to requirements. Model training parameters include but are not limited to batch size, learning rate, number of iterations, and directional propagation algorithms. Optionally, the batch size can be set to 10, the learning rate can be set to 0.00001, the number of iterations can be set to 100, and the directional propagation algorithm can be set to the stochastic gradient descent algorithm.
损失函数可以根据需求进行设置，损失函数可以采用交叉熵算法。The loss function can be set according to the requirements, and the loss function can use the cross-entropy algorithm.
步骤S503，将所述第一样本数据输入至所述汉字基本笔画识别模型进行迭代训练，根据所述损失函数计算所述汉字基本笔画识别模型输出的预测类别结果与实际类别结果之间的损失值。Step S503: Input the first sample data into the basic Chinese stroke recognition model for iterative training, and calculate the loss between the predicted category result output by the Chinese basic stroke recognition model and the actual category result according to the loss function value.
步骤S504，根据所述损失值判断所述汉字基本笔画识别模型是否训练完成。Step S504, according to the loss value, determine whether the training of the basic stroke recognition model for Chinese characters is completed.
在模型训练过程中，将手写汉字预设基本笔画的第一样本数据输入至待训练的汉字基本笔画识别模型进行迭代训练，通过损失函数计算模型输出的预测类别结果与实际类别结果之间的损失值，判断损失值是否小于预设损失阈值，若损失值小于预设损失阈值，则确定模型训练完成，保存模型训练参数，此时将提前终止学习；反之，若损失值大于或等于预设损失阈值，则更新模型训练参数，基于更新模型训练参数对汉字基本笔画识别模型重新进行训练，直至损失值小于预设损失阈值，则确定模型训练完成，保存更新后的模型训练参数。In the model training process, the first sample data of the preset basic strokes of handwritten Chinese characters is input into the basic stroke recognition model of Chinese characters to be trained for iterative training, and the loss function is used to calculate the difference between the predicted category results output by the model and the actual category results. Loss value, to determine whether the loss value is less than the preset loss threshold, if the loss value is less than the preset loss threshold, it is determined that the model training is completed, the model training parameters are saved, and the learning will be terminated in advance; otherwise, if the loss value is greater than or equal to the preset loss value If the loss threshold is set, the model training parameters are updated, and the basic stroke recognition model for Chinese characters is retrained based on the updated model training parameters. When the loss value is less than the preset loss threshold, it is determined that the model training is completed, and the updated model training parameters are saved.
步骤S505，将所述第二样本数据输入至训练完成的所述汉字基本笔画识别模型进行测试，获取所述汉字基本笔画识别模型的预测准确率。Step S505: Input the second sample data into the trained Chinese basic stroke recognition model for testing, and obtain the prediction accuracy of the Chinese basic stroke recognition model.
需要说明的是，如图5所示，步骤S501至步骤S505执行于步骤S10之前，在其他实施例中，步骤S501至步骤S505也可以执行于步骤S20，或者步骤S30之前，因此步骤S501至步骤S505仅需执行在步骤S30、步骤S20和步骤S10中的任意一个步骤之前即可。It should be noted that, as shown in FIG. 5 , steps S501 to S505 are performed before step S10. In other embodiments, steps S501 to S505 may also be performed before step S20 or step S30, so steps S501 to S501 S505 only needs to be executed before any one of steps S30, S20 and S10.
在模型测试过程中，将手写汉字预设基本笔画的第二数据样本作为训练完成的汉字基本笔画识别模型的输入参数，输入参数通过汉字基本笔画识别模型中的卷积神经网路（CNN）结构提取到特征向量，并通过三层全连接神经网路结构得到输出结果，也即手写汉字基本笔画的类别，进而获取模型的预测准确率。通过实验可以得到，采用300个第二样本数据测试汉字基本笔画识别模型，模型的预测准确率可以达到96%。可理解的，本实施例的基于FMCW雷达系统的笔画识别方法，可以精确地判别基本笔画类别。In the model testing process, the second data sample of the preset basic strokes of handwritten Chinese characters is used as the input parameters of the trained Chinese basic stroke recognition model, and the input parameters are passed through the convolutional neural network (CNN) structure in the Chinese basic stroke recognition model. The feature vector is extracted, and the output result is obtained through the three-layer fully connected neural network structure, that is, the category of the basic strokes of handwritten Chinese characters, and then the prediction accuracy of the model is obtained. It can be obtained through experiments that the basic stroke recognition model of Chinese characters is tested with 300 second sample data, and the prediction accuracy of the model can reach 96%. It is understandable that the stroke recognition method based on the FMCW radar system of this embodiment can accurately discriminate basic stroke categories.
此外，本发明实施例还提供了一种基于FMCW雷达系统的笔画识别系统，如图7所示，该系统包括数据获取模块110、预处理模块120、模型获取模块130和识别模块140，各功能模块的详细说明如下：In addition, an embodiment of the present invention also provides a stroke recognition system based on an FMCW radar system. As shown in FIG. 7 , the system includes a data acquisition module 110 , a preprocessing module 120 , a model acquisition module 130 and an identification module 140 . The detailed description of the module is as follows:
数据获取模块110，用于获取基于FMCW雷达系统手写汉字包含的至少一个待识别笔画的中频信号数据。The data acquisition module 110 is configured to acquire intermediate frequency signal data of at least one stroke to be recognized contained in handwritten Chinese characters based on the FMCW radar system.
预处理模块120，用于对各所述待识别笔画的所述中频信号数据进行预处理，获取各所述待识别笔画的特征图集合；该预处理包括特征提取和特征增强。The preprocessing module 120 is configured to preprocess the intermediate frequency signal data of each of the strokes to be recognized, and obtain a feature map set of each of the strokes to be recognized; the preprocessing includes feature extraction and feature enhancement.
模型获取模块130，用于获取训练完成的汉字基本笔画识别模型；所述汉字基本笔画识别模型是指以所述特征图集合为输入参数，以基本笔画类别为输出参数的卷积神经网络模型。The model obtaining module 130 is configured to obtain a trained Chinese basic stroke recognition model; the Chinese basic stroke recognition model refers to a convolutional neural network model that takes the feature map set as an input parameter and a basic stroke category as an output parameter.
识别模块140，用于将各所述待识别笔画的所述特征图集合输入至所述汉字基本笔画识别模型中，并获取所述汉字基本笔画识别模型输出的与每个所述待识别笔画匹配的基本笔画类别。Recognition module 140, for inputting the feature map sets of the strokes to be recognized into the basic stroke recognition model of Chinese characters, and obtaining the output of the basic stroke recognition model for Chinese characters that matches each of the strokes to be recognized The basic stroke category of .
进一步的，如图6所示，所述预处理模块120包括特征提取子模块121和特征增强子模块122，各功能子模块的详细说明如下：Further, as shown in FIG. 6 , the preprocessing module 120 includes a feature extraction sub-module 121 and a feature enhancement sub-module 122. The detailed description of each functional sub-module is as follows:
特征提取子模块121，用于通过第一算法对各所述待识别笔画的所述中频信号数据进行特征提取，获得对应的特征矩阵集合，所述特征矩阵集合包含距离-时间矩阵和角度-时间矩阵。The feature extraction sub-module 121 is used to perform feature extraction on the intermediate frequency signal data of each of the strokes to be recognized by the first algorithm, and obtain a corresponding feature matrix set, and the feature matrix set includes a distance-time matrix and an angle-time matrix matrix.
特征增强子模块122，用于通过第二算法对各所述待识别笔画的所述特征矩阵集合进行特征增强，获得对应的所述特征图集合。The feature enhancement sub-module 122 is configured to perform feature enhancement on the feature matrix set of each of the strokes to be recognized through the second algorithm to obtain the corresponding feature map set.
进一步的，如图6所示，所述特征提取子模块121包括以下单元，各功能单元的详细说明如下：Further, as shown in FIG. 6 , the feature extraction sub-module 121 includes the following units, and the detailed description of each functional unit is as follows:
距离估计单元1211，用于根据所述中频信号数据的格式和笔画挥动过程的趋势变化特征，通过第三算法获取所述中频信号数据的频率矩阵，并根据所述频率信息的变化获得距离随时间变化的距离-时间矩阵。The distance estimation unit 1211 is used to obtain the frequency matrix of the intermediate frequency signal data through the third algorithm according to the format of the intermediate frequency signal data and the trend change characteristics of the stroke waving process, and obtain the distance over time according to the change of the frequency information. Changed distance-time matrix.
角度估计单元1212，用于根据所述中频信号数据的格式和笔画挥动过程的趋势变化特征，通过第四算法获取所述距离矩阵的相位数据矩阵，并将所述相位数据矩阵转换为角度数据矩阵之后获得角度随时间变化的角度-时间矩阵-时间矩阵。The angle estimation unit 1212 is used to obtain the phase data matrix of the distance matrix through the fourth algorithm according to the format of the intermediate frequency signal data and the trend change characteristics of the stroke waving process, and convert the phase data matrix into an angle data matrix. Then get the angle-time matrix-time matrix where the angle varies with time.
进一步的，所述角度估计单元包括以下单元，各功能单元的详细说明如下：Further, the angle estimation unit includes the following units, and the detailed description of each functional unit is as follows:
变换单元，用于对所述中频信号数据进行快速傅氏变换，获得所述距离矩阵中每个矩阵元素的相位信息。A transform unit, configured to perform fast Fourier transform on the intermediate frequency signal data to obtain phase information of each matrix element in the distance matrix.
遍历单元，用于对所述相位矩阵进行协方差运算获得协方差矩阵，并对预设的空间谱函数进行遍历获得所述角度矩阵。A traversal unit, configured to perform a covariance operation on the phase matrix to obtain a covariance matrix, and traverse a preset spatial spectral function to obtain the angle matrix.
组合单元，用于将所述角度矩阵进行时间序列组合获得所述角度-时间矩阵。A combining unit, configured to perform time series combination of the angle matrix to obtain the angle-time matrix.
进一步的，如图6所示，该特征增强子模块122包括以下以下单元，各功能单元的详细说明如下：Further, as shown in FIG. 6 , the feature enhancement sub-module 122 includes the following units, and the detailed description of each functional unit is as follows:
特征框定单元1221，用于根据用两种特征矩阵表示的图像获取图像中特征区域位置坐标，并根据预设的矩阵框提取图像中笔画挥动过程的趋势变化特征。The feature framing unit 1221 is configured to obtain the position coordinates of the feature area in the image according to the images represented by two feature matrices, and extract the trend change feature of the stroke waving process in the image according to the preset matrix frame.
二值化单元1222，用于通过第五算法获得最佳阈值，根据最佳阈值将两种所述特征矩阵二值化，获得对应的二值化图像。The binarization unit 1222 is configured to obtain the optimal threshold through the fifth algorithm, and binarize the two feature matrices according to the optimal threshold to obtain the corresponding binarized image.
开操作单元1223，用于对两种所述特征矩阵对应的所述二值化图像进行开操作。The opening operation unit 1223 is configured to perform an opening operation on the binarized images corresponding to the two feature matrices.
进一步的，如图7所示，该系统还包括样本获取模块151、搭建模块152、训练模块153、测试模块154和输出模块155，各功能模块的详细说明如下：Further, as shown in FIG. 7 , the system also includes a sample acquisition module 151, a building module 152, a training module 153, a testing module 154 and an output module 155. The detailed description of each functional module is as follows:
样本获取模块151，用于获取手写汉字预设基本笔画的第一样本数据和第二样本数据；所述第一样本数据和第二样本数据均包含预设数量的预设基本笔画对应的特征图集合。The sample acquisition module 151 is used to acquire the first sample data and the second sample data of the preset basic strokes of handwritten Chinese characters; the first sample data and the second sample data both include a preset number of preset basic strokes corresponding to A collection of feature maps.
搭建模块152，用于搭建待训练的汉字基本笔画识别模型，设置模型训练参数和损失函数。The building module 152 is used for building a basic stroke recognition model of Chinese characters to be trained, and setting model training parameters and loss functions.
训练模块153，用于将所述第一样本数据输入至所述汉字基本笔画识别模型进行迭代训练，根据所述损失函数计算所述汉字基本笔画识别模型输出的预测类别结果与实际类别结果之间的损失值。The training module 153 is configured to input the first sample data into the basic Chinese character stroke recognition model for iterative training, and calculate the difference between the predicted category result output by the Chinese basic stroke recognition model and the actual category result according to the loss function. loss value in between.
输出模块154，用于根据所述损失值判断所述汉字基本笔画识别模型是否训练完成；The output module 154 is used to judge whether the basic stroke recognition model of Chinese characters has been trained according to the loss value;
测试模块155，用于将所述第二样本数据输入至训练完成的所述汉字基本笔画识别模型进行测试，获取所述汉字基本笔画识别模型的预测准确率。The testing module 155 is configured to input the second sample data into the trained Chinese basic stroke recognition model for testing, and obtain the prediction accuracy of the Chinese basic stroke recognition model.
进一步的，所述模型获取模块130中所述汉字基本笔画识别模型包括具有多个卷积层和多个池化层的卷积神经网路结构和三层全连接神经网路结构。Further, the basic stroke recognition model for Chinese characters in the model obtaining module 130 includes a convolutional neural network structure with multiple convolution layers and multiple pooling layers and a three-layer fully connected neural network structure.
进一步的，所述识别模块140中所述基本笔画类别为：Further, the basic stroke categories described in the identification module 140 are:
其中，
本发明实施例中提供的基于FMCW雷达系统的笔画识别系统，能够通过对手写汉字包含的待识别笔画的中频信号数据进行特征提取、特征增强等预处理，得到两种特征图，降低了用于表征手势运动趋势的数据量，提高了特征提取效率，同时节约了模型训练时间；且能够通过汉字基本笔画网络模型对待识别笔画的两种特征图进行识别，得到基本笔画类别，能够精确地判别出基本笔画类别。此外，本发明有利于非接触式人机交互设备后续发展。The stroke recognition system based on the FMCW radar system provided in the embodiment of the present invention can obtain two feature maps by performing feature extraction, feature enhancement and other preprocessing on the intermediate frequency signal data of the strokes to be recognized contained in the handwritten Chinese characters, which reduces the need for The amount of data representing the movement trend of gestures improves the efficiency of feature extraction and saves model training time; and can identify the two feature maps of the strokes to be recognized through the basic stroke network model of Chinese characters to obtain the basic stroke categories, which can accurately identify Basic stroke categories. In addition, the present invention is beneficial to the subsequent development of the non-contact human-computer interaction device.
所属领域的普通技术人员应当理解：以上任何实施例的讨论仅为示例性的，并非旨在暗示本公开的范围(包括权利要求)被限于这些例子；在本发明的思路下，以上实施例或者不同实施例中的技术特征之间也可以进行组合，并存在如上所述的本发明的不同方面的许多其它变化，为了简明它们没有在细节中提供。因此，凡在本发明的精神和原则之内，所做的任何省略、修改、等同替换、改进等，均应包含在本发明的保护范围之内。Those of ordinary skill in the art should understand that the discussion of any of the above embodiments is only exemplary, and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples; under the spirit of the present invention, the above embodiments or Combinations between technical features in different embodiments are also possible, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omission, modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims (7)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011420741.7ACN112198966B (en) | 2020-12-08 | 2020-12-08 | Stroke identification method and system based on FMCW radar system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011420741.7ACN112198966B (en) | 2020-12-08 | 2020-12-08 | Stroke identification method and system based on FMCW radar system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112198966A CN112198966A (en) | 2021-01-08 |
| CN112198966Btrue CN112198966B (en) | 2021-03-16 |
Family
ID=74033862
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011420741.7AActiveCN112198966B (en) | 2020-12-08 | 2020-12-08 | Stroke identification method and system based on FMCW radar system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112198966B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113361367B (en)* | 2021-06-01 | 2022-04-26 | 中南大学 | Underground target electromagnetic inversion method and system based on deep learning |
| CN114092938B (en)* | 2022-01-19 | 2022-04-19 | 腾讯科技（深圳）有限公司 | Image recognition processing method and device, electronic equipment and storage medium |
| CN115497107B (en)* | 2022-09-30 | 2023-04-18 | 江西师范大学 | Zero-sample Chinese character recognition method based on stroke and radical decomposition |
| CN117608399B (en)* | 2023-11-23 | 2024-06-14 | 首都医科大学附属北京天坛医院 | Track fitting method and device based on Chinese character strokes |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108509910A (en)* | 2018-04-02 | 2018-09-07 | 重庆邮电大学 | Deep learning gesture identification method based on fmcw radar signal |
| CN109271838A (en)* | 2018-07-19 | 2019-01-25 | 重庆邮电大学 | A kind of three parameter attributes fusion gesture identification method based on fmcw radar |
| CN109829509A (en)* | 2019-02-26 | 2019-05-31 | 重庆邮电大学 | Radar gesture identification method based on fused neural network |
| CN110348288A (en)* | 2019-05-27 | 2019-10-18 | 哈尔滨工业大学（威海） | A kind of gesture identification method based on 77GHz MMW RADAR SIGNAL USING |
| CN110765974A (en)* | 2019-10-31 | 2020-02-07 | 复旦大学 | Micro-motion gesture recognition method based on millimeter wave radar and convolutional neural network |
| CN111027458A (en)* | 2019-08-28 | 2020-04-17 | 深圳大学 | Gesture recognition method and device based on radar three-dimensional track characteristics and storage medium |
| CN111399642A (en)* | 2020-03-09 | 2020-07-10 | 深圳大学 | Gesture recognition method and device, mobile terminal and storage medium |
| CN111427031A (en)* | 2020-04-09 | 2020-07-17 | 浙江大学 | Identity and gesture recognition method based on radar signals |
| CN111461037A (en)* | 2020-04-07 | 2020-07-28 | 电子科技大学 | End-to-end gesture recognition method based on FMCW radar |
| WO2020176105A1 (en)* | 2019-02-28 | 2020-09-03 | Google Llc | Smart-device-based radar system detecting user gestures in the presence of saturation |
| CN111680539A (en)* | 2020-04-14 | 2020-09-18 | 北京清雷科技有限公司 | Dynamic gesture radar recognition method and device |
| WO2020187397A1 (en)* | 2019-03-19 | 2020-09-24 | HELLA GmbH & Co. KGaA | A method for a detection and classification of gestures using a radar system |
| CN111722706A (en)* | 2019-03-21 | 2020-09-29 | 英飞凌科技股份有限公司 | Method and system for air-written character recognition based on radar network |
| CN111813222A (en)* | 2020-07-09 | 2020-10-23 | 电子科技大学 | A Fine Dynamic Gesture Recognition Method Based on Terahertz Radar |
| CN111814743A (en)* | 2020-07-30 | 2020-10-23 | 深圳壹账通智能科技有限公司 | Handwriting recognition method, device and computer-readable storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109085548B (en)* | 2018-09-26 | 2021-03-26 | 湖南时变通讯科技有限公司 | A surface penetrating radar hyperbolic target detection method and device |
| CN110579746A (en)* | 2019-10-11 | 2019-12-17 | 湖南时变通讯科技有限公司 | Echo signal processing method, device, equipment and storage medium |
| CN111157988B (en)* | 2020-02-27 | 2023-04-07 | 中南大学 | Gesture radar signal processing method based on RDTM and ATM fusion |
- 2020
- 2020-12-08CNCN202011420741.7Apatent/CN112198966B/enactiveActive
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108509910A (en)* | 2018-04-02 | 2018-09-07 | 重庆邮电大学 | Deep learning gesture identification method based on fmcw radar signal |
| CN109271838A (en)* | 2018-07-19 | 2019-01-25 | 重庆邮电大学 | A kind of three parameter attributes fusion gesture identification method based on fmcw radar |
| CN109829509A (en)* | 2019-02-26 | 2019-05-31 | 重庆邮电大学 | Radar gesture identification method based on fused neural network |
| WO2020176105A1 (en)* | 2019-02-28 | 2020-09-03 | Google Llc | Smart-device-based radar system detecting user gestures in the presence of saturation |
| WO2020187397A1 (en)* | 2019-03-19 | 2020-09-24 | HELLA GmbH & Co. KGaA | A method for a detection and classification of gestures using a radar system |
| CN111722706A (en)* | 2019-03-21 | 2020-09-29 | 英飞凌科技股份有限公司 | Method and system for air-written character recognition based on radar network |
| CN110348288A (en)* | 2019-05-27 | 2019-10-18 | 哈尔滨工业大学（威海） | A kind of gesture identification method based on 77GHz MMW RADAR SIGNAL USING |
| CN111027458A (en)* | 2019-08-28 | 2020-04-17 | 深圳大学 | Gesture recognition method and device based on radar three-dimensional track characteristics and storage medium |
| CN110765974A (en)* | 2019-10-31 | 2020-02-07 | 复旦大学 | Micro-motion gesture recognition method based on millimeter wave radar and convolutional neural network |
| CN111399642A (en)* | 2020-03-09 | 2020-07-10 | 深圳大学 | Gesture recognition method and device, mobile terminal and storage medium |
| CN111461037A (en)* | 2020-04-07 | 2020-07-28 | 电子科技大学 | End-to-end gesture recognition method based on FMCW radar |
| CN111427031A (en)* | 2020-04-09 | 2020-07-17 | 浙江大学 | Identity and gesture recognition method based on radar signals |
| CN111680539A (en)* | 2020-04-14 | 2020-09-18 | 北京清雷科技有限公司 | Dynamic gesture radar recognition method and device |
| CN111813222A (en)* | 2020-07-09 | 2020-10-23 | 电子科技大学 | A Fine Dynamic Gesture Recognition Method Based on Terahertz Radar |
| CN111814743A (en)* | 2020-07-30 | 2020-10-23 | 深圳壹账通智能科技有限公司 | Handwriting recognition method, device and computer-readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112198966A (en) | 2021-01-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112198966B (en) | Stroke identification method and system based on FMCW radar system | |
| Parvathy et al. | RETRACTED ARTICLE: Development of hand gesture recognition system using machine learning | |
| Nair et al. | Hand gesture recognition system for physically challenged people using IOT | |
| CN103984416B (en) | Gesture recognition method based on acceleration sensor | |
| US8391613B2 (en) | Statistical online character recognition | |
| CN103150019A (en) | Handwriting input system and method | |
| CN107657233A (en) | Static sign language real-time identification method based on modified single multi-target detection device | |
| Patel et al. | Handwritten character recognition using multiresolution technique and euclidean distance metric | |
| CN106502390B (en) | A kind of visual human's interactive system and method based on dynamic 3D Handwritten Digit Recognition | |
| Dai Nguyen et al. | Recognition of online handwritten math symbols using deep neural networks | |
| CN103927555B (en) | Static manual alphabet identifying system and method based on Kinect sensor | |
| Choudhury et al. | A CNN-LSTM based ensemble framework for in-air handwritten Assamese character recognition | |
| CN118247850B (en) | Man-machine interaction method and interaction system based on gesture recognition | |
| CN112749646A (en) | Interactive point-reading system based on gesture recognition | |
| Hamdan et al. | Deep learning based handwriting recognition with adversarial feature deformation and regularization | |
| CN110353693A (en) | A kind of hand-written Letter Identification Method and system based on WiFi | |
| CN117809320A (en) | Test paper handwriting mathematical formula identification method and system based on deep learning | |
| Zhang et al. | Hand gesture recognition with SURF-BOF based on Gray threshold segmentation | |
| CN114397963B (en) | Gesture recognition method and device, electronic equipment and storage medium | |
| Agab et al. | New combined DT-CWT and HOG descriptor for static and dynamic hand gesture recognition | |
| CN112507863B (en) | Handwritten character and picture classification method based on quantum Grover algorithm | |
| Nakkach et al. | Hybrid approach to features extraction for online Arabic character recognition | |
| Yana et al. | Fusion networks for air-writing recognition | |
| Tormási et al. | Comparing the efficiency of a fuzzy single-stroke character recognizer with various parameter values | |
| Houtinezhad et al. | Off-line signature verification system using features linear mapping in the candidate points |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |