CN108491068B - Interaction method for eye movement signal control - Google Patents
Interaction method for eye movement signal controlDownload PDFInfo
- Publication number
- CN108491068B CN108491068BCN201810127171.9ACN201810127171ACN108491068BCN 108491068 BCN108491068 BCN 108491068BCN 201810127171 ACN201810127171 ACN 201810127171ACN 108491068 BCN108491068 BCN 108491068B
- Authority
- CN
- China
- Prior art keywords
- eye movement
- eye
- movement signal
- digital
- function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本发明涉及一种交互方法，具体涉及一种眼动信号控制的交互方法。The invention relates to an interaction method, in particular to an interaction method for eye movement signal control.
背景技术Background technique
随着计算机技术及人工智能的迅猛发展，符合人类自然认知习惯的交互手段成为人机交互技术发展的主流，产生了诸如动作追踪、语音识别、感觉反馈、眼动控制、脑机接口等多通道交互方式，信号输入装置也由鼠标、键盘转变为肢体动作、语音、眼动和脑电波。根据Berger的研究，在五大感知系统中眼睛所获取的信息约占80％，人类不需要专门的学习和训练，就能自由灵活的控制眼球转动以注视不同的对象，眼动追踪技术具有自然性和直观性的优点，成为诸多学科领域的共同关注焦点。With the rapid development of computer technology and artificial intelligence, interaction methods that conform to human's natural cognitive habits have become the mainstream of human-computer interaction technology development. In the channel interaction mode, the signal input device is also changed from mouse and keyboard to body movements, voice, eye movements and brain waves. According to Berger's research, among the five perception systems, the eyes acquire about 80% of the information. Humans can freely and flexibly control eye movements to focus on different objects without special learning and training. Eye tracking technology is natural And the advantages of intuitiveness have become the common focus of many disciplines.
眼机交互是用眼睛来控制计算机或装备的技术，采用眼动仪获取用户的视线信息，通过捕捉用户的注视点，识别出用户注视点的实时位置以及轨迹，以眨眼、凝视、眼球移动、平滑跟踪运动、瞳孔变化等指标作为输入指令，实现“所视即所得”。Eye-computer interaction is a technology that uses the eyes to control the computer or equipment. The eye tracker is used to obtain the user's line of sight information. Smooth tracking motion, pupil changes and other indicators are used as input commands to realize "what you see is what you get".
眼控系统中交互技术分为两种：基于视线的交互(Gaze based Interface)和视线辅助的交互(Gaze added Interface)，前者仅用眼睛和计算机或设备做交互，无其他输入设备，后者是将眼动辅以鼠控、语音交互和体感交互等多通道技术，有其他输入设备。There are two types of interaction technologies in the eye control system: Gaze based Interface and Gaze added Interface. The former only uses eyes to interact with computers or devices without other input devices, while the latter is Eye movements are supplemented with multi-channel technologies such as mouse control, voice interaction and somatosensory interaction, and there are other input devices.
传统的眼机交互的直接输入者是人眼，眼睛本身是一个感知器官，人类不习惯用眼动来控制目标，导致了一个瓶颈问题：系统难以辨识用户的注视行为是在检查信息，还是想触发操作，即“米达斯接触”效应。一旦出现“米达斯接触”，系统将出现输入冲突，引起误判，甚至会导致严重失误。采用通用的眼睛注视行为作为触发手段，很容易出现“米达斯接触”效应，导致系统崩溃。The direct input of the traditional eye-computer interaction is the human eye. The eye itself is a sensory organ. Humans are not used to using eye movements to control the target, which leads to a bottleneck problem: the system is difficult to identify whether the user's gaze behavior is checking information or thinking. Trigger action, the "Midas touch" effect. Once the "Midas contact" occurs, the system will have input conflicts, causing misjudgment and even serious mistakes. Using a generic eye gaze behavior as a trigger, the "Midas contact" effect is prone to occur, causing the system to crash.
发明内容SUMMARY OF THE INVENTION
发明目的：为了克服现有技术的不足，本发明提供一种眼动信号控制的交互方法，该方法可以解决数字界面交互的低效和“米达斯接触”效应的问题。Purpose of the invention: In order to overcome the deficiencies of the prior art, the present invention provides an interaction method for eye movement signal control, which can solve the problems of low efficiency of digital interface interaction and "Midas touch" effect.
技术方案：本发明所述的眼动信号控制的交互方法，该方法包括以下步骤：Technical solution: the interactive method for eye movement signal control according to the present invention, the method comprises the following steps:
(1)对获取的功能图标进行预处理；(1) Preprocess the acquired function icons;
(2)对眼动信号进行数字编码；(2) Digitally encode the eye movement signal;
(3)存储功能图标的眼动信号数组；(3) Store the eye movement signal array of the function icon;
(4)采集、输出、计算和识别所述眼动信号；(4) collecting, outputting, calculating and identifying the eye movement signal;
(5)被试者确认眼动指令；(5) The subject confirms the eye movement command;
所述功能图标预处理包括对所述功能图标进行数字编码，每个功能图标对应一个唯一的数字编号i，所述数字编号i的位数w为[1，4]区间上的整数，当w≥2时，数字编号的相邻数字间不重复，按照所述功能图标的使用频率m对所述功能图标进行数字编码。The function icon preprocessing includes digitally encoding the function icons, each function icon corresponds to a unique number i, and the number w of the number i is an integer in the interval [1, 4], when w When greater than or equal to 2, the adjacent numbers of the number numbers are not repeated, and the function icons are digitally coded according to the usage frequency m of the function icons.
优选的，所述功能图标的数字编码具体为：Preferably, the digital code of the function icon is specifically:
所述功能图标的使用频率为完成Z个主要操作任务所需某个功能图标的总单击次数X与完成Z个所述主要操作任务所需所有功能图标的总单击次数Y的比值，即m＝X/Y，m∈[0，1]，根据m的值从大到小对功能图标进行数字编码；当w取不同值时，数字编号有4/12/36/108种：The frequency of use of the function icons is the ratio of the total number of clicks X of a certain function icon required to complete the Z main operation tasks to the total number of clicks Y of all function icons required to complete the Z main operation tasks, that is, m=X/Y, m∈[0, 1], according to the value of m, the function icons are digitally encoded; when w takes different values, there are 4/12/36/108 kinds of digital numbers:
当w＝1时，数字编号i＝4，m的值排在前4位的功能图标的数字编码依次标定为：1、2、3、4；When w=1, the digital number i=4, and the digital codes of the function icons whose value m ranks in the first 4 digits are sequentially calibrated as: 1, 2, 3, 4;
当w＝2时，数字编号i＝12，m的值排在第5-16位的功能图标的数字编码依次标定为：12、13、14、21、23、24、31、32、34、41、42、43；When w=2, the digital number i=12, and the digital codes of the function icons whose values of m are ranked in the 5th to 16th positions are sequentially calibrated as: 12, 13, 14, 21, 23, 24, 31, 32, 34, 41, 42, 43;
当w＝3时，数字编号i＝36，m的值排在第17-52位的功能图标的数字编码依次标定为：121、123、124…431、432、434；When w=3, the digital number i=36, and the digital codes of the function icons whose value m ranks in the 17th-52nd position are sequentially calibrated as: 121, 123, 124...431, 432, 434;
当w＝4时，数字编号i＝108种，m的值排在第53-160位的功能图标的数字编码依次标定为：1212、1213、1214…4341、4342、4343。When w=4, the number i=108 kinds, and the digital codes of the function icons whose values of m are ranked in the 53rd to 160th positions are sequentially calibrated as: 1212, 1213, 1214...4341, 4342, 4343.
优选的，步骤(2)中，所述对眼动信号进行数字编码包括：Preferably, in step (2), the digital encoding of the eye movement signal includes:
(21)建立眼睛二维图像的直角坐标系；(21) establishing the Cartesian coordinate system of the two-dimensional image of the eye;
(22)根据瞳孔中心点的实时坐标，对所述眼动信号进行数字编码；(22) according to the real-time coordinates of the pupil center point, digitally encode the eye movement signal;
(23)眼球在不同象限下的有效连续运动会产生不同的数字编码序列，按照眼睛在不同象限有效运动的先后顺序，根据眼睛动作信号的数字编码，依次标定眼睛连续动作信号的数字编码。(23) The effective continuous movement of the eyeball in different quadrants will generate different digital coding sequences. According to the sequence of the effective eye movement in different quadrants, and according to the digital coding of the eye movement signal, the digital coding of the eye continuous movement signal is calibrated in turn.
优选的，步骤(22)中，所述对眼动信号进行数字编码的方法为：Preferably, in step (22), the method for digitally encoding the eye movement signal is:
瞳孔中心点的实时横坐标x0和实时纵坐标y0，当眼睛运动到第一象限，满足x0＞0，y0＞0且x02+y02≥nmm2条件时，标定所述眼动信号数字编码为“1”；其中，n为最小有效面积范围；The real-time abscissa x0 and the real-time ordinate y0 of the pupil center point, when the eye moves to the first quadrant and satisfies the conditions of x0 > 0, y0 > 0 and x02 +y02 ≥ nmm2 , the calibration The digital code of the eye movement signal is "1"; wherein, n is the minimum effective area range;
当眼睛运动到第二象限，满足x0＜0，y0＞0且x02+y02≥nmm2条件时，标定所述眼动信号数字编码为“2”；When the eye moves to the second quadrant and satisfies the conditions of x0 <0, y0 >0 and x02 +y02 ≥ nmm2 , the digital code of the eye movement signal is calibrated as "2";
当眼睛运动到第三象限，满足x0＜0，y0＜0且x02+y02≥nmm2条件时，标定所述眼动信号数字编码为“3”；When the eye moves to the third quadrant and the conditions of x0 <0, y0 <0 and x02 +y02 ≥nmm2 are satisfied, the digital code of the eye movement signal is calibrated as "3";
当眼睛运动到第四象限，满足x0＞0，y0＜0且x02+y02≥nmm2条件时，标定所述眼动信号数字编码为“4”；When the eye moves to the fourth quadrant and satisfies the conditions of x0 >0, y0 <0 and x02 +y02 ≥ nmm2 , the digital code of the eye movement signal is calibrated as "4";
当所述瞳孔中心点和坐标轴线重合时，即x0＝0或y0＝0，认为眼球未做有效运动，不予标定。When the center point of the pupil coincides with the coordinate axis, that is, x0 =0 or y0 =0, it is considered that the eyeball has not performed effective movement and is not calibrated.
优选的，所述步骤(3)中存储功能图标的眼动信号数组包括：Preferably, the eye movement signal array storing the function icon in the step (3) includes:
(31)将所述功能图标数字编码与所述眼动信号数字编码进行一一对应的匹配；(31) performing a one-to-one matching of the digital code of the function icon and the digital code of the eye movement signal;
(32)将匹配后的所有所述眼动信号的数字编码，以数组形式存储于微处理器中。(32) Store the digital codes of all the matched eye movement signals in the microprocessor in the form of an array.
有益效果：本发明通过对功能图标和眼睛动作进行数字编码，用户根据功能图标右上角的数字编码，在四个象限进行有序眼睛运动，实现眼睛对功能图标的触发，更加直接和简洁，有效规避了用户的注视行为，有较高的准确率，可节省操作时间，直接避免了“米达斯接触”效应的发生，减少了误触发率，显著提高了数字界面的交互效率。Beneficial effects: the present invention performs digital coding on the functional icons and eye movements, and the user performs orderly eye movements in four quadrants according to the digital coding on the upper right corner of the functional icons, so as to realize the triggering of the functional icons by the eyes, which is more direct, concise and effective It avoids the user's gaze behavior, has a high accuracy rate, saves operation time, directly avoids the occurrence of the "Midas contact" effect, reduces the false trigger rate, and significantly improves the interactive efficiency of the digital interface.
附图说明Description of drawings
图1为本发明的方法流程图；Fig. 1 is the method flow chart of the present invention;
图2为本发明的功能图标的数字编码示意图；Fig. 2 is the digital coding schematic diagram of the function icon of the present invention;
图3为本发明的图标的一种预处理示意图；Fig. 3 is a kind of preprocessing schematic diagram of the icon of the present invention;
图4为本发明的眼睛图像区域划分示意图；Fig. 4 is the schematic diagram of eye image area division of the present invention;
图5为本发明的方法原理示意图；5 is a schematic diagram of the method principle of the present invention;
图6为本发明的功能图标的确认对话框窗口图。FIG. 6 is a diagram of a confirmation dialog window of a function icon of the present invention.
具体实施方式Detailed ways
如图1，本发明涉及一种眼动信号控制的交互方法，本方法不需要任何语言和动作，直接通过图标的眼动信号实现对数字界面的控制，因此，适用于残疾人、老年人和飞行员等特定人群，在某些不便用手协调的场合，如宇航员在飞船或太空舱操作计算机、汽车驾驶、手术过程等，也可使用基于眼动信息的人机接口来执行动作或进行交流。首先进行图标的预处理，包括：As shown in Figure 1, the present invention relates to an interactive method of eye movement signal control. This method does not require any language and action, and directly controls the digital interface through the eye movement signal of the icon. Therefore, it is suitable for the disabled, the elderly and the Certain groups of people, such as pilots, can also use the human-machine interface based on eye movement information to perform actions or communicate in situations where it is inconvenient to coordinate with their hands, such as astronauts operating computers in spacecraft or space capsules, driving cars, and surgical procedures. . First, preprocess the icon, including:
(1)获取功能图标(1) Get the function icon
从任一开源软件的系统文件中获取PNG格式功能图标，即用户界面中的功能标识符号，约定某开源软件的功能图标总个数不超过108个，108个以上的功能图标眼动信号的控制方法，也可根据原理对本发明的技术方案做相应改变，从而实现眼动信号的控制作用。Obtain the PNG format function icon from the system file of any open source software, that is, the function identifier in the user interface. It is stipulated that the total number of function icons of an open source software does not exceed 108, and the control of the eye movement signal of the function icons above 108 According to the principle, the technical solution of the present invention can also be changed accordingly, so as to realize the control effect of the eye movement signal.
(2)功能图标的数字编码(2) The digital code of the function icon
每个功能图标对应一个唯一的数字编号，定义该数字编号为i，且i为正整数，数字编号i的位数w为[1，4]区间上的整数，数字编号i可表示为a、ab、abc或abcd，a、b、c、d为[1，4]区间上的整数，其中，w≥2时，数字编号i的相邻数字间不重复，数字编号i共计可标定为160种情况。上述涉及的某款开源软件有Z个主要操作任务，此处的主要操作任务为完成某款开源软件的某个主要功能触发时，所需要单个或多个功能图标的触发组合，Z的数量根据某款开源软件的主要功能来确定，其中每个任务操作流程主要是对不同功能图标单击的组合。Each function icon corresponds to a unique number number, which is defined as i, and i is a positive integer, the number of digits w of the number number i is an integer in the interval [1, 4], and the number number i can be expressed as a, ab, abc or abcd, a, b, c, d are integers in the interval [1, 4], where w≥2, the adjacent numbers of the number i are not repeated, and the number i can be calibrated to 160 in total a situation. A certain open source software mentioned above has Z main operation tasks. The main operation task here is the trigger combination of single or multiple function icons required to complete a certain main function trigger of a certain open source software. The number of Z is based on The main function of a certain open source software is determined, in which each task operation process is mainly a combination of clicks on different function icons.
如图2，按照功能图标的使用频率m，对功能图标进行数字编码。所述的功能图标的使用频率m为完成Z个操作任务所需某个功能图标的总单击次数X与完成Z个操作任务所需所有功能图标的总单击次数Y的比值，即m＝X/Y，m∈[0，1]，根据m的值从大到小对功能图标进行数字编码：As shown in Figure 2, according to the use frequency m of the function icons, the function icons are digitally coded. The use frequency m of the function icons is the ratio of the total number of clicks X of a certain function icon required to complete the Z operation tasks to the total number of clicks Y of all the function icons required to complete the Z operation tasks, that is, m= X/Y, m ∈ [0, 1], digitally encode the function icons according to the value of m from large to small:
w＝1时，数字编号i有1×4＝4种，m的值排在前4位的功能图标的数字编码依次标定为：1、2、3、4。When w=1, there are 1×4=4 kinds of digital numbers i, and the digital codes of the function icons whose value m ranks in the first 4 digits are demarcated as: 1, 2, 3, and 4 in turn.
w＝2时，数字编号i有4×3＝12种，m的值排在第5-16位的功能图标的数字编码依次标定为：12、13、14、21、23、24、31、32、34、41、42、43。When w=2, there are 4×3=12 kinds of digital numbers i, and the digital codes of the function icons whose value m ranks in the 5th to 16th positions are calibrated as follows: 12, 13, 14, 21, 23, 24, 31, 32, 34, 41, 42, 43.
w＝3时，数字编号i有4×3×3＝36种，m的值排在第17-52位的功能图标的数字编码依次标定为：121、123、124…431、432、434。When w=3, there are 4×3×3=36 kinds of digital numbers i, and the digital codes of the function icons whose values of m are ranked in the 17th to 52nd positions are sequentially calibrated as: 121, 123, 124...431, 432, 434.
w＝4时，数字编号i有4×3×3×3＝108种，m的值排在第53-160位的功能图标的数字编码依次标定为：1212、1213、1214…4341、4342、4343。When w=4, there are 4×3×3×3=108 kinds of digital numbers i, and the digital codes of the function icons whose value m ranks in the 53rd-160th position are calibrated as follows: 1212, 1213, 1214…4341, 4342, 4343.
(3)功能图标的处理(3) Processing of function icons
运用Photoshop，Illustrator或Coreldraw图形图像处理软件，将前一步骤标定好的数字编号i，放置于与之对应的PNG格式功能图标的右上角，如图3为一种处理方式，经过处理后的PNG格式功能图标的像素和处理前的原始图标保持一致。Using Photoshop, Illustrator or Coreldraw graphics and image processing software, place the number i calibrated in the previous step in the upper right corner of the corresponding PNG format function icon, as shown in Figure 3 for a processing method, the processed PNG The pixel of the format function icon is the same as the original icon before processing.
(4)功能图标的替换(4) Replacement of function icons
手动找到原始PNG格式功能图标的系统文件夹，在该文件夹的根目录下，将经过处理的PNG功能图标进行同文件名覆盖或替换。Manually find the system folder of the original PNG format function icon, and in the root directory of the folder, overwrite or replace the processed PNG function icon with the same file name.
其次，对眼睛动作信号的数字编码；Second, the digital encoding of the eye action signal;
(1)建立眼睛二维图像的直角坐标系(1) Establish the Cartesian coordinate system of the two-dimensional image of the eye
眼睛动作信号通过瞳孔中心点的坐标(x，y)进行表征，将眼睛的二维图像定义为平面直角坐标系，向右为x轴正向，向左为x轴负向，向上为y轴正向，向下为y轴负向，x轴和y轴将坐标空间划分为四个区域：The eye action signal is characterized by the coordinates (x, y) of the pupil center point, and the two-dimensional image of the eye is defined as a plane rectangular coordinate system, with the positive x-axis to the right, the negative x-axis to the left, and the y-axis upward. Positive, downward is the negative direction of the y-axis, the x-axis and the y-axis divide the coordinate space into four areas:
x轴正向和y轴正向组成的区域为第一区域，The area composed of the positive x-axis and the positive y-axis is the first area,
x轴负向和y轴正向组成的区域为第二区域，The area composed of the negative x-axis and the positive y-axis is the second area,
x轴负向和y轴负向组成的区域为第三区域，The area composed of the negative x-axis and the negative y-axis is the third area,
x轴正向和y轴负向组成的区域为第四区域。The area composed of the positive x-axis and the negative y-axis is the fourth area.
眼睛正视前方时瞳孔中心点为坐标系零点(0，0)，眼睛运动过程中瞳孔中心点的实时坐标为(x0,y0)。When the eye looks straight ahead, the center point of the pupil is the zero point of the coordinate system (0, 0), and the real-time coordinates of the center point of the pupil during the movement of the eye are (x0 , y0 ).
(2)眼睛动作信号的数字编码(2) Digital coding of eye movement signal
瞳孔中心点的实时横坐标x0和实时纵坐标y0，当眼睛运动到第一象限，满足x0＞0，y0＞0且x02+y02≥9mm2条件时，标定所述眼动信号数字编码为“1”；The real-time abscissa x0 and real-time ordinate y0 of the pupil center point, when the eye moves to the first quadrant, when x0 > 0, y0 > 0 and x02+ y02 ≥ 9mm2, the calibration The digital code of the eye movement signal is "1";
当眼睛运动到第二象限，满足x0＜0，y0＞0且x02+y02≥9mm2条件时，标定所述眼动信号数字编码为“2”；When the eye moves to the second quadrant and satisfies the conditions of x0 <0, y0 >0 and x02 +y02 ≥ 9mm2 , the digital code of the eye movement signal is calibrated as “2”;
当眼睛运动到第三象限，满足x0＜0，y0＜0且x02+y02≥9mm2条件时，标定所述眼动信号数字编码为“3”；When the eye moves to the third quadrant and satisfies the conditions of x0 <0, y0 <0 and x02 +y02 ≥ 9mm2 , the digital code of the eye movement signal is calibrated as “3”;
当眼睛运动到第四象限，满足x0＞0，y0＜0且x02+y02≥9mm2条件时，标定所述眼动信号数字编码为“4”；When the eye moves to the fourth quadrant and the conditions of x0 >0, y0 <0 and x02 +y02 ≥ 9mm2 are satisfied, the digital code of the eye movement signal is calibrated as "4";
当所述瞳孔中心点和坐标轴线重合时，即x0＝0或y0＝0，认为眼球未做有效运动，不予标定。When the center point of the pupil coincides with the coordinate axis, that is, x0 =0 or y0 =0, it is considered that the eyeball has not performed effective movement and is not calibrated.
上述公式中的9mm2是当采用300HZ采样率的眼动仪的最小有效面积范围为9mm2，即以眼球为中心的最小有效面积范围，眼睛在此范围之内的运动，是无效的眼动信号编码。在本发明的优选方案中，9mm2可以根据眼动仪的采样精度来进行调整，眼动仪精度越高，可适当缩小该范围。9mm2 in the above formula is when the minimum effective area range of an eye tracker with a sampling rate of 300HZ is 9mm2 , that is, the minimum effective area range centered on the eyeball, and the movement of the eye within this range is invalid eye movement. signal encoding. In the preferred solution of the present invention, 9mm2 can be adjusted according to the sampling accuracy of the eye tracker, and the higher the accuracy of the eye tracker, the narrower the range can be.
若2000HZ采样率的眼动仪的最小有效面积范围则可降为4mm2，即眼动信号的数字编码表示为：If the minimum effective area of the eye tracker with a sampling rate of 2000Hz can be reduced to 4mm2 , that is, the digital code of the eye movement signal is expressed as:
瞳孔中心点的实时横坐标x0和实时纵坐标y0，当眼睛运动到第一象限，满足x0＞0，y0＞0且x02+y02≥4mm2条件时，标定所述眼动信号数字编码为“1”；The real-time abscissa x0 and the real-time ordinate y0 of the pupil center point, when the eye moves to the first quadrant and satisfies the conditions of x0 > 0, y0 > 0 and x02+ y02 ≥ 4mm2, the calibration The digital code of the eye movement signal is "1";
当眼睛运动到第二象限，满足x0＜0，y0＞0且x02+y02≥4mm2条件时，标定所述眼动信号数字编码为“2”；When the eye moves to the second quadrant, and satisfies the conditions of x0 <0, y0 >0 and x02 +y02 ≥ 4mm2 , the digital code of the eye movement signal is calibrated as “2”;
当眼睛运动到第三象限，满足x0＜0，y0＜0且x02+y02≥4mm2条件时，标定所述眼动信号数字编码为“3”；When the eye moves to the third quadrant and satisfies the conditions of x0 <0, y0 <0 and x02 +y02 ≥ 4mm2 , the digital code of the eye movement signal is calibrated as “3”;
当眼睛运动到第四象限，满足x0＞0，y0＜0且x02+y02≥4mm2条件时，标定所述眼动信号数字编码为“4”；When the eye moves to the fourth quadrant and the conditions of x0 >0, y0 <0 and x02 +y02 ≥ 4mm2 are satisfied, the digital code of the eye movement signal is calibrated as "4";
当所述瞳孔中心点和坐标轴线重合时，即x0＝0或y0＝0，认为眼球未做有效运动，不予标定。When the center point of the pupil coincides with the coordinate axis, that is, x0 =0 or y0 =0, it is considered that the eyeball has not performed effective movement and is not calibrated.
(3)眼睛连续动作信号的数字编码(3) Digital coding of eye continuous motion signal
如图4，眼球在不同象限下的有效连续运动会产生不同的数字编码序列，按照眼睛在不同象限有效运动的先后顺序，根据眼睛动作信号的数字编码，依次标定眼睛连续动作信号的数字编码，即为不同眼睛动作信号的数字编码的组合形式。As shown in Figure 4, the effective continuous movement of the eyeball in different quadrants will generate different digital coding sequences. According to the sequence of the effective eye movement in different quadrants, and according to the digital coding of the eye action signal, the digital coding of the eye continuous action signal is calibrated in turn, that is, Combination of digital encodings for different eye movement signals.
然后，对功能图标眼动信号数字编码进行存储。Then, the digital code of the eye movement signal of the function icon is stored.
(1)功能图标数字编码与眼睛动作信号数字编码的匹配(1) Matching between the digital code of the function icon and the digital code of the eye movement signal
以1、2、3、4为最小编码，进行数字组合，相邻数字间不重复，即4*3*3*3＝108种组合方式，且1、2、3、4代表不同的眼睛运动形式。Use 1, 2, 3, and 4 as the minimum codes to combine numbers without repeating adjacent numbers, that is, 4*3*3*3=108 combinations, and 1, 2, 3, and 4 represent different eye movements form.
功能图标右上角的数字“1”代表眼睛运动到第一象限，且瞳孔中心点坐标满足x0＞0,y0＞0且x02+y02≥9mm2。The number "1" in the upper right corner of the function icon represents that the eye moves to the first quadrant, and the coordinates of the pupil center point satisfy x0 >0, y0 >0 and x02 +y02 ≥9mm2 .
功能图标右上角的数字“2”代表眼睛运动到第二象限，且瞳孔中心点坐标满足x0＜0,y0＞0且x02+y02≥9mm2。The number "2" in the upper right corner of the function icon represents that the eye moves to the second quadrant, and the coordinates of the pupil center point satisfy x0 <0, y0 >0 and x02 +y02 ≥9mm2 .
功能图标右上角的数字“3”代表眼睛运动到第三象限，且瞳孔中心点坐标满足x0＜0,y0＜0且x02+y02≥9mm2。The number "3" in the upper right corner of the function icon represents that the eye moves to the third quadrant, and the coordinates of the pupil center point satisfy x0 <0, y0 <0 and x02 +y02 ≥9mm2 .
功能图标右上角的数字“4”代表眼睛运动到第四象限，且瞳孔中心点坐标满足x0＞0,y0＜0且x02+y02≥9mm2。The number "4" in the upper right corner of the function icon represents that the eye moves to the fourth quadrant, and the coordinates of the pupil center point satisfy x0 >0, y0 <0 and x02 +y02 ≥9mm2 .
同时，可根据实际情况进行扩展，比如以1、2、3、4为最小编码，进行五位数的组合12341，相邻数字间不重复，即4*3*3*3*3＝324种组合方式，且1、2、3、4代表不同的眼睛运动形式。但实际上，一个软件一般不可能有300个功能图标。At the same time, it can be expanded according to the actual situation. For example, with 1, 2, 3, and 4 as the minimum codes, the combination of five digits is 12341, and the adjacent digits are not repeated, that is, 4*3*3*3*3=324 kinds Combinations, and 1, 2, 3, 4 represent different forms of eye movement. But in fact, it is generally impossible for a software to have 300 function icons.
本发明只用“1、2、3、4”是基于眼睛在四个象限的运动为编码来考虑的，不用“1、2、3、4、5、...、108”，因为如果每个数字代表一种眼动运动形式，用户根本记不住那么多眼动形式，而且也没有那么多眼动形式，以组合的方式呈现最佳。The present invention only uses "1, 2, 3, 4" based on the movement of the eyes in the four quadrants for coding, and does not use "1, 2, 3, 4, 5, ..., 108", because if each Each number represents one form of eye movement, the user simply cannot remember so many eye movement forms, and there are not so many eye movement forms, which are best presented in a combined way.
同理，若2000HZ采样率的眼动仪的最小有效面积范围则可降为4mm2，功能图标数字编码与眼睛动作信号数字编码的匹配为：Similarly, if the minimum effective area of an eye tracker with a sampling rate of 2000HZ can be reduced to 4mm2 , the matching between the digital code of the function icon and the digital code of the eye movement signal is:
功能图标右上角的数字“1”代表眼睛运动到第一象限，且瞳孔中心点坐标满足x0＞0,y0＞0且x02+y02≥4mm2。The number "1" in the upper right corner of the function icon represents that the eye moves to the first quadrant, and the coordinates of the pupil center point satisfy x0 >0, y0 >0 and x02 +y02 ≥4mm2 .
功能图标右上角的数字“2”代表眼睛运动到第二象限，且瞳孔中心点坐标满足x0＜0,y0＞0且x02+y02≥4mm2。The number "2" in the upper right corner of the function icon represents that the eye moves to the second quadrant, and the coordinates of the pupil center point satisfy x0 <0, y0 >0 and x02 +y02 ≥4mm2 .
功能图标右上角的数字“3”代表眼睛运动到第三象限，且瞳孔中心点坐标满足x0＜0,y0＜0且x02+y02≥4mm2。The number "3" in the upper right corner of the function icon represents that the eye moves to the third quadrant, and the coordinates of the pupil center point satisfy x0 <0, y0 <0 and x02 +y02 ≥4mm2 .
功能图标右上角的数字“4”代表眼睛运动到第四象限，且瞳孔中心点坐标满足x0＞0,y0＜0且x02+y02≥4mm2The number "4" in the upper right corner of the function icon means that the eye moves to the fourth quadrant, and the coordinates of the pupil center point satisfy x0 >0, y0 <0 and x02 +y02 ≥4mm2
(2)软件所有功能图标的眼动信号数组的存储(2) Storage of eye movement signal arrays of all function icons of the software
将软件中的所有功能图标眼动信号的数字编码，以数组b[j]的形式存储于微处理器中，其中，j表示功能图标的总个数。The digital codes of the eye movement signals of all function icons in the software are stored in the microprocessor in the form of an array b[j], where j represents the total number of function icons.
此时存储的功能图标编码和眼动信号编码必须一一对应，因为只有编码一一对应才能触发目标图标，即用户想要触发的那个功能图标，否则，可能会出现误触发或不响应的情况发生。At this time, the stored function icon code and the eye movement signal code must be in one-to-one correspondence, because only the one-to-one correspondence of the codes can trigger the target icon, that is, the function icon that the user wants to trigger, otherwise, false triggering or no response may occur. occur.
然后，对眼睛运动信号进行采集、输出、计算与识别。Then, the eye movement signal is collected, output, calculated and recognized.
(1)眨眼检测器与眼动仪的通信(1) Communication between the blink detector and the eye tracker
眼睛运动信号的记录与采集过程，由眨眼检测器与Tobii X2-30便携式眼动仪共同通信完成，其中眨眼检测器用于记录眨眼事件的次数以及眼睛闭合事件的时长，TobiiX2-30便携式眼动仪用于采集被试者的眼动数据。当眨眼检测器检测到眨眼事件且眼睛闭合时长在3s以上时，眼动仪开始采集被试者的眼球运动数据。The process of recording and collecting eye movement signals is completed by the blink detector and Tobii X2-30 portable eye tracker. The blink detector is used to record the number of blink events and the duration of eye closure events. Used to collect eye movement data of subjects. When the blink detector detects the blink event and the eyes are closed for more than 3s, the eye tracker starts to collect the subject's eye movement data.
(2)采集被试者的眼动数据(2) Collect the eye movement data of the subjects
如图5，被试者眼睛注视软件界面的功能图标，同时观察功能图标的数字编码，依次按照该数字编码做眼睛运动，被试者看到数字编码“1”，眼睛朝右上运动，被试者看到数字编码“2”，眼睛朝左上运动，被试者看到数字编码“3”，眼睛朝左下运动，被试者看到数字编码“4”，眼睛朝右下运动。As shown in Figure 5, the subjects looked at the function icons of the software interface, observed the digital codes of the function icons, and followed the digital codes to make eye movements. The subjects saw the digital code "2", and their eyes moved to the upper left, the subjects saw the digital code "3", and their eyes moved to the lower left, and the subjects saw the digital code "4", and their eyes moved to the lower right.
(3)眼睛运动信号停止记录的触发事件(3) Trigger event for eye movement signal to stop recording
当眨眼检测器检测到连续3次闭合眼睛事件发生时，眼动仪停止对眼睛运动信号的记录。When the eye-blink detector detects the occurrence of three consecutive eye-closing events, the eye-tracker stops recording the eye movement signal.
人类自然连续眨眼一般为1-2次，3次是有目的性眨眼的最低次数，同理，本发明也可针对3次以上的眨眼，但3次以上的眨眼会给用户带来视觉疲劳，增加眼动的复杂性。Humans naturally blink 1-2 times continuously, and 3 times is the minimum number of blinks for purpose. Similarly, the present invention can also be aimed at blinking more than 3 times, but blinking more than 3 times will bring visual fatigue to the user. Increases the complexity of eye movements.
(4)输出瞳孔中心点的实时坐标(4) Output the real-time coordinates of the pupil center point
在Tobii Studio软件的Text Export模块下，输出眼睛运动过程中瞳孔中心点的实时坐标(x0,y0)，将该坐标值输送到象限计算器。Under the Text Export module of Tobii Studio software, the real-time coordinates (x0 , y0 ) of the pupil center point during eye movement are output, and the coordinate values are sent to the quadrant calculator.
象限计算器与眨眼检测器、Tobii Studio软件等三个模块相互通信，象限计算器用于计算被试者瞳孔中心点坐标所在的象限位置，约定象限计算器对于Tobii Studio软件连续输送过来的同一象限内的多个不同瞳孔坐标点，只输出1次象限数字代码，直至象限发生变化时，输出新的象限数字代码。象限计算器的数字代码有四个：1、2、3、4，分别对应第一象限、第二象限、第三象限和第四象限。The quadrant calculator communicates with the blink detector and the Tobii Studio software. The quadrant calculator is used to calculate the quadrant position where the coordinates of the subject's pupil center point are located. For multiple different pupil coordinate points, the quadrant digital code is output only once, and when the quadrant changes, a new quadrant digital code is output. The quadrant calculator has four numerical codes: 1, 2, 3, and 4, which correspond to the first, second, third, and fourth quadrants, respectively.
瞳孔中心点坐标(x，y)所在象限位置的计算由Matlab软件编程完成，程序代码如下：The calculation of the quadrant position of the pupil center point coordinates (x, y) is done by Matlab software programming, and the program code is as follows:
function[results]＝myfun(x,y)function[results]=myfun(x,y)
if(x>0&y>0&x^2+y^2>＝9)if(x>0&y>0&x^2+y^2>=9)
results＝1；results=1;
endend
说明：此段为第一象限位置的计算程序。Description: This section is the calculation program for the position of the first quadrant.
if(x<0&y>0&x^2+y^2>＝9)if(x<0&y>0&x^2+y^2>=9)
results＝2；results=2;
endend
说明：此段为第二象限位置的计算程序。Description: This section is the calculation program of the second quadrant position.
if(x<0&y<0&x^2+y^2>＝9)if(x<0&y<0&x^2+y^2>=9)
results＝3；results=3;
endend
说明：此段为第三象限位置的计算程序。Description: This section is the calculation program of the third quadrant position.
if(x>0&y<0&x^2+y^2>＝9)if(x>0&y<0&x^2+y^2>=9)
results＝4；results=4;
endend
说明：此段为第四象限位置的计算程序。Description: This section is the calculation program of the fourth quadrant position.
disp(results)disp(results)
endend
(5)眼睛运动信号的识别(5) Identification of eye movement signals
将象限计算器输出的象限数字代码组合p在微处理器中，与数组b[j]进行匹配，如果p存在于数组b[j]中，输出功能图标的触发指令q。Combine the quadrant digital code p output by the quadrant calculator in the microprocessor, and match it with the array b[j], if p exists in the array b[j], output the trigger instruction q of the function icon.
数字p与数组b[j]的匹配过程如下，用Matlab软件编程实现：The matching process between the number p and the array b[j] is as follows, which is implemented by Matlab software programming:
[a,b,c,d]＝ndgrid(1:4)；[a,b,c,d]=ndgrid(1:4);
bb1＝[a(:),b(:),c(:),d(:)]；bb1=[a(:),b(:),c(:),d(:)];
[e f g]＝ndgrid(1:4)；[e f g] = ndgrid(1:4);
bb2＝[e(:)f(:)g(:)]；bb2=[e(:)f(:)g(:)];
[h i]＝ndgrid(1:4)；[hi]=ndgrid(1:4);
bb3＝[h(:)i(:)]；bb3=[h(:)i(:)];
LUT1＝nan(2,1)；LUT2＝nan(2,1)；LUT3＝nan(2,1)；LUT1=nan(2,1); LUT2=nan(2,1); LUT3=nan(2,1);
说明：此段为变量的定义Description: This section is the definition of a variable
说明：此段为四位数字编码的眼动信号Description: This segment is the eye movement signal encoded by four digits
说明：此段为三位数字编码的眼动信号Description: This segment is an eye movement signal encoded by three digits.
说明：此段为两位数字编码的眼动信号Description: This segment is the eye movement signal encoded by two digits
说明：此段为数字p与数组b[j]匹配的计算过程。Description: This section is the calculation process of matching the number p with the array b[j].
最后，被试者确认眼睛动作指令；Finally, the subjects confirmed the eye movement instructions;
(1)功能图标触发指令窗口的编译(1) The function icon triggers the compilation of the command window
当用户想触发某特定功能图标时，功能图标需要接收到触发指令，本实施例以“放大功能”图标为例。放大功能指通过眼睛观看“放大镜”图标，实现触发“放大镜”图标，该案例的最终效果是实现对软件窗口内信息或者文字的放大。如图5中，软件窗口内的文字是“xxxxxx”，在触发特定功能图标后，首先弹出是否实现该功能的确认对话窗口，之后窗口内的文字“xxxxxx”即实现了该功能图标所指的特定功能。When a user wants to trigger a specific function icon, the function icon needs to receive a trigger instruction, and this embodiment takes the "zoom in function" icon as an example. The magnifying function refers to viewing the "magnifying glass" icon through the eyes to trigger the "magnifying glass" icon. The final effect of this case is to enlarge the information or text in the software window. As shown in Figure 5, the text in the software window is "xxxxxx". After triggering a specific function icon, a confirmation dialog window will pop up whether to implement the function, and then the text "xxxxxx" in the window will realize the function icon. specific function.
以放大功能为例，如图6，接受到功能图标的触发指令q后，弹出确认对话框窗口，该窗口由VB语言编译生成，代码如下：Taking the zoom function as an example, as shown in Figure 6, after receiving the trigger command q of the function icon, a confirmation dialog window pops up, which is compiled and generated by VB language, and the code is as follows:
(2)功能图标触发指令的确认(2) Confirmation of function icon trigger command
功能图标触发指令的确认过程由眨眼检测器与窗口编译模块共同通信完成。The confirmation process of the command triggered by the function icon is completed by the blink detector and the window compiling module through joint communication.
当眨眼检测器检测到被试者3秒内眨眼次数少于4次，则认为无误，计算机运行d:\1.exe，d:\1.exe为实现功能图标的触发的可执行程序。When the blink detector detects that the subject blinks less than 4 times within 3 seconds, it is considered correct, and the computer runs d:\1.exe, which is an executable program that triggers the function icon.
当检测到3秒内连续4次眨眼，则认为指令错误，计算机运行d:\2.exe，d:\2.exe为删除该触发指令并重新提取采集和识别眼睛运动信号的可执行程序。When it detects 4 consecutive blinks within 3 seconds, it is considered that the instruction is wrong, and the computer runs d:\2.exe, d:\2.exe is an executable program to delete the trigger instruction and re-extract to collect and identify eye movement signals.
Claims (3)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810127171.9ACN108491068B (en) | 2018-02-08 | 2018-02-08 | Interaction method for eye movement signal control |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810127171.9ACN108491068B (en) | 2018-02-08 | 2018-02-08 | Interaction method for eye movement signal control |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108491068A CN108491068A (en) | 2018-09-04 |
| CN108491068Btrue CN108491068B (en) | 2020-11-17 |
Family
ID=63339948
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810127171.9AActiveCN108491068B (en) | 2018-02-08 | 2018-02-08 | Interaction method for eye movement signal control |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108491068B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109799908B (en)* | 2019-01-02 | 2022-04-01 | 东南大学 | Image zooming and dragging method based on eye movement signal |
| CN114253398B (en)* | 2021-12-08 | 2024-08-20 | 深圳市多狗乐智能研发有限公司 | High-simulation biological eye interactive image display module and method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103259971A (en)* | 2012-02-16 | 2013-08-21 | 由田信息技术(上海)有限公司 | Eye control device in vehicle and method for eye control |
| CN103336581A (en)* | 2013-07-30 | 2013-10-02 | 黄通兵 | Human eye movement characteristic design-based human-computer interaction method and system |
| CN103782131A (en)* | 2011-07-25 | 2014-05-07 | 莱卡地球系统公开股份有限公司 | Measuring device that can be operated without contact and control method for such a measuring device |
| CN104360751A (en)* | 2014-12-05 | 2015-02-18 | 三星电子（中国）研发中心 | Method and equipment realizing intelligent control |
| CN106303640A (en)* | 2016-08-15 | 2017-01-04 | 张伟 | Interaction method of remote controller and smart television |
| CN106547360A (en)* | 2016-11-29 | 2017-03-29 | 珠海格力电器股份有限公司 | Information processing method and electronic equipment |
| CN106774862A (en)* | 2016-12-03 | 2017-05-31 | 西安科锐盛创新科技有限公司 | VR display methods and VR equipment based on sight line |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8593375B2 (en)* | 2010-07-23 | 2013-11-26 | Gregory A Maltz | Eye gaze user interface and method |
| KR101989893B1 (en)* | 2012-10-29 | 2019-09-30 | 엘지전자 주식회사 | A Head Mounted Display and A Method of Outputting Audio Signal Using the Same |
- 2018
- 2018-02-08CNCN201810127171.9Apatent/CN108491068B/enactiveActive
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103782131A (en)* | 2011-07-25 | 2014-05-07 | 莱卡地球系统公开股份有限公司 | Measuring device that can be operated without contact and control method for such a measuring device |
| CN103259971A (en)* | 2012-02-16 | 2013-08-21 | 由田信息技术(上海)有限公司 | Eye control device in vehicle and method for eye control |
| CN103336581A (en)* | 2013-07-30 | 2013-10-02 | 黄通兵 | Human eye movement characteristic design-based human-computer interaction method and system |
| CN104360751A (en)* | 2014-12-05 | 2015-02-18 | 三星电子（中国）研发中心 | Method and equipment realizing intelligent control |
| CN106303640A (en)* | 2016-08-15 | 2017-01-04 | 张伟 | Interaction method of remote controller and smart television |
| CN106547360A (en)* | 2016-11-29 | 2017-03-29 | 珠海格力电器股份有限公司 | Information processing method and electronic equipment |
| CN106774862A (en)* | 2016-12-03 | 2017-05-31 | 西安科锐盛创新科技有限公司 | VR display methods and VR equipment based on sight line |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108491068A (en) | 2018-09-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11493993B2 (en) | Systems, methods, and interfaces for performing inputs based on neuromuscular control | |
| Chen et al. | A review of myoelectric control for prosthetic hand manipulation | |
| CN108459710B (en) | An interactive device controlled by eye movement signals | |
| Palaniappan et al. | A new brain-computer interface design using fuzzy ARTMAP | |
| WO2017196618A1 (en) | Changing an application state using neurological data | |
| US8205165B2 (en) | Apparatus to create, save and format text documents using gaze control and method associated based on the optimized positioning of cursor | |
| US11307665B2 (en) | Wink gesture based control system | |
| CN107239137B (en) | Character input method and device based on virtual keyboard | |
| DE102013226244A1 (en) | Medical control | |
| Adnan et al. | Measurement of the flexible bending force of the index and middle fingers for virtual interaction | |
| DE10056291A1 (en) | Visual display of objects in field of view for man-machine communication by acquiring information input by user using signal or pattern recognition | |
| CN108491068B (en) | Interaction method for eye movement signal control | |
| Allison | The I of BCIs: Next generation interfaces for brain–computer interface systems that adapt to individual users | |
| Xu et al. | Boosters of the metaverse: a review of augmented reality-based brain-computer interface | |
| Williamson et al. | Efficient human-machine control with asymmetric marginal reliability input devices | |
| CN107229330A (en) | A text input method and device based on steady-state visual evoked potential | |
| EP1756700B1 (en) | System and method for bodily controlled data input | |
| Prashun et al. | Investigating the trend of virtual reality-based stroke rehabilitation systems | |
| Avola et al. | Toward the future of surgery: an immersive, virtual-reality-based endoscopic prototype | |
| US20210225525A1 (en) | Virtual reality-based systems and methods | |
| Vasiljevas | Development of a concept-based EMG-based speller | |
| Zhou et al. | Deep learning based gesture recognition and its application in interactive control of intelligent wheelchair | |
| Yang et al. | Study on Finger gesture interface using 1-channel EMG | |
| Liang et al. | A Review of Multimodal Interaction | |
| Wang et al. | Comparative study on gesture recognition using multiple kernel learning via multi-mode information fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |