技术领域technical field
本发明涉及图像处理技术领域，具体而言，涉及一种标记物的识别方法、装置及识别跟踪系统。The present invention relates to the technical field of image processing, and in particular, to a marker identification method, device and identification tracking system.
背景技术Background technique
近年来，随着科技的进步，增强现实(AR，Augmented Reality)和虚拟现实(VR，Virtual Reality)等技术已逐渐成为国内外研究的热点。以增强现实为例，增强现实是通过计算机系统提供的信息增加用户对现实世界感知的技术，其将计算机生成的虚拟物体、场景或系统提示信息叠加到真实场景中，来增强或修改对现实世界环境或表示现实世界环境的数据的感知。In recent years, with the advancement of science and technology, technologies such as augmented reality (AR, Augmented Reality) and virtual reality (VR, Virtual Reality) have gradually become the focus of research at home and abroad. Taking augmented reality as an example, augmented reality is a technology that increases users' perception of the real world through information provided by a computer system. It superimposes computer-generated virtual objects, scenes or system prompt information on the real scene to enhance or modify the real world Perception of the environment or data representing the real-world environment.
在虚拟现实系统及增强现实系统等交互系统中，需要对视觉交互装置进行识别跟踪。现有的识别跟踪方法，通常是采用磁传感器、光传感器、超声波、惯性传感器、视觉交互装置图像处理等方式实现，但是这些识别跟踪的方法，通常，识别跟踪效果并不理想，如磁传感器、光传感器、超声波等通常受到环境较大的影响，惯性传感器对精度要求极高，市场上急需一种全新的识别方法，以实现低成本、高精度的交互，而对视觉交互装置的图像的处理，如通过对视觉交互装置的图像处理以识别视觉交互装置的身份信息，作为识别跟踪的重要技术也需要一套完美有效的解决方法。In interactive systems such as virtual reality systems and augmented reality systems, visual interaction devices need to be identified and tracked. The existing identification and tracking methods are usually realized by means of magnetic sensors, optical sensors, ultrasonic waves, inertial sensors, and image processing of visual interaction devices. Optical sensors, ultrasonic waves, etc. are usually greatly affected by the environment. Inertial sensors have extremely high requirements for accuracy. A new identification method is urgently needed in the market to achieve low-cost and high-precision interaction. , such as identifying the identity information of the visual interaction device through image processing of the visual interaction device, as an important technology for identification and tracking, a set of perfect and effective solutions is also required.
发明内容SUMMARY OF THE INVENTION
有鉴于此，本发明实施例提供了一种标记物的识别方法、装置及识别跟踪系统，识别标记物的身份信息以通过标记物进行跟踪定位，以改善上述问题。In view of this, embodiments of the present invention provide a marker identification method, device, and identification tracking system, which identify the identity information of the marker to track and locate the marker, so as to improve the above problems.
为了实现上述目的，本发明采用的技术方案如下：In order to achieve the above object, the technical scheme adopted in the present invention is as follows:
一种标记物的识别方法，应用于一识别跟踪系统，所述系统包括：一个或多个标记物，每个标记物包括多个相互分离的子标记物，所述系统预先存储有标记物的特征和身份信息；所述方法包括：获取包括标记物的目标图像；对所述目标图像进行处理,并获取所述目标图像中多个连通域之间的包围关系；根据所述目标图像中多个连通域之间的包围关系，以及预存储的标记物的特征，确定所述目标图像中标记物的身份信息为对应的预存储标记物的身份信息。A marker identification method, applied to an identification tracking system, the system includes: one or more markers, each marker includes a plurality of mutually separated sub-markers, the system pre-stores the marker's feature and identity information; the method includes: acquiring a target image including markers; processing the target image, and acquiring the enclosing relationship between multiple connected domains in the target image; The enclosing relationship between the connected domains and the features of the pre-stored markers determine the identity information of the markers in the target image as the identity information of the corresponding pre-stored markers.
一种标记物的识别装置，应用于一识别跟踪系统，所述系统包括一个或多个标记物，每个标记物包括多个相互分离的子标记物，所述系统预先存储有标记物的特征和身份信息；其中，所述识别装置包括：图像获取模块，用于获取包括标记物的目标图像；图像处理模块，用于对所述目标图像进行处理,并获取所述目标图像中多个连通域之间的包围关系；身份确定模块，根据所述目标图像中多个连通域之间的包围关系，以及预存储的标记物的特征，确定所述目标图像中标记物的身份信息为对应的预存储标记物的身份信息。A marker identification device, applied to an identification tracking system, the system includes one or more markers, each marker includes a plurality of sub-markers separated from each other, the system pre-stores the characteristics of the markers and identity information; wherein, the identification device includes: an image acquisition module for acquiring a target image including markers; an image processing module for processing the target image and acquiring a plurality of connected images in the target image The enclosing relationship between domains; the identity determination module determines the identity information of the marker in the target image as the corresponding Pre-stored marker identification information.
一种识别跟踪系统，包括：图像采集装置以及一个或多个标记物，每个标记物包括多个相互分离的子标记物，所述系统预先存储有标记物的特征和身份信息，所述图像采集装置与一处理器连接；所述图像采集装置用于：采集包括标记物的目标图像；所述处理器用于：获取所述图像采集装置采集的目标图像；对所述目标图像进行处理,并获取所述目标图像中多个连通域之间的包围关系；根据所述目标图像中多个连通域之间的包围关系，以及预存储的标记物的特征，确定所述目标图像中标记物的身份信息为对应的预存储标记物的身份信息。An identification and tracking system, comprising: an image acquisition device and one or more markers, each marker includes a plurality of sub-markers separated from each other, the system pre-stores the features and identity information of the markers, the image The acquisition device is connected to a processor; the image acquisition device is used for: acquiring a target image including markers; the processor is used for: acquiring the target image acquired by the image acquisition device; processing the target image, and Obtain the enclosing relationship between multiple connected domains in the target image; determine the enclosing relationship between the multiple connected domains in the target image and the pre-stored features of the marker to determine the enclosing relationship of the marker in the target image. The identity information is the identity information of the corresponding pre-stored marker.
本发明实施例提供的识别跟踪系统中的标记物的识别方法、装置及识别跟踪系统，通过目标图像中连通域之间的包围关系以及预存储的标记物的特征，确定标记物对应的连通域的身份信息，从而获得标记物的身份信息，实现对标记物的识别，以使在该识别跟踪系统中可以根据识别出的标记物进行跟踪定位。The identification method, device, and identification tracking system for identifying markers in a tracking system provided by the embodiments of the present invention determine the connected domains corresponding to the markers through the enclosing relationship between the connected domains in the target image and the features of the pre-stored markers. Therefore, the identification information of the marker can be obtained, and the identification of the marker can be realized, so that the identification and tracking system can perform tracking and positioning according to the identified marker.
为使本发明的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below, and are described in detail as follows in conjunction with the accompanying drawings.
附图说明Description of drawings
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
图1示出了本发明实施例提供的识别跟踪系统的结构示意图；1 shows a schematic structural diagram of an identification and tracking system provided by an embodiment of the present invention;
图2示出了本发明实施例提供的标记物的第一种示意图；Fig. 2 shows the first schematic diagram of the marker provided by the embodiment of the present invention;
图3示出了本发明实施例提供的标记物的第二种示意图；FIG. 3 shows a second schematic diagram of the marker provided in the embodiment of the present invention;
图4示出了本发明实施例提供的视觉交互装置的第一种示意图；FIG. 4 shows a first schematic diagram of a visual interaction device provided by an embodiment of the present invention;
图5示出了本发明实施例提供的视觉交互装置的第二种示意图；FIG. 5 shows a second schematic diagram of a visual interaction device provided by an embodiment of the present invention;
图6示出了本发明实施例提供的视觉交互装置的第三种示意图；FIG. 6 shows a third schematic diagram of a visual interaction device provided by an embodiment of the present invention;
图7示出了本发明实施例提供的视觉交互装置的第四种示意图；FIG. 7 shows a fourth schematic diagram of a visual interaction device provided by an embodiment of the present invention;
图8示出了本发明实施例提供的视觉交互装置的第五种示意图；FIG. 8 shows a fifth schematic diagram of a visual interaction device provided by an embodiment of the present invention;
图9示出了本发明实施例提供的视觉交互装置的第六种示意图；FIG. 9 shows a sixth schematic diagram of a visual interaction device provided by an embodiment of the present invention;
图10示出了本发明实施例提供的标记物的第三种示意图；Fig. 10 shows a third schematic diagram of the marker provided by the embodiment of the present invention;
图11示出了本发明实施例提供的标记物的第四种示意图；FIG. 11 shows a fourth schematic diagram of the marker provided in the embodiment of the present invention;
图12示出了本发明实施例提供的标记物的识别方法流程图；12 shows a flowchart of a method for identifying a marker provided by an embodiment of the present invention;
图13示出了本发明实施例提供的一种包围关系的树状图；FIG. 13 shows a tree diagram of an enclosing relationship provided by an embodiment of the present invention;
图14示出了本发明实施例提供的标记物的识别装置的功能模块图。FIG. 14 shows a functional block diagram of an apparatus for identifying a marker provided by an embodiment of the present invention.
具体实施方式Detailed ways
下面将结合本发明实施例中附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本发明实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本发明的实施例的详细描述并非旨在限制要求保护的本发明的范围，而是仅仅表示本发明的选定实施例。基于本发明的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. The components of the embodiments of the invention generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Thus, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present invention.
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。同时，在本发明的描述中，术语“第一”、“第二”等仅用于区分描述，而不能理解为指示或暗示相对重要性。It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.
下面将结合本发明实施例中附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本发明实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本发明的实施例的详细描述并非旨在限制要求保护的本发明的范围，而是仅仅表示本发明的选定实施例。基于本发明的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. The components of the embodiments of the invention generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Thus, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present invention.
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。同时，在本发明的描述中，术语“第一”、“第二”等仅用于区分描述，而不能理解为指示或暗示相对重要性。It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.
请参阅图1，示出了本发明实施例提供的识别跟踪系统。识别跟踪系统包括头戴显示装置100和视觉交互装置。Referring to FIG. 1 , an identification and tracking system provided by an embodiment of the present invention is shown. The identification tracking system includes a head mounted display device 100 and a visual interaction device.
该视觉交互装置包括第一背景和按照特定规则分布于第一背景的至少一个标记物。所述标记物包括第二背景以及按照特定规则分布于第二背景的若干子标记物，每个子标记物具有一个或多个特征点。其中，第一背景和第二背景有一定的区分度，例如，可以是第一背景为黑色，第二背景为白色。本实施方式中，每个标记物内的子标记物的分布规则不同，因此，每个标记物所对应的图像互不相同。The visual interaction device includes a first background and at least one marker distributed on the first background according to a specific rule. The marker includes a second background and several sub-markers distributed in the second background according to a specific rule, and each sub-marker has one or more feature points. The first background and the second background have a certain degree of distinction. For example, the first background may be black and the second background may be white. In this embodiment, the distribution rules of the sub-markers in each marker are different, so the images corresponding to each marker are different from each other.
子标记物为具有一定形状的图案，且该子标记物的颜色与标记物内的第二背景有一定的区分度，例如，第二背景为白色，而子标记物的颜色为黑色。子标记物可以是由一个或多个特征点构成，且特征点的形状不做限定，可以是圆点、圆环，也可以是三角形、其他形状。The sub-marker is a pattern with a certain shape, and the color of the sub-marker has a certain degree of distinction from the second background in the marker. For example, the second background is white, and the color of the sub-marker is black. The sub-marker may be composed of one or more feature points, and the shape of the feature point is not limited, and may be a dot, a ring, a triangle, or other shapes.
作为一种实施方式，如图2所示，标记物210内包括多个子标记物220，而每个子标记物220由一个或多个特征点221构成，图2中的每个白色圆形图案为一个特征点221。标记物210的轮廓为矩形，当然，标记物的形状也可以是其他形状，在此不做限定，图2中，矩形的白色区域以及该白色区域内的多个子标记物构成一个标记物。As an embodiment, as shown in FIG. 2 , the marker 210 includes a plurality of sub-markers 220, and each sub-marker 220 is composed of one or more feature points 221. Each white circular pattern in FIG. 2 is A feature point 221. The outline of the marker 210 is a rectangle. Of course, the shape of the marker can also be other shapes, which are not limited here. In FIG. 2 , the white area of the rectangle and a plurality of sub-markers in the white area constitute a marker.
作为另一种实施方式，如图3所示，标记物310内包括多个子标记物340，而每个子标记物340由一个或多个特征点341构成；其中，多个黑色圆点341构成一个子标记物340。具体地，图3中，每个白色圆形图案和每个黑色圆点图案均为特征点341。As another embodiment, as shown in FIG. 3 , the marker 310 includes a plurality of sub-markers 340, and each sub-marker 340 is composed of one or more feature points 341; wherein, a plurality of black dots 341 constitute a Submarker 340. Specifically, in FIG. 3 , each white circular pattern and each black circular dot pattern are feature points 341 .
具体地，视觉交互装置包括平面标记物体和多面标记结构体。该平面标记物体包括第一标记板200和第二标记板500，该多面标记结构体包括六面标记结构体400和二十六面标记结构体300，当然还可以是其他面数的标记结构体，在此不一一列举。Specifically, the visual interaction device includes a plane marking object and a multi-faceted marking structure. The plane marking object includes a first marking board 200 and a second marking board 500 , the multi-faceted marking structure includes a six-sided marking structure 400 and a twenty-six-sided marking structure 300 , and of course can also be marking structures with other faces , not listed here.
第一标记板200上设置有多个标记物，多个标记物的内容互不相同，且第一标记板200上的多个标记物设置在同一个平面上，即第一标记板200设有一个标记面，且所有的标记物均设置在第一标记板200的标记面上，第一标记板200上的特征点均在标记面上；第二标记板500上设置有一个标记物，第二标记板500上的特征点也全部是在标记面上的，第二标记板500的数量可以是多个，且每个第二标记板500的标记物的内容互不相同，多个第二标记板500可以组合使用，如在该识别跟踪系统对应的增强现实，或虚拟现实等应用领域组合使用。The first marker board 200 is provided with multiple markers, the contents of the multiple markers are different from each other, and the multiple markers on the first marker board 200 are arranged on the same plane, that is, the first marker board 200 is provided with One marking surface, and all markers are arranged on the marking surface of the first marking board 200, and the feature points on the first marking board 200 are all on the marking surface; the second marking board 500 is provided with a marking object, and the first marking board 500 is provided with a marking object. The feature points on the two marking boards 500 are also all on the marking surface. The number of the second marking boards 500 can be multiple, and the contents of the markers on each second marking board 500 are different from each other. The marker board 500 can be used in combination, for example, in the application fields such as augmented reality or virtual reality corresponding to the identification and tracking system.
多面标记结构体包括多个标记面，且其中至少两个不共面的标记面上设置有标记物，如图1中所示，该多面标记结构体包括六面标记结构体400和二十六面标记结构体300，其中，六面标记结构体400包括6个标记面，且每个标记面上均设置有标记物，且每个面上的标记物的图案互不相同。The multi-faceted marking structure includes a plurality of marking surfaces, and at least two non-coplanar marking surfaces are provided with markers, as shown in FIG. 1 , the multi-sided marking structure includes a six-sided marking structure 400 and twenty-six The surface marking structure 300, wherein the six-sided marking structure 400 includes 6 marking surfaces, and each marking surface is provided with a marker, and the patterns of the markers on each surface are different from each other.
而二十六面标记结构体300包括二十六个面，其中，包括17个标记面，且每个标记面上均设置有标记物，且每个面上的标记物的图案互不相同。当然，上述的多面标记结构体的总面数以及标记面的描述和标记物的设置，可以根据实际使用而设置，在此不做限定。The twenty-six-face marking structure 300 includes twenty-six faces, including 17 marking faces, and each marking face is provided with a marker, and the patterns of the markers on each face are different from each other. Of course, the total number of faces of the above-mentioned multi-faceted marker structure, the description of the marker faces, and the setting of markers can be set according to actual use, which are not limited herein.
需要说明的是，视觉交互装置并不限定于上述平面标记物体和多面标记结构体，视觉交互装置可以是任何具有标记物的载体，载体可以根据实际场景设置，如玩具枪、游戏枪等模型枪，在模型枪等视觉交互装置上设置相应的标记物，通过识别追踪模型枪上的标记物，能够获取模型枪的位置和旋转信息，用户通过握持该模型枪在虚拟场景中进行游戏操作，实现增强现实的效果。It should be noted that the visual interaction device is not limited to the above-mentioned plane marking objects and multi-faceted marking structures. The visual interaction device can be any carrier with markers, and the carrier can be set according to the actual scene, such as toy guns, game guns and other model guns. , Set corresponding markers on visual interaction devices such as model guns. By identifying and tracking the markers on the model gun, the position and rotation information of the model gun can be obtained. The user can play the game in the virtual scene by holding the model gun. Realize the effect of augmented reality.
头戴显示装置100包括壳体(未标识)、图像采集装置110、处理器140、显示装置120、光学组件130和照明装置150。The head-mounted display device 100 includes a housing (not identified), an image capture device 110 , a processor 140 , a display device 120 , an optical assembly 130 and an illumination device 150 .
其中，视觉里程相机160、显示装置120和图像采集装置110均与处理器电连接；在一些实施方式中，照明装置150和图像采集装置110均通过滤光板(未标识)装设并覆盖在壳体内，该滤光板可过滤环境光等干扰光线，如照明装置150发射红外光，则该滤光板可为过滤除红外光线外的光线的元件。Wherein, the visual odometry camera 160, the display device 120 and the image capture device 110 are all electrically connected to the processor; in some embodiments, the lighting device 150 and the image capture device 110 are both installed through a filter plate (not identified) and covered in the casing Inside the body, the filter plate can filter interfering light such as ambient light. For example, if the lighting device 150 emits infrared light, the filter plate can be an element that filters light other than infrared light.
图像采集装置110用于采集待拍摄物体的图像并发送至处理器。具体地，采集包含有上述标记板或多面标记结构体中至少一个的图像，并发送至处理器。作为一种实施方式，该图像采集装置110为单目的近红外成像相机。当前实施方式中，图像采集装置110为采用红外接收方式且为单目的相机，不仅成本低，无需双目相机之间的外参，而且功耗低，同等带宽下帧率更高。The image acquisition device 110 is used for acquiring an image of the object to be photographed and sending it to the processor. Specifically, an image containing at least one of the marking plate or the multi-faceted marking structure described above is acquired and sent to the processor. As an embodiment, the image acquisition device 110 is a single-purpose near-infrared imaging camera. In the current embodiment, the image acquisition device 110 adopts the infrared receiving method and is a single-purpose camera, which not only has low cost, does not require external parameters between binocular cameras, but also has low power consumption and a higher frame rate under the same bandwidth.
处理器140用于根据图像输出对应的显示内容至显示装置120，还用于对视觉交互装置进行识别跟踪的运算。The processor 140 is configured to output the corresponding display content to the display device 120 according to the image, and is further configured to perform an operation of identifying and tracking the visual interaction device.
处理器140可以包括任何适当类型的通用或专用微处理器、数字信号处理器或微控制器。处理器140可以被配置为经由例如网络从系统的各种组件接收数据和/或信号。处理器140还可处理数据和/或信号以确定系统中的一个或多个操作条件。例如，当处理器140应用于头戴显示装置时，处理器根据预先存储的图像数据生成虚拟世界的图像数据，将其发送至显示装置并通过光学组件进行显示；也可以通过有线或无线网络接收智能终端或计算机的发送的图像数据，根据所接收的图像数据生成虚拟世界的图像，通过光学组件进行显示；还可以根据图像采集装置采集的图像进行识别跟踪运算而确定在虚拟世界中对应的显示内容，发送至显示装置并通过光学组件进行显示。可以理解的是，处理器140并不限定于装设在头戴显示装置内。Processor 140 may comprise any suitable type of general or special purpose microprocessor, digital signal processor or microcontroller. The processor 140 may be configured to receive data and/or signals from various components of the system via, for example, a network. The processor 140 may also process data and/or signals to determine one or more operating conditions in the system. For example, when the processor 140 is applied to a head-mounted display device, the processor generates image data of the virtual world according to the pre-stored image data, sends it to the display device and displays it through an optical component; it can also be received through a wired or wireless network The image data sent by the intelligent terminal or computer generates an image of the virtual world according to the received image data, and displays it through the optical component; it can also perform identification and tracking calculation according to the image collected by the image acquisition device to determine the corresponding display in the virtual world. The content is sent to the display device and displayed through the optical assembly. It can be understood that the processor 140 is not limited to be installed in the head-mounted display device.
在一些实施方式中，头戴显示装置100还包括设置在壳体上的视觉里程相机160，其中，视觉里程相机160与处理器电连接，该视觉里程相机160用于采集外界真实场景的场景图像，将场景图像发送至处理器。在用户穿戴头戴显示装置100时，处理器根据该视觉里程相机160采集的场景图像，根据视觉里程技术获取用户的头部与真实场景的位置及旋转关系，具体地，系统通过该相机获取的图像序列，经过特征提取、特征匹配与跟踪和运动估计的处理，得出具体的位置和方向的变化，完成导航定位，进而获得头戴显示装置与真实场景的相对位置及旋转关系；再根据视觉交互装置相对于头戴显示装置的位置及旋转信息，可以推算出视觉交互装置与真实场景之间的相对位置及旋转关系，从而可以实现更复杂的交互形式与体验。In some embodiments, the head-mounted display device 100 further includes a visual odometry camera 160 disposed on the housing, wherein the visual odometry camera 160 is electrically connected to the processor, and the visual odometry camera 160 is used to collect scene images of real scenes in the outside world , which sends the scene image to the processor. When the user wears the head-mounted display device 100, the processor obtains the position and rotation relationship between the user's head and the real scene according to the scene image collected by the visual odometry camera 160 and the visual odometry technology. The image sequence is processed by feature extraction, feature matching and tracking and motion estimation to obtain the specific position and direction changes, complete the navigation and positioning, and then obtain the relative position and rotation relationship between the head-mounted display device and the real scene; The position and rotation information of the interaction device relative to the head-mounted display device can calculate the relative position and rotation relationship between the visual interaction device and the real scene, so that more complex interaction forms and experiences can be realized.
显示装置120用于将显示内容显示。在一些实施例中，显示装置可以是智能终端的一部分，即智能终端的显示屏，例如手机和平板电脑的显示屏。在另一些实施例中，显示装置还可以是独立的显示器(例如，LED，OLED或LCD)等，则此时显示装置固定安装在壳体上。The display device 120 is used to display the display content. In some embodiments, the display device may be a part of a smart terminal, that is, a display screen of a smart terminal, such as a display screen of a mobile phone and a tablet computer. In other embodiments, the display device may also be an independent display (eg, LED, OLED, or LCD), etc., and then the display device is fixedly mounted on the casing at this time.
需要说明的是，当显示装置120为智能终端的显示屏时，壳体上设置有用于安装该智能终端的安装结构。在使用时，将智能终端通过安装结构安装在壳体上。则处理器140可以是智能终端内的处理器，也可以是独立设置在壳体内的处理器，并与智能终端通过数据线或通信接口电连接。另外，当显示装置120为与智能终端等终端设备分离的显示装置时，固定安装在壳体上。It should be noted that, when the display device 120 is a display screen of an intelligent terminal, an installation structure for installing the intelligent terminal is provided on the casing. When in use, the intelligent terminal is installed on the casing through the installation structure. Then the processor 140 may be a processor in the smart terminal, or may be a processor independently provided in the casing, and is electrically connected to the smart terminal through a data line or a communication interface. In addition, when the display device 120 is a display device separated from a terminal device such as a smart terminal, it is fixedly mounted on the casing.
光学组件130用于将显示装置120的出光面发出的入射光射向预设位置处。其中，预设位置处为用户双眼的观察位置。The optical component 130 is used for emitting the incident light emitted from the light emitting surface of the display device 120 to a preset position. The preset position is the observation position of the user's eyes.
照明装置150用于为图像采集装置110采集待拍摄物体的图像时提供光线。具体地，照明装置150的照明角度以及照明装置150的数量，可以根据实际使用而设定，以使所发射的照明光线能够覆盖待拍摄物体。其中，照明装置150采用红外光照明装置，能够发出红外光线，此时图像采集装置为近红外相机，可以接收红外光线。通过主动照明的方式，提高图像采集装置110采集的目标图像的图像质量，具体地，照明装置150的数量不限，可以是一个，也可以是多个。在一些实施方式中，照明装置150设置在图像采集装置110的附近，其中，可以是多个照明装置150周向设置在图像采集装置110的摄像头的附近。The lighting device 150 is used to provide light for the image capturing device 110 to capture the image of the object to be captured. Specifically, the lighting angle of the lighting device 150 and the number of the lighting devices 150 can be set according to actual use, so that the emitted lighting light can cover the object to be photographed. The illuminating device 150 adopts an infrared light illuminating device capable of emitting infrared light. At this time, the image acquisition device is a near-infrared camera and can receive infrared light. The image quality of the target image captured by the image capturing device 110 is improved by means of active illumination. Specifically, the number of illumination devices 150 is not limited, and may be one or multiple. In some embodiments, the lighting device 150 is arranged near the image capturing device 110 , wherein a plurality of lighting devices 150 may be arranged circumferentially near the camera of the image capturing device 110 .
用户在佩戴头戴显示装置100，进入预设的虚拟场景时，当该视觉交互装置在图像采集装置110的视野范围内时，图像采集装置110采集到包含有该视觉交互装置的目标图像；处理器140获取到该目标图像及相关信息，运算识别出该视觉交互装置并获取到该目标图像内的标记物与图像采集装置之间的位置与旋转关系，进而得到视觉交互装置相对于头戴显示装置的位置及旋转关系，使得用户观看到的虚拟场景在相应的位置及旋转角度上；用户还可以通过多个视觉交互装置的结合以在虚拟场景内进一步产生的新的虚拟图像，给用户带来更好的体验效果；用户还可以通过视觉交互装置实现与虚拟场景的交互；此外，该识别追踪系统还可以通过视觉里程相机获取头戴显示装置与真实场景的位置与旋转关系，进而可以获取视觉交互装置与真实场景的位置和旋转关系，当视觉交互装置对应的虚拟场景和真实场景有一定的对应关系时，可以构建出一个与真实场景类似的虚拟场景，可以提高更真实的增强现实体验。When the user wears the head-mounted display device 100 and enters a preset virtual scene, when the visual interaction device is within the field of view of the image capture device 110, the image capture device 110 captures a target image containing the visual interaction device; processing The device 140 obtains the target image and related information, recognizes the visual interaction device by operation, and obtains the position and rotation relationship between the marker in the target image and the image acquisition device, and then obtains the visual interaction device relative to the head-mounted display. The position and rotation relationship of the device make the virtual scene viewed by the user at the corresponding position and rotation angle; the user can also generate a new virtual image in the virtual scene through the combination of multiple visual interaction devices to bring to the user. The user can also interact with the virtual scene through the visual interaction device; in addition, the identification and tracking system can also obtain the position and rotation relationship between the head-mounted display device and the real scene through the visual mileage camera, and then obtain the The position and rotation relationship between the visual interaction device and the real scene. When the virtual scene corresponding to the visual interaction device has a certain corresponding relationship with the real scene, a virtual scene similar to the real scene can be constructed, which can improve a more realistic augmented reality experience. .
在该识别跟踪系统中，视觉交互装置为用于实现用户与虚拟世界交互的物体，处理器通过对图像采集装置采集的图像进行处理，获得其中视觉交互装置中标记物的身份信息及视觉交互装置的位置信息，可根据标记物的身份信息确定对应标记物的显示内容，根据视觉交互装置的位置信息确定显示内容呈现给用户的显示位置。当该识别跟踪系统应用于增强现实技术领域或者虚拟现实技术领域，可实现通过视觉交互装置定位虚拟场景的显示，并且可以实现与虚拟场景的交互。因此，本发明实施例提供了一种标记物的识别方法，该识别方法可用于如图1所示的识别跟踪系统，用于对视觉交互装置进行身份识别，获取视觉交互装置的身份信息。In the identification and tracking system, the visual interaction device is an object used to realize the interaction between the user and the virtual world, and the processor obtains the identity information of the marker in the visual interaction device and the visual interaction device by processing the image collected by the image acquisition device. According to the location information of the marker, the display content of the corresponding marker can be determined according to the identity information of the marker, and the display position of the display content to be presented to the user can be determined according to the location information of the visual interaction device. When the identification and tracking system is applied to the field of augmented reality technology or the field of virtual reality technology, the display of positioning the virtual scene through the visual interaction device can be realized, and the interaction with the virtual scene can be realized. Therefore, an embodiment of the present invention provides a marker identification method, which can be used in the identification and tracking system as shown in FIG. 1 to identify the visual interaction device and obtain the identity information of the visual interaction device.
其中，本发明实施例中识别方法所识别的视觉交互装置，可以是任何具有标记物的载体，如本实施方式中介绍的标记物在一个平面上的平面标记物体，也可以是标记物在多个平面上的多面标记结构体。Wherein, the visual interaction device identified by the identification method in the embodiment of the present invention may be any carrier with markers, such as a plane marker object with markers on one plane introduced in this embodiment, or a marker on multiple planes. A faceted marker structure on a plane.
如图4所示，视觉交互装置为标记物在一个平面上的平面标记物体。视觉交互装置200包括目标物本体201，以及设置于所述目标物本体201表面的一个或多个标记物210。如图5及图6所示，视觉交互装置为标记物在多个平面上的多面标记结构体，视觉交互装置300包括目标物本体301，以及设置于所述目标物本体301表面的一个或多个标记物310。分别是二十六面体和六面体。当然，多面标记结构体的具体面数以及各面之间的位置关系在本发明实施例中并不限定。目标物本体的具体形态并不限定，其表面还可以包括曲面、弧面或者凹凸不平的表面等，可以根据具体需要设定。As shown in FIG. 4 , the visual interaction device is a plane marking object in which the marker is on a plane. The visual interaction device 200 includes a target body 201 and one or more markers 210 disposed on the surface of the target body 201 . As shown in FIG. 5 and FIG. 6 , the visual interaction device is a multi-faceted marking structure with markers on multiple planes. The visual interaction device 300 includes a target body 301 and one or more objects disposed on the surface of the target body 301 . marker 310. They are the icosahedron and the hexahedron, respectively. Of course, the specific number of faces of the multi-faceted marking structure and the positional relationship between the faces are not limited in the embodiments of the present invention. The specific shape of the target body is not limited, and its surface may also include a curved surface, an arc surface, or an uneven surface, etc., which can be set according to specific needs.
另外，若视觉交互装置包括的标记物310为一个，则标记物310可以设置于目标物本体301的任意一个表面，如图6所示；或者是设置于不同平面的两个相邻表面的交界处，如图7所示。In addition, if there is only one marker 310 included in the visual interaction device, the marker 310 can be arranged on any surface of the target body 301, as shown in FIG. 6; or it can be arranged at the junction of two adjacent surfaces of different planes , as shown in Figure 7.
若视觉交互装置包括的标记物为多个，其设置方式可以包括多种。可以是，该多个标记物310设置于目标物本体301的同一表面，该同一表面可以是在同一个平面的同一表面，如图4所示；或者是在不同平面的同一表面，如设置于球形表面、弧形表面等，如图8所示。也可以是，该多个标记物310分别设置于目标物本体301在不同平面的不同表面，如图5所示，且在设置有标记物310的不同平面的每个表面，可以设置一个或多个标记物；又或者是，多个标记物310中至少一个标记物设置于所述目标物本体301在不同平面的两个相邻表面的交界处，如图9所示。另外，对于多面标记结构体，在不同表面均可以设置一个或多个标记物310。其中，目标物本体用于设置标记物的表面为对应的标记面。If the visual interaction device includes a plurality of markers, the setting manners may include a plurality of markers. It can be that the plurality of markers 310 are arranged on the same surface of the target body 301, and the same surface can be the same surface on the same plane, as shown in FIG. 4; or the same surface on different planes, such as being arranged on Spherical surface, curved surface, etc., as shown in Figure 8. Alternatively, the plurality of markers 310 are respectively disposed on different surfaces of the target body 301 on different planes, as shown in FIG. 5 , and on each surface of the different planes on which the markers 310 are disposed, one or more markers can be disposed. or, at least one of the plurality of markers 310 is disposed at the junction of two adjacent surfaces of the target body 301 on different planes, as shown in FIG. 9 . Additionally, for a multi-faceted marker structure, one or more markers 310 may be provided on different surfaces. Wherein, the surface of the target body used for setting the marker is the corresponding marker surface.
该视觉交互装置中的一个或多个标记物，可以突出于目标物本体的表面设置，即为设置在目标物本体表面的一个层结构。另外，也可以是在目标物本体用于设置标记物的表面设置有对应所述标记物的凹槽，设置于目标物本体的一个或多个标记物分别设置于对应表面的凹槽，也就是说，目标物本体设置的标记物，设置于该目标物本体表面的凹槽内，凹槽的深度可以等于标记物的厚度，使标记物的外表面与凹槽顶部齐平，当然，凹槽的深度在本发明实施例中并不限定。One or more markers in the visual interaction device can be arranged protruding from the surface of the target body, that is, a layer structure arranged on the surface of the target body. In addition, grooves corresponding to the markers may be provided on the surface of the target body for setting the markers, and one or more markers provided on the target body are respectively provided in the corresponding grooves on the surface, that is, Said, the marker set on the target body is arranged in the groove on the surface of the target body, the depth of the groove can be equal to the thickness of the marker, so that the outer surface of the marker is flush with the top of the groove, of course, the groove The depth is not limited in the embodiments of the present invention.
在本发明实施例中，每个标记物602包括多个相互分离的子标记物603，如图10所示，每个标记物602包括的子标记物603的具体个数并不限定，可以根据标记物602的大小范围设定，也可以根据具体识别需求确定。In this embodiment of the present invention, each marker 602 includes a plurality of sub-markers 603 that are separated from each other. As shown in FIG. 10 , the specific number of the sub-markers 603 included in each marker 602 is not limited, and can be determined according to The size range of the marker 602 can also be determined according to specific identification requirements.
另外，子标记物603与标记物602的背景604有一定区分度，进一步的，子标记物的颜色与标记物602中背景604的颜色可以色差较大，如图10所示，子标记物603为黑色，背景604为白色。目标物本体601的表面与标记物602的背景604也可以有不同，如图4所示，标记物602的背景604为白色，目标物本体601表面为黑色。其中，目标物本体601的表面为上述的第一背景，标记物602的背景为上述的第二背景。In addition, the sub-marker 603 and the background 604 of the marker 602 have a certain degree of distinction. Further, the color of the sub-marker and the color of the background 604 in the marker 602 may have a large color difference. As shown in FIG. 10, the sub-marker 603 is black and the background 604 is white. The surface of the target body 601 and the background 604 of the marker 602 may also be different. As shown in FIG. 4 , the background 604 of the marker 602 is white, and the surface of the target body 601 is black. The surface of the target body 601 is the aforementioned first background, and the background of the marker 602 is the aforementioned second background.
其中，每个子标记物603包括一个或多个特征点605，如图10所示，每个子标记物603中的各个特征点相互分离。其中，每个子标记物603所包括的特征点的数量并不限定，可以根据实际识别需求及标记物602所占区域大小确定。每个特征点605的形状在本发明实施例中并不限定，可以是三角形、四边形等多边形，也可以是圆形。Wherein, each sub-marker 603 includes one or more feature points 605. As shown in FIG. 10, each feature point in each sub-marker 603 is separated from each other. The number of feature points included in each sub-marker 603 is not limited, and may be determined according to actual identification requirements and the size of the area occupied by the marker 602 . The shape of each feature point 605 is not limited in this embodiment of the present invention, and may be a polygon such as a triangle or a quadrilateral, or may be a circle.
在本发明实施例中，子标记物603可以是空心图形，包括一个或多个空心部分，其中，每个空心部分作为一个特征点，如图10中的包括白色圆点的黑色子标记物所示。作为一种具体的实施方式，子标记物可以是多个相互连通的圆环构成，每个圆环中的空心部分作为该子标记物中的特征点。In the embodiment of the present invention, the sub-marker 603 may be a hollow figure, including one or more hollow parts, wherein each hollow part serves as a feature point, as shown by the black sub-marker including white dots in FIG. 10 . Show. As a specific implementation manner, the sub-marker may be composed of a plurality of interconnected rings, and the hollow part in each ring serves as a feature point in the sub-marker.
进一步的，在子标记物的任意一个空心部分，还可以设置实心图形，以该实心图形作为该子标记物中该空心部分对应的特征点，如图10中子标记物a所示。Further, in any hollow part of the sub-marker, a solid figure can also be set, and the solid figure is used as the feature point corresponding to the hollow part in the sub-marker, as shown in sub-marker a in FIG. 10 .
进一步的，在子标记物的空心部分，设置的还可以是空心图形，如圆环，以空心部分的空心图形的空心部分，作为子标记物中对应的一个特征点。在本发明实施例中，可以以此类推，在子标记物中设置层层嵌套的空心图形，如层层嵌套的圆环，以最后被嵌套的空心部分作为特征点。其中，子标记物中空心图形的嵌套层数可以根据实际识别需求设定，或根据图像采集装置的分辨率确定，在本发明实施例中并不限定。Further, in the hollow part of the sub-marker, a hollow figure, such as a ring, can also be set, and the hollow part of the hollow figure of the hollow part is used as a corresponding feature point in the sub-marker. In this embodiment of the present invention, it can be deduced by analogy. Layer-by-layer nested hollow graphics, such as layers of nested rings, can be set in the sub-marker, and the last nested hollow part is used as a feature point. The number of nesting layers of the hollow graphics in the sub-marker may be set according to actual identification requirements, or determined according to the resolution of the image acquisition device, which is not limited in this embodiment of the present invention.
在本发明实施例中，在标记物的子标记物中，可以是，至少一个子标记物由相互分离的实心图形构成，每个实心图形为一个特征点。如图10中相互分离的各个黑色实心圆，可以构成一个子标记物，各个实心圆为该子标记物中的特征点。In the embodiment of the present invention, among the sub-markers of the marker, at least one sub-marker may be composed of solid figures separated from each other, and each solid figure is a feature point. The black solid circles separated from each other in Fig. 10 can constitute a sub-marker, and each solid circle is a feature point in the sub-marker.
进一步的，在本发明实施例中，为便于对各个标记物602进行区分识别，识别跟踪系统中各个标记物602彼此不同。其中，各个标记物602的彼此不同，可以是，该多个标记物602包括的子标记物603的数量不同，例如，识别跟踪系统中包括3个标记物602，各个标记物602的子标记物603的数量分别为x、y、z，x、y、z彼此不相等。Further, in this embodiment of the present invention, in order to facilitate distinguishing and identifying each marker 602, each marker 602 in the identification tracking system is different from each other. Wherein, the markers 602 are different from each other, and the number of sub-markers 603 included in the plurality of markers 602 may be different. For example, the identification and tracking system includes three markers 602, and the sub-markers of each marker 602 The numbers of 603 are respectively x, y, z, and x, y, z are not equal to each other.
或者是，标记物602中至少一个子标记物603的特征点605的数量与其他标记物602中子标记物603的特征点605数量不同，例如，某标记物602中有一个子标记物603的特征点605为3个，而其他任意一个标记物602中，都不包括标特征点605为3个的子标记物603。Alternatively, the number of feature points 605 of at least one sub-marker 603 in the marker 602 is different from the number of feature points 605 of the sub-markers 603 in other markers 602, for example, there is one sub-marker 603 in a marker 602. There are three feature points 605 , and any other marker 602 does not include the sub-marker 603 marked with three feature points 605 .
或者是，标记物602中至少一个子标记物603的特征点605的形状与其他标记物602中子标记物603的特征点605的形状不同，如某标记物602中有一个子标记物603包括的特征点为实心圆，其他任意一个标记物602中都不包括特征点形状为实心圆的子标记物603。Alternatively, the shape of the feature point 605 of at least one sub-marker 603 in the marker 602 is different from the shape of the feature point 605 of the sub-marker 603 in other markers 602, for example, one of the sub-markers 603 in a marker 602 includes The feature points of are solid circles, and none of the other markers 602 include sub-markers 603 whose feature points are in the shape of solid circles.
或者是，作为一种实施方式，至少一个子标记物中空心图形的嵌套层数与其他子标记物的嵌套层数不同，因此，当识别到具有该嵌套层数的空心图形对应的特征点，可以确定该子标记物对应的标记物的身份。例如，只有一个子标记物的空心部分设置了一个实心圆点作为特征点，则当识别到在空心部分设置有一个实心圆点子标记物时，可以确定该子标记物对应的标记物为预设标记物模型中设置有一个实心圆点子标记物对应的标记物。Or, as an embodiment, the number of nesting levels of hollow figures in at least one sub-marker is different from the number of nesting levels of other sub-markers. The feature point can determine the identity of the marker corresponding to the sub-marker. For example, only a hollow part of a sub-marker is set with a solid dot as a feature point, then when it is recognized that a sub-marker with a solid dot is set in the hollow part, it can be determined that the marker corresponding to the sub-marker is a predetermined sub-marker. It is assumed that a marker corresponding to a solid dot sub-marker is set in the marker model.
另外，标记物602的不同，也可以是多个标记物602对应的数量组合不同。其中，每个标记物602中各个子标记物603的特征点604的数量构成该标记物602中的数量组合。以图10为例，该标记物602中包括四个子标记物603，一个子标记物的特征点数量为3，一个子标记物603的特征点数量为2，一个子标记物的特征点数量为5，一个子标记物的特征点数量为1，该四个子标记物的特征点的数量形成了该标记物602中的数量组合。In addition, the difference of the markers 602 may also be due to the different combinations of the numbers corresponding to the plurality of markers 602 . The number of the feature points 604 of each sub-marker 603 in each marker 602 constitutes a combination of the numbers in the marker 602 . Taking FIG. 10 as an example, the marker 602 includes four sub-markers 603, the number of feature points of one sub-marker is 3, the number of feature points of one sub-marker 603 is 2, and the number of feature points of one sub-marker is 5. The number of feature points of one sub-marker is 1, and the number of feature points of the four sub-markers forms the number combination in the marker 602 .
当然，在本发明实施例中，对标记物602进行区分的方式并不限定，也可以是上述各种方式的任意组合。例如，识别跟踪系统中，一个标记物包括的子标记物的数量与其他标记物包括的子标记物数量不同，当识别到具有该数量子标记物的标记物时，可以确定该标记物的身份；另一个标记物中子标记物603的特征点605的数量与其他标记物602中子标记物603的特征点605数量不同，当识别到具有该数量的特征点的子标记物时，可以确定该子标记物对应的标记物的身份。Of course, in the embodiment of the present invention, the manner of distinguishing the marker 602 is not limited, and it may be any combination of the above-mentioned various manners. For example, in the identification tracking system, the number of sub-markers included in one marker is different from that of other markers. When a marker with this number of sub-markers is identified, the identity of the marker can be determined. ; The number of feature points 605 of the sub-marker 603 in another marker is different from the number of feature points 605 of the sub-marker 603 in the other marker 602. When a sub-marker with this number of feature points is identified, it can be determined The identity of the marker to which this sub-marker corresponds.
请参见图12，本发明实施例中对上述视觉交互装置进行识别的识别方法可以包括：Referring to FIG. 12, an identification method for identifying the above-mentioned visual interaction device in an embodiment of the present invention may include:
步骤S110：获取包括标记物的目标图像。Step S110: Acquire a target image including the marker.
获取的目标图像可以是由图像采集装置获得，所述目标图像包括标记物210，如图2所示，还包括标记物之间的部分，即部分第一背景。The acquired target image may be obtained by an image acquisition device, and the target image includes markers 210, as shown in FIG. 2, and also includes a portion between the markers, that is, a portion of the first background.
步骤S120：对所述目标图像进行预处理。Step S120: Preprocess the target image.
对目标图像进行处理，以能够从目标图像中分辨出第一背景、第二背景以及子标记物、特征点各自对应的连通域。作为一种具体的实施方式，可以是将所述目标图像处理为二值化图像，其中，二值化阈值可以根据标记物的明暗特性灵活设置,或者采用自适应的阈值二值化方法。或者是，在连续多帧图像中，对于除首帧以外的每一帧，以历史帧图像处理后获得阈值图像，作为当前帧图像二值化时的二值化阈值。作为一种具体的实施方式，目标图像进行二值化后，将目标图像中标记物之间的部分以及子标记物处理为第一颜色，所述标记物中除子标记物以外的部分为第二颜色。The target image is processed so that the first background, the second background, and the connected domains corresponding to the sub-markers and feature points can be distinguished from the target image. As a specific implementation manner, the target image may be processed into a binarized image, wherein the binarization threshold may be flexibly set according to the light and dark characteristics of the marker, or an adaptive threshold binarization method may be adopted. Alternatively, in the continuous multi-frame image, for each frame except the first frame, a threshold image is obtained after processing the historical frame image, which is used as the binarization threshold when the current frame image is binarized. As a specific embodiment, after the target image is binarized, the part between the markers and the sub-markers in the target image is processed into the first color, and the part of the marker except the sub-markers is the first color. Two colors.
也就是说，将标记物中依次呈包围关系的各个部分，处理成具有颜色层次，使各部分之间形成依次包围的连通域。以图10所示为例，将目标图像中第一背景对应的部分处理为第一颜色，将标记物602中的第二背景604处理为第二颜色，将子标记物603处理为第一颜色，将子标记物围成的空心部分处理为第二颜色。若子标记物的空心部分中还包括实心图形，如图11中子标记物a所示，将该实心图形处理为第二颜色。其中，第一颜色与第二颜色可以是像素值差别较大的颜色，如第一颜色为黑色，第二颜色为白色。当然，二值化后的图像，第一背景、第二背景以及子标记物、特征点之间的区分也可以通过对比度等其他方式，本发明实施例主要以颜色层次为例进行说明。That is to say, each part of the marker that is in a surrounding relationship in sequence is processed to have a color hierarchy, so that each part forms a connected domain enclosed in sequence. Taking the example shown in FIG. 10 , the part corresponding to the first background in the target image is processed as the first color, the second background 604 in the marker 602 is processed as the second color, and the sub-marker 603 is processed as the first color , the hollow part enclosed by the sub-markers is treated as the second color. If the hollow part of the sub-marker also includes a solid figure, as shown in the sub-marker a in FIG. 11 , the solid figure is treated as the second color. Wherein, the first color and the second color may be colors with a large difference in pixel value, for example, the first color is black and the second color is white. Of course, in the binarized image, the first background, the second background, the sub-markers, and the feature points can also be distinguished by other methods such as contrast, and the embodiment of the present invention mainly uses color gradation as an example for description.
步骤S130：获取连通域信息，基于该连通域信息获取所有连通域的包围关系。Step S130: Obtain connected domain information, and obtain the enclosing relationships of all connected domains based on the connected domain information.
获取目标图像中的连通域信息，可以用4路或8路连通性计算标记为布尔图像的连通分量，输出连通域的数量，其中，可以根据包围关系输出各个连通域的类型，即输出对应目标图像第一背景、第二背景、子标记物、特征点等各个部分的连通域。To obtain the connected domain information in the target image, 4-way or 8-way connectivity can be used to calculate the connected components marked as Boolean images, and the number of connected domains can be output. Among them, the type of each connected domain can be output according to the enclosing relationship, that is, the corresponding target can be output. The connected domain of each part of the image, such as the first background, the second background, the sub-markers, and the feature points.
如图2所示的目标图像中，第一背景为一个连通域，标记物中第二背景为一个连通域，每一不包含黑点的子标记物是一个连通域信息，子标记物中的白点是一个连通域信息，包括黑点的子标记物中，每一个黑点是一个连通域信息。其中，不包含黑点的子标记物为空心图形的子标记物，其中的白点为特征点，包括黑点的子标记物，黑点为特征点。In the target image shown in Figure 2, the first background is a connected domain, the second background in the marker is a connected domain, each sub-marker that does not contain black dots is a connected domain information, and the The white dot is a piece of connected domain information, and each black dot is a connected domain information in the sub-markers including the black dots. Among them, the sub-markers that do not contain black dots are sub-markers of hollow figures, and the white dots are feature points, and the sub-markers that include black dots are feature points.
再基于目标图像中的连通域获取各个连通域之间的包围关系。以图2中包含有3个白点的子标记物为例，该子标记物为一个连通域信息，该连通域内包括三个白点信息，该三个白点分别为一个连通域，各个白点对应的连通域被子标记物对应的连通域包围。Then, the enclosing relationship between each connected domain is obtained based on the connected domain in the target image. Taking the sub-marker containing three white points in Figure 2 as an example, the sub-marker is a connected domain information, and the connected domain includes three white point information, and the three white points are respectively a connected domain, and each white point is a connected domain. The connected domain corresponding to the point is surrounded by the connected domain corresponding to the sub-marker.
具体的，如图2所示，在目标图像中，第一背景、第二背景以及子标记物之间形成了包围关系，若子标记物为空心图形，子标记物与空心部分对应还具有包围关系，如图2中包括白点的子标记物，与白点之间形成包围关系。Specifically, as shown in FIG. 2 , in the target image, a surrounding relationship is formed between the first background, the second background and the sub-marker. If the sub-marker is a hollow figure, the sub-marker corresponds to the hollow part and also has a surrounding relationship. , as shown in Figure 2, the sub-markers including the white dots form a surrounding relationship with the white dots.
其中，第一背景包围第二背景，第二背景包围子标记物，子标记物还包围其中的白点，即空心部分。也就是说，第一背景、第二背景以及子标记物分别对应的连通域具有包围关系，子标记物对应的连通域与其空心部分对应的连通域之间也有包围关系，其包围关系可以通过树状图表示，如图10中对应的各个连通域的包围关系可以由如图13所示的树状图表示。其中，连通域是指图像中具有相同像素值且位置相邻的像素点组成的图像区域。Wherein, the first background surrounds the second background, the second background surrounds the sub-markers, and the sub-markers also surround the white dots, that is, the hollow parts. That is to say, the connected domains corresponding to the first background, the second background, and the sub-markers respectively have an enclosing relationship, and the connected domains corresponding to the sub-markers and the connected domains corresponding to their hollow parts also have an enclosing relationship, and the enclosing relationship can be determined by the tree The enclosing relationship of each connected domain corresponding to FIG. 10 can be represented by a tree diagram as shown in FIG. 13 . The connected domain refers to the image area composed of adjacent pixels with the same pixel value in the image.
具体的，可以将第一背景定义为第四连通域，也就是说，确定第四连通域，在目标图像中，第一背景包围了所有标记物，因此，可以将目标图像中包围其他所有连通域的连通域作为第四连通域。若以二值化后的目标图像包括第一颜色和第二颜色为例，其中确定的第四连通域满足如下条件：颜色为第一颜色、包围有第二颜色的连通域并且，未被第二颜色的连通域包围。Specifically, the first background can be defined as the fourth connected domain, that is, the fourth connected domain is determined. In the target image, the first background surrounds all markers, so all other connected objects can be surrounded by the target image. The connected domain of the domain is used as the fourth connected domain. If the binarized target image includes the first color and the second color as an example, the determined fourth connected domain satisfies the following conditions: the color is a connected domain of the first color, surrounded by the second color, and is not connected by the first color. The two-color connected domain is surrounded.
另外，第一背景包围标记物，则第四连通域包围标记物中第二背景对应的连通域，定义该第二背景对应的连通域为第一连通域。也就是说，可以将第四连通域包围、且与第四连通域相邻的连通域作为第一连通域，第四连通域包围的每个第一连通域对应一个标记物，标记物中包围其他连通域的连通域为第一连通域。In addition, if the first background surrounds the marker, the fourth connected domain surrounds the connected domain corresponding to the second background in the marker, and the connected domain corresponding to the second background is defined as the first connected domain. That is to say, a connected domain surrounded by the fourth connected domain and adjacent to the fourth connected domain can be used as the first connected domain, each first connected domain surrounded by the fourth connected domain corresponds to a marker, and the marker surrounded by The connected domain of other connected domains is the first connected domain.
若以二值化后的目标图像包括第一颜色和第二颜色为例，确定被第四连通域包围且颜色为第二颜色的连通域为第一连通域。If the binarized target image includes the first color and the second color as an example, it is determined that the connected domain surrounded by the fourth connected domain and the color is the second color is the first connected domain.
另外，由于标记物中包括各个子标记物，每个子标记物具有特征点，则可以确定第一连通域包围的连通域为第二连通域，即定义子标记物对应的连通域为第二连通域。确定第二连通域包围的连通域为第三连通域，即若子标记物为如图10中所示的包围白点的空心图形，空心部分(即包围的白色部分，也就是白色特征点)对应的连通域定义为第三连通域，每个第三连通域是一个特征点。若第二连通域不包围第三连通域，则确定每个不包围第三连通域的第二连通域为一个特征点。In addition, since the marker includes various sub-markers, and each sub-marker has a feature point, it can be determined that the connected domain surrounded by the first connected domain is the second connected domain, that is, the connected domain corresponding to the sub-marker is defined as the second connected domain area. It is determined that the connected domain enclosed by the second connected domain is the third connected domain, that is, if the sub-marker is a hollow figure surrounded by white points as shown in Figure 10, the hollow part (that is, the enclosed white part, that is, the white feature point) corresponds to The connected domain of is defined as the third connected domain, and each third connected domain is a feature point. If the second connected domain does not enclose the third connected domain, each second connected domain that does not enclose the third connected domain is determined as a feature point.
也就是说，在目标图像中，可以获取到如图13所示的包围关系树状图，各个第一连通域被第四连通域包围，各个第二连通域被相应的第一连通域包围，各个第三连通域被相应的第二连通域包围。因此，如图13的包围关系树状图所示，可以获取到每个第一连通域包围的第二连通域，以及每个第一连通域中包围的第二连通域的数量；每个第二连通域包围的第三连通域，以及每个第二连通域包围的第三连通域的数量。That is to say, in the target image, the enclosing relationship tree diagram as shown in Figure 13 can be obtained, each first connected domain is surrounded by the fourth connected domain, and each second connected domain is surrounded by the corresponding first connected domain, Each third connected domain is surrounded by a corresponding second connected domain. Therefore, as shown in the dendrogram of the enclosing relationship in FIG. 13 , the second connected domains enclosed by each first connected domain and the number of second connected domains enclosed in each first connected domain can be obtained; Third connected domains enclosed by two connected domains, and the number of third connected domains enclosed by each second connected domain.
步骤S140：根据目标图像子标记物的包围关系，以及预存储的标记物的特征，判断目标图像中的子标记物是否包含预存储标记物的包含样式。Step S140: According to the enclosing relationship of the sub-markers in the target image and the characteristics of the pre-stored markers, determine whether the sub-markers in the target image contain the inclusion pattern of the pre-stored markers.
根据目标图像中连通域的包围关系可以区分出各个标记物，其中，每个第一连通域对应一个标记物，或者说，每个第一连通域及其包围的第二连通域、第三连通域构成目标图像中的一个标记物。Each marker can be distinguished according to the enclosing relationship of the connected domains in the target image, wherein each first connected domain corresponds to a marker, or in other words, each first connected domain and its enclosed second connected domain and third connected domain The domain constitutes a marker in the target image.
系统中预先存储有标记物的特征和身份信息，再根据预存储的标记物的特征，可以确定所述目标图像中标记物的身份信息为对应的预存储标记物的身份信息。其中，预存储标记物的身份信息与标记物的特征可以对应存储。The feature and identity information of the marker are pre-stored in the system, and then according to the pre-stored feature of the marker, the identity information of the marker in the target image can be determined as the identity information of the corresponding pre-stored marker. Wherein, the identity information of the pre-stored marker and the feature of the marker may be stored correspondingly.
其中，预存储的标记物的特征包括标记物中对应的连通域，该连通域分别包括第一连通域、第二连通域以及第三连通域，其中，预存储特征信息还包括连通域之间的包围关系：第一连通域包围的第二连通域以及包围的各第二连通域的数量，每个第二连通域包围的第三连通域以及包围的第三连通域的数量。The features of the pre-stored markers include the corresponding connected domains in the markers, and the connected domains respectively include a first connected domain, a second connected domain, and a third connected domain, and the pre-stored feature information also includes the connected domains between the connected domains. The enclosing relationship of : the second connected domain enclosed by the first connected domain and the number of each enclosed second connected domain, the third connected domain enclosed by each second connected domain and the number of enclosed third connected domains.
在第一种具体的实施方式中，若多个预存储标记物对应的各个子标记物中特征点形成的数量组合不同，则在目标图像中的标记物，与数理组合相同的预存储标记物对应。具体的，在对标记物进行身份识别可以是：对于目标图像中每个第一连通域，在预先存储标记物的特征信息中确定对应的第一连通域，其中，相互对应的第一连通域包围有相同数量的第二连通域、且包围的各个第二连通域所包围的第三连通域的数量一一对应。例如，以图10中标记物为例，在目标图像中的该标记物的第二背景对应的第一连通域，包括8个第二连通域，其中5个第二连通域不包括有第三连通域，该5个第二连通域对应5个特征点，构成一个子标记物；其中3个第二连通域包括有第三连通域，该3个第二连通域各对应一个子标记物，分别包围1个第三连通域、3个第三连通域、两个第三连通域，即是分别具有1个特征点，3个特征点，2个特征点，且各个特征点为白点。则在预存储的预存储标记物的包围关系中，查找标记物，其包括4个子标记物，该4个子标记物的特征点分别为1个白点、3个白点、2个白点、5个黑点。In the first specific embodiment, if the number of feature points formed in each sub-marker corresponding to the plurality of pre-stored markers is different, the markers in the target image are the same as the mathematical combination of the pre-stored markers. correspond. Specifically, the identification of the marker may be: for each first connected domain in the target image, the corresponding first connected domain is determined in the feature information of the pre-stored marker, wherein the first connected domains corresponding to each other The same number of second connected domains are enclosed, and the number of third connected domains enclosed by each of the enclosed second connected domains is in one-to-one correspondence. For example, taking the marker in FIG. 10 as an example, the first connected domain corresponding to the second background of the marker in the target image includes 8 second connected domains, of which 5 second connected domains do not include the third connected domain Connected domains, the 5 second connected domains correspond to 5 feature points and constitute a sub-marker; the 3 second connected domains include a third connected domain, and the three second connected domains each correspond to a sub-marker, Surrounding 1 third connected domain, 3 third connected domains, and two third connected domains respectively, that is, there are 1 feature point, 3 feature points, 2 feature points, and each feature point is a white point. Then, in the enclosing relationship of the pre-stored pre-stored markers, find the markers, which include 4 sub-markers, and the feature points of the 4 sub-markers are 1 white point, 3 white points, 2 white points, 5 black dots.
在第二种具体的实施方式中，可以是，预存储的多个标记物的特征中，连通域之间的包围关系通过编码表示，其中，每个第二连通域对应一个编码，第二连通域包围的第三连通域数量不同，对应的编码不同。In a second specific implementation, it may be that, in the pre-stored features of multiple markers, the enclosing relationship between connected domains is represented by codes, wherein each second connected domain corresponds to a code, and the second connected domain corresponds to a code. The number of third connected domains enclosed by the domain is different, and the corresponding codes are different.
在该实施方式中，获取处理后的所述目标图像中多个连通域之间的包围关系还可以包括：在所述目标图像中，包围不同数量第三连通域的第二连通域，设置不同的对应编码，其中第三连通域数量与编码的对应关系相同于预存储标记物的第三连通域数量与编码的对应关系。例如，若预存储标记物中，包围有一个第三连通域的第二连通域编码为B1，包围有两个第三连通域的第二连通域编码为B2，包围有两个第三连通域的第二连通域编码为B3，以此类推。则在对目标图像中的第二连通域进行编码时，包围有一个第三连通域的第二连通域编码为B1，包围有两个第三连通域的第二连通域编码为B2，包围有两个第三连通域的第二连通域编码为B3，以此类推。In this implementation manner, acquiring the enclosing relationship between the plurality of connected domains in the target image after processing may further include: in the target image, second connected domains enclosing different numbers of third connected domains, set different The corresponding coding of , wherein the corresponding relationship between the number of the third connected domain and the coding is the same as the corresponding relationship between the number of the third connected domain of the pre-stored marker and the coding. For example, if in the pre-stored marker, the second connected domain surrounded by one third connected domain is encoded as B1, the second connected domain surrounded by two third connected domains is encoded as B2, and the second connected domain surrounded by two third connected domains is encoded as B2 The second connected domain of is encoded as B3, and so on. Then, when encoding the second connected domain in the target image, the second connected domain surrounded by one third connected domain is coded as B1, the second connected domain surrounded by two third connected domains is coded as B2, and the second connected domain surrounded by two third connected domains is coded as B2. The second connected domain of the two third connected domains is coded as B3, and so on.
当然，在对连通域进行编码时，如预存储的多个标记物对应的编码中，第四连通域通过第一编码表示，第一连通域通过第二编码表示。则获取处理后的所述目标图像中多个连通域之间的包含关系时，可以通过第一编码表示第四连通域，通过第二编码表示第一连通域。其中，仍然主要通过第一连通域中各个第二连通域的编码确定第一连通域的身份信息，从而确定第一连通域对应的标记物的身份信息。Of course, when encoding the connected domain, for example, in the encoding corresponding to the pre-stored multiple markers, the fourth connected domain is represented by the first encoding, and the first connected domain is represented by the second encoding. Then, when acquiring the inclusion relationship between multiple connected domains in the processed target image, the fourth connected domain can be represented by the first code, and the first connected domain can be represented by the second code. Wherein, the identity information of the first connected domain is still mainly determined through the coding of each second connected domain in the first connected domain, so as to determine the identity information of the marker corresponding to the first connected domain.
在本实施方式中，确定所述目标图像中标记物与预存储标记物的对应则可以是，确定所述目标图像中标记物的与编码相同的预存储标记物对应。具体的，每个第一连通域包围一个或多个第二连通域，每个第一连通域对应一个标记物，则标记物对应的编码可以是包围的各个第二连通域对应的各个编码。同样的，预存储标记物对应的编码可以是预存储的该预存储标记物的各个第二连通域的编码。在预存储的预存储标记物对应的编码中，获取与标记物对应的编码相同的编码，该相同编码的预存储标记物对应的身份信息即为目标图像中标记物的身份信息。其中，编码顺序并不限定，例如，对于编码B0B1B2B3的标记物，在预存储标记物中获取与B0B1B2B3相同的编码，其中B0B1B2B3的顺序不限定，例如获取到的B1B2B0B3也认为与B0B1B2B3相同。In this embodiment, determining the correspondence between the marker in the target image and the pre-stored marker may be to determine that the marker in the target image corresponds to the pre-stored marker with the same encoding. Specifically, each first connected domain surrounds one or more second connected domains, and each first connected domain corresponds to a marker, and the code corresponding to the marker may be each code corresponding to each enclosed second connected domain. Likewise, the code corresponding to the pre-stored marker may be the pre-stored code of each second connected domain of the pre-stored marker. In the code corresponding to the pre-stored pre-stored marker, the same code as the code corresponding to the marker is obtained, and the identity information corresponding to the pre-stored marker with the same code is the identity information of the marker in the target image. Wherein, the coding order is not limited, for example, for the marker encoding B0B1B2B3, the same code as B0B1B2B3 is obtained in the pre-stored marker, wherein the order of B0B1B2B3 is not limited, for example, the obtained B1B2B0B3 is also considered to be the same as B0B1B2B3.
在本发明实施例中，标记物之间的不同，也可以是该多个标记物包括的子标记物的数量不同，例如，多个预设预存储标记物中，只有一个预存储标记物对应有第二数量的子标记物。则本发明实施例提供了第三种实施方式，在目标图像中，若某个标记物中包括第二数量的子标记物，该标记物与有第二数量的子标记物的预存储标记物对应。在识别过程中，若预存储标记物中，只有一个第一连通域包围有第二数量的第二连通域，且在目标图像中，若某个第一连通域包围第二数量的第二连通域，则该第一连通域对应的标记物与预存储标记物中包围有第二数量的第二连通域的标记物对应。In this embodiment of the present invention, the difference between the markers may also be the number of sub-markers included in the multiple markers. For example, among the multiple preset pre-stored markers, only one pre-stored marker corresponds to There is a second number of sub-markers. The embodiment of the present invention provides a third implementation. In the target image, if a marker includes a second number of sub-markers, the marker is related to the pre-stored marker with the second number of sub-markers. correspond. During the identification process, if there is only one first connected domain surrounded by a second number of second connected domains in the pre-stored markers, and in the target image, if a certain first connected domain surrounds a second number of second connected domains domain, the marker corresponding to the first connected domain corresponds to the marker surrounded by the second number of second connected domains in the pre-stored markers.
作为一种具体的实施方式，由于如图10所示，可以将每个不包括白色特征点的黑色点作为一个特征点，所有不包括白色特征点的黑色点作为一个子标记物。也就是说，每个不包围第三连通域的第二连通域作为一个特征点，所有不包围第三连通域的第二连通域作为一个子标记物，则在该识别过程中，将包围有第三连通域的每个第二连通域统计数量为1，不包围第三连通域的所有第二连通域统计数量为1。As a specific implementation manner, as shown in FIG. 10 , each black point that does not include a white feature point can be used as a feature point, and all black points that do not include a white feature point can be used as a sub-marker. That is to say, each second connected domain that does not enclose the third connected domain is taken as a feature point, and all second connected domains that do not enclose the third connected domain are taken as a sub-marker, then in the identification process, there will be surrounded by The statistical quantity of each second connected domain of the third connected domain is 1, and the statistical quantity of all second connected domains that do not enclose the third connected domain is 1.
进一步的，在本发明实施例中，目标图像中的标记物并不一定为完整的标记物，若只获取到标记物的一部分，且该标记物与其他标记物差异化较大，具有其他标记物所没有的特征，可以根据该标记物中的特征判定该标记物的身份。Further, in the embodiment of the present invention, the marker in the target image is not necessarily a complete marker. If only a part of the marker is obtained, and the marker is greatly different from other markers, it has other markers. The identity of the marker can be determined according to the features in the marker.
具体的，在本发明实施例提供的第四种具体的实施方式中，若多个预存储标记物中，某预存储标记物的至少一个子标记物的特征点的数量与其他标记物中子标记物的特征点数量不同，也就是说，所述多个预存储标记物中只有一个第一连通域包围有一个特定的第二连通域，该特定的第二连通域包围有第一数量的第三连通域。则所述目标图像中，若存在某个第一连通域包围的第二连通域包围有第一数量的第三连通域，该第一连通域对应的标记物与该特定的第二连通域所对应的预存储标记物对应。Specifically, in the fourth specific implementation provided by the embodiment of the present invention, if among multiple pre-stored markers, the number of feature points of at least one sub-marker of a pre-stored marker is the same as that of other markers in the The number of feature points of the markers is different, that is, only one first connected domain in the plurality of pre-stored markers is surrounded by a specific second connected domain, and the specific second connected domain is surrounded by a first number of The third connected domain. In the target image, if there is a second connected domain surrounded by a first connected domain and surrounded by a first number of third connected domains, the marker corresponding to the first connected domain is the same as the specific second connected domain. The corresponding pre-stored markers correspond to.
或者是，多个预存储标记物中只有一个第一连通域包围有预设数量个不包围第三连通域的第二连通域。则所述目标图像中，若存在某个第一连通域，包围有预设数量个不包围第三连通域的第二连通域，该第一连通域对应的标记物与该预设数量的第二连通域所对应的预存储标记物对应。Alternatively, only one first connected domain in the plurality of pre-stored markers is surrounded by a preset number of second connected domains that do not surround the third connected domain. Then, in the target image, if there is a certain first connected domain, there is a preset number of second connected domains that do not enclose the third connected domain, and the marker corresponding to the first connected domain is the same as the preset number of second connected domains. The pre-stored markers corresponding to the two connected domains correspond.
另外，本发明实施例还提供了第五种实施方式，若多个预存储标记物的特征中，某个预存储标记物的至少一个子标记物中空心图形的嵌套层数与其他子标记物的嵌套层数不同，则在目标图像中，若某个子标记物的图形嵌套层数与该预存储标记物中该子标记物的嵌套层数相同，该子标记物对应的标记物与该预存储标记物对应。也就是说，预存储的多个预存储标记物中，若只有一个预存储标记物包括依次包围的第三数量的连通域，则若所述目标图像中包括依次包围的第三数量的连通域，确定该第三数量的连通域对应的标记物与该预存储标记物对应。In addition, the embodiment of the present invention also provides a fifth implementation, if among the features of multiple pre-stored markers, the number of nesting levels of hollow figures in at least one sub-marker of a certain pre-stored marker is the same as that of other sub-markers. In the target image, if the number of nesting levels of a sub-marker is the same as the nesting level of the sub-marker in the pre-stored marker, the marker corresponding to the sub-marker corresponding to the pre-stored marker. That is to say, among the pre-stored multiple pre-stored markers, if only one pre-stored marker includes a third number of connected domains enclosed in sequence, then if the target image includes a third number of connected domains enclosed in sequence , it is determined that the marker corresponding to the third quantity of connected domains corresponds to the pre-stored marker.
步骤S150:确定所述目标图像中标记物的身份信息具体对应预存储的预存储标记物的身份信息，识别视觉交互装置。Step S150: Determine that the identity information of the marker in the target image specifically corresponds to the pre-stored identity information of the pre-stored marker, and identify the visual interaction device.
预存储标记物为预存储有身份信息的标记物，其身份信息对应预存储标记物的各种信息，如预存储标记物中各个特征点的物理坐标、预存储标记物所设置的目标物本体等。对于目标图像中的第一连通域，以预存储标记物的包围关系中对应的第一连通域的身份信息作为其身份信息，获得该第一连通域对应的标记物的身份信息，从而可以获得目标图像中各个标记物中特征点的物理坐标、对应的视觉交互装置等所需要的信息。The pre-stored marker is a marker with pre-stored identity information, and its identity information corresponds to various information of the pre-stored marker, such as the physical coordinates of each feature point in the pre-stored marker, and the target body set by the pre-stored marker. Wait. For the first connected domain in the target image, the identity information of the corresponding first connected domain in the enclosing relationship of the pre-stored marker is used as its identity information, and the identity information of the marker corresponding to the first connected domain is obtained, thereby obtaining Information required by the physical coordinates of the feature points in each marker in the target image, the corresponding visual interaction device, etc.
如图14所示，本实施例提供了一种标记物的识别装置600。其中，所述系统包括一个或多个标记物，每个标记物包括多个相互分离的子标记物，所述系统预先存储有标记物的特征和身份信息。具体的，所述识别装置包括：图像获取模块610，用于获取包括标记物的目标图像；图像处理模块620，用于对所述目标图像进行处理,并获取所述目标图像中多个连通域之间的包围关系；身份确定模块630，用于根据所述目标图像中多个连通域之间的包围关系，以及预存储的标记物的特征，确定所述目标图像中标记物的身份信息为对应的预存储标记物的身份信息。As shown in FIG. 14 , this embodiment provides an apparatus 600 for identifying a marker. Wherein, the system includes one or more markers, each marker includes a plurality of sub-markers separated from each other, and the system pre-stores the characteristics and identity information of the markers. Specifically, the identification device includes: an image acquisition module 610 for acquiring a target image including markers; an image processing module 620 for processing the target image and acquiring multiple connected domains in the target image The identity determination module 630 is configured to determine the identity information of the marker in the target image according to the surrounding relationship between multiple connected domains in the target image and the characteristics of the pre-stored marker as: The identity information of the corresponding pre-stored marker.
其中，图像处理模块620用于将所述目标图像处理为二值化图像，使标记物中子标记物与除子标记物以外的部分具有区分度。The image processing module 620 is configured to process the target image into a binarized image, so that the sub-marker in the marker has a degree of distinction from the part other than the sub-marker.
另外，预先存储的标记物特征信息包括：第一连通域包围的第二连通域的数量，每个第二连通域包围的第三连通域的数量。In addition, the pre-stored marker feature information includes: the number of second connected domains enclosed by the first connected domain, and the number of third connected domains enclosed by each second connected domain.
图像处理模块620可以包括，连通域确定单元，用于确定标记物中包围其他连通域的连通域为第一连通域，确定第一连通域包围的连通域为第二连通域，确定第二连通域包围的连通域为第三连通域。连通域关系获取单元，用于获取每个第一连通域中包围的第二连通域的数量以及每个第二连通域包围的第三连通域的数量。身份确定模块630可以用于对于目标图像中每个第一连通域，在预先存储的标记物的特征信息中确定对应的第一连通域，其中，相互对应的第一连通域包围有相同数量的第二连通域、且包围的各个第二连通域所包围的第三连通域的数量一一对应。The image processing module 620 may include a connected domain determination unit, configured to determine a connected domain surrounded by other connected domains in the marker as a first connected domain, determine a connected domain surrounded by the first connected domain as a second connected domain, and determine the second connected domain. The connected domain enclosed by the domain is the third connected domain. A connected domain relationship obtaining unit, configured to obtain the number of second connected domains enclosed in each first connected domain and the number of third connected domains enclosed by each second connected domain. The identity determination module 630 may be configured to, for each first connected domain in the target image, determine the corresponding first connected domain from the pre-stored feature information of the marker, wherein the first connected domain corresponding to each other is surrounded by the same number of connected domains. The number of second connected domains and the number of third connected domains enclosed by each of the enclosed second connected domains is in one-to-one correspondence.
进一步的，若所述预存储的标记物中只有一个第一连通域包围的第二连通域包围有第一数量的第三连通域，身份确定模块可以用于，若所述目标图像中存在一个第一连通域，该第一连通域包围的第二连通域包围有第一数量的第三连通域，确定该第一连通域对应的标记物的身份信息为所述第一数量的第三连通域对应的预存储标记物的身份信息。Further, if only a second connected domain surrounded by a first connected domain is surrounded by a first number of third connected domains in the pre-stored markers, the identity determination module can be used to, if there is a first connected domain in the target image, a first connected domain, the second connected domain surrounded by the first connected domain is surrounded by a first number of third connected domains, and the identity information of the marker corresponding to the first connected domain is determined to be the first number of third connected domains The identity information of the pre-stored marker corresponding to the domain.
可选的，若所述多个预存储标记物中只有一个第一连通域包围有第二数量的第二连通域，身份确定模块可以用于，若所述目标图像中存在一个第一连通域，该第一连通域包围有第二数量的第二连通域，确定该第一连通域对应的标记物的身份信息为所述第二数量的第二连通域对应的预存储标记物的身份信息。Optionally, if only one first connected domain in the plurality of pre-stored markers is surrounded by a second number of second connected domains, the identity determination module can be used to, if there is one first connected domain in the target image , the first connected domain is surrounded by a second number of second connected domains, and it is determined that the identity information of the marker corresponding to the first connected domain is the identity information of the pre-stored marker corresponding to the second number of second connected domains .
可选的，预存储的多个预存储标记物中，只有一个预存储标记物包括依次包围的第三数量的连通域，身份确定模块可以用于，若所述目标图像中包括依次包围的第三数量的连通域，确定该第三数量的连通域对应的标记物的身份信息为预存储的所述第三数量的连通域对应的标记物的身份信息。Optionally, among the pre-stored multiple pre-stored markers, only one of the pre-stored markers includes a third number of connected domains surrounded in sequence, and the identity determination module can be used to, if the target image includes the sequentially surrounded third number of connected domains. There are three connected domains, and it is determined that the identity information of the markers corresponding to the third number of connected domains is the pre-stored identity information of the markers corresponding to the third number of connected domains.
可选的，预存储的标记物特征信息中还包括第四连通域，所述第四连通域包围所述第一连通域。身份确定模块可以用于确定第四连通域包围的每个第一连通域对应一个标记物。Optionally, the pre-stored marker feature information further includes a fourth connected domain, and the fourth connected domain surrounds the first connected domain. The identity determination module may be configured to determine that each first connected domain surrounded by the fourth connected domain corresponds to a marker.
在本发明实施例中，获取目标图像中连通域的包围关系，根据连通域之间的包围关系以及预存储的标记物的特征信息及身份信息，将包围关系相同的预存储标记物的身份信息确定为目标图像中相应标记物的身份信息。其中，预存储的身份信息中，可以包括标记物中各个标记图形的像素位置、物理坐标位置等信息，从而可以进一步根据获得的标记物的身份信息获取标记物的姿态信息，对标记物实现识别跟踪。In the embodiment of the present invention, the enclosing relationship of the connected domains in the target image is obtained, and according to the enclosing relationship between the connected domains and the feature information and identity information of the pre-stored markers, the identity information of the pre-stored markers with the same enclosing relationship is It is determined as the identity information of the corresponding marker in the target image. Wherein, the pre-stored identity information may include information such as the pixel position and physical coordinate position of each marked figure in the marker, so that the gesture information of the marker can be further obtained according to the obtained identity information of the marker, and the marker can be recognized. track.
需要说明的是，本说明书中的各个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似的部分互相参见即可。对于装置类实施例而言，由于其与方法实施例基本相似，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。It should be noted that, each embodiment in this specification focuses on the differences from other embodiments, and it is sufficient to refer to each other for the same and similar parts among the various embodiments. As for the apparatus type embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant part, please refer to the partial description of the method embodiment.
以上所述仅为本发明的优选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention. It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应所述以权利要求的保护范围为准。The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention. should be included within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810119854.XACN110120060B (en) | 2018-02-06 | 2018-02-06 | Marker identification method, device and identification tracking system |
| PCT/CN2019/073578WO2019154169A1 (en) | 2018-02-06 | 2019-01-29 | Method for tracking interactive apparatus, and storage medium and electronic device |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810119854.XACN110120060B (en) | 2018-02-06 | 2018-02-06 | Marker identification method, device and identification tracking system |
| Publication Number | Publication Date |
|---|---|
| CN110120060Atrue CN110120060A (en) | 2019-08-13 |
| CN110120060B CN110120060B (en) | 2023-07-14 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810119854.XAActiveCN110120060B (en) | 2018-02-06 | 2018-02-06 | Marker identification method, device and identification tracking system |
| Country | Link |
|---|---|
| CN (1) | CN110120060B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110659587A (en)* | 2019-09-02 | 2020-01-07 | 广东虚拟现实科技有限公司 | Marker, marker identification method, marker identification device, terminal device and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6351559B1 (en)* | 1998-12-22 | 2002-02-26 | Matsushita Electric Corporation Of America | User-enclosed region extraction from scanned document images |
| US20030202709A1 (en)* | 2002-04-25 | 2003-10-30 | Simard Patrice Y. | Clustering |
| US20090196502A1 (en)* | 2008-02-04 | 2009-08-06 | Craig Sullender | Connected component labeling system and method |
| CN104484871A (en)* | 2014-11-27 | 2015-04-01 | 小米科技有限责任公司 | Edge extraction method and device |
| CN106249881A (en)* | 2016-07-21 | 2016-12-21 | 江苏奥格视特信息科技有限公司 | Augmented reality view field space and virtual three-dimensional target dynamic method for registering |
| CN106529550A (en)* | 2016-10-25 | 2017-03-22 | 凌云光技术集团有限责任公司 | Multidimensional characteristic extraction method and device based on connected domain analysis |
| CN107093171A (en)* | 2016-02-18 | 2017-08-25 | 腾讯科技（深圳）有限公司 | A kind of image processing method and device, system |
| CN107194968A (en)* | 2017-05-18 | 2017-09-22 | 腾讯科技（上海）有限公司 | Recognition and tracking method, device, intelligent terminal and the readable storage medium storing program for executing of image |
| CN107450714A (en)* | 2016-05-31 | 2017-12-08 | 大唐电信科技股份有限公司 | Man-machine interaction support test system based on augmented reality and image recognition |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6351559B1 (en)* | 1998-12-22 | 2002-02-26 | Matsushita Electric Corporation Of America | User-enclosed region extraction from scanned document images |
| US20030202709A1 (en)* | 2002-04-25 | 2003-10-30 | Simard Patrice Y. | Clustering |
| US20090196502A1 (en)* | 2008-02-04 | 2009-08-06 | Craig Sullender | Connected component labeling system and method |
| CN104484871A (en)* | 2014-11-27 | 2015-04-01 | 小米科技有限责任公司 | Edge extraction method and device |
| CN107093171A (en)* | 2016-02-18 | 2017-08-25 | 腾讯科技（深圳）有限公司 | A kind of image processing method and device, system |
| CN107450714A (en)* | 2016-05-31 | 2017-12-08 | 大唐电信科技股份有限公司 | Man-machine interaction support test system based on augmented reality and image recognition |
| CN106249881A (en)* | 2016-07-21 | 2016-12-21 | 江苏奥格视特信息科技有限公司 | Augmented reality view field space and virtual three-dimensional target dynamic method for registering |
| CN106529550A (en)* | 2016-10-25 | 2017-03-22 | 凌云光技术集团有限责任公司 | Multidimensional characteristic extraction method and device based on connected domain analysis |
| CN107194968A (en)* | 2017-05-18 | 2017-09-22 | 腾讯科技（上海）有限公司 | Recognition and tracking method, device, intelligent terminal and the readable storage medium storing program for executing of image |
| Title |
|---|
| 仲思东,高智: "序列图像编码的自动识别", 计算机应用研究* |
| 张恒等: "面向目标特征提取的连通域标记算法", 《计算机与网络》* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110659587A (en)* | 2019-09-02 | 2020-01-07 | 广东虚拟现实科技有限公司 | Marker, marker identification method, marker identification device, terminal device and storage medium |
| CN110659587B (en)* | 2019-09-02 | 2022-08-12 | 广东虚拟现实科技有限公司 | Marker, marker identification method, device, terminal equipment and storage medium |
| Publication number | Publication date |
|---|---|
| CN110120060B (en) | 2023-07-14 |
| Publication | Publication Date | Title |
|---|---|---|
| CN106843460B (en) | Multiple target position capture positioning system and method based on multi-cam | |
| CN106372702B (en) | Positioning identifier and positioning method thereof | |
| CN110120099A (en) | Localization method, device, recognition and tracking system and computer-readable medium | |
| CN110119190A (en) | Localization method, device, recognition and tracking system and computer-readable medium | |
| CN207780718U (en) | Visual interactive device | |
| CN110119194A (en) | Virtual scene processing method, device, interactive system, head-wearing display device, visual interactive device and computer-readable medium | |
| CN110443853A (en) | Scaling method, device, terminal device and storage medium based on binocular camera | |
| CN104364733A (en) | Position-of-interest detection device, position-of-interest detection method, and position-of-interest detection program | |
| CN111310575A (en) | Face living body detection method, related device, equipment and storage medium | |
| CN208126341U (en) | Visual interactive device | |
| CN108564662A (en) | The method and device that augmented reality digital culture content is shown is carried out under a kind of remote scene | |
| CN109697444B (en) | Object recognition method and device, device and storage medium based on depth image | |
| CN209821887U (en) | Mark | |
| CN107481318A (en) | Method, device and terminal equipment for replacing user avatar | |
| Mikawa et al. | Dynamic projection mapping for robust sphere posture tracking using uniform/biased circumferential markers | |
| CN110555879A (en) | Space positioning method, device, system and computer readable medium thereof | |
| CN206400472U (en) | A kind of virtual reality device and its alignment system | |
| CN110120100B (en) | Image processing method, device and recognition tracking system | |
| US20040001074A1 (en) | Image display apparatus and method, transmitting apparatus and method, image display system, recording medium, and program | |
| CN110120060B (en) | Marker identification method, device and identification tracking system | |
| CN110119193B (en) | visual interactive device | |
| CN207909071U (en) | Visual interactive device | |
| CN110659587B (en) | Marker, marker identification method, device, terminal equipment and storage medium | |
| US20140098991A1 (en) | Game doll recognition system, recognition method and game system using the same | |
| CN209591427U (en) | Marker and interactive device |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |