技术领域technical field
本发明涉及电子设备技术领域，特别涉及一种图像处理方法、系统、可读存储介质及电子设备。The invention relates to the technical field of electronic equipment, in particular to an image processing method, system, readable storage medium and electronic equipment.
背景技术Background technique
随着电子及通信技术的不断发展，电子设备的功能不断完善，性能也不断提升，其中最具代表性的就是智能平板，智能平板广泛地应用于教育、办公、医疗等领域。With the continuous development of electronics and communication technology, the functions and performance of electronic equipment are constantly improving. The most representative one is the smart tablet, which is widely used in education, office, medical and other fields.
智能平板通过数据线/无线与电脑、手机等设备连接，能够将设备上的画面同步投影到平板电脑上。在智能平板显示画面的过程当，通常会使用到标记功能，即将用户通过手写、鼠标、笔等方式添加的标记图像(如圆圈)叠加到投影画面的相应位置上。The smart tablet is connected to computers, mobile phones and other devices through data cables/wireless, and can simultaneously project the screen on the device to the tablet computer. In the process of displaying images on the smart panel, the marker function is usually used, that is, the marker image (such as a circle) added by the user through handwriting, mouse, pen, etc. is superimposed on the corresponding position of the projected image.
然而，现有技术当中，标记图像是作为一个静态图像叠加在投影画面上的，当作为背景的投影画面发生移动时，标记图像并没有作出跟随移动，而依然停留在原先的位置，这就导致标记图像与原本标记的图像区域发生错位，导致用户体验感差。However, in the prior art, the marker image is superimposed on the projection screen as a static image. When the projection screen as the background moves, the marker image does not follow the movement, but still stays at the original position, which leads to The marked image is misaligned with the original marked image area, resulting in poor user experience.
发明内容Contents of the invention
基于此，本发明的目的是提供一种图像处理方法、系统、可读存储介质及电子设备，以使标记图像能够跟随背景一起移动。Based on this, the object of the present invention is to provide an image processing method, system, readable storage medium and electronic equipment, so that the marker image can move along with the background.
根据本发明实施例的一种图像处理方法，包括：An image processing method according to an embodiment of the present invention, comprising:
判断显示界面中的当前显示图像上是否存在与标记背景图像一致的目标子图像，所述标记背景图像存于标记关联集合当中，所述标记关联集合用于存储用户每次对所述显示界面显示的图像进行标记所产生的标记信息；Judging whether there is a target sub-image consistent with the marked background image on the currently displayed image in the display interface, the marked background image is stored in the mark association set, and the mark association set is used to store the user's display of the display interface every time The tag information generated by tagging the image;
若是，则从所述标记关联集合当中获取与所述标记背景图像关联的标记图像；If so, obtaining a marker image associated with the marker background image from the marker association set;
在所述目标子图像的位置上显示所述标记图像。The marker image is displayed at the position of the target sub-image.
另外，根据本发明上述实施例的一种图像处理方法，还可以具有如下附加的技术特征：In addition, an image processing method according to the above-mentioned embodiment of the present invention may also have the following additional technical features:
进一步地，所述判断显示界面中的当前显示图像上是否存在与标记背景图像一致的目标子图像的步骤包括：Further, the step of judging whether there is a target sub-image consistent with the marked background image on the currently displayed image in the display interface includes:
以所述标记背景图像为模板在所述当前显示图像中进行扫描，以计算出所述标记背景图像与所述当前显示图像中的每个子图像的相似度；Scanning the currently displayed image using the marked background image as a template to calculate the similarity between the marked background image and each sub-image in the currently displayed image;
判断是否存在与所述标记背景图像的相似度高于第一阈值的所述目标子图像。Judging whether there is the target sub-image whose similarity with the marked background image is higher than a first threshold.
进一步地，在所述从所述标记关联集合当中获取与所述标记背景图像关联的标记图像的步骤当中，还获取与所述标记背景图像关联的标记原始位置，所述标记原始位置为用户在标记所述标记背景图像时所述标记背景图像在所述显示界面中的位置，所述在所述目标子图像的位置上显示所述标记图像的步骤包括：Further, in the step of acquiring the marker image associated with the marker background image from the marker association set, the marker original position associated with the marker background image is also acquired, and the marker original position is the When marking the marked background image, the position of the marked background image in the display interface, the step of displaying the marked image at the position of the target sub-image comprises:
获取所述目标子图像在所述显示界面中的当前位置，并计算出所述当前位置相对所述标记原始位置的偏离量；Acquire the current position of the target sub-image in the display interface, and calculate the deviation of the current position relative to the original position of the marker;
将所述标记图像投影到所述显示界面的所述标记原始位置上，并将所述标记图像偏移所述偏离量。projecting the mark image onto the original position of the mark on the display interface, and offsetting the mark image by the offset amount.
进一步地，所述图像处理方法还包括：Further, the image processing method also includes:
当监听到标记事件时，获取所述当前显示图像上的新增标记图像；When a marker event is detected, acquire a newly added marker image on the currently displayed image;
获取所述新增标记图像在所述显示界面中的位置，并从所述当前显示图像中截取所述新增标记图像标记的区域，以分别作为所述新增标记图像对应的新增标记原始位置及新增标记背景图像；Acquiring the position of the newly added marker image in the display interface, and intercepting the area marked by the newly added marker image from the currently displayed image, so as to serve as the newly added marker original corresponding to the newly added marker image position and add background images of markers;
将所述新增标记图像、所述新增标记原始位置及所述新增标记背景图像存入到所述标记关联集合当中。Store the newly added marker image, the original position of the added marker and the background image of the added marker into the marker association set.
进一步地，所述将所述新增标记图像、所述新增标记原始位置及所述新增标记背景图像存入到所述标记关联集合当中的步骤包括：Further, the step of storing the added marker image, the original position of the added marker and the background image of the added marker into the marker association set includes:
判断所述标记关联集合当中是否存在与所述新增标记背景图像的相似度高于第二阈值的相似标记背景图像；Judging whether there is a similar tagged background image in the tag association set whose similarity with the newly added tagged background image is higher than a second threshold;
若否，则将所述新增标记图像、所述新增标记原始位置及所述新增标记背景图像相互关联后存入到所述标记关联集合当中；If not, correlating the newly added marker image, the original position of the added marker and the background image of the added marker and storing them in the marker association set;
若是，则将所述新增标记图像叠加到与所述相似标记背景图像关联的所述标记图像上，以形成新的标记图像，并将与所述相似标记背景图像关联的所述标记原始位置替换为所述新增标记原始位置，并将所述相似标记背景图像替换为所述新增标记背景图像。If so, superimposing the newly added marker image on the marker image associated with the similar marker background image to form a new marker image, and setting the original position of the marker associated with the similar marker background image The original position of the newly added mark is replaced, and the background image of the similar mark is replaced with the background image of the newly added mark.
根据本发明实施例的一种图像处理系统，包括：An image processing system according to an embodiment of the present invention, comprising:
图像判断模块，用于判断显示界面中的当前显示图像上是否存在与标记背景图像一致的目标子图像，所述标记背景图像存于标记关联集合当中，所述标记关联集合用于存储用户每次对所述显示界面显示的图像进行标记所产生的标记信息；The image judging module is used to judge whether there is a target sub-image consistent with the marked background image on the currently displayed image in the display interface. The marked background image is stored in the marked association set, and the marked associated set is used to store the marking information generated by marking the image displayed on the display interface;
信息获取模块，用于当判断到所述当前显示图像上存在与所述标记背景图像一致的所述目标子图像时，从所述标记关联集合当中获取与所述标记背景图像关联的标记图像；An information acquisition module, configured to acquire a marker image associated with the marker background image from the marker association set when it is determined that the target sub-image consistent with the marker background image exists on the currently displayed image;
图像显示模块，用于在所述目标子图像的位置上显示所述标记图像。An image display module, configured to display the mark image at the position of the target sub-image.
另外，根据本发明上述实施例的一种图像处理系统，还可以具有如下附加的技术特征：In addition, an image processing system according to the above-mentioned embodiments of the present invention may also have the following additional technical features:
进一步地，所述图像判断模块包括：Further, the image judging module includes:
相度计算单元，用于以所述标记背景图像为模板在所述当前显示图像中进行扫描，以计算出所述标记背景图像与所述当前显示图像中的每个子图像的相似度；A phase calculation unit, configured to use the marked background image as a template to scan in the currently displayed image to calculate the similarity between the marked background image and each sub-image in the currently displayed image;
相度判断单元，用于判断是否存在与所述标记背景图像的相似度高于第一阈值的所述目标子图像。A phase degree judging unit, configured to judge whether there is the target sub-image whose similarity with the marked background image is higher than a first threshold.
进一步地，在所述从所述标记关联集合当中获取与所述标记背景图像关联的标记图像的步骤当中，还获取与所述标记背景图像关联的标记原始位置，所述标记原始位置为用户在标记所述标记背景图像时所述标记背景图像在所述显示界面中的位置，所述图像显示模块包括：Further, in the step of acquiring the marker image associated with the marker background image from the marker association set, the marker original position associated with the marker background image is also acquired, and the marker original position is the When marking the marked background image, the position of the marked background image in the display interface, the image display module includes:
偏量计算单元，用于获取所述目标子图像在所述显示界面中的当前位置，并计算出所述当前位置相对所述标记原始位置的偏离量；An offset calculation unit, configured to acquire the current position of the target sub-image in the display interface, and calculate the deviation of the current position relative to the original position of the mark;
图像投影单元，用于将所述标记图像投影到所述显示界面的所述标记原始位置上，并将所述标记图像偏移所述偏离量。An image projecting unit, configured to project the mark image onto the original position of the mark on the display interface, and offset the mark image by the offset amount.
本发明还提出一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如上述的图像处理方法。The present invention also proposes a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned image processing method is realized.
本发明还提出一种电子设备，包括存储器、处理器以及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现如上述的图像处理方法。The present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor. The processor implements the above-mentioned image processing method when executing the program.
上述图像处理方法、系统、计算机可读存储介质及电子设备，通过将每次标记产生的标记图像及标记图像所标记的图像区域(标记背景图像)存入到所述标记关联集合当中，然后每次用存入的标记背景图像与显示界面中的当前显示图像进行匹配，若发现当前显示图像存在目标子图像与标记背景图像一致时，自动将标记背景图像对应的标记图像投影到目标子图像上。因此，即使当前显示图像发生移动等变化，以致原先标记的图像区域发送移动，所述图像处理方法、系统、计算机可读存储介质及电子设备通过图像匹配，仍然能够找到原先标记的图像区域的当前位置，并自动将标记图像投影到该当前位置上，达到了标记图像跟随背景一起移动的效果，用户体验感好。The image processing method, system, computer-readable storage medium, and electronic device described above store the tagged image generated each time and the image region marked by the tagged image (marked background image) into the tag association set, and then each time Match the stored marked background image with the currently displayed image in the display interface. If the target sub-image is found to be consistent with the marked background image in the current displayed image, the marked image corresponding to the marked background image will be automatically projected onto the target sub-image. . Therefore, even if the currently displayed image changes such as movement, so that the originally marked image area is sent to move, the image processing method, system, computer-readable storage medium and electronic device can still find the current position of the originally marked image area through image matching. position, and automatically project the marked image to the current position, achieving the effect that the marked image moves with the background, and the user experience is good.
附图说明Description of drawings
图1为本发明第一实施例中的图像处理方法的流程图；Fig. 1 is the flowchart of the image processing method in the first embodiment of the present invention;
图2为本发明第二实施例中的图像处理方法的流程图；Fig. 2 is the flowchart of the image processing method in the second embodiment of the present invention;
图3为本发明第三实施例中的图像处理方法的流程图；3 is a flowchart of an image processing method in a third embodiment of the present invention;
图4为图3中步骤S23的具体实施流程图。FIG. 4 is a specific implementation flowchart of step S23 in FIG. 3 .
图5为本发明第四实施例中的图像处理系统的结构示意图；5 is a schematic structural diagram of an image processing system in a fourth embodiment of the present invention;
图6为本发明第五实施例中的电子设备的结构示意图。FIG. 6 is a schematic structural diagram of an electronic device in a fifth embodiment of the present invention.
主要元件符号说明：Description of main component symbols:
以下具体实施方式将结合上述附图进一步说明本发明。The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings.
具体实施方式Detailed ways
为了便于理解本发明，下面将参照相关附图对本发明进行更全面的描述。附图中给出了本发明的若干实施例。但是，本发明可以以许多不同的形式来实现，并不限于本文所描述的实施例。相反地，提供这些实施例的目的是使对本发明的公开内容更加透彻全面。In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the associated drawings. Several embodiments of the invention are shown in the drawings. However, the present invention can be embodied in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of the present invention will be thorough and complete.
需要说明的是，当元件被称为“固设于”另一个元件，它可以直接在另一个元件上或者也可以存在居中的元件。当一个元件被认为是“连接”另一个元件，它可以是直接连接到另一个元件或者可能同时存在居中元件。本文所使用的术语“垂直的”、“水平的”、“左”、“右”以及类似的表述只是为了说明的目的。It should be noted that when an element is referred to as being “fixed on” another element, it may be directly on the other element or there may be an intervening element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or intervening elements may also be present. The terms "vertical," "horizontal," "left," "right," and similar expressions are used herein for purposes of illustration only.
除非另有定义，本文所使用的所有的技术和科学术语与属于本发明的技术领域的技术人员通常理解的含义相同。本文中在本发明的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本发明。本文所使用的术语“及/或”包括一个或多个相关的所列项目的任意的和所有的组合。Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field of the invention. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only, and are not intended to limit the present invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
请参阅图1，所示为本发明第一实施例中的图像处理方法，应用于电子设备当中，包括步骤S01至步骤S03。Please refer to FIG. 1 , which shows the image processing method in the first embodiment of the present invention, which is applied to an electronic device, including steps S01 to S03 .
步骤S01，判断显示界面中的当前显示图像上是否存在与标记背景图像一致的目标子图像，所述标记背景图像存于标记关联集合当中，所述标记关联集合用于存储用户每次对所述显示界面显示的图像进行标记所产生的标记信息。Step S01, judging whether there is a target sub-image consistent with the marked background image on the currently displayed image in the display interface, the marked background image is stored in the marked association set, and the marked associated set is used to store the The marking information generated by marking the image displayed on the interface is displayed.
其中，所述显示界面为电子设备用作显示投影图像的界面，例如智能平板的显示屏幕。Wherein, the display interface is an interface used by an electronic device to display projected images, such as a display screen of a smart tablet.
此外，当判断到当前显示图像上存在与标记背景图像一致的目标子图像时，则执行步骤S02，当判断到当前显示图像上不存在与标记背景图像一致的目标子图像时，则代表当前显示图像上不存在图像区域曾被标记过，此时不动作并继续检测。In addition, when it is determined that there is a target sub-image consistent with the marked background image on the currently displayed image, step S02 is executed; when it is determined that there is no target sub-image consistent with the marked background image on the currently displayed image, the If there is no image area on the image that has been marked, no action will be taken at this time and detection will continue.
步骤S02，从所述标记关联集合当中获取与所述标记背景图像关联的标记图像。Step S02 , acquiring a marker image associated with the marker background image from the marker association set.
需要指出的是，所述标记关联集合当中存入的每个标记背景图像和标记图像，均为用户在此前对显示界面显示的图像进行某个标记时产生的，而后被存入到标记关联集合当中。其中，标记图像为用户通过手写、鼠标、笔等方式添加到显示界面当时显示的画面上的图像，而标记背景图像为从当时显示的画面上截取的标记图像所标记的图像区域。It should be pointed out that each marker background image and marker image stored in the marker association set is generated when the user marks an image displayed on the display interface before, and is then stored in the marker association set among. Wherein, the marked image is an image added by the user to the screen currently displayed on the display interface through handwriting, mouse, pen, etc., and the marked background image is the image area marked by the marked image intercepted from the currently displayed screen.
故在具体实施时，可以将用户每次标记产生的标记图像及标记图像所标记的图像区域(标记背景图像)存入到所述标记关联集合当中。举例来说，当用户通过手写、鼠标、笔等方式在当前显示图像的标题处画上一个圆圈时，该圆圈圈中该标题，此时，电子设备将截取该圆圈圈中的图像区域，以作为标记背景图像，并和该圆圈图像一同存入到标记关联集合当中。Therefore, during specific implementation, the mark image generated each time the user marks and the image area marked by the mark image (mark background image) can be stored in the mark association set. For example, when the user draws a circle on the title of the currently displayed image by handwriting, mouse, pen, etc., the title is enclosed in the circle, and at this time, the electronic device will intercept the image area in the circle to It is used as the background image of the mark and stored in the mark association collection together with the circle image.
步骤S03，在所述目标子图像的位置上显示所述标记图像。Step S03, displaying the mark image at the position of the target sub-image.
在具体实施时，可以通过图像叠加技术将所述标记图像叠加到当前显示图像的所述目标子图像的位置上，以实现投影显示效果。During specific implementation, the marker image may be superimposed on the position of the target sub-image of the currently displayed image through an image overlay technology, so as to achieve a projection display effect.
综上，本发明上述实施例当中的图像处理方法，通过将每次标记产生的标记图像及标记图像所标记的图像区域(标记背景图像)存入到所述标记关联集合当中，然后每次用存入的标记背景图像与显示界面中的当前显示图像进行匹配，若发现当前显示图像存在目标子图像与标记背景图像一致时，自动将标记背景图像对应的标记图像投影到目标子图像上。因此，即使当前显示图像发生移动等变化，以致原先标记的图像区域发送移动，所述图像处理方法、系统、计算机可读存储介质及电子设备通过图像匹配，仍然能够找到原先标记的图像区域的当前位置，并自动将标记图像投影到该当前位置上，达到了标记图像跟随背景一起移动的效果，用户体验感好。To sum up, the image processing method in the above-mentioned embodiments of the present invention stores the marked image generated each time and the image area marked by the marked image (marked background image) into the marked association set, and then uses The stored marked background image is matched with the currently displayed image in the display interface. If the target sub-image is found to be consistent with the marked background image in the currently displayed image, the marked image corresponding to the marked background image is automatically projected onto the target sub-image. Therefore, even if the currently displayed image changes such as movement, so that the originally marked image area is sent to move, the image processing method, system, computer-readable storage medium and electronic device can still find the current position of the originally marked image area through image matching. position, and automatically project the marked image to the current position, achieving the effect that the marked image moves with the background, and the user experience is good.
请参阅图2，所示为本发明第二实施例中的图像处理方法，应用于电子设备当中，包括步骤S11至步骤S15。Please refer to FIG. 2 , which shows the image processing method in the second embodiment of the present invention, which is applied to electronic equipment, including steps S11 to S15 .
步骤S11，以标记背景图像为模板在当前显示图像上进行扫描，以计算出所述标记背景图像与所述当前显示图像中的每个子图像的相似度，所述标记背景图像存于标记关联集合当中。Step S11, using the marked background image as a template to scan on the currently displayed image to calculate the similarity between the marked background image and each sub-image in the currently displayed image, the marked background image is stored in the marked association set among.
其中，当前显示图像为电子设备的显示界面中当前显示的图像，所述标记关联集合用于存储用户每次对所述显示界面显示的图像进行标记所产生的标记信息，所述标记信息包括标记背景图像、标记图像及标记原始位置。Wherein, the currently displayed image is the image currently displayed in the display interface of the electronic device, and the tag association set is used to store tag information generated each time the user tags the image displayed on the display interface, and the tag information includes tag Background image, marker image and marker origin.
步骤S12，判断是否存在与所述标记背景图像的相似度高于第一阈值的目标子图像。Step S12, judging whether there is a target sub-image whose similarity with the marked background image is higher than a first threshold.
其中，当判断到存在与所述标记背景图像的相似度高于第一阈值的目标子图像时，则执行步骤S13至步骤S15，当判断到不存在与所述标记背景图像的相似度高于第一阈值的目标子图像时，则代表当前显示图像上不存在图像区域曾被标记过，此时不动作并继续检测。Wherein, when it is judged that there is a target sub-image whose similarity with the marked background image is higher than the first threshold, then execute step S13 to step S15, and when it is judged that there is no target sub-image whose similarity with the marked background image is higher than When the first threshold is the target sub-image, it means that there is no image area that has been marked on the currently displayed image. At this time, no action will be taken and the detection will continue.
步骤S13，从所述标记关联集合当中获取与所述标记背景图像关联的标记图像及标记原始位置。Step S13, acquiring the marker image associated with the marker background image and the marker original position from the marker association set.
其中，所述标记原始位置为用户在标记所述标记背景图像时所述标记背景图像在所述显示界面中的位置。Wherein, the marked original position is the position of the marked background image in the display interface when the user marks the marked background image.
需要指出的是，所述标记关联集合当中存入的每个标记原始位置，均为用户在此前进行某个标记时产生的，而后被存入到标记关联集合当中。其中，标记原始位置与对应的标记图像及标记背景图像相互关联，标记原始位置为对应的标记背景图像那时在显示界面中的位置。It should be pointed out that the original position of each marker stored in the marker association set is generated when the user performs a certain marker before, and then stored in the marker association set. Wherein, the original position of the mark is related to the corresponding mark image and the mark background image, and the mark original position is the position of the corresponding mark background image in the display interface at that time.
故在具体实施时，可以将用户每次标记产生的标记图像、标记图像所标记的图像区域(标记背景图像)及标记区域在显示界面中的位置(标记原始位置)存入到所述标记关联集合当中。举例来说，当用户通过手写、鼠标、笔等方式在当前显示图像的标题处画上一个圆圈时，该圆圈圈中该标题，此时，电子设备将截取该圆圈圈中的图像区域，以作为标记背景图像，并获取该标题在显示界面当中的位置，以作为标记原始位置，并和该圆圈图像一同存入到标记关联集合当中。Therefore, during specific implementation, the mark image generated by the user each mark, the image area marked by the mark image (mark background image) and the position of the mark area in the display interface (mark original position) can be stored in the mark association in the collection. For example, when the user draws a circle on the title of the currently displayed image by handwriting, mouse, pen, etc., the title is enclosed in the circle, and at this time, the electronic device will intercept the image area in the circle to It is used as the background image of the mark, and the position of the title in the display interface is obtained as the original position of the mark, and stored together with the circle image in the mark association collection.
步骤S14，获取所述目标子图像在显示界面中的当前位置，并计算出所述当前位置相对所述标记原始位置的偏离量。Step S14, acquiring the current position of the target sub-image in the display interface, and calculating the deviation of the current position relative to the original position of the marker.
其中，所述偏移量包括水平的偏移距离和偏移方向，及竖直的偏移距离和偏移方向。Wherein, the offset includes a horizontal offset distance and an offset direction, and a vertical offset distance and an offset direction.
在具体实施例，可以在获取位置的过程当中获取该位置的坐标，因此根据所述当前位置和所述标记原始位置的坐标数据，可以计算出两者之间的偏移量。In a specific embodiment, the coordinates of the position can be obtained during the process of obtaining the position, so according to the coordinate data of the current position and the original position of the marker, the offset between the two can be calculated.
步骤S15，将所述标记图像投影到所述显示界面的所述标记原始位置上，并将所述标记图像偏移所述偏离量。Step S15 , projecting the mark image onto the original position of the mark on the display interface, and offsetting the mark image by the offset amount.
可以理解的，所述步骤S15的目的在于，将所述标记图像投影到所述目标子图像的位置上，本实施例中的具体操作过程为，先将所述标记图像投影到原先标记的位置(相对显示界面)上，然后在将所述标记图像偏移到所述目标子图像的位置上，由于该过程用时极短，肉眼无法分辨，能够达到像直接投影到目标子图像上的效果。It can be understood that the purpose of the step S15 is to project the marked image to the position of the target sub-image, and the specific operation process in this embodiment is to first project the marked image to the original marked position (relative to the display interface), and then shift the marked image to the position of the target sub-image. Since this process takes a very short time, the naked eye cannot distinguish it, and the effect of directly projecting the image onto the target sub-image can be achieved.
同时，这种做法的好处在于，由于在直接将标记图像投影到目标子图像上的过程当中，没有位置参考，精度较低，导致标记图像在投影后所标记的区域与原先所标记的区域(标记背景图像)会发生变动，而通过偏移的方法，存在位置参考，且所述偏离量为通过精确的坐标数据计算而来的，使得投影精度高，从而使得标记图像在投影后所标记的区域与原先所标记的区域(标记背景图像)不会发生变动。At the same time, the advantage of this approach is that since there is no position reference in the process of directly projecting the marked image onto the target sub-image, the accuracy is low, resulting in the marked area after the marked image is projected to be different from the original marked area ( The marked background image) will change, and through the offset method, there is a position reference, and the deviation is calculated through precise coordinate data, so that the projection accuracy is high, so that the marked image after projection The area will not change from the previously marked area (marked background image).
请参阅图3，所示为本发明第三实施例中的图像处理方法，包括步骤S21至步骤S24。Please refer to FIG. 3 , which shows the image processing method in the third embodiment of the present invention, including steps S21 to S24.
步骤S21，当监听到标记事件时，获取当前显示图像上的新增标记图像。Step S21, when a marker event is detected, acquire a newly added marker image on the currently displayed image.
其中，当前显示图像为电子设备的显示界面中当前显示的图像。Wherein, the currently displayed image is an image currently displayed on the display interface of the electronic device.
可以理解的，当用户启动标注功能，并通过手写、鼠标、笔等方式在当前显示图像上开始添加标记图像时，电子设备将监听到该标记事件，并不断监听书写规则，以生成该新增标记图像。It is understandable that when the user activates the marking function and starts to add a marked image on the currently displayed image by handwriting, mouse, pen, etc., the electronic device will monitor the marking event and continuously monitor the writing rules to generate the newly added image. Tag the image.
步骤S22，获取所述新增标记图像在显示界面中的位置，并从所述当前显示图像中截取所述新增标记图像标记的区域，以分别作为所述新增标记图像对应的新增标记原始位置及新增标记背景图像。Step S22, acquiring the position of the newly-added marker image in the display interface, and intercepting the region marked by the newly-added marker image from the currently displayed image, as newly-added markers corresponding to the newly-added marker image Original position and new marker background image.
步骤S23，将所述新增标记图像、所述新增标记原始位置及所述新增标记背景图像存入到所述标记关联集合当中。Step S23, storing the newly added marker image, the original position of the added marker, and the newly added marker background image into the marker association set.
请查阅图4，所示为步骤S23的具体实施流程图，包括步骤S231至步骤S235。Please refer to FIG. 4 , which shows a specific implementation flowchart of step S23 , including step S231 to step S235 .
步骤S231，判断所述标记关联集合当中是否存在与所述新增标记背景图像的相似度高于第二阈值的相似标记背景图像。Step S231 , judging whether there is a similar tagged background image in the tag association set whose similarity with the newly added tagged background image is higher than a second threshold.
其中，当判断到所述标记关联集合当中不存在与所述新增标记背景图像的相似度高于第二阈值的相似标记背景图像时，则执行步骤S232，当判断到所述标记关联集合当中存在与所述新增标记背景图像的相似度高于第二阈值的相似标记背景图像时，代表标记关联集合当中存在重复标记的背景图像，则执行步骤S233。Wherein, when it is judged that there is no similar tagged background image in the tag association set whose similarity with the newly added tag background image is higher than the second threshold, step S232 is executed, and when it is judged that in the tag associated set When there is a similar tagged background image whose similarity with the newly added tagged background image is higher than the second threshold, it means that there are duplicate tagged background images in the tag association set, and step S233 is executed.
步骤S232，将所述新增标记图像、所述新增标记原始位置及所述新增标记背景图像相互关联后存入到所述标记关联集合当中。Step S232, correlating the newly added marker image, the original position of the added marker and the background image of the added marker and storing them in the marker association set.
可以理解的，当判断到标记关联集合当中不存在与新增标记背景图像的相似度高于第二阈值的相似标记背景图像时，代表标记关联集合当中不存在相同的标记背景图像，则直接将数据相互关联后存入标记关联集合当中。It can be understood that when it is judged that there is no similar tagged background image in the tag association set whose similarity with the newly added tag background image is higher than the second threshold, it means that the same tag background image does not exist in the tag association set, and the After the data is associated with each other, it is stored in the tag association collection.
步骤S233，将所述新增标记图像叠加到与所述相似标记背景图像关联的所述标记背景图像上，以形成新的标记背景图像，并将与所述相似标记背景图像关联的所述标记原始位置替换为所述新增标记原始位置，并将所述相似标记背景图像替换为所述新增标记背景图像。Step S233, superimposing the newly added tag image on the tag background image associated with the similar tag background image to form a new tag background image, and adding the tag associated with the similar tag background image The original position is replaced with the original position of the added mark, and the background image of the similar mark is replaced with the background image of the added mark.
可以理解的，当判断到标记关联集合当中存在与新增标记背景图像的相似度高于第二阈值的相似标记背景图像时，代表标记关联集合当中存在相同的标记背景图像，则将新增标记图像和与相似标记背景图像关联的标记图像相互叠加，以形成新的标记图像，并对相似标记背景图像及对应的标记原始位置进行更新。It can be understood that when it is determined that there is a similar tagged background image in the tag association set whose similarity with the newly added tag background image is higher than the second threshold, it means that there is the same tag background image in the tag associated set, and the new tag will be added. The image and the marker image associated with the similar marker background image are superimposed on each other to form a new marker image, and the similar marker background image and the corresponding marker original position are updated.
本发明另一方面还提供一种图像处理系统，请查阅图5，所示为本发明第四实施例中的图像处理系统10，包括：Another aspect of the present invention also provides an image processing system, please refer to FIG. 5, which shows an image processing system 10 in the fourth embodiment of the present invention, including:
图像判断模块11，用于判断显示界面中的当前显示图像上是否存在与标记背景图像一致的目标子图像，所述标记背景图像存于标记关联集合当中，所述标记关联集合用于存储用户每次对所述显示界面显示的图像进行标记所产生的标记信息；The image judging module 11 is used to judge whether there is a target sub-image consistent with the marked background image on the currently displayed image in the display interface. The marked background image is stored in the marked association set, and the marked associated set is used to store the marking information generated by marking the image displayed on the display interface for the second time;
信息获取模块12，用于当判断到所述当前显示图像上存在与所述标记背景图像一致的所述目标子图像时，从所述标记关联集合当中获取与所述标记背景图像关联的标记图像；An information acquisition module 12, configured to acquire a marker image associated with the marker background image from the marker association set when it is determined that the target sub-image consistent with the marker background image exists on the currently displayed image ;
图像显示模块13，用于在所述目标子图像的位置上显示所述标记图像。An image display module 13, configured to display the mark image at the position of the target sub-image.
进一步地，所述图像判断模块11包括：Further, the image judging module 11 includes:
相度计算单元111，用于以所述标记背景图像为模板在所述当前显示图像中进行扫描，以计算出所述标记背景图像与所述当前显示图像中的每个子图像的相似度；A phase calculation unit 111, configured to use the marked background image as a template to scan in the currently displayed image to calculate the similarity between the marked background image and each sub-image in the currently displayed image;
相度判断单元112，用于判断是否存在与所述标记背景图像的相似度高于第一阈值的所述目标子图像。A phase degree judging unit 112, configured to judge whether there is the target sub-image whose similarity with the marked background image is higher than a first threshold.
进一步地，在所述从所述标记关联集合当中获取与所述标记背景图像关联的标记图像的步骤当中，还获取与所述标记背景图像关联的标记原始位置，所述标记原始位置为用户在标记所述标记背景图像时所述标记背景图像在所述显示界面中的位置，所述图像显示模块13包括：Further, in the step of acquiring the marker image associated with the marker background image from the marker association set, the marker original position associated with the marker background image is also acquired, and the marker original position is the When marking the marked background image, the position of the marked background image in the display interface, the image display module 13 includes:
偏量计算单元131，用于获取所述目标子图像在所述显示界面中的当前位置，并计算出所述当前位置相对所述标记原始位置的偏离量；An offset calculation unit 131, configured to acquire the current position of the target sub-image in the display interface, and calculate the deviation of the current position relative to the original position of the mark;
图像投影单元132，用于将所述标记图像投影到所述显示界面的所述标记原始位置上，并将所述标记图像偏移所述偏离量。The image projecting unit 132 is configured to project the mark image onto the original position of the mark on the display interface, and offset the mark image by the offset amount.
进一步地，所述图像处理系统还包括：Further, the image processing system also includes:
标记获取模块14，用于当监听到标记事件时，获取所述当前显示图像上的新增标记图像；A marker acquisition module 14, configured to acquire a newly added marker image on the currently displayed image when a marker event is monitored;
信息获取模块15，用于获取所述新增标记图像在所述显示界面中的位置，并从所述当前显示图像中截取所述新增标记图像标记的区域，以分别作为所述新增标记图像对应的新增标记原始位置及新增标记背景图像；An information acquisition module 15, configured to obtain the position of the newly added marker image in the display interface, and intercept the region marked by the newly added marker image from the currently displayed image, as the newly added marker The original position of the newly added marker corresponding to the image and the background image of the newly added marker;
信息存储模块16，用于将所述新增标记图像、所述新增标记原始位置及所述新增标记背景图像存入到所述标记关联集合当中。The information storage module 16 is configured to store the added marker image, the original position of the added marker and the background image of the added marker into the marker association set.
进一步地，所述信息存储模块16包括：Further, the information storage module 16 includes:
相似判断单元161，用于判断所述标记关联集合当中是否存在与所述新增标记背景图像的相似度高于第二阈值的相似标记背景图像；A similarity judging unit 161, configured to judge whether there is a similar tagged background image whose similarity with the newly added tagged background image is higher than a second threshold in the tag association set;
第一处理单元162，用于当判断到所述标记关联集合当中不存在所述相似标记背景图像时，将所述新增标记图像、所述新增标记原始位置及所述新增标记背景图像相互关联后存入到所述标记关联集合当中；The first processing unit 162 is configured to, when it is determined that the background image of the similar mark does not exist in the mark association set, combine the newly added mark image, the original position of the newly added mark, and the newly added mark background image After being correlated with each other, they are stored in the tag association set;
第一处理单元163，用于当判断到所述标记关联集合当中存在所述相似标记背景图像时，将所述新增标记图像叠加到与所述相似标记背景图像关联的所述标记图像上，以形成新的标记图像，并将与所述相似标记背景图像关联的所述标记原始位置替换为所述新增标记原始位置，并将所述相似标记背景图像替换为所述新增标记背景图像。The first processing unit 163 is configured to superimpose the newly added marker image on the marker image associated with the similar marker background image when it is determined that the similar marker background image exists in the marker association set, to form a new tag image, and replace the original position of the tag associated with the background image of the similar tag with the original position of the added tag, and replace the background image of the similar tag with the background image of the added tag .
本发明还提出一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如上述的图像处理方法。The present invention also proposes a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned image processing method is realized.
本发明还提出一种电子设备，请查阅图6，所示为本发明第五实施例当中的电子设备，包括处理器100、存储器200以及存储在存储器上并可在处理器上运行的计算机程序300，所述处理器100执行所述程序300时实现如上述的图像处理方法。The present invention also proposes an electronic device, please refer to FIG. 6, which shows the electronic device in the fifth embodiment of the present invention, including a processor 100, a memory 200, and a computer program stored in the memory and operable on the processor 300. When the processor 100 executes the program 300, implement the above-mentioned image processing method.
此外，所述电子设备还包括显示装置400、输入装置500及电源600。所述显示装置400为显示屏，其可用于显示通过数据线/无线与所述电子设备连接的电脑、手机等设备传来的投影图像，所述输入装置500为供用户交互输入用的设备，如鼠标、触摸屏等，用户通过所述输入装置500可以在当前显示画面上添加标注图像。所述电源600则为所述电子设备提供电源。In addition, the electronic device further includes a display device 400 , an input device 500 and a power supply 600 . The display device 400 is a display screen, which can be used to display projected images from devices such as computers and mobile phones connected to the electronic device through data lines/wireless, and the input device 500 is a device for user interactive input, Such as a mouse, a touch screen, etc., through the input device 500, the user can add an annotation image on the current display screen. The power supply 600 provides power for the electronic equipment.
可以理解的，所述电子设备包括但是不限于为智能平板、电脑、智能电视、魔镜等。It can be understood that the electronic device includes, but is not limited to, a smart tablet, a computer, a smart TV, a magic mirror, and the like.
本领域技术人员可以理解，在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。就本说明书而言，“计算机可读介质”可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。Those skilled in the art will understand that the logic and/or steps shown in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, which can be specifically implemented in on any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, apparatus, or device and execute them), or Used in conjunction with these instruction execution systems, devices or equipment. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use in or in conjunction with an instruction execution system, device or device.
计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。另外，计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection with one or more wires (electronic device), portable computer disk case (magnetic device), random access memory (RAM), Read Only Memory (ROM), Erasable and Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, since the program can be read, for example, by optically scanning the paper or other medium, followed by editing, interpretation or other suitable processing if necessary. The program is processed electronically and stored in computer memory.
应当理解，本发明的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。例如，如果用硬件来实现，和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或它们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。It should be understood that various parts of the present invention can be realized by hardware, software, firmware or their combination. In the embodiments described above, various steps or methods may be implemented by software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques known in the art: Discrete logic circuits, ASICs with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
以上所述实施例仅表达了本发明的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干变形和改进，这些都属于本发明的保护范围。因此，本发明专利的保护范围应以所附权利要求为准。The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present invention. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention should be based on the appended claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810130137.7ACN108366238A (en) | 2018-02-08 | 2018-02-08 | Image processing method, system, readable storage medium and electronic device |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810130137.7ACN108366238A (en) | 2018-02-08 | 2018-02-08 | Image processing method, system, readable storage medium and electronic device |
| Publication Number | Publication Date |
|---|---|
| CN108366238Atrue CN108366238A (en) | 2018-08-03 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810130137.7APendingCN108366238A (en) | 2018-02-08 | 2018-02-08 | Image processing method, system, readable storage medium and electronic device |
| Country | Link |
|---|---|
| CN (1) | CN108366238A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109547845A (en)* | 2018-12-24 | 2019-03-29 | 武汉烽火众智智慧之星科技有限公司 | A kind of video marker method and device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1943229A (en)* | 2004-04-08 | 2007-04-04 | 皇家飞利浦电子股份有限公司 | Mobile projectable gui |
| CN101859208A (en)* | 2009-04-10 | 2010-10-13 | 船井电机株式会社 | Image display device, method for displaying image and the recording medium that has image display program stored therein |
| CN103218854A (en)* | 2013-04-01 | 2013-07-24 | 成都理想境界科技有限公司 | Method for realizing component marking during augmented reality process and augmented reality system |
| CN104423560A (en)* | 2013-09-09 | 2015-03-18 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105095362A (en)* | 2015-06-25 | 2015-11-25 | 深圳码隆科技有限公司 | Image display method and device based on target object |
| CN105573702A (en)* | 2015-12-16 | 2016-05-11 | 广州视睿电子科技有限公司 | Method and system for synchronizing movement and scaling of remote annotation |
| CN106445208A (en)* | 2016-09-30 | 2017-02-22 | 南京巨鲨医疗科技有限公司 | Method for following display of labeling graph based on serial port |
| CN106650965A (en)* | 2016-12-30 | 2017-05-10 | 触景无限科技（北京）有限公司 | Remote video processing method and apparatus |
| CN107194968A (en)* | 2017-05-18 | 2017-09-22 | 腾讯科技（上海）有限公司 | Recognition and tracking method, device, intelligent terminal and the readable storage medium storing program for executing of image |
| CN107454433A (en)* | 2017-08-09 | 2017-12-08 | 广州视源电子科技股份有限公司 | Live broadcasting annotation method and device, terminal and live broadcasting system |
| CN107479791A (en)* | 2017-08-18 | 2017-12-15 | 广州视源电子科技股份有限公司 | Associated annotation information determination method and device, intelligent teaching equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1943229A (en)* | 2004-04-08 | 2007-04-04 | 皇家飞利浦电子股份有限公司 | Mobile projectable gui |
| CN101859208A (en)* | 2009-04-10 | 2010-10-13 | 船井电机株式会社 | Image display device, method for displaying image and the recording medium that has image display program stored therein |
| CN103218854A (en)* | 2013-04-01 | 2013-07-24 | 成都理想境界科技有限公司 | Method for realizing component marking during augmented reality process and augmented reality system |
| CN104423560A (en)* | 2013-09-09 | 2015-03-18 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105095362A (en)* | 2015-06-25 | 2015-11-25 | 深圳码隆科技有限公司 | Image display method and device based on target object |
| CN105573702A (en)* | 2015-12-16 | 2016-05-11 | 广州视睿电子科技有限公司 | Method and system for synchronizing movement and scaling of remote annotation |
| CN106445208A (en)* | 2016-09-30 | 2017-02-22 | 南京巨鲨医疗科技有限公司 | Method for following display of labeling graph based on serial port |
| CN106650965A (en)* | 2016-12-30 | 2017-05-10 | 触景无限科技（北京）有限公司 | Remote video processing method and apparatus |
| CN107194968A (en)* | 2017-05-18 | 2017-09-22 | 腾讯科技（上海）有限公司 | Recognition and tracking method, device, intelligent terminal and the readable storage medium storing program for executing of image |
| CN107454433A (en)* | 2017-08-09 | 2017-12-08 | 广州视源电子科技股份有限公司 | Live broadcasting annotation method and device, terminal and live broadcasting system |
| CN107479791A (en)* | 2017-08-18 | 2017-12-15 | 广州视源电子科技股份有限公司 | Associated annotation information determination method and device, intelligent teaching equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109547845A (en)* | 2018-12-24 | 2019-03-29 | 武汉烽火众智智慧之星科技有限公司 | A kind of video marker method and device |
| CN109547845B (en)* | 2018-12-24 | 2021-10-01 | 武汉烽火众智智慧之星科技有限公司 | Video marking method and device |
| Publication | Publication Date | Title |
|---|---|---|
| CN107491174B (en) | Method, device and system for remote assistance and electronic equipment | |
| US9405182B2 (en) | Image processing device and image processing method | |
| JP6007497B2 (en) | Image projection apparatus, image projection control apparatus, and program | |
| CN110852332B (en) | Method, device, storage medium and electronic device for generating training samples | |
| CN110986969B (en) | Map fusion method and device, equipment and storage medium | |
| US20140351718A1 (en) | Information processing device, information processing method, and computer-readable medium | |
| US10691878B2 (en) | Presenting associations of strokes with content | |
| US9830336B2 (en) | Information processing device, information processing method and information processing program | |
| JP2013080326A (en) | Image processing device, image processing method, and program | |
| CN110648363A (en) | Camera posture determining method and device, storage medium and electronic equipment | |
| US11361532B1 (en) | System and method for OCR based object registration | |
| JP2017091328A (en) | Information processing apparatus and information processing program | |
| US10281804B2 (en) | Image processing apparatus, image processing method, and program | |
| CN104680393A (en) | Interactive advertisement method based on image contents and matching | |
| US9304618B2 (en) | Creating a summary of content and stroke association | |
| US10372318B2 (en) | Associating strokes with content | |
| WO2022123929A1 (en) | Information processing device and information processing method | |
| CN108366238A (en) | Image processing method, system, readable storage medium and electronic device | |
| US10796435B2 (en) | Image processing method and image processing apparatus | |
| CN113011298A (en) | Truncated object sample generation method, target detection method, road side equipment and cloud control platform | |
| CN113177607B (en) | A method and client for labeling training objects | |
| CN113766147B (en) | Method for embedding image in video, and method and device for acquiring plane prediction model | |
| JP2016045538A (en) | Information processing apparatus, image determination method, and program | |
| CN107831896A (en) | Audio-frequency information player method, device, storage medium and electronic equipment | |
| CN115471416A (en) | Target identification method, storage medium and equipment |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20180803 | |
| RJ01 | Rejection of invention patent application after publication |