CN114398511A - A medical system visualization device and its labeling method - Google Patents

Abstract

The invention relates to a medical system visualization device and a label labeling method thereof, wherein the medical system visualization device is in communication connection with an image labeling system; the medical system visualization equipment at least comprises an imaging part, an image display part, an image analysis part and an image labeling part, wherein the image labeling system is matched with the labels in at least one label body library based on the keywords of the labeling request and independently distributes the medical images to be labeled to the labeling operators; the imaging part converts the image information sent by the medical image labeling part into a medical image and presents the medical image to a labeling operator at the image display part, and the image analysis part records the labeling information of at least one labeling operator and counts intersection labels of the same medical image; the image labeling part completes the matching of the medical image and the label according to the received medical image, the label content matched with the medical image and the matching relation code of the medical image. The invention automatically recommends the label for the user and supports the cooperative work of a plurality of users.

Description

Translated fromChinese

一种医疗系统可视化设备及其标签标注方法A medical system visualization device and its labeling method

本发明是申请号为201580084598.X，申请名称为一种医学图像标注方法及系统，申请日为2015年12月24日的发明专利的分案申请。The present invention is a divisional application of an invention patent with the application number of 201580084598.X, the application name is a medical image annotation method and system, and the application date is December 24, 2015.

技术领域technical field

本发明涉及图像处理技术领域，尤其涉及一种医疗系统可视化设备及其标签标注方法。The invention relates to the technical field of image processing, in particular to a medical system visualization device and a label labeling method thereof.

背景技术Background technique

迄今为止，世界上各种医疗学术机构都积累了大量的医学图像，这些图像数量巨大，种类繁多，对这些医学图像的管理、检索与再利用一直是个难题。So far, various medical academic institutions in the world have accumulated a large number of medical images, which are huge in number and variety. The management, retrieval and reuse of these medical images has always been a difficult problem.

目前在图像检索方面，基于内容的图像搜索(Content-Based Image Retrieval，CBIR)是一种常用的解决方案。CBIR通过将图像的视觉特征与用户输入的检索条件(如图片)的视觉特征做对比来进行检索。然而由于底层视觉特征与人类对视觉认知的高层语义之间存在“语义鸿沟”的问题，因此在医学领域，CBIR的检索结果往往并不理想。因此图像检索的主要方法依然需要基于图像的文本信息，图像的文本标签标注就显得至关重要。Currently, Content-Based Image Retrieval (CBIR) is a commonly used solution in image retrieval. CBIR performs retrieval by comparing the visual features of the image with the visual features of the retrieval conditions (such as pictures) entered by the user. However, due to the problem of "semantic gap" between the low-level visual features and the high-level semantics of human visual cognition, the retrieval results of CBIR are often not ideal in the medical field. Therefore, the main method of image retrieval still needs the text information based on the image, and the text label annotation of the image is very important.

目前的图像文本标签标注方式主要是人工标注，或者纯粹的自动标注。人工标注效率较低，人力成本高，而且完全依赖于标注操作人员的专业知识，长时间标注的标签质量也无法保证。而自动标注虽然效率高，但是目前还没有可以完全保证质量的标签推荐方法。The current image and text label labeling methods are mainly manual labeling, or purely automatic labeling. The manual labeling efficiency is low, the labor cost is high, and it is completely dependent on the professional knowledge of the labeling operator, and the quality of labels labelled for a long time cannot be guaranteed. Although automatic labeling is highly efficient, there is currently no label recommendation method that can fully guarantee the quality.

中国专利(CN 104462738 A)公开了一种标注医学图像的方法，，该方法包括：将未标注医学图像集划分为至少两个未标注医学图像子集；将所述至少两个未标注医学图像子集分配给至少两个标注终端，以供各标注终端对各自分配到的未标注医学图像子集中的医学图像进行标注；接收各标注终端上传的标注信息。虽然该专利能够使用户在任意时间和地点对医学图像进行协同标注，但是纯手动的标注完全依赖标注操作人员的专业知识，无法保证标签的质量，纯手动标签标注较慢，效率低下。因此，市场上需要一种半自动的医学图像标注方法来提高标注的质量和效率。Chinese patent (CN 104462738 A) discloses a method for labeling medical images, the method includes: dividing an unlabeled medical image set into at least two unlabeled medical image subsets; dividing the at least two unlabeled medical images The subset is allocated to at least two labeling terminals, so that each labeling terminal can label the medical images in the respectively assigned unlabeled medical image subsets; and the labeling information uploaded by each labeling terminal is received. Although this patent enables users to collaboratively label medical images at any time and place, the purely manual labeling completely relies on the expertise of the labeling operator and cannot guarantee the quality of the labels. Purely manual labeling is slow and inefficient. Therefore, a semi-automatic medical image annotation method is needed in the market to improve the quality and efficiency of annotation.

现有技术中，没有任何技术方案公开本发明的基于标签的标注权重变化来调节标签推荐顺序的方法。In the prior art, there is no technical solution that discloses the method of the present invention for adjusting the label recommendation order based on the change of the labeling weight of the label.

发明内容SUMMARY OF THE INVENTION

针对现有技术之不足，本发明提供一种医学图像标注方法，其特征在于，该方法包括：Aiming at the deficiencies of the prior art, the present invention provides a medical image labeling method, characterized in that the method includes:

基于待标注医学图像的关键字与标签本体库中的标签的匹配值向至少两个标注终端推荐至少两个可供标注操作人员选择的选择标签；Based on the matching value between the keyword of the medical image to be annotated and the tag in the tag ontology database, recommend at least two selection tags to the at least two tagging terminals for selection by the tagging operator;

由至少两个标注终端的标注操作人员基于所述选择标签对待标注医学图像分别独立进行标注；labeling the medical images to be labelled independently by labeling operators of at least two labeling terminals based on the selection label;

基于至少两个标注操作人员的标注内容的对比结果确认待标注医学图像的标注标签。The labeling label of the medical image to be labelled is confirmed based on the comparison result of labeling contents of at least two labeling operators.

根据一个优选实施方式，所述方法还包括：According to a preferred embodiment, the method further comprises:

分配待标注医学图像给至少两个标注操作人员独立进行标注，基于至少两个标注操作人员标注的标签的交集确认标注结果。Allocate the medical image to be labeled to at least two labeling operators to label independently, and confirm the labeling result based on the intersection of labels labelled by the at least two labeling operators.

根据一个优选实施方式，所述方法还包括：According to a preferred embodiment, the method further comprises:

分配待标注医学图像给至少两个标注操作人员独立进行标注，基于至少两个标注操作人员标注的标签的权值融合得到确认标注结果。Allocate the medical images to be labeled to at least two labeling operators to label them independently, and obtain a confirmation labeling result based on the fusion of the weights of the labels labelled by the at least two labeling operators.

根据一个优选实施方式，所述方法还包括：According to a preferred embodiment, the method further comprises:

分配待标注医学图像给至少两个标注操作人员独立进行标注，对比所述至少两个标注操作人员的标注标签，向所述至少两个标注操作人员分别显示差异较大的标注结果，并且由至少两个标注操作人员协商确认标注结果。Allocate the medical images to be labeled to at least two labeling operators to label independently, compare the labeling labels of the at least two labeling operators, and display the labeling results with large differences to the at least two labeling operators, respectively. The two annotation operators negotiate and confirm the annotation results.

根据一个优选实施方式，所述方法还包括：According to a preferred embodiment, the method further comprises:

基于来源于权威期刊与书籍中的医学影像，通过使用全文索引从所述权威期刊与书籍中的全文中自动找出与医学影像相关的语句，以便基于所述语句来生成至少两个供标注操作人员选择的标签。Based on medical images from authoritative journals and books, sentences related to medical images are automatically found from the full text in authoritative journals and books by using full-text indexing, so as to generate at least two annotation operations based on the sentences The label selected by the person.

根据一个优选实施方式，所述方法还包括：According to a preferred embodiment, the method further comprises:

所述标签以可供选择的按钮形式显示在标注操作人员的标注终端设备上。The label is displayed on the labeling terminal equipment of the labeling operator in the form of a selectable button.

根据一个优选实施方式，所述方法还包括：According to a preferred embodiment, the method further comprises:

所述标签以与医学图像相关语句相组合的方式显示在标注操作人员的终端设备上。The label is displayed on the terminal device of the labeling operator in a manner of being combined with a sentence related to the medical image.

根据一个优选实施方式，所述方法还包括：According to a preferred embodiment, the method further comprises:

将关键字和标签的匹配值进行排序，选择大于匹配阈值的至少两个标签作为推荐的选择标签。The matching values of keywords and tags are sorted, and at least two tags greater than the matching threshold are selected as the recommended selection tags.

根据一个优选实施方式，所述方法还包括：According to a preferred embodiment, the method further comprises:

基于公众标注人员的标注内容生成动态变动的标签队列，将小于顺序阈值的标签由专家确认后加入标签本体库；其中，Based on the annotation content of the public annotators, a dynamically changing tag queue is generated, and tags smaller than the sequence threshold are confirmed by experts and added to the tag ontology library; among them,

标签队列中的标签顺序基于的标注权重变化，The label order in the label queue changes based on the label weight,

公众标注人员的标注能力值基于标签队列中的标签顺序变化，The labeling ability value of public labelers changes based on the order of labels in the label queue,

标签加入标签本体库的数量相应增加对应的公众标注人员的标注能力值。The number of tags added to the tag ontology library will increase the corresponding public tagger's tagging ability value accordingly.

一种医学图像标注系统，其特征在于，所述标注系统包括导入医学图像的导入单元、存储医学图像及其关键字的第一存储服务器、存储标签的第二存储服务器、匹配单元、将待标注医学图像及相应的选择标签分配给至少两个标注终端的分配单元、确认至少两个标注操作人员标注结果的确认单元、至少两个标注终端；A medical image labeling system, characterized in that the labeling system comprises an import unit for importing medical images, a first storage server for storing medical images and their keywords, a second storage server for storing labels, a matching unit, a Medical images and corresponding selection labels are allocated to at least two allocation units for labeling terminals, a confirmation unit for confirming at least two labeling operators' labeling results, and at least two labeling terminals;

所述导入单元将待标注医学图像导入并存储至所述第一存储单元；The importing unit imports and stores the medical images to be marked into the first storage unit;

所述匹配单元提取所述第一存储服务器存储的待标注医学图像的关键字和所述第二存储单元存储的至少一个标签进行匹配，The matching unit extracts the keyword of the medical image to be marked stored in the first storage server and matches at least one label stored in the second storage unit,

所述分配单元基于待标注医学图像的关键字与标签本体库中的标签的匹配值向至少两个标注终端推荐至少两个可供标注操作人员选择的选择标签；The allocating unit recommends at least two selection labels that can be selected by the labeling operator to the at least two labeling terminals based on the matching value of the keyword of the medical image to be labelled and the label in the label ontology library;

所述确认单元基于至少两个标注操作人员的标注内容的对比结果确认待标注医学图像的标注标签。The confirmation unit confirms the labeling label of the medical image to be labelled based on the comparison result of labeling contents of at least two labeling operators.

根据本发明的一个独立方面，本发明公开了一种多人协同半自动医学图像标注方法，其特征在于，所述方法是通过将待标注医学图像分配给至少两个标注终端执行标注操作，所述标注操作是基于多人协同半自动医学图像系统推荐的至少两个可供所述标注终端的标注操作人员选择的标签来完成的，并由所述多人协同半自动医学图像系统对所述至少两个标注终端的标注操作人员独立完成的标注结果进行融合或比对以确定对所述待标注医学图像的标注标签。According to an independent aspect of the present invention, the present invention discloses a multi-person collaborative semi-automatic medical image labeling method, characterized in that the method is to perform labeling operations by allocating medical images to be labelled to at least two labeling terminals, the The labeling operation is completed based on at least two labels recommended by the multi-person collaborative semi-automatic medical image system that can be selected by the labeling operator of the labeling terminal, and the at least two labels are evaluated by the multi-person collaborative semi-automatic medical image system. The labeling results independently completed by the labeling operator of the labeling terminal are fused or compared to determine the labeling label of the medical image to be labelled.

根据一个优选实施方式，所述多人协同半自动医学图像系统通过如下方式确定对所述待标注医学图像的标注标签：According to a preferred embodiment, the multi-person collaborative semi-automatic medical image system determines the labeling label for the medical image to be labelled in the following manner:

所述待标注医学图像的标注标签为所述多人协同半自动医学图像系统对所述至少两个标注终端的标注结果所取的交集；对交集为空的所述待标注医学图像，由所述多人协同半自动医学图像系统将所述待标注医学图像重新发给所述至少两个标注终端执行标注操作，直至所述多人协同半自动医学图像系统确定所述待标注医学图像的标注标签；或者The labeling label of the medical image to be labelled is the intersection taken by the multi-person collaborative semi-automatic medical image system on the labeling results of the at least two labeling terminals; for the medical image to be labelled with an empty intersection set, the The multi-person collaborative semi-automatic medical image system re-sends the to-be-labeled medical image to the at least two labeling terminals to perform the labeling operation, until the multi-person collaborative semi-automatic medical image system determines the label label of the to-be-labeled medical image; or

对交集为空的所述待标注医学图像，由所述多人协同半自动医学图像系统将所述至少两个标注终端的标注结果同时显示给所述至少两个标注终端的标注操作人员，由所述至少两个标注操作人员协商后确定所述待标注医学图像的标注标签。For the medical images to be labeled with an empty intersection, the multi-person collaborative semi-automatic medical image system simultaneously displays the labeling results of the at least two labeling terminals to the labeling operators of the at least two labeling terminals, and the labeling results of the at least two labeling terminals are simultaneously displayed by the multi-person collaborative semi-automatic medical image system. The at least two labeling operators determine the labeling label of the medical image to be labelled after negotiation.

根据一个优选实施方式，所述多人协同半自动医学图像系统通过如下方式确定对所述待标注医学图像的标注标签：According to a preferred embodiment, the multi-person collaborative semi-automatic medical image system determines the labeling label for the medical image to be labelled in the following manner:

对所述至少两个标注终端的标注结果由所述多人协同半自动医学图像系统进行比对，对比对结果出现差距的标注，由所述多人协同半自动医学图像系统将所述至少两个标注终端的标注结果同时显示给所述至少两个标注终端的标注操作人员，由所述至少两个标注操作人员协商后确定所述待标注医学图像的标注标签。The labeling results of the at least two labeling terminals are compared by the multi-person collaborative semi-automatic medical image system, and the labels with discrepancies in the comparison results are compared, and the at least two labels are labelled by the multi-person collaborative semi-automatic medical image system. The labeling results of the terminals are simultaneously displayed to the labeling operators of the at least two labeling terminals, and the labeling labels of the medical images to be labelled are determined after negotiation by the at least two labeling operators.

根据一个优选实施方式，所述多人协同半自动医学图像系统通过如下方式为所述至少两个标注终端的标注操作人员推荐至少两个可供选择的标签：According to a preferred embodiment, the multi-person collaborative semi-automatic medical image system recommends at least two optional labels for the labeling operators of the at least two labeling terminals in the following manner:

所述多人协同半自动医学图像系统基于来源于权威期刊与书籍中的医学图像，通过使用全文索引从所述权威期刊与书籍中的全文中自动找出与医学图像相关的语句，以便基于所述相关的语句来生成至少两个可供标注操作人员选择的标签，并且，所述标签以可供选择的按钮形式显示在所述标注操作人员的标注终端上和/或所述标签以与所述相关语句组合的方式显示在标注操作人员的标注终端上。The multi-person collaborative semi-automatic medical image system is based on medical images from authoritative journals and books, and automatically finds sentences related to medical images from the full text of the authoritative journals and books by using full-text indexing, so as to based on the related sentences to generate at least two labels that can be selected by the labeling operator, and the labels are displayed on the labeling terminal of the labeling operator in the form of selectable buttons and/or the labels are associated with the labeling operator. The way the related sentences are combined is displayed on the annotation terminal of the annotation operator.

根据一个优选实施方式，将所述自动找出的与医学图像相关的语句提取关键字，并将所述关键字与标签本体库中的标签进行匹配，依据所述关键字与所述标签的匹配度来生成至少两个可供标注操作人员选择的标签。According to a preferred embodiment, keywords are extracted from the automatically found sentences related to medical images, and the keywords are matched with the tags in the tag ontology database, and the keywords are matched with the tags according to the matching of the keywords and the tags. degree to generate at least two labels that can be selected by the annotation operator.

根据一个优选实施方式，所述至少两个标注终端的标注操作人员基于所选择的标签以及与所述标签相关的语句在所述待标注医学图像中标出兴趣区的边界。According to a preferred embodiment, the labeling operators of the at least two labeling terminals mark the boundaries of the regions of interest in the medical images to be labelled based on the selected labels and sentences related to the labels.

根据一个优选实施方式，所述多人协同半自动医学图像标注系统将所述待标注医学图像分配给至少两个标注终端的分配方式为：According to a preferred embodiment, the multi-person collaborative semi-automatic medical image labeling system allocates the medical image to be labelled to at least two labeling terminals in the following manner:

基于预先设置的标注终端的优先级顺序将所述待标注医学图像分配给所述至少两个标注终端；或者Allocate the medical image to be annotated to the at least two annotation terminals based on a preset priority order of the annotation terminals; or

基于所述至少两个标注终端的终端处理能力顺序将所述待标注医学图像分配给所述至少两个标注终端；或者allocating the medical image to be annotated to the at least two annotating terminals in sequence based on the terminal processing capabilities of the at least two annotating terminals; or

基于负载均衡原理将所述待标注医学图像分配给所述至少两个标注终端。The to-be-labeled medical image is allocated to the at least two labeling terminals based on the load balancing principle.

根据一个优选实施方式，医学图像存储于第一服务器上，与所述医学图像匹配的标注存储于第二服务器，并将所述医学图像与所述标签间的匹配关系作为数据记录存储于所述第二服务器上；在用户提取已标注的医学图像时，由所述第一服务器和所述第二服务器分别并发发送数据后并由用户在本地根据来自所述第二服务器的数据记录将所述标签与所述医学图像进行匹配并在本地进行展示。According to a preferred embodiment, the medical image is stored on the first server, the label matching the medical image is stored on the second server, and the matching relationship between the medical image and the label is stored as a data record in the On the second server; when the user extracts the marked medical image, the first server and the second server send data concurrently, and the user locally records the medical image according to the data record from the second server. Labels are matched to the medical image and displayed locally.

根据一个优选实施方式，所述标注终端为功能手机、智能手机、掌上电脑、个人电脑、平板电脑或个人数字助理。According to a preferred embodiment, the labeling terminal is a feature phone, a smart phone, a palmtop computer, a personal computer, a tablet computer or a personal digital assistant.

根据一个优选实施方式，所述方法包括如下步骤：According to a preferred embodiment, the method comprises the steps of:

由所述多人协同半自动医学图像标注系统将待标注医学图像分配给至少两个标注终端执行标注操作；The multi-person collaborative semi-automatic medical image labeling system allocates the medical images to be labelled to at least two labeling terminals to perform labeling operations;

由所述多人协同半自动医学图像系统基于来源于权威期刊与书籍中的医学图像，并通过使用全文索引从所述权威期刊与书籍中的全文中自动找出与医学图像相关的语句，以便基于所述相关的语句来生成至少两个可供标注操作人员选择的标签，并由所述系统将所述标签显示给所述至少两个标注终端；The multi-person collaborative semi-automatic medical image system is based on medical images from authoritative journals and books, and automatically finds sentences related to medical images from the full text in the authoritative journals and books by using full-text indexing, so as to be based on generating at least two labels that can be selected by the labeling operator, and the system displays the labels to the at least two labeling terminals;

所述至少两个标注终端的标注操作人员基于所述系统推荐的标签独立完成所述待标注医学图像的标注；The labeling operators of the at least two labeling terminals independently complete the labeling of the medical image to be labelled based on the labels recommended by the system;

对所述至少两个标注终端的标注操作人员独立完成的标注结果由所述系统采用融合或比对的方式以确定对所述待标注医学图像的标注标签。The labeling results independently completed by the labeling operators of the at least two labeling terminals are fused or compared by the system to determine labeling labels for the medical images to be labelled.

根据本发明的另一个独立方面，本发明公开了一种多人协同半自动医学图像标注系统，其特征在于，所述系统基于医学图像的文本与标签本体库单元中的标签和/或文章进行匹配，将匹配的至少两个标签和/或相关语句自动推荐给至少两个标注终端。According to another independent aspect of the present invention, the present invention discloses a multi-person collaborative semi-automatic medical image labeling system, characterized in that the system matches the labels and/or articles in the label ontology library unit based on the text of the medical image , automatically recommending at least two matching tags and/or related sentences to at least two tagging terminals.

根据一个优选实施方式，所述标签本体库单元包括已标注标签单元和医学期刊与书籍数据单元，当所述标注终端操作人员将未标注的图像导入系统后，根据图像文本信息在所述已标注标签单元和/或医学期刊与书籍数据单元进行检索，并基于检索结果匹配分数，生成至少两个标签和/或相关语句。According to a preferred embodiment, the label ontology library unit includes a labelled label unit and a medical journal and book data unit. After the labeling terminal operator imports the unlabeled image into the system, the labelled label is stored in the labelled image according to the image text information. The tag unit and/or the medical journal and the book data unit are searched, and based on the search result match score, at least two tags and/or related sentences are generated.

根据一个优选实施方式，生成的所述至少两个标签以可选择按钮形式显示在所述操作人员的标注终端上。According to a preferred embodiment, the generated at least two labels are displayed on the operator's labeling terminal in the form of selectable buttons.

根据一个优选实施方式，生成的所述至少两个标签以与所述相关语句相组合的方式显示在所述操作人员的标注终端上。According to a preferred embodiment, the generated at least two labels are displayed on the operator's labeling terminal in a combined manner with the related sentences.

根据一个优选实施方式，所述系统还包括手动输入标签单元，所述手动输入标签单元基于所述至少两个标签和/或相关语句由所述操作人员手动输入准确的标签。According to a preferred embodiment, the system further comprises a manual input label unit for manually inputting an accurate label by the operator based on the at least two labels and/or related sentences.

根据一个优选实施方式，所述系统还包括分配单元和对比单元，所述分配单元用于将未标注的医学图像分配给至少两个标注终端操作人员，并由所述至少两个操作人员分别独立完成标注；所述对比单元用于将所述至少两个操作人员的标注结果进行对比分析，若针对同一医学图像对比结果显示不同，所述对比单元经对比分析后将至少两个操作人员的标注结果同时发送给所述至少两个操作人员，由所述至少两个操作人员协商确定准确的标签。According to a preferred embodiment, the system further comprises an assignment unit and a comparison unit, the assignment unit is used for assigning the unlabeled medical images to at least two labeling terminal operators, and the at least two operators are respectively independent Complete the labeling; the comparison unit is used to compare and analyze the labeling results of the at least two operators. If the comparison results for the same medical image are different, the comparison unit will label the at least two operators after the comparison and analysis. The results are sent to the at least two operators at the same time, and the at least two operators negotiate to determine the exact label.

根据一个优选实施方式，所述系统还包括高速远程服务器单元和标注内容服务器单元，医学图像存储于所述高速远程服务器单元，与所述医学图像相匹配的标签存储于所述标注内容服务器单元，所述医学图像和与其相匹配的所述标签之间的匹配关系数据记录存储于所述标注内容服务器单元。According to a preferred embodiment, the system further comprises a high-speed remote server unit and an annotation content server unit, where medical images are stored in the high-speed remote server unit, and tags matching the medical images are stored in the annotation content server unit, The matching relationship data record between the medical image and the label matched therewith is stored in the labeling content server unit.

根据一个优选实施方式，所述高速远程服务器单元和所述标注内容服务器单元接收到提取已标注的医学图像的命令后，分别从不同位置发出相关数据，并显示于标注终端，操作人员根据来自所述标注内容服务器的匹配关系数据记录，将所述标签与所述医学图像进行匹配并显示于所述标注终端。According to a preferred embodiment, after the high-speed remote server unit and the annotated content server unit receive the command to extract the annotated medical image, relevant data are respectively sent from different positions and displayed on the annotating terminal. The matching relationship data record of the labeling content server is matched, and the label is matched with the medical image and displayed on the labeling terminal.

根据一个优选实施方式，所述标注内容服务器单元具有加密系统。根据一个优选实施方式，所述系统还包括导入和导出单元，所述导入和导出单元用于对未标注的医学图像进行导入，以及用于将已标注的医学图像导出成本地文件。According to a preferred embodiment, the annotation content server unit has an encryption system. According to a preferred embodiment, the system further includes an import and export unit for importing unlabeled medical images and for exporting labeled medical images to local files.

根据本发明的又一个独立方面，本发明公开了一种医学图像标注方法，其特征在于，该方法的步骤包括：According to another independent aspect of the present invention, the present invention discloses a medical image labeling method, characterized in that the steps of the method include:

响应至少一个标注终端的请求，提取待标注医学图像的描述信息的关键字；In response to the request of at least one labeling terminal, extracting the keywords of the description information of the medical image to be labelled;

将所述关键字与至少一个标签本体库中的标签进行匹配；matching the keyword with tags in at least one tag ontology library;

向至少一个标注终端单独分配未标注医学图像；individually assigning unlabeled medical images to at least one labeling terminal;

基于关键字与基于至少一个标签的推荐值向相应标注终端的标注操作人员推荐至少一个选择标签；Recommend at least one selection tag to the tagging operator of the corresponding tagging terminal based on the keyword and the recommendation value based on the at least one tag;

以全文检索的方式检索与医学图像/或选择标签相关联的语句并向标注操作人员标记显示；Retrieve sentences associated with medical images and/or select labels in a full-text search manner and mark them for display by the labeling operator;

记录至少一个标注操作人员的标注信息并统计同一个医学图像的交集标签。Record the annotation information of at least one annotation operator and count the intersection labels of the same medical image.

根据一个优选实施方式，所述待标注医学图像储存于医学图像数据库，并根据其描述信息至少划分为至少两个未标注的医学图像子集，每个子集至少包括一个待标注医学图像。According to a preferred embodiment, the medical images to be labeled are stored in a medical image database, and are at least divided into at least two unlabeled medical image subsets according to their description information, and each subset includes at least one medical image to be labeled.

根据一个优选实施方式，所述未标注医学图像子集根据待标注医学图像描述信息，并基于生物解剖结构或者生物生理系统或则结合了生物解剖结构与生物生理系统进行划分。According to a preferred embodiment, the subset of unlabeled medical images is divided according to the description information of the medical images to be labeled and based on biological anatomy structure or biological physiological system or a combination of biological anatomical structure and biological physiological system.

根据一个优选实施方式，所述医学图像数据库包括未标注医学图像数据库和已标注医学图像数据库，所述已标注医学图像数据库基于已标注图像的标签或标注信息划分为至少两个已标注医学图像子集，每个子集至少包括一个待标注医学图像。According to a preferred embodiment, the medical image database includes an unlabeled medical image database and a labeled medical image database, and the labeled medical image database is divided into at least two labeled medical image subsections based on labels or labeling information of the labeled images. Sets, each subset includes at least one medical image to be labeled.

根据一个优选实施方式，所述已标注医学图像子集根据已标注医学图像的标签或标注信息，并基于生物解剖结构和/或生物生理系统进行划分。According to a preferred embodiment, the labeled medical image subsets are divided according to labels or labeling information of the labeled medical images and based on biological anatomy and/or biological physiological system.

根据一个优选实施方式，所述选择标签包括基于关键字与至少一个标签本体库中的标签进行匹配产生的选择标签和基于关键字与来源于权威期刊和书籍中的医学影像，通过使用全文索引从所述权威期刊和书籍中的全文中自动找出与关键字和医学影像相关的语句，生成至少两个可供标注操作人员选择的标签。According to a preferred embodiment, the selection tags include selection tags generated based on keywords matched with tags in at least one tag ontology database and based on keywords and medical images from authoritative journals and books, by using full-text indexing from Sentences related to keywords and medical images are automatically found in the full text of the authoritative journals and books, and at least two tags that can be selected by the annotation operator are generated.

根据一个优选实施方式，所述生成的至少两个可供标注操作人员选择的标签以可供选择的按钮形式显示在标注操作人员的标注终端上，同时，与所述标签相关的语句显示在标注操作人员的标注终端上，标注操作人员基于所选择的标签以及与所述标签相关的语句在待标注医学影像中标出兴趣区的边界。According to a preferred embodiment, the generated at least two labels that can be selected by the labeling operator are displayed on the labeling terminal of the labeling operator in the form of selectable buttons, and at the same time, the sentences related to the labels are displayed on the labeling terminal. On the operator's labeling terminal, the labeling operator marks the boundary of the region of interest in the medical image to be labelled based on the selected label and sentences related to the label.

根据一个优选实施方式，该方法还包括：主动向至少一个标注终端发送未标注医学图像，包括将有待标注的医学影像同时发给至少两个标注操作人员，由所述至少两个标注操作人员分别独立完成标注。According to a preferred embodiment, the method further includes: actively sending unlabeled medical images to at least one labeling terminal, including simultaneously sending the medical images to be labelled to at least two labeling operators, and the at least two labeling operators respectively Complete labeling independently.

根据一个优选实施方式，针对所述至少两个标注操作人员分别独立完成的标注，对其进行比对；如若比对结果差距很大，则将双方的标注内容同时显示给两人，由双方协商确定最为准确的标注标签。According to a preferred embodiment, the at least two labels completed independently by the at least two labeling operators are compared; if the comparison results are very different, the labeling contents of the two parties are displayed to the two at the same time, and the two parties negotiate. Determine the most accurate callout labels.

一种医疗系统可视化设备，其特征在于，所述可视化设备包括图像展示部、图像分析部和图像标注部，所述可视化设备与医学图像标注系统连接，A medical system visualization device, characterized in that the visualization device includes an image display part, an image analysis part and an image annotation part, and the visualization device is connected to a medical image annotation system,

可视化设备做为一个图像标注终端，向图像标注系统发出图像标注请求；As an image annotation terminal, the visualization device sends an image annotation request to the image annotation system;

图像标注系统基于所述标注请求的关键字与至少一个标签本体库中的标签进行匹配，并向所述可视化设备单独分配待标注医学图像；The image tagging system matches tags in at least one tag ontology library based on the keyword of the tagging request, and individually assigns the medical image to be tagged to the visualization device;

所述图像标注系统基于所述标注请求的关键字与至少一个标签的匹配值向可视化设备的标注操作人员推荐至少一个选择标签；The image tagging system recommends at least one selection tag to an annotation operator of the visualization device based on a matching value of the keyword of the tagging request and at least one tag;

所述图像标注系统通过所述可视化设备以全文检索的方式检索与选择标签相关联的语句并向标注操作人员标记显示；The image tagging system retrieves the sentences associated with the selected tags in a full-text retrieval manner through the visualization device, and marks and displays them to the tagging operator;

所述标注操作人员基于所述图像展示部展示的所述选择标签和所述选择标签相关联的语句，在图像标注部完成待标注医学图像的标注，所述图像标注部发送标注内容至所述图像标注系统；The labeling operator completes labeling of the medical image to be labelled in the image labeling unit based on the selection label displayed by the image display unit and the sentence associated with the selection label, and the image labeling unit sends labeling content to the image labeling unit. Image annotation system;

所述图像分析部记录至少一个标注操作人员的标注信息并统计同一个医学图像的交集标签。The image analysis unit records the labeling information of at least one labeling operator and counts the intersection labels of the same medical image.

本发明还提供另一种医疗系统可视化设备，所述医疗系统可视化设备与图像标注系统建立通信连接；所述医疗系统可视化设备至少包括成像部、图像展示部、图像分析部和图像标注部，所述图像标注系统基于标注请求的关键字与至少一个标签本体库中的标签进行匹配，并向标注操作人员单独分配待标注医学图像；所述成像部将医学图像标注部发送的图像信息转化为医学图像并在图像展示部呈现给标注操作人员，所述图像分析部记录至少一个标注操作人员的标注信息并统计同一个医学图像的交集标签；所述图像标注部在本地根据接收的医学图像、与医学图像匹配的标注内容和医学图像的匹配关系代码完成医学图像与标注的匹配。The present invention also provides another medical system visualization device, the medical system visualization device establishes a communication connection with an image annotation system; the medical system visualization device at least includes an imaging part, an image display part, an image analysis part and an image annotation part, so The image labeling system matches the label in at least one label ontology library based on the keyword of the labeling request, and assigns the medical image to be labelled separately to the labeling operator; the imaging unit converts the image information sent by the medical image labeling unit into medical images. The image is presented to the labeling operator in the image display part, and the image analysis part records the labeling information of at least one labeling operator and counts the intersection labels of the same medical image; the image labeling part locally according to the received medical image, and The annotation content of the medical image matching and the matching relationship code of the medical image complete the matching between the medical image and the annotation.

优选地，所述图像标注系统包括标注内容服务器单元，在待标注的医学图像初步标注后，所述标注内容服务器单元基于公众标注人员的标注内容生成动态变动的标签队列，将小于顺序阈值的标签由专家确认后加入标签本体库；所述标签队列中的标签顺序基于的标注权重变化，所述公众标注人员的标注能力值基于标签队列中的标签顺序变化。Preferably, the image labeling system includes a labeling content server unit. After the medical image to be labelled is preliminarily labelled, the labeling content server unit generates a dynamically changing label queue based on the labeling content of the public labelers, and labels the labels smaller than the order threshold. After being confirmed by experts, it is added to the tag ontology library; the tag order in the tag queue changes based on the tag weight, and the tagging ability value of the public tagger changes based on the tag order in the tag queue.

优选地，所述标注内容服务器单元基于标签在标签队列的动态变动情况对相应的公众标注人员的标注能力值进行评估；若标签的顺序不断向前变动，则其公众标注人员的标注能力值会增加；若标签的顺序不断向后变动，则其公众标注人员的标注能力值会减少。Preferably, the tagging content server unit evaluates the tagging ability value of the corresponding public taggers based on the dynamic changes of tags in the tag queue; if the order of tags keeps moving forward, the tagging ability value of the public taggers will be Increase; if the order of labels keeps changing backwards, the labeling ability value of its public labelers will decrease.

优选地，在所述标签被纳入标签实体库后，与标签对应的所述公众标注人员的标注能力值会增加，增加的分值由管理人员设定；所述标签实体库中纳入的标签越多，与标签对应的公众标注人员的标注能力值增加越多。Preferably, after the tag is included in the tag entity library, the tagging ability value of the public tagger corresponding to the tag will increase, and the increased score is set by the administrator; the more tags included in the tag entity library, the more The more the tagging ability value of the public tagger corresponding to the tag increases.

优选地，所述图像标注系统中的匹配单元基于标注操作人员的标注请求的关键字与至少一个标签的匹配值向可视化设备的标注操作人员推荐至少一个选择标签。Preferably, the matching unit in the image annotation system recommends at least one selection tag to the annotation operator of the visualization device based on a matching value between the keyword of the annotation operator's annotation request and the at least one tag.

优选地，标注操作人员基于所述图像展示部展示的选择标签和与选择标签相关联的语句完成待标注医学图像标注，并发送至图像标注系统。Preferably, the labeling operator completes labeling of the medical image to be labelled based on the selection label displayed by the image display unit and the sentence associated with the selection label, and sends the label to the image labeling system.

优选地，所述标注权重是基于公众标注人员的资质和标注历史对标注内容进行加权处理得到的。Preferably, the labeling weight is obtained by weighting the labeling content based on the qualifications and labeling history of public labelers.

本发明还提供一种医疗系统可视化设备的标签标注方法，所述方法至少包括：基于标注请求的关键字与至少一个标签本体库中的标签进行匹配，并向标注操作人员单独分配待标注医学图像；将医学图像标注部发送的图像信息转化为医学图像并在图像展示部呈现给标注操作人员，记录至少一个标注操作人员的标注信息并统计同一个医学图像的交集标签；在本地根据接收的医学图像、与医学图像匹配的标注内容和医学图像的匹配关系代码完成医学图像与标注的匹配。The present invention also provides a label labeling method for a medical system visualization device, the method at least comprising: matching a label in at least one label ontology library based on a keyword of a labeling request, and individually assigning a medical image to be labelled to the labeling operator ; Convert the image information sent by the medical image labeling unit into a medical image and present it to the labeling operator in the image display unit, record the labeling information of at least one labeling operator and count the intersection labels of the same medical image; The image, the annotation content matched with the medical image, and the matching relationship code of the medical image complete the matching between the medical image and the annotation.

所述方法还包括：在待标注的医学图像初步标注后，基于公众标注人员的标注内容生成动态变动的标签队列，将小于顺序阈值的标签由专家确认后加入标签本体库；所述标签队列中的标签顺序基于的标注权重变化，所述公众标注人员的标注能力值基于标签队列中的标签顺序变化。The method further includes: after the medical image to be labeled is initially labeled, generating a dynamically changing label queue based on the labeling content of the public labelers, and adding labels smaller than the sequence threshold to the label ontology library after being confirmed by experts; The tag order of the tag changes based on the tag weight, and the tagging ability value of the public taggers changes based on the tag order in the tag queue.

所述方法还包括：基于标签在标签队列的动态变动情况对相应的公众标注人员的标注能力值进行评估；若标签的顺序不断向前变动，则其公众标注人员的标注能力值会增加；若标签的顺序不断向后变动，则其公众标注人员的标注能力值会减少。The method further includes: evaluating the labeling ability value of the corresponding public labelers based on the dynamic changes of labels in the labeling queue; if the order of labels keeps changing forward, the labeling ability value of the public labelers will increase; If the order of the labels keeps changing backwards, the labeling ability value of its public labelers will decrease.

本发明的有益技术效果：Beneficial technical effects of the present invention:

首先，本发明可以为用户自动推荐标签，支持多个用户协同进行工作，这是最主要的功能。First of all, the present invention can automatically recommend tags for users, and support multiple users to work collaboratively, which is the most important function.

其次，本发明提供了用户管理功能，管理员可以使用管理工具轻松管理标注用户的信息，并为标注者分配需要标注的图像。Secondly, the present invention provides a user management function, the administrator can use the management tool to easily manage the information of the annotated users, and assign the annotated images to the annotators.

另外，本发明提供了数据导入导出功能。标注者可以登录上传自己的图片，并由管理员负责分配图片。除了将数据保存在系统数据库中，用户也可以将指定数据导出成本地文件，本地文件支持.csv和.xml两种格式。In addition, the present invention provides data import and export functions. Annotators can log in to upload their own pictures, and the administrator is responsible for assigning pictures. In addition to saving the data in the system database, the user can also export the specified data to a local file. The local file supports both .csv and .xml formats.

最后，由于图片都是来源于一些文章，所以本发明还支持查看图像关联的文章，并自动找出和标签相关的句子，高亮显示。Finally, since the pictures are all derived from some articles, the present invention also supports viewing articles associated with images, and automatically finds and highlights sentences related to tags.

附图说明Description of drawings

图1是本发明优选的一种医学图像标注方法的示意图；1 is a schematic diagram of a preferred medical image labeling method of the present invention;

图2是本发明的一种多人协同半自动医学图像标注方法的示意图；2 is a schematic diagram of a multi-person collaborative semi-automatic medical image labeling method of the present invention;

图3是本发明优选的另一种医学图像标注方法的示意图；3 is a schematic diagram of another preferred medical image labeling method of the present invention;

图4是本发明优选的一种医学图像标注系统的示意图；4 is a schematic diagram of a preferred medical image labeling system of the present invention;

图5是本发明的一种多人协同半自动医学图像标注系统的示意图；5 is a schematic diagram of a multi-person collaborative semi-automatic medical image labeling system of the present invention;

图6是本发明的一种医疗系统可视化设备的示意图；和Figure 6 is a schematic diagram of a medical system visualization device of the present invention; and

图7是本发明的医学图像标注系统的构架示意图。FIG. 7 is a schematic structural diagram of the medical image labeling system of the present invention.

具体实施方式Detailed ways

下面结合附图进行详细说明。The following detailed description is given in conjunction with the accompanying drawings.

在本发明中，医学图片也称医学影像，是指为了医疗或医学研究，对动物躯体、人体或人体某部分，以非侵入方式取得内部组织的图片或影像。In the present invention, a medical picture is also called a medical image, which refers to a non-invasive way to obtain a picture or image of an internal tissue of an animal body, human body or a certain part of the human body for medical treatment or medical research.

实施例一Example 1

本实施例提供一种医学图像标注方法，其特征在于，该方法包括基于待标注医学图像的关键字与标签本体库中的标签的匹配值向至少两个标注终端推荐至少两个选择标签，由至少两个标注终端的标注操作人员基于标注操作人员选择标签对待标注医学图像分别独立进行标注。或者，将关键字与至少一个标签的匹配值排序，按照一定的规律选择至少两个标签，发送至对应的标注终端并显示为可供标注操作人员选择的选择标签。This embodiment provides a medical image labeling method, characterized in that the method includes recommending at least two selection labels to at least two labeling terminals based on a matching value between a keyword of a medical image to be labelled and a label in a label ontology library, and the The labeling operators of at least two labeling terminals independently label the medical images to be labelled based on the labels selected by the labeling operator. Or, sort the matching value of the keyword and at least one tag, select at least two tags according to a certain rule, send them to the corresponding tagging terminal, and display them as selection tags that can be selected by the tagging operator.

如图1所示，至少一个标注操作人员在至少一个标注终端输入标注请求。响应标注操作人员的标注要求，向至少两个标注终端分配至少一个待标注医学图像，使得标注操作人员在标注终端独立进行标注。优选地，基于预先设置的标注终端优先级顺序将待标注医学图像分配给至少两个标注终端。或者，基于标注终端的终端处理能力顺序将未标注医学图像子集分配给至少两个标注终端。或者，基于负载均衡原理将未标注医学图像子集分配给至少两个标注终端。或者，同时基于标注终端优先级顺序，终端处理能力顺序和负载均衡原理将未标注医学图像子集分配给至少两个标注终端。As shown in FIG. 1 , at least one labeling operator inputs a labeling request at at least one labeling terminal. In response to the labeling request of the labeling operator, at least one medical image to be labelled is allocated to at least two labeling terminals, so that the labeling operator can label the labeling terminal independently. Preferably, the medical images to be annotated are allocated to at least two annotating terminals based on a preset priority order of the annotating terminals. Alternatively, the subset of unlabeled medical images is sequentially assigned to at least two labeling terminals based on the terminal processing capabilities of the labeling terminals. Alternatively, the subset of unlabeled medical images is allocated to at least two labeling terminals based on the load balancing principle. Alternatively, the subset of unlabeled medical images is allocated to at least two labeling terminals based on the priority order of labeling terminals, the order of terminal processing capabilities, and the load balancing principle at the same time.

根据一个优选实施方式，基于预先设置的标注终端优先级顺序将未标注医学图像子集分配给多个标注终端。比如，假设有三个标注终端，分别为标注终端1、标注终端2和标注终端3，每个标注终端可以处理的医学图像为10张。标注终端1的优先级最高，标注终端2的优先级次之，标注终端3的优先级最低。假设需要分配的未标注医学图像子集一共有两个，分别为未标注医学图像子集1和未标注医学图像子集2，每个未标注医学图像子集中包含有10张图像。那么，任务分配结果可以为：向标注终端1发送未标注医学图像子集1(或未标注医学图像子集2)中的10张图像，向标注终端1发送剩下的未标注医学图像子集中的10张图像，而不向标注终端3发送未标注图像。According to a preferred embodiment, a subset of unlabeled medical images is allocated to a plurality of labeling terminals based on a preset priority order of labeling terminals. For example, suppose there are three labeling terminals, namely labeling terminal 1, labeling terminal 2, and labeling terminal 3, and each labeling terminal can process 10 medical images. The priority of marking the terminal 1 is the highest, the priority of marking the terminal 2 is the second, and the priority of marking the terminal 3 is the lowest. It is assumed that there are two unlabeled medical image subsets to be allocated, namely unlabeled medical image subset 1 and unlabeled medical image subset 2, and each unlabeled medical image subset contains 10 images. Then, the task assignment result can be: sending 10 images in the unlabeled medical image subset 1 (or unlabeled medical image subset 2) to the labeling terminal 1, and sending the remaining unlabeled medical image subsets to the labeling terminal 1 10 images, without sending unlabeled images to the labeling terminal 3.

根据一个优选实施方式，基于标注终端的终端处理能力顺序将未标注医学图像子集分配给标注终端。比如，假设有三个标注终端，分别为标注终端1、标注终端2和标注终端3。标注终端1可以处理的医学图像为10张，标注终端2可以处理的医学图像为10张，标注终端3可以处理的医学图像为5张。假设需要分配的未标注医学图像子集一共有两个，分别为未标注医学图像子集1和未标注医学图像子集2，每个未标注医学图像子集中包含有10张图像。那么，任务分配结果可以为：向标注终端1发送未标注医学图像子集1(或未标注医学图像子集2)中的10张图像，向标注终端1发送剩下的未标注医学图像子集中的10张图像，而不向标注终端3发送未标注图像。According to a preferred embodiment, the unlabeled medical image subsets are assigned to the labeling terminals in order based on the terminal processing capabilities of the labeling terminals. For example, it is assumed that there are three labeling terminals, namely labeling terminal 1, labeling terminal 2, and labeling terminal 3. The number of medical images that can be processed by the labeling terminal 1 is 10, the number of medical images that can be processed by the labeling terminal 2 is 10, and the number of medical images that can be processed by the labeling terminal 3 is 5. It is assumed that there are two unlabeled medical image subsets to be allocated, namely unlabeled medical image subset 1 and unlabeled medical image subset 2, and each unlabeled medical image subset contains 10 images. Then, the task assignment result can be: sending 10 images in the unlabeled medical image subset 1 (or unlabeled medical image subset 2) to the labeling terminal 1, and sending the remaining unlabeled medical image subsets to the labeling terminal 1 10 images, without sending unlabeled images to the labeling terminal 3.

从待标注医学图像的描述文字中提取关键字，并且将关键字与标签本体库中至少一个标签进行匹配。记录关键字与至少一个标签的匹配值。将匹配值大于预设的匹配阈值的至少两个标签推荐至相应的标注终端，供标注操作人员选择。标签以可供选择的按钮形式显示在标注操作人员的标注终端上。或者，标签以与医学图像相关语句相组合的方式显示在标注操作人员的标注终端上Keywords are extracted from the description text of the medical image to be labeled, and the keywords are matched with at least one label in the label ontology library. Records the matching value of the keyword with at least one tag. At least two tags whose matching value is greater than the preset matching threshold are recommended to the corresponding tagging terminal for selection by the tagging operator. Labels are displayed on the labeling operator's labeling terminal as a selectable button. Alternatively, the label is displayed on the labeling terminal of the labeling operator in combination with a sentence related to the medical image

或者，基于来源于权威期刊与书籍中的医学图像，通过使用全文索引从权威期刊与书籍中的全文中自动找出与医学图像相关的语句。将相关语句生成可供标注操作人员选择的标签，由标注操作人员选择进行标注。Or, based on medical images sourced from authoritative journals and books, automatically find sentences related to medical images from the full text of authoritative journals and books by using full-text indexing. The relevant sentences are generated into labels that can be selected by the labeling operator, and the labeling operator chooses to label.

在至少两个标注操作人员标注相同的医学图像后，对比至少两个标注操作人员的标注内容。基于至少两个标注内容的交集确认标注结果。After at least two labeling operators label the same medical image, the labeling contents of the at least two labeling operators are compared. The annotation results are confirmed based on the intersection of at least two annotation contents.

若至少两个标注操作人员的标注内容差异较大，在标注终端分别向标注操作人员显示其他标注操作人员的标注内容或选择的标签。请求标注操作人员重新对待标注医学图像进行标注。或者，在至少两个标注操作人员的标注内容差异较大的情况下，为标注同一幅医学图像的至少两个标注操作人员建立通讯连接或者即时通讯连接。由至少两个标注操作人员通过协商的方式确认最终的标注标签。If the labeling contents of at least two labeling operators are quite different, the labeling contents or selected labels of the other labeling operators are respectively displayed to the labeling operators on the labeling terminal. Request the annotation operator to re-annotate the medical image to be annotated. Or, in the case that the labeling contents of the at least two labeling operators are quite different, a communication connection or an instant communication connection is established for the at least two labeling operators labeling the same medical image. The final annotation label is confirmed by negotiation between at least two annotation operators.

或者，在至少两个标注操作人员标注相同的医学图像后，基于至少两个标注操作人员选择的标签的权值融合来确认标注结果。从而得到最终的标注标签。Alternatively, after at least two labeling operators label the same medical image, the labeling result is confirmed based on weight fusion of labels selected by at least two labeling operators. So as to get the final label.

实施例二Embodiment 2

本实施例提供一种医学图像标注方法，其特征在于，该方法包括基于待标注医学图像的关键字与标签本体库中的标签的匹配值向至少两个标注终端推荐至少两个选择标签，由至少两个标注终端的标注操作人员基于标注操作人员选择标签对待标注医学图像分别独立进行标注。This embodiment provides a medical image labeling method, characterized in that the method includes recommending at least two selection labels to at least two labeling terminals based on a matching value between a keyword of a medical image to be labelled and a label in a label ontology library, and the The labeling operators of at least two labeling terminals independently label the medical images to be labelled based on the labels selected by the labeling operator.

首先，将大量与已标注医学图像、相关的关键字和标注的标签分类存储，作为样本。将已标注医学图像、关键字和标签分别建立映射关系，并将映射关系存储在数据库中。First, a large number of annotated medical images, related keywords and annotated labels are classified and stored as samples. A mapping relationship is established for the labeled medical images, keywords and labels, and the mapping relationship is stored in the database.

然后，分别计算数据库中每张已标注医学图像与待标注医学图像的图像相似度。其中，图像相似度主要是对于两幅图片的图像内容的相似程度进行计算，得出一个图像相似度值，该图像相似度值越高，说明这两幅图片的内容越相似。图像相似度可以通过两幅图片的视觉特征来计算，视觉特征具体可以为颜色RGB(Red Green Blue，三原色)特征、纹理特征和直方图特征和SIFT(Scale-invariant feature transform，尺度不变特征转换)特征等。Then, the image similarity between each labeled medical image in the database and the medical image to be labeled is calculated separately. Among them, the image similarity mainly calculates the similarity of the image content of the two pictures to obtain an image similarity value. The higher the image similarity value is, the more similar the content of the two pictures is. The image similarity can be calculated by the visual features of the two pictures. The visual features can be color RGB (Red Green Blue, three primary colors) features, texture features, histogram features, and SIFT (Scale-invariant feature transform, scale-invariant feature transform). ) features, etc.

选择与待标注医学图像的图像相似度大于第一阈值的已标注医学图像组成图片组。其中，第一阈值为预先设定的相似度值。具体的，第一阈值可以由标注操作人员来自行设定。相对而言，将第一阈值设定的越高，在数据库中找到的已标注图像与待标注图像就越相似，但是找到的已标注图片的数量也会相对较少。The labeled medical images whose image similarity with the medical image to be labeled is greater than the first threshold are selected to form a picture group. The first threshold is a preset similarity value. Specifically, the first threshold may be set by the labeling operator. Relatively speaking, the higher the first threshold is set, the more similar the marked images found in the database are to the images to be marked, but the number of marked images found will be relatively small.

提取医学图像组中每张已标注图像对应的标签组成标签词组。对应的关键字组成关键词组。根据已标注图像的映射关系提取关键字和标签。如果在数据库中存储打印有标签的已标注图像，那么先识别已标注图像的描述文字的关键字，然后后再提取该标签。The label corresponding to each labeled image in the medical image group is extracted to form a label phrase. The corresponding keywords form a keyword group. Extract keywords and tags based on the mapping relationship of annotated images. If an annotated image printed with a label is stored in the database, the keyword of the descriptive text of the annotated image is identified first, and then the label is extracted.

输出标签词组中的至少一个标签作为待标注图片的选择标签。由于标签词组中的标签可能有很多个，但是用户可能不希望输出过多的标签，仅希望输出预设数量的标签，所以可通过下述步骤来实现：判断标签词组中的标签的数量是否大于第三阈值。当标签词组中的标签的数量大于第三阈值时，则输出标签词组中预设数量的标签作为待标注图片的选择标签，预设数量小于等于第三阈值。具体的，预设数量和第三阈值为标注操作人员自行设定的标签数量。At least one tag in the output tag phrase is used as the selection tag of the image to be tagged. Since there may be many tags in the tag phrase, the user may not want to output too many tags, but only want to output a preset number of tags, so it can be achieved by the following steps: judging whether the number of tags in the tag phrase is greater than third threshold. When the number of tags in the tag phrase is greater than the third threshold, a preset number of tags in the tag phrase are output as selection tags of the image to be tagged, and the preset number is less than or equal to the third threshold. Specifically, the preset number and the third threshold are the number of labels set by the labeling operator.

将对应的至少一个标签以可选择按钮的形式显示在标注终端上。The corresponding at least one label is displayed on the labeling terminal in the form of a selectable button.

根据一个优选实施方式，从待标注医学图像的描述文字中提取关键字的方法包括：对文字信息进行分词以获取至少一个分词，并获取至少一个分词的语义内容和语义类型。语义内容是分词对应的具有含义的语义信息，语义类型是语义信息的类型，例如，分词的词性、分词所表示的意义等。According to a preferred embodiment, the method for extracting keywords from the description text of a medical image to be marked includes: segmenting the text information to obtain at least one segment, and acquiring the semantic content and semantic type of the at least one segment. Semantic content is the semantic information with meaning corresponding to the participle, and the semantic type is the type of semantic information, for example, the part of speech of the participle, the meaning represented by the participle, and so on.

根据语义内容和语义类型对至少一个分词在对应的关键词组进行筛选，以筛选出与待标注医学图像相关的关键字。与待标注医学图像相似度极高的已标注医学图像对应多个关键字。选择关键词组中与描述文字中分词语义相似度最高的关键词作为待标注医学图像的关键词。语义相似度主要是对于两个词语的语义的相似程度进行计算，得出一个语义相似度值，该语义相似度值越高，说明这两个词语的语义越相似。第二阈值具体可以为用户预先设定的语义相似度值。依据关键词和标签词组的映射关系得到至少一个标签。According to the semantic content and the semantic type, at least one segmented word is filtered in the corresponding keyword group, so as to filter out keywords related to the medical image to be marked. Labeled medical images that are highly similar to the medical images to be labelled correspond to multiple keywords. The keyword with the highest semantic similarity with the word segmentation in the description text in the keyword group is selected as the keyword of the medical image to be annotated. The semantic similarity is mainly to calculate the semantic similarity of two words to obtain a semantic similarity value. The higher the semantic similarity value is, the more similar the semantics of the two words are. Specifically, the second threshold may be a semantic similarity value preset by the user. At least one tag is obtained according to the mapping relationship between the keyword and the tag phrase.

类似的，标签可以为与待标注医学图像相关的相关语句。相关语句通过对权威期刊或书籍进行全文索引得到。若已标注医学图像来源于权威期刊与书籍，则将权威期刊和书籍的文章建立为相关语句组。对与待标注医学图像的描述文字，通过使用全文索引从权威期刊与书籍中的全文中自动找出与医学图像相关的语句作为推荐的标签。相关语句以可选择的按钮形式显示在标注操作人员的标注终端。Similarly, the tags can be related sentences related to the medical image to be annotated. Relevant sentences are obtained by full-text indexing of authoritative journals or books. If the annotated medical images come from authoritative journals and books, the articles of authoritative journals and books are established as related sentence groups. For the description text of the medical image to be labeled, the sentence related to the medical image is automatically found from the full text of authoritative journals and books by using the full-text index as the recommended label. Relevant sentences are displayed on the annotation terminal of the annotation operator in the form of selectable buttons.

实施例三Embodiment 3

本实施例提供一种多人协同半自动医学图像标注方法。通过将待标注医学图像分配给至少两个标注终端执行标注操作，标注操作是基于多人协同半自动医学图像系统推荐的至少两个可供标注终端的标注操作人员选择的标签来完成的，并由多人协同半自动医学图像系统对至少两个标注终端的标注操作人员独立完成的标注结果进行融合或比对以确定对待标注医学图像的标注标签。This embodiment provides a multi-person collaborative semi-automatic medical image labeling method. The labeling operation is performed by assigning the medical images to be labelled to at least two labeling terminals. The labeling operation is based on at least two labels recommended by the multi-person collaborative semi-automatic medical image system that can be selected by the labeling operator of the labeling terminal, and is completed by The multi-person collaborative semi-automatic medical image system fuses or compares the labeling results independently completed by the labeling operators of at least two labeling terminals to determine the labeling label of the medical image to be labelled.

如图2所示，标注操作人员在标注终端发出标注请求。响应标注终端的标注请求，向至少两个标注终端分配至少一个待标注医学图像。分配方式包括：基于预先设置的标注终端的优先级顺序将待标注医学图像分配给标注操作人员至少两个标注终端；或者，基于至少两个标注终端的终端处理能力顺序将待标注医学图像分配给至少两个标注终端；或者，基于负载均衡原理将待标注医学图像分配给至少两个标注终端。As shown in Figure 2, the labeling operator sends a labeling request at the labeling terminal. In response to the labeling request of the labeling terminal, at least one medical image to be labelled is allocated to at least two labeling terminals. The allocation method includes: allocating the medical images to be labeled to at least two labeling terminals based on the preset order of the labeling terminals; At least two labeling terminals; or, based on the load balancing principle, the medical images to be labelled are allocated to at least two labeling terminals.

基于来源于权威期刊与书籍中的医学图像，通过使用全文索引从权威期刊与书籍中的全文中自动找出与医学图像相关的语句，以便基于相关的语句来生成至少两个可供标注操作人员选择的标签，并且，标签以可供选择的按钮形式显示在标注操作人员的标注终端上和/或标签以与标注操作人员相关语句组合的方式显示在标注操作人员的标注终端上。Based on medical images from authoritative journals and books, automatically find out sentences related to medical images from the full text in authoritative journals and books by using full-text indexing, so as to generate at least two labels for operators based on the relevant sentences. The selected label, and the label is displayed on the labeling operator's labeling terminal in the form of a selectable button and/or the label is displayed on the labeling operator's labeling terminal in the form of a combination of sentences related to the labeling operator.

或者，将自动找出的与医学图像相关的语句提取关键字，并将关键字与标签本体库中的标签进行匹配，依据关键字与标签的匹配度来生成至少两个可供标注操作人员选择的标签。Or, extract keywords from automatically found sentences related to medical images, match the keywords with the tags in the tag ontology library, and generate at least two options for the annotation operator to select according to the matching degree between the keywords and the tags. Tag of.

待标注医学图像的标注标签为多人协同半自动医学图像系统对至少两个标注终端的标注结果所取的交集。对交集为空的待标注医学图像，由多人协同半自动医学图像系统将待标注医学图像重新发给至少两个标注终端执行标注操作，直至多人协同半自动医学图像系统确定待标注医学图像的标注标签。The labeling label of the medical image to be labelled is the intersection taken by the multi-person collaborative semi-automatic medical image system on the labeling results of at least two labeling terminals. For the medical images to be labeled whose intersection is empty, the multi-person collaborative semi-automatic medical image system re-sends the medical images to be labeled to at least two labeling terminals to perform labeling operations, until the multi-person collaborative semi-automatic medical image system determines the medical images to be labeled. Label.

或者，对交集为空的待标注医学图像，由多人协同半自动医学图像系统将至少两个标注终端的标注结果同时显示给至少两个标注终端的标注操作人员，由至少两个标注操作人员协商后确定待标注医学图像的标注标签。Or, for the medical images to be labeled whose intersection is empty, the multi-person collaborative semi-automatic medical image system simultaneously displays the labeling results of at least two labeling terminals to the labeling operators of at least two labeling terminals, and the at least two labeling operators negotiate. Then, the labeling label of the medical image to be labelled is determined.

或者，对至少两个标注终端的标注结果由多人协同半自动医学图像系统进行比对，对比对结果出现差距的标注，由多人协同半自动医学图像系统将至少两个标注终端的标注结果同时显示给至少两个标注终端的标注操作人员，由至少两个标注操作人员协商后确定待标注医学图像的标注标签。Alternatively, the labeling results of at least two labeling terminals are compared by a multi-person collaborative semi-automatic medical image system, and the labels with discrepancies in the comparison results are compared, and the labeling results of at least two labeling terminals are displayed simultaneously by the multi-person collaborative semi-automatic medical image system. For the labeling operators of at least two labeling terminals, the labeling labels of the medical images to be labelled are determined after negotiation between the at least two labeling operators.

至少两个标注终端的标注操作人员基于所选择的标签以及与标签相关的语句在待标注医学图像中标出兴趣区(ROI)的边界。The labeling operators of at least two labeling terminals mark the boundaries of regions of interest (ROI) in the medical image to be labelled based on the selected labels and sentences related to the labels.

根据一个优选实施方式，医学图像存储于第一服务器上。与医学图像匹配的标签存储于第二服务器。医学图像与标签间的匹配关系作为数据记录存储于第二服务器上。在用户提取已标注的医学图像时，由第一服务器和第二服务器分别并发发送数据后并由用户在本地根据来自第二服务器的数据记录将标签与医学图像进行匹配并在本地进行展示。According to a preferred embodiment, the medical images are stored on the first server. The tags matching the medical images are stored on the second server. The matching relationship between the medical image and the label is stored on the second server as a data record. When the user extracts the labeled medical image, the first server and the second server respectively send data concurrently, and the user locally matches the label with the medical image according to the data record from the second server and displays it locally.

实施例四Embodiment 4

本实施例提供一种医学图像标注方法。该方法的步骤包括；This embodiment provides a medical image labeling method. The steps of the method include;

如图3所示，响应至少一个标注终端的请求，提取待标注医学图像的描述信息的关键字。将关键字与至少一个标签本体库中的标签进行匹配。向至少一个标注终端单独分配未标注医学图像。基于关键字与基于至少一个标签的匹配值向相应标注终端的标注操作人员推荐至少一个选择标签。以全文检索的方式检索与医学图像/或选择标签相关联的语句并向标注操作人员标记显示。记录至少一个标注操作人员的标注信息并统计同一个医学图像的交集。As shown in FIG. 3 , in response to a request from at least one labeling terminal, a keyword of the description information of the medical image to be labelled is extracted. Match keywords with tags in at least one tag ontology library. The unlabeled medical images are individually assigned to at least one labeling terminal. At least one selection tag is recommended to the tagging operator of the corresponding tagging terminal based on the matching value based on the keyword and based on the at least one tag. The sentences associated with the medical images and/or selection labels are retrieved in a full-text search manner and marked for display by the labeling operator. Record the annotation information of at least one annotation operator and count the intersection of the same medical image.

具体地，一种医学图像标注方法的步骤包括：Specifically, the steps of a medical image labeling method include:

S01：响应至少一个标注终端的请求，提取待标注医学图像的描述信息的关键字。S01: In response to a request of at least one labeling terminal, extract keywords of description information of the medical image to be labelled.

至少一个标注操作人员在标注终端发出标注请求。响应至少一个标注终端的请求，提取待标注医学图像的描述信息中的关键字。待标注医学图像附带有描述文字，描述文字中包含有关键字。提取描述文字中的关键字。At least one labeling operator issues a labeling request at the labeling terminal. In response to a request from at least one labeling terminal, keywords in the description information of the medical image to be labelled are extracted. The medical image to be labeled is accompanied by a description text, and the description text contains keywords. Extract keywords from description text.

S02：将关键字与至少一个标签本体库中的标签进行匹配。S02: Match the keyword with the tags in at least one tag ontology library.

S03：基于关键字与基于至少一个标签的匹配值向相应标注终端的标注操作人员推荐至少一个选择标签。S03: Recommend at least one selection tag to the tagging operator of the corresponding tagging terminal based on the matching value based on the keyword and the at least one tag.

将关键字与至少一个标签本体库中的标签进行匹配并计算匹配值。根据匹配值的大小顺序，向相应标注终端的标注操作人员推荐至少一个可供标注操作人员选择的选择标签。Match keywords with tags in at least one tag ontology library and calculate the matching value. According to the magnitude order of the matching values, at least one selection label that can be selected by the labeling operator is recommended to the labeling operator of the corresponding labeling terminal.

主动向至少一个标注终端发送未标注医学图像，包括将有待标注的医学图像同时发给至少两个标注操作人员。由至少两个标注操作人员分别独立完成标注。供标注操作人员选择的选择标签与对应的待标注医学图像同时显示在标注终端。同时，标注终端还显示有手动标注输入栏。当标注操作人员对显示的选择标签均不满意时，可以在手动标注输入栏输入人工标注。Actively sending unlabeled medical images to at least one labeling terminal includes simultaneously sending the medical images to be labelled to at least two labeling operators. Annotation is completed independently by at least two annotation operators. The selection label selected by the labeling operator and the corresponding medical image to be labelled are simultaneously displayed on the labeling terminal. At the same time, the annotation terminal also displays a manual annotation input field. When the labeling operator is not satisfied with the displayed selection labels, they can input manual labeling in the manual labeling input field.

S04：以全文检索的方式检索与医学图像/或选择标签相关联的语句并向标注操作人员标记显示。S04: Retrieve sentences associated with medical images/or selection labels by full-text search, and mark and display them to the labeling operator.

基于来源于权威期刊与书籍中的医学图像，通过使用全文索引从权威期刊与书籍中的全文中自动找出与医学图像相关的语句，以便基于语句来生成至少两个可供标注操作人员选择的标签。Based on the medical images from authoritative journals and books, automatically find sentences related to medical images from the full text of authoritative journals and books by using full-text indexing, so as to generate at least two sentences based on sentences that can be selected by the annotation operator. Label.

S05：记录至少一个标注操作人员的标注信息并统计同一个医学图像的交集。S05: Record the labeling information of at least one labeling operator and count the intersection of the same medical image.

记录至少一个标注操作人员的标注信息。统计至少一个标注操作人员对同一个医学图像标注的标签的交集。将交集的标签作为待标注医学图像最终的标注标签。Record the annotation information of at least one annotation operator. Count the intersection of labels annotated by at least one annotation operator on the same medical image. The label of the intersection is used as the final label of the medical image to be labelled.

根据一个优选实施方式，待标注医学图像储存于医学图像数据库，并根据待标注医学图像描述信息至少划分为至少两个未标注的医学图像子集，每个子集至少包括一个待标注医学图像。According to a preferred embodiment, the medical images to be labeled are stored in a medical image database, and are divided into at least two unlabeled medical image subsets according to the description information of the medical images to be labeled, and each subset includes at least one medical image to be labeled.

医学图像数据库中存储有未标注医学图像集和已标注医学图像集。已标注医学图像集中包含已经被标注的医学图像。未标注医学图像集中包含尚未被标注的医学图像。医学图像数据库可以具有中心结构或者分布式结构。而且，医学图像数据库的存储容量还可以随着医学图像数目的增多进行相应扩展。The medical image database stores unlabeled medical image sets and labeled medical image sets. The labeled medical image set contains medical images that have been labeled. The unlabeled medical image set contains medical images that have not been labeled. The medical image database may have a central structure or a distributed structure. Moreover, the storage capacity of the medical image database can also be expanded correspondingly as the number of medical images increases.

当收到对医学图像数据库中未标注医学图像集进行标注的任务之后，可以将未标注医学图像集划分为多个(至少两个)未标注医学图像子集，每个未标注医学图像子集可以包括一幅或多幅医学图像。After receiving the task of labeling the unlabeled medical image set in the medical image database, the unlabeled medical image set can be divided into multiple (at least two) unlabeled medical image subsets, each unlabeled medical image subset One or more medical images may be included.

根据一个优选实施方式，基于生物(例如人体)解剖结构划分未标注医学图像子集。比如，可以按照人体的解剖结构，将未标注医学图像集具体划分为：脑部、胸、心脏、腹部、上肢、下肢等图像子集。According to a preferred embodiment, the subset of unlabeled medical images is divided based on biological (eg human) anatomy. For example, according to the anatomical structure of the human body, the unlabeled medical image set can be specifically divided into image subsets such as brain, chest, heart, abdomen, upper limbs, and lower limbs.

根据一个优选实施方式，按照生物生理系统结构划分未标注医学图像子集。比如，将未标注医学图像集具体划分为消化系统、神经系统、运动系统、内分泌系统、泌尿系统、生殖系统、循环系统、呼吸系统、免疫系统等图像子集。According to a preferred embodiment, a subset of unlabeled medical images is divided according to the biophysiological system structure. For example, the unlabeled medical image set is specifically divided into image subsets such as digestive system, nervous system, motor system, endocrine system, urinary system, reproductive system, circulatory system, respiratory system, and immune system.

实际上，还可以对未标注医学图像集进行多层次的细化区分。比如，对于脑部图像，还可以将其划分为脑核(Central Core)、脑缘系统(Limbic System)和大脑皮质(Cerebral Cortex)等图像子集。In fact, multiple levels of refinement can also be performed on unlabeled medical image sets. For example, for a brain image, it can also be divided into image subsets such as the Central Core, the Limbic System, and the Cerebral Cortex.

根据一个优选实施方式，结合相关的心脏图像分为主动脉、左心房、左心室、右心房、右心室等图像子集。According to a preferred embodiment, the associated cardiac images are divided into image subsets such as aorta, left atrium, left ventricle, right atrium, right ventricle, etc.

根据一个优选实施方式，为了便于后续对医学图像子集进行组合，可以为每个医学图像子集分配一个标识符。相同医学图像子集中的所有医学图像共享相同的标识符。According to a preferred embodiment, in order to facilitate subsequent combination of the medical image subsets, each medical image subset may be assigned an identifier. All medical images in the same medical image subset share the same identifier.

根据一个优选实施方式，已标注医学图像数据库基于已标注图像的标签或标注信息划分为至少两个已标注医学图像子集，每个子集至少包括一幅待标注医学图像。According to a preferred embodiment, the labeled medical image database is divided into at least two labeled medical image subsets based on the labels or labeling information of the labeled images, and each subset includes at least one medical image to be labeled.

根据一个优选实施方式，已标注医学图像子集根据已标注医学图像的标签或标注信息，并且基于生物解剖结构和/或者生物生理系统或进行划分。According to a preferred embodiment, the subset of annotated medical images is divided according to labels or annotation information of the annotated medical images, and based on biological anatomy and/or biological physiological systems.

实施例五Embodiment 5

本实施例提供一种医学标注系统。如图4所示，标注系统包括导入医学图像的导入单元、存储医学图像的第一存储服务器、存储标签和/或医学图像相关语句的第二存储服务器、匹配单元、将待标注医学图像及相应的选择标签分配给至少两个标注终端的分配单元、确认至少两个标注结果的确认单元、至少两个标注终端。This embodiment provides a medical labeling system. As shown in FIG. 4 , the labeling system includes an import unit for importing medical images, a first storage server for storing medical images, a second storage server for storing labels and/or sentences related to medical images, a matching unit, a medical image to be labelled and corresponding The selection tags are assigned to at least two assignment units of annotating terminals, a confirmation unit that confirms at least two annotation results, and at least two annotating terminals.

导入单元将可视医学图像设备生成的医学图像导入并存储至第一存储单元。The importing unit imports and stores the medical image generated by the visual medical image device into the first storage unit.

第一存储服务器将医学图像划分为至少两个医学图像子集并分类存储医学图像及其关键字信息。The first storage server divides the medical images into at least two medical image subsets and stores the medical images and their keyword information by category.

匹配单元提取第一存储服务器存储的待标注医学图像的关键字和第二存储单元存储的至少一个标签和/或医学图像相关语句并在本地进行匹配和计算匹配值，将符合条件的至少一个标签和/或医学图像相关语句作为选择标签分配至相应的标注终端。The matching unit extracts the keywords of the medical images to be marked stored in the first storage server and at least one label and/or medical image-related sentences stored in the second storage unit, and performs matching locally and calculates the matching value, and matches the at least one label that meets the conditions. and/or medical image-related sentences are assigned to corresponding labeling terminals as selection labels.

分配单元基于预先设置的标注终端优先级顺序将待标注医学图像分配给至少两个标注终端，或者，分配单元基于终端处理能力顺序/基于负载均衡原理将待标注医学图像分配给至少两个标注终端。The assigning unit assigns the medical images to be labeled to at least two labeling terminals based on a preset priority order of labeling terminals, or the assigning unit assigns the medical images to be labeled to at least two labeling terminals based on the order of terminal processing capabilities/based on the load balancing principle .

确认单元基于至少两个标注操作人员的标注内容确认待标注医学图像的标注结果。The confirmation unit confirms the labeling result of the medical image to be labelled based on the labeling contents of the at least two labeling operators.

实施例六Embodiment 6

本实施例提供一种多人协同半自动医学图像标注系统。如图5所示，多人协同半自动医学图像标注系统至少包括存储标签和/或与医学图像相关的文章/相关语句的标签本体库单元、手动输入标签单元、分配单元、对比单元、高速远程服务器单元和标注内容服务器单元。This embodiment provides a multi-person collaborative semi-automatic medical image labeling system. As shown in FIG. 5 , the multi-person collaborative semi-automatic medical image labeling system at least includes a label ontology library unit for storing labels and/or articles/related sentences related to medical images, a manual input label unit, an allocation unit, a comparison unit, and a high-speed remote server. Units and Annotation Content Server Units.

多人协同半自动医学图像标注系统基于医学图像的文本与标签本体库单元中的标签和/或文章进行匹配，将匹配的至少两个标签和/或相关语句自动推荐给至少两个标注终端。The multi-person collaborative semi-automatic medical image annotation system matches the text of the medical image with the tags and/or articles in the tag ontology library unit, and automatically recommends at least two matched tags and/or related sentences to at least two tagging terminals.

标签本体库单元包括已标注标签单元和医学期刊与书籍数据单元，当标注终端的标注操作人员将未标注的图像导入系统后，根据图像文本信息在已标注标签单元和/或医学期刊与书籍数据单元进行检索，并基于检索结果匹配分数，生成至少两个标签和/或相关语句。生成的至少两个标签以可选择按钮形式显示在标注操作人员的标注终端上。或者，生成的至少两个标签以与相关语句相组合的方式显示在标注操作人员的标注终端上。The label ontology library unit includes the labeled label unit and the medical journal and book data unit. When the labeling operator at the labeling terminal imports the unlabeled image into the system, the labeled label unit and/or the medical journal and book data will be stored in the labeled label unit and/or medical journal and book data according to the image text information. The unit performs a search and generates at least two tags and/or related sentences based on the search result match scores. The generated at least two labels are displayed on the labeling terminal of the labeling operator in the form of selectable buttons. Alternatively, the generated at least two labels are displayed on the labeling terminal of the labeling operator in a manner of being combined with the relevant sentences.

手动输入标签单元基于至少两个标签和/或相关语句由标注操作人员手动输入准确的标签。The manual input label unit manually inputs an accurate label by a labeling operator based on at least two labels and/or related sentences.

分配单元用于将未标注的医学图像分配给至少两个标注终端操作人员，并由至少两个操作人员分别独立完成标注。对比单元用于将至少两个操作人员的标注结果进行对比分析，若针对同一医学图像对比结果显示不同，对比单元经对比分析后将至少两个操作人员的标注结果同时发送给至少两个操作人员，由至少两个操作人员协商确定准确的标签。The assigning unit is used for assigning the unlabeled medical images to at least two labeling terminal operators, and the at least two operators respectively complete the labeling independently. The comparison unit is used to compare and analyze the annotation results of at least two operators. If the comparison results for the same medical image are different, the comparison unit will send the annotation results of the at least two operators to the at least two operators after the comparison and analysis. , the exact label is negotiated by at least two operators.

根据一个优选实施方式，医学图像存储于高速远程服务器单元。与医学图像相匹配的标签存储于标注内容服务器单元。医学图像和与其相匹配的标签之间的匹配关系数据记录存储于标注内容服务器单元。高速远程服务器单元和标注内容服务器单元接收到提取已标注的医学图像的命令后，分别从不同位置发出相关数据，并显示于标注终端，标注操作人员根据来自标注内容服务器的匹配关系数据记录，将标签与医学图像进行匹配并显示于标注终端。其中，标注内容服务器单元具有加密系统。According to a preferred embodiment, the medical images are stored on a high-speed remote server unit. The tags matching the medical images are stored in the tagging content server unit. The matching relationship data record between the medical image and its matching label is stored in the labeling content server unit. After the high-speed remote server unit and the labeling content server unit receive the command to extract the labelled medical images, they respectively send relevant data from different positions and display them on the labeling terminal. Labels are matched to medical images and displayed on the labeling terminal. Wherein, the labeling content server unit has an encryption system.

根据一个优选实施方式，多人协同半自动医学图像标注系统还包括导入和导出单元，导入和导出单元用于对未标注的医学图像进行导入，以及用于将已标注的医学图像导出生成本地文件。According to a preferred embodiment, the multi-person collaborative semi-automatic medical image labeling system further includes an import and export unit, which is used for importing unlabeled medical images, and for exporting labeled medical images to generate local files.

实施例七Embodiment 7

本实施例提供一种医疗系统可视化设备。如图6所示，可视化设备包括成像部、图像展示部、图像分析部和图像标注部。图像标注部基于成像部生成的图像结合生物解剖结构或者生物生理系统划分完成所有图像标注。This embodiment provides a medical system visualization device. As shown in FIG. 6 , the visualization apparatus includes an imaging part, an image presentation part, an image analysis part and an image annotation part. The image labeling unit completes all image labeling based on the images generated by the imaging unit combined with biological anatomical structure or biological physiological system division.

同时，可视化设备作为一个图像标注终端，向图像标注系统发出图像标注请求。At the same time, the visualization device, as an image annotation terminal, sends an image annotation request to the image annotation system.

图像标注系统基于标注请求的关键字与至少一个标签本体库中的标签进行匹配，并向可视化设备单独分配待标注医学图像。成像部将医学图像标注系统发送的图像信息转化为医学图像并在图像展示部呈现给标注操作人员。The image tagging system matches tags in at least one tag ontology library based on the keyword of the tagging request, and individually assigns the medical images to be tagged to the visualization device. The imaging unit converts the image information sent by the medical image labeling system into medical images and presents them to the labeling operator in the image display unit.

图像标注系统基于标注请求的关键字与至少一个标签的匹配值向可视化设备的推荐至少一个选择标签。The image tagging system recommends at least one selection tag to the visualization device based on a matching value of the keyword of the tagging request and the at least one tag.

或者，图像标注系统通过可视化设备以全文检索的方式检索与选择标签相关联的语句并向标注操作人员标记显示。Alternatively, the image tagging system retrieves the sentences associated with the selected tags in a full-text retrieval manner through a visual device and marks them for display by the tagging operator.

标注操作人员基于图像展示部展示的选择标签和选择标签相关联的语句完成待标注医学图像标注，并发送至图像标注系统。The labeling operator completes the labeling of the medical image to be labelled based on the selection label displayed by the image display unit and the sentence associated with the selection label, and sends it to the image labeling system.

图像分析部能够结合图像标注系统中已标注图像对成像标注部标注的标注内容进行处理分析。The image analysis part can process and analyze the annotated content marked by the imaging marking part in combination with the marked images in the image marking system.

图像分析部通过网络接收存储在高速远程服务器上的医学图像、存储在另一台标注内容服务器上的与医学图像匹配的标注内容、标注内容与医学影像的匹配关系代码。The image analysis part receives the medical image stored on the high-speed remote server, the annotated content stored on another annotated content server that matches the medical image, and the matching relationship code between the annotated content and the medical image through the network.

图像标注部在本地根据接收的医学图像以及与医学图像匹配的标注内容和标注内容与医学图像的匹配关系代码完成医学图像与标注的匹配，并且记录至少一个标注操作人员的标注信息，统计同一个医学图像的交集标签。The image labeling unit completes the matching between the medical image and the label locally according to the received medical image, the labeling content that matches the medical image, and the matching relationship code between the labeling content and the medical image, and records the labeling information of at least one labeling operator, and counts the same one. Intersection labels for medical images.

如图7所示，本发明的医学图像标注系统，MITagger采用B/S架构，后台使用Python+Django开发，采用了标准的Django MVC框架，并提供了形式统一的HTTP接口。前端所有服务均调用后台服务器提供的HTTP接口完成。后台基本采用分层与模块化架构，将系统分成几个层次，每个层次由一定数量模块组成，As shown in FIG. 7 , in the medical image labeling system of the present invention, MITagger adopts B/S architecture, uses Python+Django to develop the background, adopts the standard Django MVC framework, and provides a unified HTTP interface. All front-end services are completed by calling the HTTP interface provided by the back-end server. The background basically adopts a layered and modular architecture, dividing the system into several levels, each level is composed of a certain number of modules,

Http Interface将所有功能封装为接口，供前端使用。Request Auth则负责校验请求的用户身份，大部分服务只有合法用户才能访问。Network Service则完成各项功能处理。Tag Recommdation层封装了基于实体库的标签推荐引擎。Data Storage则基于DjangoModel和Elasticsearch接口实现，负责数据的存储管理。Http Interface encapsulates all functions as interfaces for front-end use. Request Auth is responsible for verifying the user identity of the request, and most services can only be accessed by legitimate users. Network Service completes the processing of various functions. The Tag Recommdation layer encapsulates an entity library-based tag recommendation engine. Data Storage is implemented based on DjangoModel and Elasticsearch interfaces and is responsible for data storage management.

实施例八Embodiment 8

本实施例是对前述任一实施例的改进。This embodiment is an improvement to any of the preceding embodiments.

本发明的医学图片被标注后，利用众包手段对医学图片相关知识的再发现和新发现信息进行更新。在待标注的医学图像初步标注后，向公众用户开放标注信息。公众用户在医学图像标注系统注册个人信息后成为公众标注人员，可以对医学图像进行标注。将公众标注人员的标注内容保存，并且增加入对应的医学图片的标签队列。标签队列中的标签包括初步标注标签和公众标注人员标注生成的标签。基于公众标注人员的资质和标注历史，对标注内容进行加权处理。在权重的影响下，多个公众标注人员标注的标签呈现动态排序的标签队列。当公众标注人员标注的标签在队列中的排序领先，排序次序小于预设的顺序阈值时，公众标注人员标注的标签会被专家核实确认后，加入标签本体库。这样，本发明将与医学图片相关的新知识纳入标签本体库，从而使标签本体库不断进行更新。例如，预设顺序阈值为5，则标签队列的顺序在前五名的标签会被专家确认核实后，加入标签本体库。After the medical pictures of the present invention are marked, the rediscovery of the relevant knowledge of the medical pictures and the newly discovered information are updated by means of crowdsourcing. After the medical images to be labeled are initially labeled, the labeled information is opened to public users. Public users become public annotators after registering their personal information in the medical image annotation system, and can annotate medical images. Save the labeling content of the public labelers and add them to the labeling queue of the corresponding medical pictures. The labels in the label queue include preliminary label labels and labels generated by public labelers. Based on the qualifications and annotation history of the public annotators, the annotation content is weighted. Under the influence of the weight, the tags annotated by multiple public annotators present a dynamically sorted tag queue. When the tags marked by the public annotators are ranked ahead in the queue and the sorting order is smaller than the preset order threshold, the tags marked by the public annotators will be verified and confirmed by experts and added to the tag ontology library. In this way, the present invention incorporates new knowledge related to medical pictures into the label ontology database, so that the label ontology database is continuously updated. For example, if the preset order threshold is 5, the top five tags in the order of the tag queue will be confirmed and verified by experts and added to the tag ontology library.

根据一个优选实施方式，基于标签在标签队列的动态变动情况对相应的公众标注人员的标注能力值进行评估。若标签的顺序不断向前变动，则其公众标注人员的标注能力值会增加。若标签的顺序不断向后变动，则其公众标注人员的标注能力值会减少。According to a preferred embodiment, the tagging ability value of the corresponding public tagger is evaluated based on the dynamic change of tags in the tag queue. If the order of tags keeps moving forward, the tagging ability value of its public taggers will increase. If the order of labels keeps changing backwards, the labeling ability value of its public labelers will decrease.

根据一个优选实施方式，至少一个标签纳入标签本体库的公众标注人员的标注能力值会相应提高。标签被纳入标签实体库后，标签对应的公众标注人员的标注能力值会增加。增加的分值由管理人员设定。标签实体库中纳入的标签越多，其公众标注人员的标注能力值增加越多。According to a preferred embodiment, the tagging ability value of the public taggers whose at least one tag is included in the tag ontology library will be correspondingly increased. After the tag is included in the tag entity library, the tagging ability value of the public tagger corresponding to the tag will increase. The added points are set by managers. The more tags are included in the tag entity library, the more the tagging ability value of its public taggers increases.

本发明各个实施例中的方法步骤可以互相组合使用。在本发明任一实施例中，标注终端为功能手机、智能手机、掌上电脑、个人电脑、平板电脑或个人数字助理等任意具有计算处理能力的实体。标注终端还具有与网络的通信功能，从而可以通过网络接收由医学图像标注系统提供的待标注医学图像以及向医学图像标注系统返回标注信息。标注操作人员通过标注终端还可以查看已标注的医学图像。优选的，可通过在智能处理设备的浏览器中安装插件来实现标注功能，浏览器例如可以包括Internet Explorer、Firefox、Safari，Opera、Google Chrome、GreenBrowser等。标注终端可以分布在不同的地理区域之中。本发明实施方式并不局限于这些浏览器，而是可以适用于任意可用于显示网页服务器或档案系统内的文件、并让用户与文件互动的应用(APP)，这些应用可以是目前常见的各种浏览器，也可以是其它的具有网页浏览功能的应用程序。本发明的医学图像包括医学影像。The method steps in the various embodiments of the present invention may be used in combination with each other. In any embodiment of the present invention, the labeling terminal is any entity with computing processing capabilities, such as a feature phone, a smart phone, a palmtop computer, a personal computer, a tablet computer, or a personal digital assistant. The labeling terminal also has a communication function with the network, so that it can receive the medical images to be labelled provided by the medical image labeling system and return labeling information to the medical image labeling system through the network. The annotation operator can also view the annotated medical images through the annotation terminal. Preferably, the annotation function can be implemented by installing a plug-in in the browser of the intelligent processing device, and the browser can include, for example, Internet Explorer, Firefox, Safari, Opera, Google Chrome, GreenBrowser, and the like. Annotation terminals can be distributed in different geographical areas. The embodiments of the present invention are not limited to these browsers, but can be applied to any application (APP) that can be used to display files in a web server or file system and allow users to interact with the files. It can be a browser or other application program with web browsing function. The medical images of the present invention include medical images.

需要注意的是，上述具体实施例是示例性的，本领域技术人员可以在本发明公开内容的启发下想出各种解决方案，而这些解决方案也都属于本发明的公开范围并落入本发明的保护范围之内。本领域技术人员应该明白，本发明说明书及其附图均为说明性而并非构成对权利要求的限制。本发明的保护范围由权利要求及其等同物限定。It should be noted that the above-mentioned specific embodiments are exemplary, and those skilled in the art can come up with various solutions inspired by the disclosure of the present invention, and these solutions also belong to the disclosure scope of the present invention and fall within the scope of the present invention. within the scope of protection of the invention. It should be understood by those skilled in the art that the description of the present invention and the accompanying drawings are illustrative rather than limiting to the claims. The protection scope of the present invention is defined by the claims and their equivalents.

Claims (10)

1. A medical system visualization device establishes a communication connection with an image annotation system; it is characterized in that the medical system visualization equipment at least comprises an imaging part, an image display part, an image analysis part and an image labeling part,

the image annotation system matches the keywords based on the annotation request with the labels in at least one label body library, and individually allocates medical images to be annotated to annotation operators;

the imaging part converts the image information sent by the medical image labeling part into a medical image and presents the medical image to a labeling operator at the image display part,

the image analysis part records the labeling information of at least one labeling operator and counts intersection labels of the same medical image;

the image labeling part completes the matching of the medical image and the label according to the received medical image, the label content matched with the medical image and the matching relation code of the medical image.

2. The medical system visualization device of claim 1, wherein the image annotation system comprises an annotation content server unit,

after the medical image to be labeled is initially labeled, the labeled content server unit generates a label queue with dynamic change based on the labeled content of public labeling personnel, and labels smaller than a sequence threshold value are added into a label body library after being confirmed by an expert; and the labeling weight based on the label sequence in the label queue is changed, and the labeling capability value of the public labeling personnel is changed based on the label sequence in the label queue.

3. The medical system visualization apparatus according to claim 1 or 2, wherein the annotation content server unit evaluates the annotation ability values of the corresponding public annotation people based on the dynamic change of the tags in the tag queue;

if the sequence of the labels continuously changes forwards, the labeling capability value of public labeling personnel can be increased;

if the sequence of labels is constantly changing backwards, the labeling ability value of its public labeling personnel will decrease.

4. The medical system visualization device according to any one of claims 1 to 3, wherein after the tag is included in the tag entity library, the labeling ability value of the public labeling personnel corresponding to the tag is increased, and the increased value is set by a manager;

the more labels are included in the label entity library, the more the labeling capability value of the public labeling personnel corresponding to the labels is increased.

5. A medical system visualization device as in any of claims 1 to 4,

the matching unit in the image annotation system recommends at least one selection label to the annotation operator of the visualization equipment based on the matching value of the keyword of the annotation request of the annotation operator and the at least one label.

6. The medical system visualization device according to any one of claims 1 to 5, wherein a labeling operator completes labeling of the medical image to be labeled based on the selection tag displayed by the image display part and the sentence associated with the selection tag, and sends the medical image to be labeled to an image labeling system.

7. The medical system visualization device according to any one of claims 1 to 6, wherein the annotation weight is obtained by weighting annotation contents based on the qualification and annotation history of public annotators.

8. A label labeling method of medical system visualization equipment is characterized in that keywords based on a labeling request are matched with labels in at least one label body library, and medical images to be labeled are individually distributed to a labeling operator;

the image information sent by the medical image labeling part is converted into a medical image and is presented to a labeling operator at the image display part,

recording the labeling information of at least one labeling operator and counting intersection labels of the same medical image;

and matching the medical image and the label locally according to the received medical image, the label content matched with the medical image and the matching relation code of the medical image.

9. The label labeling method of claim 8, further comprising:

after the medical image to be labeled is initially labeled, generating a dynamically changed label queue based on the labeling content of public labeling personnel, and adding labels smaller than a sequence threshold value into a label body library after the labels are confirmed by experts; and the labeling weight based on the label sequence in the label queue is changed, and the labeling capability value of the public labeling personnel is changed based on the label sequence in the label queue.

10. The label labeling method according to claim 8 or 9, wherein the labeling ability value of the corresponding public labeling person is evaluated based on the dynamic change condition of the label in the label queue;

if the sequence of the labels continuously changes forwards, the labeling capability value of public labeling personnel can be increased;

if the sequence of labels is constantly changing backwards, the labeling ability value of its public labeling personnel will decrease.

Images