技术领域Technical Field
本发明涉及智慧医疗技术领域，具体涉及一种基于人工智能的医学影像导诊导检系统、方法、设备及介质。The present invention relates to the field of smart medical technology, and in particular to a medical imaging diagnosis and inspection guidance system, method, equipment and medium based on artificial intelligence.
背景技术Background Art
在就诊过程中，许多患者经常遇到选择科室的困惑；医疗知识过于专业，疾病临床表现复杂，存在大量疾病具有相似症状的情况，从而造成患者在选择科室时的困惑甚至错误；目前，人工导诊，不仅耗时耗力，且大大影响医疗服务效率和患者就医体验。现有智能导诊、导检、病例结构化类别产品在技术功能点和服务应用方面覆盖已较全面，然而在服务对象的领域细分方面尚不完备，影像类应用有所空缺。During the medical consultation process, many patients often encounter confusion in choosing a department; medical knowledge is too professional, the clinical manifestations of diseases are complex, and there are a large number of diseases with similar symptoms, which causes patients to be confused or even make mistakes when choosing a department; currently, manual guidance is not only time-consuming and labor-intensive, but also greatly affects the efficiency of medical services and the patient's medical experience. Existing intelligent guidance, inspection guidance, and case structured products have comprehensive coverage in terms of technical functions and service applications, but the field segmentation of service objects is still incomplete, and imaging applications are vacant.
发明内容Summary of the invention
针对现有技术中的缺陷，本发明实施例提供一种基于人工智能的医学影像导诊导检系统、方法、设备及介质，具体涉及AI问答系统、知识图谱、风险预测、推荐系统几大人工智能技术领域的应用，采用人工智能技术针对医学影像类的专业、标准化检查项目、就诊方案与服务等进行辅助推荐，辅助患者自查与医生诊断。In response to the defects in the prior art, the embodiments of the present invention provide a medical imaging diagnosis and inspection guidance system, method, device and medium based on artificial intelligence, which specifically involve the application of several major artificial intelligence technology fields such as AI question-answering system, knowledge graph, risk prediction and recommendation system. Artificial intelligence technology is used to provide auxiliary recommendations for medical imaging specialties, standardized examination items, treatment plans and services, to assist patients in self-examination and doctors in diagnosis.
第一方面，本发明实施例提供的一种基于人工智能的医学影像导诊导检系统，包括：问诊模块、病例生成模块、数据资源模块、匹配模块、导诊模块和导检模块，In a first aspect, an embodiment of the present invention provides a medical imaging diagnosis and inspection guidance system based on artificial intelligence, comprising: a diagnosis inquiry module, a case generation module, a data resource module, a matching module, a diagnosis guidance module and an inspection guidance module.
所述问诊模块用于收集患者基本信息、主诉信息及AI问诊交流信息；The consultation module is used to collect basic patient information, chief complaint information, and AI consultation communication information;
所述病例生成模块用于将患者信息生成结构化病例，将病例分别推送给患者端和医生端；The case generation module is used to generate structured cases from patient information and push the cases to the patient end and the doctor end respectively;
所述数据资源模块用于创建、整合慢病知识图谱、影像检查图谱、影像专业诊断资料医学影像数据资源；The data resource module is used to create and integrate chronic disease knowledge graphs, imaging examination graphs, and imaging professional diagnostic materials and medical imaging data resources;
所述匹配模块用于根据整合匹配算法、匹配规则、知识图谱检索规则构建匹配模型，从医学影像数据资源中提取特征，制作影像数据语料库，采用机器学习方法训练匹配模型，将病例信息输入匹配模型进行匹配，得到多路融合粗筛召回结果集；The matching module is used to construct a matching model based on the integrated matching algorithm, matching rules, and knowledge graph retrieval rules, extract features from medical image data resources, create an image data corpus, train the matching model using a machine learning method, input case information into the matching model for matching, and obtain a multi-channel fusion rough screening recall result set;
所述导诊模块用于将多路融合粗筛召回结果集提取与患者业务相关联特征，作为补充语料送入第一精排模型，进行训练、排序和筛选，得到导诊信息、疾病知识和风险预测结果集，向患者端发送推荐结果；The guidance module is used to extract features associated with the patient's business from the multi-channel fusion rough screening recall result set, and send it to the first fine sorting model as a supplementary corpus for training, sorting and screening to obtain guidance information, disease knowledge and risk prediction result sets, and send recommendation results to the patient end;
所述导检模块用于将多路融合粗筛召回结果集提取与医生业务相关联特征，作为补充数据语料送入第二精排模型，进行训练、排序和筛选，得到诊疗方案和诊断提醒服务结果集，向医生端发送推荐结果。The inspection guidance module is used to extract features related to the doctor's business from the multi-channel fusion rough screening recall result set, and send it to the second fine sorting model as a supplementary data corpus for training, sorting and screening to obtain the diagnosis and treatment plan and diagnosis reminder service result set, and send the recommendation results to the doctor end.
第二方面，本发明实施例提供的一种基于人工智能的医学影像导诊导检方法，包括以下步骤：In a second aspect, an embodiment of the present invention provides a medical imaging diagnosis and inspection guidance method based on artificial intelligence, comprising the following steps:
获取患者基本信息、主诉信息及AI问诊交流信息；Obtain the patient's basic information, chief complaint information, and AI consultation communication information;
根据患者信息生成结构化病例，将病例分别推送给患者端和医生端；Generate structured cases based on patient information and push them to the patient and doctor ends respectively;
创建、整合慢病知识图谱、影像检查图谱、影像专业诊断资料医学影像数据资源库；Create and integrate chronic disease knowledge graphs, imaging examination graphs, and medical imaging data resource libraries for imaging professional diagnostic materials;
根据整合匹配算法、匹配规则、知识图谱检索规则构建匹配模型，从医学影像数据资源库中提取特征，制作影像数据语料库，采用机器学习方法训练匹配模型，将病例信息输入匹配模型进行匹配，得到多路融合粗筛召回结果集；A matching model is constructed based on the integrated matching algorithm, matching rules, and knowledge graph retrieval rules. Features are extracted from the medical image data resource library to create an image data corpus. The matching model is trained using machine learning methods. Case information is input into the matching model for matching to obtain a multi-channel fusion rough screening recall result set.
将多路融合粗筛召回结果集提取与患者业务相关联特征，作为补充语料送入第一精排模型，进行训练、排序和筛选，得到导诊信息、疾病知识和风险预测结果集，向患者端发送推荐结果；Extract features related to the patient's business from the multi-channel fusion rough screening recall result set and send it to the first fine sorting model as supplementary corpus for training, sorting and screening to obtain the guidance information, disease knowledge and risk prediction result set, and send the recommendation results to the patient end;
将多路融合粗筛召回结果集提取与医生业务相关联特征，作为补充数据语料送入第二精排模型，进行训练、排序和筛选，得到诊疗方案和诊断提醒服务结果集，向医生端发送推荐结果。The features related to the doctor's business are extracted from the multi-channel fusion rough screening recall result set and sent to the second fine sorting model as a supplementary data corpus for training, sorting and screening to obtain the diagnosis and treatment plan and diagnosis reminder service result set, and send the recommendation results to the doctor.
第三方面，本发明实施例提供的一种智能设备，包括处理器、输入设备、输出设备和存储器，所述处理器、输入设备、输出设备和存储器相互连接，所述存储器用于存储计算机程序，所述计算机程序包括程序指令，所述处理器被配置用于调用所述程序指令，执行上述实施例描述的方法。In a third aspect, an embodiment of the present invention provides an intelligent device, comprising a processor, an input device, an output device and a memory, wherein the processor, input device, output device and memory are interconnected, the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method described in the above embodiment.
第四方面，本发明实施例提供的一种计算机可读存储介质，所述计算机存储介质存储有计算机程序，所述计算机程序包括程序指令，所述程序指令当被处理器执行时使所述处理器执行上述实施例描述的方法。In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method described in the above embodiment.
本发明的有益效果：Beneficial effects of the present invention:
本发明实施例提供的一种基于人工智能的医学影像导诊导检系统、方法、设备及介质，具体的，系统融合使用了包括机器学习、深度学习、实体识别、相似度匹配、意图识别、知识图谱、推荐系统、问答系统、风险预测等计算机人工智能自然语言处理技术解决方案，对涵盖专业、规模、系统化的专业影像报告真实语料数据与影像检查专业学术资料在内的行业数据进行处理、训练、机器学习，可以实现针对医学影像类的专业、标准化检查项目、就诊方案与服务等人工智能技术辅助推荐与问答咨询，辅助患者自查与医生诊断。The embodiments of the present invention provide a medical imaging diagnosis and inspection guidance system, method, device and medium based on artificial intelligence. Specifically, the system integrates computer artificial intelligence natural language processing technology solutions including machine learning, deep learning, entity recognition, similarity matching, intent recognition, knowledge graphs, recommendation systems, question-and-answer systems, risk prediction, etc., and processes, trains and machines industry data including professional, large-scale and systematic real corpus data of professional imaging reports and professional academic materials on imaging examinations. It can realize artificial intelligence technology-assisted recommendation and question-and-answer consultation for professional and standardized examination items, treatment plans and services for medical imaging, and assist patients in self-examination and doctors in diagnosis.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
为了更清楚地说明本发明具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍。在所有附图中，类似的元件或部分一般由类似的附图标记标识。附图中，各元件或部分并不一定按照实际的比例绘制。In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.
图1示出了本发明第一实施例所提供的一种基于人工智能的医学影像导诊导检系统的结构框图；FIG1 shows a structural block diagram of a medical imaging diagnosis and inspection guidance system based on artificial intelligence provided by a first embodiment of the present invention;
图2示出了本发明第二实施例所提供的一种基于人工智能的医学影像导诊导检方法的流程图；FIG2 shows a flow chart of a medical imaging diagnosis and inspection guidance method based on artificial intelligence provided by a second embodiment of the present invention;
图3示出了本发明第三实施例所提供的一种智能设备的结构框图。FIG. 3 shows a structural block diagram of an intelligent device provided by a third embodiment of the present invention.
具体实施方式DETAILED DESCRIPTION
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
应当理解，当在本说明书和所附权利要求书中使用时，术语“包括”和“包含”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and/or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or combinations thereof.
还应当理解，在此本发明说明书中所使用的术语仅仅是出于描述特定实施例的目的而并不意在限制本发明。如在本发明说明书和所附权利要求书中所使用的那样，除非上下文清楚地指明其它情况，否则单数形式的“一”、“一个”及“该”意在包括复数形式。It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
还应当进一步理解，本发明说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。It should be further understood that the term "and/or" used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
如在本说明书和所附权利要求书中所使用的那样，术语“如果”可以依据上下文被解释为“当...时”或“一旦”或“响应于确定”或“响应于检测到”。类似地，短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述条件或事件]”或“响应于检测到[所描述条件或事件]”。As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.
需要注意的是，除非另有说明，本申请使用的技术术语或者科学术语应当为本发明所属领域技术人员所理解的通常意义。It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.
如图1所示，示出了本发明第一实施例所提供的一种基于人工智能的医学影像导诊导检系统的结构框图，该系统包括：问诊模块、病例生成模块、数据资源模块、匹配模块、导诊模块和导检模块，其中，As shown in FIG1 , a structural block diagram of a medical imaging diagnosis and inspection guidance system based on artificial intelligence provided by the first embodiment of the present invention is shown. The system includes: a diagnosis inquiry module, a case generation module, a data resource module, a matching module, a diagnosis guidance module and an inspection guidance module, wherein:
所述问诊模块用于收集患者基本信息、主诉信息及AI问诊交流信息；The consultation module is used to collect basic patient information, chief complaint information, and AI consultation communication information;
所述病例生成模块用于将患者信息生成结构化病例，将病例分别推送给患者端和医生端；The case generation module is used to generate structured cases from patient information and push the cases to the patient end and the doctor end respectively;
所述数据资源模块用于创建、整合慢病知识图谱、影像检查图谱、影像专业诊断资料等医学影像数据资源；The data resource module is used to create and integrate medical imaging data resources such as chronic disease knowledge graphs, imaging examination graphs, and imaging professional diagnostic materials;
所述匹配模块用于根据整合匹配算法、匹配规则、知识图谱检索规则构建匹配模型，从医学影像数据资源库中提取特征，制作影像数据语料库，采用机器学习方法训练匹配模型，将病例信息输入匹配模型进行匹配，得到多路融合粗筛召回(推荐)结果集；The matching module is used to construct a matching model based on the integrated matching algorithm, matching rules, and knowledge graph retrieval rules, extract features from the medical image data resource library, create an image data corpus, use machine learning methods to train the matching model, input case information into the matching model for matching, and obtain a multi-channel fusion rough screening recall (recommendation) result set;
所述导诊模块用于将多路融合粗筛召回结果集提取与患者业务相关联特征，作为补充语料送入第一精排模型，进行训练、排序、筛选，得到科普性导诊信息、疾病知识、风险预测等结果集，向患者端发送推荐结果；The guidance module is used to extract features associated with the patient's business from the multi-channel fusion rough screening recall result set, and send it to the first fine sorting model as a supplementary corpus for training, sorting, and screening to obtain a result set of popular science guidance information, disease knowledge, risk prediction, etc., and send the recommendation results to the patient end;
所述导检模块用于将多路融合粗筛召回结果集提取与医生业务相关联特征，作为补充数据语料送入第二精排模型，进行训练、排序、筛选，得到专业性诊疗方案、诊断提醒等服务结果集，向医生端发送推荐结果。The inspection guidance module is used to extract features related to the doctor's business from the multi-channel fusion rough screening recall result set, and send it to the second fine sorting model as a supplementary data corpus for training, sorting, and screening to obtain service result sets such as professional treatment plans and diagnosis reminders, and send recommendation results to the doctor side.
在本实施例中，问诊模块包括患者自诉单元、推荐医生单元、大数据问答检索单元、AI客服问答系统单元、医患沟通单元、推荐专家答疑单元、患者信息记录单元和影像数据语料库单元；患者自诉单元用于收集和获取用户个人信息，包括姓名、年龄、性别、地区，症状描述，症状属性描述，包括发作部位、程度、频次等，病史、过敏史等一诉五史信息；推荐医生单元用于根据患者自诉信息推荐相关联医生，方便医患沟通；大数据问答检索单元用于检索互联网主流医疗问答资讯，向患者推荐专家答疑；AI客服问答系统单元用于患者与AI客服在线沟通，咨询问题，得到答案；患者信息记录单元用于记录患者自诉信息、患者与AI客服聊天信息、医患沟通信息、专家答疑信息，并将这些存储到数据库；影像数据语料库单元用于将记录患者自诉信息、患者与AI客服聊天信息、医患沟通信息、专家答疑信息从数据库取出，分析录入影像数据语料库，用于返回AI客服问答系统单元分析与组织对话，数据循环利用，同步用于后续匹配模块训练学习。In this embodiment, the consultation module includes a patient self-report unit, a doctor recommendation unit, a big data question and answer retrieval unit, an AI customer service question and answer system unit, a doctor-patient communication unit, a recommended expert question and answer unit, a patient information recording unit and an image data corpus unit; the patient self-report unit is used to collect and obtain user personal information, including name, age, gender, region, symptom description, symptom attribute description, including onset site, degree, frequency, etc., medical history, allergy history and other one-complaint and five-history information; the doctor recommendation unit is used to recommend related doctors based on the patient's self-report information to facilitate doctor-patient communication; the big data question and answer retrieval unit is used to retrieve the mainstream medical question and answer information on the Internet. The AI customer service question and answer system unit is used for patients to communicate with AI customer service online, ask questions and get answers; the patient information recording unit is used to record patient self-report information, patient chat information with AI customer service, doctor-patient communication information, and expert question and answer information, and store them in the database; the image data corpus unit is used to retrieve patient self-report information, patient chat information with AI customer service, doctor-patient communication information, and expert question and answer information from the database, analyze and enter them into the image data corpus, which is returned to the AI customer service question and answer system unit for analysis and organization of dialogue. The data is recycled and used synchronously for subsequent matching module training and learning.
AI客服问答系统单元包括实体识别单元、意图识别单元和图谱检索单元。实体识别单元用于将患者输入的问题做严格检索匹配和相似度近似匹配实体识别，即同义置换，生成标准表述问题；意图识别单元用于将匹配或同义置换处理后的患者标准表述问题送入机器学习模型训练分类，做问题分类，如：含有“怎么治”、“吃什么药”这一类关键词的句子倾向于是咨询治疗法方法类问题，含有“多久”、“康复”类关键词的句子倾向于是咨询治疗周期类问题；图谱检索单元用于将患者问题按问题分类导向答案模板，如：治疗法方法类问题的答案模版为“疾病{}的治疗方法有{}”，询问治疗周期类问题的答案模版为“疾病{}的治愈周期为{}”，并对关键词检索慢病知识图谱、影像知识图谱等数据资源，填充模版，如针对关键词“中耳炎”，检索到治疗方法“使用滴耳剂苯酚甘油，口服消炎药”，检索到治愈周期为“10-14天”,向患者返回完整答案，“疾病中耳炎的治疗方法有：使用滴耳剂苯酚甘油，口服消炎药”，“疾病中耳炎的治愈周期为10-14天”。The AI customer service question-and-answer system unit includes an entity recognition unit, an intent recognition unit, and a graph retrieval unit. The entity recognition unit is used to perform strict search matching and similarity approximate matching entity recognition on the questions input by the patient, that is, synonym replacement, to generate standard expression questions; the intent recognition unit is used to send the patient's standard expression questions after matching or synonym replacement processing into the machine learning model training classification to perform question classification, such as: sentences containing keywords such as "how to treat" and "what medicine to take" tend to be questions about consulting treatment methods, and sentences containing keywords such as "how long" and "recovery" tend to be questions about consulting treatment cycles; the graph retrieval unit is used to guide patient questions according to question classification to answer templates, such as: treatment methods The answer template for method-related questions is "The treatment methods for disease {} are {}", and the answer template for questions about treatment cycles is "The cure period for disease {} is {}". Keywords are searched for data resources such as chronic disease knowledge graphs and imaging knowledge graphs to fill in the template. For example, for the keyword "otitis media", the treatment method is retrieved as "using ear drops phenol glycerin and oral anti-inflammatory drugs", and the cure period is retrieved as "10-14 days". The complete answer is returned to the patient, "The treatment methods for otitis media are: using ear drops phenol glycerin and oral anti-inflammatory drugs", and "The cure period for otitis media is 10-14 days".
病例生成模块包括信息获取单元和生成病例单元，信息获取模块用于读取患者主诉信息等数据，生成病例单元用于将患者信息生成结构化病例结果数据。The case generation module includes an information acquisition unit and a case generation unit. The information acquisition module is used to read data such as patient's chief complaint information, and the case generation unit is used to generate structured case result data from patient information.
数据资源模块包括慢病知识图谱单元、影像检查图谱单元、影像报告单元、大数据专家问答单元和自扩展知识图谱单元，慢病知识图谱单元用于将关系型病名、介绍、病因、病症、治疗、预防、日常等慢病科谱知识数据，生成实体与关系对儿的图数据，导入慢病知识图谱图数据库；影像检查图谱单元用于将关系型病症、检查、适应症、诊疗方案、设备、扫描方式、检查身体摆位等影像专业资料数据，生成实体与关系对儿的图数据，导入影像检查知识图谱图数据库；影像报告单元用于采集获取原始影像报告数据知识库；大数据专家问答单元用于采集互联网医疗咨询专家答疑数据；自扩展知识图谱单元用于提取患者病例、报告、互联网专家答颖数据中的实体与关系对，实时自动创建扩展知识知识图谱。The data resource module includes a chronic disease knowledge graph unit, an imaging examination graph unit, an imaging report unit, a big data expert question and answer unit and a self-expanding knowledge graph unit. The chronic disease knowledge graph unit is used to generate graph data of entity and relationship pairs from relational chronic disease knowledge data such as disease names, introductions, causes, symptoms, treatments, prevention, and daily life, and import them into the chronic disease knowledge graph database; the imaging examination graph unit is used to generate graph data of entity and relationship pairs from relational imaging professional data such as symptoms, examinations, indications, treatment plans, equipment, scanning methods, and examination body positions, and import them into the imaging examination knowledge graph database; the imaging report unit is used to collect and obtain the original imaging report data knowledge base; the big data expert question and answer unit is used to collect Internet medical consultation expert question and answer data; the self-expanding knowledge graph unit is used to extract entity and relationship pairs from patient cases, reports, and Internet expert answers, and automatically create an extended knowledge graph in real time.
匹配模块包括包括数据处理单元、图谱检索单元、神经网络模型单元、相似度特征提取单元、推荐模型单元、召回结果集单元。The matching module includes a data processing unit, a graph retrieval unit, a neural network model unit, a similarity feature extraction unit, a recommendation model unit, and a recall result set unit.
其中，数据处理单元用于获取慢病知识图谱、影像检查知识图谱、影像报告知识库、大数据专家答疑、自扩展知识图谱中的数据，分析处理，提取特征，用数学向量化符号表示，制作影像数据语料库，用于机器学习训练意图与匹配。Among them, the data processing unit is used to obtain data from chronic disease knowledge graphs, imaging examination knowledge graphs, imaging report knowledge bases, big data expert Q&A, and self-expanding knowledge graphs, analyze and process data, extract features, represent them with mathematical vectorized symbols, and create an imaging data corpus for machine learning training intent and matching.
图谱检索单元用于根据患者病例数据检索慢病管理知识图谱、影像检查知识图谱、自扩展知识图谱，找到关联匹配结果实体关系对儿，得到图谱检索召回结果集。The graph retrieval unit is used to retrieve the chronic disease management knowledge graph, imaging examination knowledge graph, and self-expanding knowledge graph based on patient case data, find the associated matching result entity relationship pairs, and obtain the graph retrieval recall result set.
神经网络模型单元用于搭建神经网络模型，将影像数据语料送入模型，进行机器学习和训练，对患者病例信息作意图分析、预测，用于生成问答对、风险预测候选集，得到神经网络结果集，辅助图谱检索与匹配。The neural network model unit is used to build a neural network model, feed the imaging data corpus into the model, perform machine learning and training, perform intent analysis and prediction on patient case information, generate question-answer pairs and risk prediction candidate sets, obtain a neural network result set, and assist in graph retrieval and matching.
相似度特征提取单元用于将影像数据语料库中的数据制作模型训练所需的匹配句子对数据，提取相似性特征。相似性特征提取包括：句长相似性提取、最长公共子串提取、最长公共子序列提取、编辑距离特征提取、n-gram特征词提取、jieba(结巴)分词特征词提取、疑问词特征提取、规则权重关键词特征提取、词向量特征提取。句长相似性提取用于提取匹配句子双方句长特征，用句长相差度来衡量意图相似性；最长公共子串提取用于提取匹配句子双方所含最长相同字符串长度特征，用句中最长重复词衡量相似性；最长公共子序列提取用于提取匹配双方所含最长相同字符串长度累加值特征，用句中重复词长度衡量相似性；编辑距离特征提取用于提取匹配双方变换复杂度，用双方变换复杂程度衡量句长、句序及重复词维度的相似性特征；n-gram特征提取用于提取匹配双方两字组合、三字组合重复词特征，用句中相临字相同个数指标衡量相似性；jieba分词特征提取用于提取匹配双方jieba分词后的常规词相同个数，用句中常规词的重复性指标衡量相似性；疑问词特征提取用于提取、比对匹配双方相同的疑问词特征，用疑问词定位问题类型，把握句子主体的相似性，如：“什么是？”代表答案倾向于一个解释性的结果，如，病的介绍，“多久？”代表答案是一个时间，如，康复时间；规则权重关键词特征提取用于为匹配双方含有的权重关键词加权重，如：双方都含有“膝盖”，说明患者本轮问题是围绕“膝盖”的核心展开，即可把语料库中含有“膝盖”、“膝关节”、及其它膝关节相关的特征词加权重处理，提高相似度计算的分值参考；词向量特征提取用于将匹配双方句子做数学向量化符号处理，用于直接送入BiLSTM(双向循环长短时记忆)深度学习模型、Attention(注意力)机制模型(Bert)学习相似性规律。The similarity feature extraction unit is used to extract similarity features from the data in the image data corpus to make matching sentence pairs required for model training. Similarity feature extraction includes: sentence length similarity extraction, longest common substring extraction, longest common subsequence extraction, edit distance feature extraction, n-gram feature word extraction, jieba (stammer) segmentation feature word extraction, question word feature extraction, rule weight keyword feature extraction, and word vector feature extraction. Sentence length similarity extraction is used to extract the sentence length features of both matching sentences, and the difference in sentence length is used to measure the similarity of intentions; the longest common substring extraction is used to extract the longest identical string length features contained in both matching sentences, and the longest repeated word in the sentence is used to measure the similarity; the longest common subsequence extraction is used to extract the cumulative value features of the longest identical string length contained in both matching sentences, and the length of repeated words in the sentence is used to measure the similarity; the edit distance feature extraction is used to extract the transformation complexity of both matching parties, and the transformation complexity of both parties is used to measure the similarity features of sentence length, sentence order and repeated word dimensions; n-gram feature extraction is used to extract the repeated word features of two-character combinations and three-character combinations of both matching parties, and the similarity is measured by the number of adjacent characters in the sentence; jieba segmentation feature extraction is used to extract the number of regular words that are the same after jieba segmentation of both matching parties, and the similarity is measured by the repetitiveness index of regular words in the sentence; question word feature extraction is used Extract and compare the same question word features of both parties, use question words to locate the question type, and grasp the similarity of the sentence body, such as: "What is?" represents that the answer tends to be an explanatory result, such as the introduction of the disease, and "How long?" represents that the answer is a time, such as recovery time; rule weight keyword feature extraction is used to add weight to the weight keywords contained in both parties, such as: both parties contain "knee", which means that the patient's current round of questions revolves around the core of "knee", and the feature words containing "knee", "knee joint", and other knee joint-related features in the corpus can be weighted to improve the score reference for similarity calculation; word vector feature extraction is used to perform mathematical vectorization symbol processing on the sentences of both parties, and directly send them to the BiLSTM (bidirectional recurrent long short-term memory) deep learning model and the Attention mechanism model (Bert) to learn similarity rules.
推荐模型单元用于将相似度特征提取单元提取的特征送入匹配模型，学习相似性特征：包括协同过滤算法、矩阵分解、GBDT+LR(决策树模型+逻辑回归)模型、Wide&Deep深度学习模型、规则权重召回；协同过滤算法用于计算基于患者(UserCF)相似性结果集与基于病症(ItemCF)相似性的结果集；矩阵分解用于将UserCF或ItemCF中的共现矩阵分解，计算患者与病症信息隐含语义的关联相似性，如：患者矩阵，张三：[胸闷0.8恶心0.6心率不齐0.9]，症状矩阵，胆源性心脏病：[胸闷0.81恶心0.61心率不齐0.91]，则张三得胆源性心脏病的得分即为张三胸闷的频次*胆源心脏病有胸闷征的概率+张三恶心的频次*胆源心脏病有恶心征的概率+张三心率不齐频次*胆源心脏病有心率不齐征的概率＝0.8*0.81+0.6*0.61+0.9*0.91，将患者矩阵与症状矩阵两个隐含矩阵共现的部分计算浓缩为一个指标，代表这个患者与这个病症的相似性，弥补协同过滤仅利用了患者与病症的交互信息，未用到患者自身属性与病症自身属性的不足，以及解决了矩阵稀疏化的问题，增强了模型的泛化能力；GBDT+LR(决策树模型+逻辑回归)模型用于利用决策树机器学习模型对上下文信息自动进行特征筛选组合，生成新的离散特征向量，通过LR(逻辑回归)模型生成预测结果，弥补协同过滤仅利用患者与病症之间关联交互信息，忽视患者自身特征与病症信息特征的不足；Wide&Deep深度学习模型用于采用深度学习模型Wide部分快速关联规则粗筛相似性，增强了模型的“直接记忆能力”，Deep部分深入抽象训练模型，增强了模型的“抽象泛化能力”，达到效率与精确性结合的目的；规则权重召回用于将相似度提取特征中的权重特征词加权召回结果，得到规则权重结果集，如“膝盖”相关问题，将影像数据语料库中“膝盖”相关的词条信息加权重计算相似度排名。The recommendation model unit is used to send the features extracted by the similarity feature extraction unit into the matching model to learn similarity features: including collaborative filtering algorithm, matrix decomposition, GBDT+LR (decision tree model + logistic regression) model, Wide&Deep deep learning model, rule weight recall; the collaborative filtering algorithm is used to calculate the result set based on patient (UserCF) similarity and the result set based on disease (ItemCF) similarity; matrix decomposition is used to decompose the co-occurrence matrix in UserCF or ItemCF to calculate the association similarity of the implicit semantics of patient and disease information, such as : Patient matrix, Zhang San: [chest tightness 0.8 nausea 0.6 arrhythmia 0.9], symptom matrix, gallbladder heart disease: [chest tightness 0.81 nausea 0.61 arrhythmia 0.91], then Zhang San's score for gallbladder heart disease is the frequency of Zhang San's chest tightness * the probability of gallbladder heart disease having chest tightness + the frequency of Zhang San's nausea * the probability of gallbladder heart disease having nausea + the frequency of Zhang San's arrhythmia * the probability of gallbladder heart disease having arrhythmia = 0.8*0.81+0.6*0.61+0.9*0.91, the co-occurrence part of the two implicit matrices of the patient matrix and the symptom matrix is calculated and condensed The GBDT+LR (decision tree model + logistic regression) model is used to automatically perform feature screening and combination of context information using the decision tree machine learning model, generate new discrete feature vectors, and generate prediction results through the LR (logistic regression) model, which makes up for the deficiency that collaborative filtering only uses the associated interactive information between patients and symptoms and ignores the characteristics of patients themselves and the characteristics of symptom information. The Wide&Deep deep learning model is used to use the Wide part of the deep learning model to quickly associate rules to roughly screen similarities, which enhances the "direct memory ability" of the model, and the Deep part to deeply abstract the training model, which enhances the "abstract generalization ability" of the model, so as to achieve the purpose of combining efficiency and accuracy. The rule weight recall is used to weight the recall results of the weighted feature words in the similarity extraction features to obtain the rule weight result set. For example, for "knee" related issues, the "knee" related term information in the image data corpus is weighted to calculate the similarity ranking.
召回结果集单元用于将图谱检索结果集、神经网络结果集、匹配模型结果集、规则权重结果集、推荐模型结果集等多路召回结果集拼接，归一化为0到1之间的去量纲、标量化指标概率值，得到多路融合粗筛召回结果集。The recall result set unit is used to splice multiple recall result sets such as graph retrieval result set, neural network result set, matching model result set, rule weight result set, and recommendation model result set, and normalize them into dimensionless and scalar indicator probability values between 0 and 1 to obtain a multi-channel fusion coarse screening recall result set.
在本实施例中，导诊模块包括第一精排单元和第一推荐应用单元，第一精排单元包括第一提取补充业务特征单元和第一精排推荐排序模型单元；第一提取补充业务特征单元用于将多路召回结果集提取加强与患者端业务相关联特征，包括科室、检查项、疾病知识、生知指导、风险预测；第一精排推荐排序模型单元用于将补充提取业务特征生成新的数据语料送入模型，做精细化机器学习、训练、计算相似度、排序、筛选得到患者端推荐结果集。第一推荐应用单元用于将患者端推荐结果，包括科室、检查项、疾病知识、生知指导、风险预测与患者病例结合，推荐给患者端应用。In this embodiment, the medical guidance module includes a first precise sorting unit and a first recommendation application unit. The first precise sorting unit includes a first unit for extracting supplementary business features and a first precise sorting recommendation ranking model unit. The first unit for extracting supplementary business features is used to extract the multi-channel recall result set to strengthen the features associated with the patient-side business, including departments, examination items, disease knowledge, bio-information guidance, and risk prediction. The first precise sorting recommendation ranking model unit is used to generate a new data corpus from the supplementary extracted business features and feed it into the model to perform refined machine learning, training, similarity calculation, sorting, and screening to obtain the patient-side recommendation result set. The first recommendation application unit is used to combine the patient-side recommendation results, including departments, examination items, disease knowledge, bio-information guidance, and risk prediction with patient cases, and recommend them to the patient-side application.
在本实施例中，医生端导检模块包括召回结果集第二精排单元和第二推荐应用单元，第二精排单元单元包括第二提取补充业务特征单元、第二精排推荐排序模型单元和重点词提醒单元；第二提取补充业务特征单元用于将多路召回结果集提取加强与医生端业务相关联特征，包括专业检查项目、项目介绍、适应症、设备、扫描方式、检查身体摆位、诊疗方案；第二精排推荐排序模型单元用于将补充提取业务特征生成新的数据语料送入模型，做精细化机器学习、训练、计算相似度、排序、筛选得到医生端推荐结果集；重点词提醒单元用于将患者病例中的病症、风险等重点词突出快捷提醒，标识出患者病例中的病症和风险重点词；第二推荐单元用于将医生端推荐结果，包括检查项目、项目介绍、适应症、设备、扫描方式、检查身体摆位、诊疗方案、重点词快捷提醒与患者病例结合，推荐给医生端应用。In this embodiment, the doctor-side inspection guidance module includes a second fine sorting unit of the recall result set and a second recommendation application unit, and the second fine sorting unit includes a second extraction and supplementary business feature unit, a second fine sorting recommendation ranking model unit and a key word reminder unit; the second extraction and supplementary business feature unit is used to extract multiple recall result sets to strengthen the features associated with the doctor-side business, including professional examination items, project introductions, indications, equipment, scanning methods, examination body positioning, and treatment plans; the second fine sorting recommendation ranking model unit is used to generate a new data corpus from the supplementary extracted business features and send it into the model to perform refined machine learning, training, calculation of similarity, sorting, and screening to obtain the doctor-side recommended result set; the key word reminder unit is used to highlight and quickly remind key words such as symptoms and risks in patient cases, and identify the symptoms and risk key words in patient cases; the second recommendation unit is used to combine the doctor-side recommendation results, including examination items, project introductions, indications, equipment, scanning methods, examination body positioning, treatment plans, and key word quick reminders with patient cases, and recommend them to the doctor-side application.
本发明实施例提供一种基于人工智能的医学影像导诊导检系统，具体的，系统融合使用了包括机器学习、深度学习、相似度匹配、意图识别、知识图谱、推荐系统、问答系统、风险预测等计算机人工智能自然语言处理(NLP)整套技术解决方案，对涵盖专业、规模、系统化的专业影像报告真实语料数据与影像检查专业学术资料在内的行业数据进行处理、训练、机器学习，可以实现针对医学影像类的专业、标准化检查项目、就诊方案与服务等人工智能技术辅助推荐与问答咨询，辅助患者自查与医生诊断。The embodiment of the present invention provides a medical imaging diagnosis and inspection guidance system based on artificial intelligence. Specifically, the system integrates a complete set of computer artificial intelligence natural language processing (NLP) technical solutions including machine learning, deep learning, similarity matching, intent recognition, knowledge graphs, recommendation systems, question-and-answer systems, risk prediction, etc., and processes, trains, and machine-learns industry data including professional, large-scale, and systematic real corpus data of professional imaging reports and professional academic materials on imaging examinations. It can realize artificial intelligence technology-assisted recommendations and question-and-answer consultations for professional and standardized examination items, treatment plans, and services for medical imaging, and assist patients in self-examination and doctors in diagnosis.
在上述第一实施例中，提供一种基于人工智能的医学影像导诊导检系统，与之对应的，本申请另一实施例提供一种基于人工智能的医学影像导诊导检方法，请参考图2，示出了本申请另一实施例提供的基于人工智能的医学影像导诊导检方法流程图。由于方法实施例基本相似于装置实施例，所以描述得比较简单，相关之处参见装置实施例的部分说明即可。下述描述的方法实施例仅仅是示意性的。In the above-mentioned first embodiment, a medical image diagnosis and inspection guidance system based on artificial intelligence is provided. Correspondingly, another embodiment of the present application provides a medical image diagnosis and inspection guidance method based on artificial intelligence. Please refer to Figure 2, which shows a flow chart of the medical image diagnosis and inspection guidance method based on artificial intelligence provided by another embodiment of the present application. Since the method embodiment is basically similar to the device embodiment, the description is relatively simple. For relevant parts, please refer to the partial description of the device embodiment. The method embodiment described below is only illustrative.
如图2所示，示出了本发明另一实施例提供的一种基于人工智能的医学影像导诊导检方法流程图，方法包括以下步骤：As shown in FIG2 , a flowchart of a medical imaging diagnosis and inspection guidance method based on artificial intelligence is shown in another embodiment of the present invention. The method includes the following steps:
S1:获取患者基本信息、主诉信息及AI问诊交流信息。S1: Obtain the patient’s basic information, chief complaint information, and AI consultation communication information.
具体地，通过获取患者在聊天窗口中输入的病症、病史、个人信息等主诉信息及其它与AI客服的聊天信息分析入库。Specifically, the patient's main complaint information such as symptoms, medical history, personal information, etc. entered in the chat window and other chat information with the AI customer service are analyzed and stored in the database.
具体包括：Specifically include:
收集用户个人信息，包括姓名、年龄、性别、地区，症状描述，症状属性描述，包括发作部位、程度、频次等，病史、过敏史等一诉五史信息；Collect user personal information, including name, age, gender, region, symptom description, symptom attribute description, including onset location, degree, frequency, etc., medical history, allergy history and other information;
根据患者自诉信息推荐相关联医生，用于医患沟通；Recommend related doctors based on the patient's self-reported information for doctor-patient communication;
检索互联网主流医疗问答资讯，用于推荐专家答疑；Search the mainstream medical Q&A information on the Internet to recommend experts to answer questions;
患者与AI客服在线沟通，咨询问题，得到答案。具体方法采用：实体识别、意图识别、图谱检索。实体识别步骤将患者输入的问题做严格检索匹配和相似度近似匹配实体识别，即同义置换，生成标准表述问题；所述意图识别步骤将匹配或同义置换处理后的患者标准表述问题送入机器学习模型训练分类，做问题分类，如：含有“怎么治”、“吃什么药”这一类关键词的句子倾向于是咨询治疗法方法类问题，含有“多久”、“康复”类关键词的句子倾向于是咨询治疗周期类问题；所述图谱检索步骤将患者问题按问题分类导向答案模板，如：治疗法方法类问题的答案模版为“疾病{}的治疗方法有{}”，询问治疗周期类问题的答案模版为“疾病{}的治愈周期为{}”，并对关键词检索慢病知识图谱、影像知识图谱等数据资源，填充模版，如针对关键词“中耳炎”，检索到治疗方法“使用滴耳剂苯酚甘油，口服消炎药”，检索到治愈周期为“10-14天”,向患者返回完整答案，“疾病中耳炎的治疗方法有：使用滴耳剂苯酚甘油，口服消炎药”，“疾病中耳炎的治愈周期为10-14天”。Patients communicate with AI customer service online, ask questions, and get answers. The specific methods used are: entity recognition, intent recognition, and graph retrieval. The entity recognition step performs strict search matching and similarity approximate matching entity recognition on the questions input by the patients, that is, synonym replacement, to generate standard statement questions; the intent recognition step sends the patient's standard statement questions after matching or synonym replacement processing to the machine learning model training classification for question classification, such as: sentences containing keywords such as "how to treat" and "what medicine to take" tend to be questions about consulting treatment methods, and sentences containing keywords such as "how long" and "recovery" tend to be questions about consulting treatment cycles; the graph retrieval step guides patient questions according to question classification to answer templates, such as: treatment methods The answer template for questions about treatment methods is "The treatment methods for disease {} are {}", and the answer template for questions about treatment cycles is "The cure cycle for disease {} is {}". The data resources such as chronic disease knowledge graph and imaging knowledge graph are searched for keywords to fill in the template. For example, for the keyword "otitis media", the treatment method "using ear drops phenol glycerin and oral anti-inflammatory drugs" is retrieved, and the cure cycle is retrieved as "10-14 days". The complete answer is returned to the patient, "The treatment methods for disease otitis media are: using ear drops phenol glycerin and oral anti-inflammatory drugs", and "The cure cycle for disease otitis media is 10-14 days".
记录患者自诉信息、患者与AI客服聊天信息、医患沟通信息、专家答疑信息，入库；Record patient self-report information, patient chat information with AI customer service, doctor-patient communication information, and expert Q&A information, and store them in the database;
将所述记录患者自己诉信息、患者与AI客服聊天信息、医患沟通信息、专家答疑信息从数据库取出，分析录入影像数据语料库，用于返回AI客服问答系统单元分析与组织对话，数据循环利用，同步用于后续匹配模块训练学习。The records of the patient's own complaints, the patient's chat information with the AI customer service, the doctor-patient communication information, and the expert Q&A information are retrieved from the database, analyzed and entered into the image data corpus, and used to return to the AI customer service question and answer system unit to analyze and organize the dialogue. The data is recycled and used synchronously for subsequent matching module training and learning.
S2:根据患者信息生成结构化病例，将病例分别推送给患者端和医生端。S2: Generate structured cases based on patient information and push the cases to the patient and doctor ends respectively.
具体地，获取患者主诉信息等数据；Specifically, data such as patient complaints are obtained;
将患者信息生成结构化病例结果数据；Generate structured case outcome data from patient information;
获取生成病例结果数据并在患者端作信息展示；Obtain and generate case result data and display the information on the patient side;
获取生成病例结果数据并在医生端作病例列表及信息展示。Obtain generated case result data and display case lists and information on the doctor's side.
S3:创建、整合慢病知识图谱、影像检查图谱、影像专业诊断资料等医学影像数据资源库。S3: Create and integrate medical imaging data resource libraries such as chronic disease knowledge graphs, imaging examination graphs, and imaging professional diagnostic materials.
具体地，将关系型病名、介绍、病因、病症、治疗、预防、日常等慢病科谱知识数据，生成实体与关系对的图数据，存入慢病知识图谱图数据库；Specifically, the relational disease name, introduction, cause, symptoms, treatment, prevention, daily life and other chronic disease knowledge data are used to generate graph data of entity and relationship pairs and stored in the chronic disease knowledge graph database;
将关系型病症、检查、适应症、诊疗方案、设备、扫描方式、检查身体摆位等影像专业资料数据，生成实体与关系对的图数据，存入影像检查知识图谱图数据库；Generate graph data of entity and relationship pairs from relational disease, examination, indication, treatment plan, equipment, scanning method, examination body position and other imaging professional data, and store them in the imaging examination knowledge graph database;
获取原始影像报告数据知识库；Obtain the original imaging report data knowledge base;
获取互联网医疗咨询专家答疑数据；Obtain data on questions and answers from Internet medical consultation experts;
提取患者病例、报告、互联网专家答疑数据中的实体与关系对，实时自动创建扩展知识知识图谱。Extract entity and relationship pairs from patient cases, reports, and Internet expert Q&A data, and automatically create an extended knowledge graph in real time.
S4:根据整合匹配算法、匹配规则、知识图谱检索规则构建匹配模型，从医学影像数据资源库中提取特征，制作影像数据语料库，采用机器学习方法训练匹配模型，将病例信息输入匹配模型进行匹配，得到多路融合粗筛召回(推荐)结果集。S4: Build a matching model based on the integrated matching algorithm, matching rules, and knowledge graph retrieval rules, extract features from the medical image data resource library, create an image data corpus, use machine learning methods to train the matching model, input case information into the matching model for matching, and obtain a multi-channel fusion coarse screening recall (recommendation) result set.
具体地，获取慢病知识图谱、影像检查知识图谱、影像报告知识库、大数据专家答疑、自扩展知识图谱中的数据，分析处理，提取特征，用数学向量化符号表示，制作影像数据语料库，用于机器学习训练意图与匹配。根据患者病例数据检索慢病管理知识图谱、影像检查知识图谱、自扩展知识图谱，找到关联匹配结果实体关系对，得到图谱检索召回结果集。搭建神经网络模型，将影像数据语料送入模型，机器学习、训练，对患者病例信息作意图分析、预测，用于生成问答对、风险预测候选集，得到神经网络结果集，辅助图谱检索与匹配。将影像数据语料库中的数据制作模型训练所需的匹配句子对数据，提取相似性特征：提取匹配句子双方句长特征，用句长相差度来衡量意图相似性；提取匹配句子双方所含最长相同字符串长度特征，用句中最长重复词衡量相似性；提取匹配双方所含最长相同字符串长度累加值特征，用句中重复词长度衡量相似性；提取匹配双方变换复杂度，用双方变换复杂程度衡量句长、句序及重复词维度的相似性特征；提取匹配双方两字组合、三字组合重复词特征，用句中相临字相同个数指标衡量相似性；提取匹配双方jieba分词后的常规词相同个数，用句中常规词的重复性指标衡量相似性；提取、比对匹配双方相同的疑问词特征，用疑问词定位问题类型，把握句子主体的相似性；提取用于为匹配双方含有的权重关键词加权重，提高相似度计算的分值参考；将匹配双方句子做数学向量化符号处理，用于直接送入BiLSTM(双向循环长短时记忆)深度学习模型、Attention(注意力)机制模型(Bert)学习相似性规律；将相似度特征提取单元提取的特征送入匹配模型，学习相似性特征：计算基于患者(UserCF)相似性结果集与基于病症(ItemCF)相似性的结果集；将UserCF或ItemCF中的共现矩阵分解，计算患者与病症信息隐含语义的关联相似性；利用决策树机器学习模型对上下文信息自动进行特征筛选组合，生成新的离散特征向量，通过LR(逻辑回归)模型生成预测结果，弥补协同过滤仅利用患者与病症之间关联交互信息，忽视患者自身特征与病症信息特征的不足；采用深度学习模型Wide部分快速关联规则粗筛相似性，增强模型的“直接记忆能力”，Deep部分深入抽象训练模型，增强模型的“抽象泛化能力”，达到效率与精确性结合的目的；将相似度提取特征中的权重特征词加权召回结果；将图谱检索结果集、神经网络结果集、匹配模型结果集、规则权重结果集、推荐模型结果集等多路召回结果集拼接，归一化为0到1之间的去量纲、标量化指标概率值，得到多路融合粗筛召回结果集。Specifically, data from the chronic disease knowledge graph, imaging examination knowledge graph, imaging report knowledge base, big data expert Q&A, and self-expanding knowledge graph are obtained, analyzed and processed, features are extracted, and represented with mathematical vectorized symbols to create an imaging data corpus for machine learning training intent and matching. According to the patient case data, the chronic disease management knowledge graph, imaging examination knowledge graph, and self-expanding knowledge graph are retrieved to find the entity relationship pairs of the associated matching results and obtain the graph retrieval recall result set. A neural network model is built, and the imaging data corpus is fed into the model for machine learning and training. The patient case information is analyzed and predicted for intent, which is used to generate question-answer pairs and risk prediction candidate sets, and a neural network result set is obtained to assist in graph retrieval and matching. The data in the image data corpus are used to make the matching sentence pair data required for model training, and similarity features are extracted: the sentence length features of both sides of the matching sentences are extracted, and the sentence length difference is used to measure the similarity of intention; the longest identical string length features of both sides of the matching sentences are extracted, and the longest repeated word in the sentence is used to measure the similarity; the cumulative value features of the longest identical string lengths contained in both sides of the matching sentences are extracted, and the length of repeated words in the sentence is used to measure the similarity; the transformation complexity of both sides is extracted, and the transformation complexity of both sides is used to measure the similarity features of sentence length, sentence order and repeated word dimensions; the repeated word features of two-character combinations and three-character combinations of both sides are extracted, and the length of adjacent words in the sentence is used to measure the similarity of similar words in the sentence. The similarity is measured by the number of identical characters; the number of identical regular words after Jieba segmentation of the two matching parties is extracted, and the repetition index of regular words in the sentence is used to measure the similarity; the interrogative word features that are the same in the two matching parties are extracted and compared, and the question words are used to locate the question type and grasp the similarity of the main body of the sentence; the weights of the weighted keywords contained in the two matching parties are extracted to improve the score reference for similarity calculation; the sentences of the two matching parties are mathematically vectorized and symbolized, and are directly sent to the BiLSTM (bidirectional recurrent long short-term memory) deep learning model and the Attention mechanism model (Bert) to learn the similarity law; the features extracted by the similarity feature extraction unit are sent to the matching model to learn the similarity features: the similarity result set based on the patient (UserCF) and the result set based on the similarity of the symptom (ItemCF) are calculated; the co-occurrence matrix in UserCF or ItemCF is decomposed to calculate the association similarity of the implicit semantics of the patient and symptom information; the decision tree machine learning model is used to automatically perform feature screening and combination on the context information to generate a new discrete feature vector, and the prediction result is generated through the LR (logistic regression) model to make up for the fact that collaborative filtering only uses the association interaction information between the patient and the symptom, and ignores the patient's own characteristics The shortcomings of the symptom and symptom information features are overcome; the Wide part of the deep learning model is used to quickly associate rules to roughly screen similarities and enhance the "direct memory ability" of the model; the Deep part is used to deeply abstract the training model and enhance the "abstract generalization ability" of the model to achieve the purpose of combining efficiency and accuracy; the weighted feature words in the similarity extraction features are weighted to weight the recall results; the graph retrieval result set, neural network result set, matching model result set, rule weight result set, recommendation model result set and other multiple recall result sets are spliced and normalized to dimensionless and scalar indicator probability values between 0 and 1 to obtain a multi-channel fusion rough screening recall result set.
S5:将多路融合粗筛召回结果集提取与患者业务相关联特征，作为补充数据语料送入第一精排模型，进行训练、排序、筛选，得到科普性导诊信息、疾病知识、风险预测等结果集，向患者端发送推荐结果。S5: Extract the features related to the patient's business from the multi-channel fusion rough screening recall result set, and send it to the first fine sorting model as a supplementary data corpus for training, sorting, and screening to obtain a result set of popular science guidance information, disease knowledge, risk prediction, etc., and send the recommendation results to the patient end.
具体地，从多路召回结果集提取加强与患者端业务相关联特征作为第一提取补充业务特征，包括科室、检查项、疾病知识、生知指导、风险预测；Specifically, the features associated with the patient-side business are extracted from the multi-channel recall result set as the first extracted supplementary business features, including department, examination items, disease knowledge, bio-information guidance, and risk prediction;
将第一补充提取业务特征生成新的数据语料送入第一精排模型，做精细化机器学习、训练、计算相似度、排序、筛选得到患者端科普性推荐结果集；The first supplementary extraction of business features generates a new data corpus and feeds it into the first refined sorting model to perform refined machine learning, training, similarity calculation, sorting, and screening to obtain a patient-side popular science recommendation result set;
向患者端推荐结果，包括科室、检查项、疾病知识、生知指导、风险预测与患者病例结合，推荐给患者端应用。Recommend results to patients, including departments, examination items, disease knowledge, biomedical guidance, risk prediction and patient cases, and recommend them to patients for application.
S6:将多路融合粗筛召回结果集提取与医生业务相关联特征，作为补充数据语料送入第二精排模型，进行训练、排序、筛选，得到专业性诊疗方案、诊断提醒等服务结果集，向医生端发送推荐结果。S6: Extract the features related to the doctor's business from the multi-channel fusion rough screening recall result set, and send it to the second fine sorting model as a supplementary data corpus for training, sorting, and screening to obtain service result sets such as professional treatment plans and diagnosis reminders, and send recommendation results to the doctor.
具体地，将多路召回结果集提取加强与医生端业务相关联特征作为第二提取补充业务特征，包括专业检查项目、项目介绍、适应症、设备、扫描方式、检查身体摆位、诊疗方案；Specifically, the multi-channel recall result set is extracted to strengthen the features associated with the doctor's business as the second extraction of supplementary business features, including professional examination items, project introduction, indications, equipment, scanning methods, examination body positioning, and diagnosis and treatment plans;
将第二补充提取业务特征生成新的数据语料送入第二精排模型，做精细化机器学习、训练、计算相似度、排序、筛选得到专业性医生端推荐结果集；The second supplementary extraction of business features generates a new data corpus and feeds it into the second refined ranking model for refined machine learning, training, similarity calculation, sorting, and screening to obtain a professional doctor-side recommendation result set;
将患者病例中的病症、风险等重点词突出快捷提醒；Highlight key words such as symptoms and risks in patient records for quick reminders;
向医生端推荐结果，包括检查项目、项目介绍、适应症、设备、扫描方式、检查身体摆位、诊疗方案、重点词快捷提醒与患者病例结合，推荐给医生端应用。The results are recommended to doctors, including examination items, project introductions, indications, equipment, scanning methods, body positioning, treatment plans, quick reminders of key words, and patient cases, and recommended to doctors.
以上，为本发明第二实施例提供的一种基于人工智能的医学影像导诊导检方法的实施例说明。The above is a description of an embodiment of a medical imaging diagnosis and inspection guidance method based on artificial intelligence provided by the second embodiment of the present invention.
本发明提供的一种基于人工智能的医学影像导诊导检方法与上述基于人工智能的医学影像导诊导检系统出于相同的发明构思，具有相同的有益效果，此处不再赘述。The artificial intelligence-based medical image diagnosis and inspection guidance method provided by the present invention is based on the same inventive concept as the above-mentioned artificial intelligence-based medical image diagnosis and inspection guidance system and has the same beneficial effects, which will not be repeated here.
如图3所示，示出了本发明实施例还提供一种智能设备的结构框图，该设备包括处理器、输入设备、输出设备和存储器，所述处理器、输入设备、输出设备和存储器相互连接，所述存储器用于存储计算机程序，所述计算机程序包括程序指令，所述处理器被配置用于调用所述程序指令，执行上述第二实施例描述的方法。As shown in FIG3 , an embodiment of the present invention further provides a structural block diagram of an intelligent device, the device comprising a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory are interconnected, the memory is used to store a computer program, the computer program comprises program instructions, the processor is configured to call the program instructions and execute the method described in the second embodiment above.
应当理解，在本发明实施例中，所称处理器可以是中央处理单元(CentralProcessing Unit，CPU)，该处理器还可以是其他通用处理器、数字信号处理器(DigitalSignal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。It should be understood that in the embodiments of the present invention, the processor referred to may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
输入设备可以包括触控板、指纹采传感器(用于采集用户的指纹信息和指纹的方向信息)、麦克风等，输出设备可以包括显示器(LCD等)、扬声器等。Input devices may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint direction information), a microphone, etc., and output devices may include a display (LCD, etc.), a speaker, etc.
该存储器可以包括只读存储器和随机存取存储器，并向处理器提供指令和数据。存储器的一部分还可以包括非易失性随机存取存储器。例如，存储器还可以存储设备类型的信息。The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.
具体实现中，本发明实施例中所描述的处理器、输入设备、输出设备可执行本发明实施例提供的方法实施例所描述的实现方式，也可执行本发明实施例所描述的系统实施例的实现方式，在此不再赘述。In a specific implementation, the processor, input device, and output device described in the embodiments of the present invention may execute the implementation described in the method embodiment provided in the embodiments of the present invention, or may execute the implementation described in the system embodiment described in the embodiments of the present invention, which will not be described in detail here.
在本发明还提供一种计算机可读存储介质的实施例，所述计算机存储介质存储有计算机程序，所述计算机程序包括程序指令，所述程序指令当被处理器执行时使所述处理器执行上述第二实施例描述的方法。The present invention further provides an embodiment of a computer-readable storage medium, wherein the computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method described in the second embodiment.
所述计算机可读存储介质可以是前述实施例所述的终端的内部存储单元，例如终端的硬盘或内存。所述计算机可读存储介质也可以是所述终端的外部存储设备，例如所述终端上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(SecureDigital,SD)卡，闪存卡(Flash Card)等。进一步地，所述计算机可读存储介质还可以既包括所述终端的内部存储单元也包括外部存储设备。所述计算机可读存储介质用于存储所述计算机程序以及所述终端所需的其他程序和数据。所述计算机可读存储介质还可以用于暂时地存储已经输出或者将要输出的数据。The computer-readable storage medium may be an internal storage unit of the terminal described in the foregoing embodiments, such as a hard disk or memory of the terminal. The computer-readable storage medium may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the computer-readable storage medium may also include both an internal storage unit of the terminal and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，上述描述的终端和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the terminals and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.
在本申请所提供的几个实施例中，应该理解到，所揭露终端和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口、装置或单元的间接耦合或通信连接，也可以是电的，机械的或其它的形式连接。In the several embodiments provided in the present application, it should be understood that the disclosed terminal and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.
最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围，其均应涵盖在本发明的权利要求和说明书的范围当中。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110121696.3ACN112786194B (en) | 2021-01-28 | 2021-01-28 | Medical image diagnosis guiding and guiding system, method and equipment based on artificial intelligence |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110121696.3ACN112786194B (en) | 2021-01-28 | 2021-01-28 | Medical image diagnosis guiding and guiding system, method and equipment based on artificial intelligence |
| Publication Number | Publication Date |
|---|---|
| CN112786194A CN112786194A (en) | 2021-05-11 |
| CN112786194Btrue CN112786194B (en) | 2024-11-01 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110121696.3AActiveCN112786194B (en) | 2021-01-28 | 2021-01-28 | Medical image diagnosis guiding and guiding system, method and equipment based on artificial intelligence |
| Country | Link |
|---|---|
| CN (1) | CN112786194B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113270168B (en)* | 2021-05-19 | 2024-01-02 | 中科芯未来微电子科技成都有限公司 | Method and system for improving medical image processing capability |
| CN113327691B (en)* | 2021-06-01 | 2022-08-12 | 平安科技（深圳）有限公司 | Query method and device based on language model, computer equipment and storage medium |
| CN113343100B (en)* | 2021-06-25 | 2024-01-30 | 中关村智慧城市产业技术创新战略联盟 | Smart city resource recommendation method and system based on knowledge graph |
| CN113407841B (en)* | 2021-06-25 | 2023-04-28 | 陈亮 | Method and system for automatically recommending AI (automatic indicator) scheme based on performance analysis of structured report |
| CN113656706A (en)* | 2021-08-31 | 2021-11-16 | 平安医疗健康管理股份有限公司 | Information pushing method and device based on multi-mode deep learning model |
| CN113946651B (en)* | 2021-09-27 | 2024-05-10 | 盛景智能科技(嘉兴)有限公司 | Maintenance knowledge recommendation method and device, electronic equipment, medium and product |
| CN113963234B (en)* | 2021-10-25 | 2024-02-23 | 北京百度网讯科技有限公司 | Data annotation processing method, device, electronic equipment and medium |
| CN113851219A (en)* | 2021-11-29 | 2021-12-28 | 山东交通学院 | Intelligent diagnosis guiding method based on multi-mode knowledge graph |
| CN114496208A (en)* | 2022-02-16 | 2022-05-13 | 平安国际智慧城市科技股份有限公司 | Online consultation method, device and system and computer readable storage medium |
| CN114550915A (en)* | 2022-02-22 | 2022-05-27 | 深圳市医未医疗科技有限公司 | Method and system for automatically generating report in image diagnosis |
| CN114628012B (en)* | 2022-03-21 | 2023-09-05 | 中国人民解放军西部战区总医院 | Emergency department's preliminary examination sorting system |
| CN114925160B (en)* | 2022-04-18 | 2024-06-21 | 北京急救中心 | Pre-hospital emergency instruction recommendation system and method based on knowledge graph technology |
| CN114817505A (en)* | 2022-05-10 | 2022-07-29 | 国网江苏省电力有限公司南通供电分公司 | A fast reply method for power supply work order based on historical work order matching system |
| CN114822826A (en)* | 2022-05-16 | 2022-07-29 | 河南艾玛医疗科技有限公司 | Internet medical diagnosis system |
| CN115098651A (en)* | 2022-05-19 | 2022-09-23 | 四川大学华西医院 | Intelligent question-answering system for prostate cancer and implementation method thereof |
| CN115036034B (en)* | 2022-08-11 | 2022-11-08 | 之江实验室 | Similar patient identification method and system based on patient characterization map |
| CN115062165B (en)* | 2022-08-18 | 2022-12-06 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Medical image diagnosis method and device based on film reading knowledge graph |
| CN115905960B (en)* | 2023-03-08 | 2023-05-12 | 安徽通灵仿生科技有限公司 | Adverse event detection method and device based on ventricular assist device |
| CN115995287B (en)* | 2023-03-23 | 2023-06-13 | 山东远程分子互联网医院有限公司 | Cloud image data receiving and transmitting system and method |
| CN116631558B (en)* | 2023-05-29 | 2024-03-22 | 武汉大学人民医院（湖北省人民医院） | A construction method for Internet-based medical testing projects |
| CN116825312B (en)* | 2023-07-24 | 2024-07-02 | 广州腾方医信科技有限公司 | Triage system and triage method based on credit-invasive environment |
| CN116955733A (en)* | 2023-07-28 | 2023-10-27 | 上海联影智能医疗科技有限公司 | Image diagnosis learning recommendation method, device and storage medium |
| CN117153378B (en)* | 2023-10-31 | 2024-03-01 | 北京博晖创新生物技术集团股份有限公司 | Diagnosis guiding method and device, electronic equipment and storage medium |
| CN117688226B (en)* | 2024-02-02 | 2024-05-03 | 徐州医科大学 | Intelligent pre-diagnosis self-service ordering method and system based on matching similar pediatric patients |
| CN117711635B (en)* | 2024-02-05 | 2024-05-03 | 神州医疗科技股份有限公司 | Medical image inspection result analysis method and device |
| CN118522469B (en)* | 2024-07-22 | 2024-11-05 | 宁波紫湾科技有限公司 | Big data analysis medical decision method and system |
| CN119028566B (en)* | 2024-07-25 | 2025-06-10 | 上海诺诚电气股份有限公司 | Lung pathology information processing method, device, electronic device and computer medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110111886A (en)* | 2019-05-16 | 2019-08-09 | 闻康集团股份有限公司 | A kind of intelligent interrogation system and method based on XGBoost disease forecasting |
| CN110929016A (en)* | 2019-12-10 | 2020-03-27 | 北京爱医生智慧医疗科技有限公司 | Intelligent question and answer method and device based on knowledge graph |
| CN111449665A (en)* | 2020-03-02 | 2020-07-28 | 上海昊博影像科技有限公司 | Intelligent image diagnosis system |
| CN111816301A (en)* | 2020-07-07 | 2020-10-23 | 平安科技（深圳）有限公司 | Medical inquiry assisting method, device, electronic equipment and medium |
| CN111897967A (en)* | 2020-07-06 | 2020-11-06 | 北京大学 | A medical consultation recommendation method based on knowledge graph and social media |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106557653B (en)* | 2016-11-15 | 2017-09-22 | 合肥工业大学 | A kind of portable medical intelligent medical guide system and method |
| CN111613311A (en)* | 2020-06-09 | 2020-09-01 | 广东珠江智联信息科技股份有限公司 | Intelligent AI (Artificial intelligence) diagnosis guide realization technology |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110111886A (en)* | 2019-05-16 | 2019-08-09 | 闻康集团股份有限公司 | A kind of intelligent interrogation system and method based on XGBoost disease forecasting |
| CN110929016A (en)* | 2019-12-10 | 2020-03-27 | 北京爱医生智慧医疗科技有限公司 | Intelligent question and answer method and device based on knowledge graph |
| CN111449665A (en)* | 2020-03-02 | 2020-07-28 | 上海昊博影像科技有限公司 | Intelligent image diagnosis system |
| CN111897967A (en)* | 2020-07-06 | 2020-11-06 | 北京大学 | A medical consultation recommendation method based on knowledge graph and social media |
| CN111816301A (en)* | 2020-07-07 | 2020-10-23 | 平安科技（深圳）有限公司 | Medical inquiry assisting method, device, electronic equipment and medium |
| Publication number | Publication date |
|---|---|
| CN112786194A (en) | 2021-05-11 |
| Publication | Publication Date | Title |
|---|---|---|
| CN112786194B (en) | Medical image diagnosis guiding and guiding system, method and equipment based on artificial intelligence | |
| CN113345577B (en) | Diagnosis and treatment auxiliary information generation method, model training method, device, equipment and storage medium | |
| CN116383413B (en) | Knowledge graph updating method and system based on medical data extraction | |
| CN109785927A (en) | Clinical document structuring processing method based on internet integration medical platform | |
| TWI521467B (en) | Nursing decision support system | |
| CN113688255A (en) | Knowledge graph construction method based on Chinese electronic medical record | |
| CN118013001A (en) | Interactive knowledge interaction system based on knowledge base and large language model | |
| CN111477320B (en) | Construction system of treatment effect prediction model, treatment effect prediction system and terminal | |
| CN115394393A (en) | Intelligent diagnosis and treatment data processing method, device, electronic equipment and storage medium | |
| CN116992839A (en) | Automatic generation method, device and equipment for medical records front page | |
| CN119230090B (en) | Knowledge-graph-based medical records diagnosis and operation ICD coding method | |
| CN113593669A (en) | Intelligent medication recommendation method, system and device | |
| CN120144730B (en) | Auxiliary detection method, device, equipment and medium for oral diseases | |
| CN112562808A (en) | Patient portrait generation method and device, electronic equipment and storage medium | |
| CN113553840A (en) | Text information processing method, device, equipment and storage medium | |
| Gaur et al. | “Who can help me?”: Knowledge Infused Matching of Support Seekers and Support Providers during COVID-19 on Reddit | |
| CN119578497A (en) | A medical guidance model training method, system, terminal and medium | |
| CN119541887A (en) | A medical AI question answering method based on retrieval enhancement | |
| Chandra et al. | Natural language processing and ontology based decision support system for diabetic patients | |
| Kumar et al. | Natural language processing: Healthcare achieving benefits via NLP | |
| Nair et al. | Automated clinical concept-value pair extraction from discharge summary of pituitary adenoma patients | |
| CN117038026B (en) | Recommendation method, electronic equipment and medium for hospital specialists | |
| Sabra et al. | Performance evaluation for semantic-based risk factors extraction from clinical narratives | |
| TW202309928A (en) | System and method for automatic analysis of texts in psychotherapy, counseling, and other mental health management activities | |
| Chrimes et al. | Text mining using clinical terms in electronic records of annual falls of patients in home community care |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |