技术领域Technical Field
本发明涉及客户环境信息档案建立技术领域，特别涉及一种基于信息融合特征建立环境信息档案的方法、系统及设备。The present invention relates to the technical field of establishing a customer environment information file, and in particular to a method, system and device for establishing an environment information file based on information fusion features.
背景技术Background technique
当前金融业大量使用远程音视频技术，部分中介机构通过非法手段，集中办理金融业务，给银行带来大量经济和信誉损失。监控与排查这些风险，在千万级甚至上亿级别的图像数据库中，需要将具有相同背景的图像挑选出来，需要很好的记忆和极高的效率，人工排查显然很难做到。Currently, the financial industry makes extensive use of remote audio and video technology. Some intermediaries use illegal means to centrally handle financial business, causing huge economic and reputation losses to banks. To monitor and troubleshoot these risks, it is necessary to select images with the same background from a database of tens of millions or even hundreds of millions of images, which requires good memory and extremely high efficiency. Manual troubleshooting is obviously difficult to do.
因此，由于现有技术的限制，人们无法解决。Therefore, due to the limitations of existing technology, people cannot solve it.
发明内容Summary of the invention
（一）发明目的：为解决上述现有技术中存在的问题，本发明的目的是提供一种基于信息融合特征建立环境信息档案的方法、系统及设备，能够快速、全面分析场景信息，建立档案、检索和比对。(I) Purpose of the invention: In order to solve the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a method, system and device for establishing an environmental information archive based on information fusion features, which can quickly and comprehensively analyze scene information, establish archives, search and compare.
（二）技术方案：为了解决上述技术问题，本技术方案提供基于信息融合特征建立环境信息档案的方法，具体包括：(II) Technical solution: In order to solve the above technical problems, this technical solution provides a method for establishing an environmental information archive based on information fusion features, which specifically includes:
步骤1、获取远程图像的信息特征；Step 1, obtaining information features of remote images;
步骤2、将远程图像的各信息特征通过矩阵排列进行特征融合，将高疏离度、低密度的设备编码放置于高位；低疏离度、高密度的设备编码放置于低位，得到第一环境信息特征；根据第一环境信息特征建立环境信息特征库；Step 2: various information features of the remote image are arranged in a matrix for feature fusion, and the device codes with high alienation and low density are placed in a high position; the device codes with low alienation and high density are placed in a low position, so as to obtain the first environmental information feature; and an environmental information feature library is established according to the first environmental information feature;
步骤3、再次接收到新的图像，通过特征融合得到第二环境信息特征；在所述环境信息特征库中根据所述第二环境信息特征与所述第一环境信息特征的内积检索与所述第二环境信息特征相似的第一环境信息特征；Step 3: receiving a new image again, obtaining a second environment information feature by feature fusion; retrieving a first environment information feature similar to the second environment information feature in the environment information feature library according to the inner product of the second environment information feature and the first environment information feature;
步骤4、根据环境信息特征内积绝对值，输出相似环境告警或安全的结论。Step 4: Output similar environmental alarm or safety conclusion based on the absolute value of the inner product of the environmental information feature.
其中，所述步骤1具体包括：Wherein, the step 1 specifically includes:
步骤101、提取远程图像的背景信息，并利用视觉大模型对远程图像编码得到背景信息特征；Step 101, extracting background information of a remote image, and encoding the remote image using a visual macro model to obtain background information features;
步骤102、对采集的远程图像的远程设备结构化信息编码得到设备信息特征。Step 102: Encode the remote device structured information of the collected remote image to obtain device information features.
其中，所述高疏离度、低密度的设备编码指的是向量元素更少的一项；所述低疏离度、高密度的设备编码指的是向量元素更多的一项。The high-isolation, low-density device code refers to an item with fewer vector elements; the low-isolation, high-density device code refers to an item with more vector elements.
其中，所述高位指设备信息特征和背景信息特征在特征融合时所形成的矩阵的前端或上端部分；所述低位指的是设备信息特征和背景信息特征在特征融合时所形成的矩阵的后端或下端部分。Among them, the high position refers to the front end or upper part of the matrix formed by the device information features and the background information features when the features are fused; the low position refers to the back end or lower part of the matrix formed by the device information features and the background information features when the features are fused.
其中，所述步骤3还包括：Wherein, the step 3 further comprises:
步骤301、获取新的远程图像的信息特征并进行特征融合，得到第二环境信息特征；Step 301: Acquire information features of a new remote image and perform feature fusion to obtain second environment information features;
步骤302、根据所述第二环境信息特征在所述环境信息特征库中检索相似的第一环境信息特征。Step 302: Retrieve similar first environmental information features from the environmental information feature library according to the second environmental information features.
其中，当所述环境信息特征库中的第一环境信息特征与所述第二环境信息特征的环境信息特征内积绝对值小于等于第一阈值，判定有风险，输出相似环境信息提示或告警的结论；当所述环境信息特征库中的第一环境信息特征与所述第二环境信息特征的环境信息特征内积绝对值大于第一阈值，判定为无风险，输出安全的结论。Among them, when the absolute value of the inner product of the environmental information features of the first environmental information feature in the environmental information feature library and the second environmental information feature is less than or equal to a first threshold, it is determined that there is a risk and a conclusion of similar environmental information prompt or alarm is output; when the absolute value of the inner product of the environmental information features of the first environmental information feature in the environmental information feature library and the second environmental information feature is greater than the first threshold, it is determined that there is no risk and a safe conclusion is output.
其中，所述方法还包括：步骤5、根据环境信息特征内积绝对值，更新所述环境信息特征库。Wherein, the method further includes: step 5, updating the environmental information feature library according to the absolute value of the inner product of the environmental information feature.
其中，当所述环境信息特征库中的第一环境信息特征与所述第二环境信息特征的环境信息特征内积绝对值小于等于第一阈值，将所述第一环境信息特征向量和第二环境信息特征向量加权取平均值，得到第三环境信息特征，将所述第三环境信息特征替换所述第一环境信息特征，输入所述环境信息特征库中；Wherein, when the absolute value of the inner product of the first environmental information feature and the second environmental information feature in the environmental information feature library is less than or equal to a first threshold, the first environmental information feature vector and the second environmental information feature vector are weighted and averaged to obtain a third environmental information feature, and the third environmental information feature is used to replace the first environmental information feature and input into the environmental information feature library;
当所述环境信息特征库中的第一环境信息特征与所述第二环境信息特征的环境信息特征内积绝对值大于第一阈值，所述第二环境信息特征作为补充，直接输入所述环境信息特征库中。When the absolute value of the inner product of the first environmental information feature and the second environmental information feature in the environmental information feature library is greater than a first threshold, the second environmental information feature is directly input into the environmental information feature library as a supplement.
基于信息融合特征建立环境信息档案的系统，适用于所述基于信息融合特征建立环境信息档案的方法，包括编码单元、特征融合单元、结果判定单元和特征修正单元；所述编码单元将远程设备以及所述远程设备采集到的远程图像进行编码，得到对应的信息特征；The system for establishing an environmental information archive based on information fusion features is applicable to the method for establishing an environmental information archive based on information fusion features, comprising an encoding unit, a feature fusion unit, a result determination unit and a feature correction unit; the encoding unit encodes a remote device and a remote image collected by the remote device to obtain corresponding information features;
所述特征融合单元将所述编码单元得到的各信息特征进行融合，得到第一环境信息特征，并存储在所述环境信息特征库中；The feature fusion unit fuses the information features obtained by the encoding unit to obtain a first environmental information feature, and stores the first environmental information feature in the environmental information feature library;
所述结果判定单元将新输入远程图像通过特征融合得到的第二环境信息特征在所述环境信息特征库中与第一环境信息特征求内积得到环境信息特征内积绝对值；The result determination unit calculates the inner product of the second environment information feature obtained by feature fusion of the newly input remote image with the first environment information feature in the environment information feature library to obtain the absolute value of the inner product of the environment information feature;
所述特征修正单元对环境信息特征库中存储的环境信息特征进行优化更新。The feature correction unit optimizes and updates the environmental information features stored in the environmental information feature library.
基于信息融合特征建立环境信息档案的设备，适用于所述基于信息融合特征建立环境信息档案的方法和系统，包括处理器，存储器和显示器；所述处理器，存储器和显示器通过通信总线和/或其他形式的连接机构互连并相互通讯。A device for establishing an environmental information archive based on information fusion features, suitable for the method and system for establishing an environmental information archive based on information fusion features, includes a processor, a memory and a display; the processor, the memory and the display are interconnected and communicate with each other through a communication bus and/or other forms of connection mechanisms.
（三）有益效果：本发明基于信息融合特征建立环境信息档案的方法、系统及设备，通过将远程图像的背景信息特征和设备信息特征融合成一个环境信息特征，实现快速、全面分析场景信息，建立档案、检索和比对的效果。(III) Beneficial effects: The method, system and device of the present invention for establishing an environmental information archive based on information fusion features can achieve the effects of rapid and comprehensive analysis of scene information, establishment of archives, retrieval and comparison by fusing the background information features and device information features of a remote image into one environmental information feature.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1是本发明基于信息融合特征建立环境信息档案的方法流程图；FIG1 is a flow chart of a method for establishing an environmental information archive based on information fusion features according to the present invention;
图2是本发明基于信息融合特征建立环境信息档案的方法流程框图；2 is a flowchart of a method for establishing an environmental information archive based on information fusion features according to the present invention;
图3是本发明步骤1的详细流程图；FIG3 is a detailed flow chart of step 1 of the present invention;
图4是本发明步骤3的详细流程图；FIG4 is a detailed flow chart of step 3 of the present invention;
图5是本发明基于信息融合特征建立环境信息档案的系统结构图；5 is a system structure diagram of establishing an environmental information archive based on information fusion features of the present invention;
图6是本发明基于信息融合特征建立环境信息档案的设备结构图。FIG6 is a structural diagram of a device for establishing an environmental information archive based on information fusion features according to the present invention.
具体实施方式Detailed ways
下面结合优选的实施例对本发明做进一步详细说明，在以下的描述中阐述了更多的细节以便于充分理解本发明，但是，本发明显然能够以多种不同于此描述的其他方式来实施，本领域技术人员可以在不违背本发明内涵的情况下根据实际应用情况作类似推广、演绎，因此不应以此具体实施例的内容限制本发明的保护范围。The present invention is further described in detail below in conjunction with preferred embodiments. More details are elaborated in the following description to facilitate a full understanding of the present invention. However, the present invention can obviously be implemented in a variety of other ways different from the description. Those skilled in the art can make similar generalizations and deductions based on actual application situations without violating the connotation of the present invention. Therefore, the protection scope of the present invention should not be limited by the content of this specific embodiment.
附图是本发明的实施例的示意图，需要注意的是，此附图仅作为示例，并非是按照等比例的条件绘制的，并且不应该以此作为对本发明的实际要求保护范围构成限制。The accompanying drawings are schematic diagrams of embodiments of the present invention. It should be noted that the drawings are only examples and are not drawn to scale, and should not be used to limit the actual protection scope of the present invention.
基于信息融合特征建立环境信息档案的方法，通过将图像特征和位置及信道信息进行融合，实现在不降低精度的情况下，相似背景的快速配对，从而得到安全与否的结论。如图1、图2所示，具体包括以下步骤：The method of establishing environmental information archive based on information fusion features, by fusing image features with position and channel information, can achieve rapid matching of similar backgrounds without reducing accuracy, thereby obtaining a conclusion on safety or not. As shown in Figures 1 and 2, the method specifically includes the following steps:
步骤1、获取远程图像的信息特征。Step 1: Obtain information features of remote images.
步骤2、将远程图像的各信息特征通过矩阵排列进行特征融合，将高疏离度、低密度的设备编码放置于高位；低疏离度、高密度的设备编码放置于低位，得到第一环境信息特征；根据第一环境信息特征建立环境信息特征库。Step 2: The information features of the remote image are arranged in a matrix for feature fusion, and the device codes with high alienation and low density are placed at high positions; the device codes with low alienation and high density are placed at low positions to obtain the first environmental information feature; and an environmental information feature library is established based on the first environmental information feature.
步骤3、再次接收到新的图像，通过特征融合得到第二环境信息特征；在所述环境信息特征库中根据所述第二环境信息特征与所述第一环境信息特征的内积检索与所述第二环境信息特征相似的第一环境信息特征。Step 3: Receive a new image again, and obtain a second environmental information feature through feature fusion; retrieve a first environmental information feature similar to the second environmental information feature in the environmental information feature library based on the inner product of the second environmental information feature and the first environmental information feature.
步骤4、根据环境信息特征内积绝对值，输出相似环境告警或安全的结论。Step 4: Output similar environmental alarm or safety conclusion based on the absolute value of the inner product of the environmental information feature.
步骤5、根据环境信息特征内积绝对值，更新所述环境信息特征库。Step 5: Update the environmental information feature library according to the absolute value of the inner product of the environmental information feature.
需要注意的是，所述步骤4和步骤5不分先后顺序，可以同时进行，也可以选择其一进行，根据实际情况进行选择，在此不做具体限制。It should be noted that there is no particular order for step 4 and step 5, and they can be performed simultaneously or one of them can be selected based on the actual situation. No specific restrictions are made here.
更为具体的，如图3所示，所述步骤1具体包括：More specifically, as shown in FIG3 , the step 1 specifically includes:
步骤101、提取远程图像的背景信息，并利用视觉大模型对远程图像编码得到背景信息特征。Step 101: extract background information of the remote image, and use the visual macro model to encode the remote image to obtain background information features.
图像编码：对输入的远程图像，首先提取所述远程图像的背景环境，利用视觉大模型对所述远程图像的背景环境进行归一化编码，得到背景信息特征向量。Image coding: For the input remote image, first extract the background environment of the remote image, and use the visual big model to normalize and encode the background environment of the remote image to obtain the background information feature vector .
本发明所述视觉大模型包括但不限于ViT、CLIP、SimCLR等基于transformer结构的视觉预训练模型在内所有的视觉大模型。The visual large model described in the present invention includes but is not limited to all visual large models including ViT, CLIP, SimCLR and other visual pre-training models based on transformer structure.
步骤102、对采集的远程图像的远程设备结构化信息编码得到设备信息特征。Step 102: Encode the remote device structured information of the collected remote image to obtain device information features.
设备编码：对采集到的远程设备信息进行编码，包括位置坐标、设备ID等。并且如果所述设备有其他的特殊标识，也可以进行单独的编码，例如：摄像头、屏幕或者元器件等。所述远程设备信息都是结构化的信息，可以直接进行编码，得到对应的特征向量，依次类推分别命名为、/>、/>……。Device encoding: Encode the collected remote device information, including location coordinates, device ID, etc. And if the device has other special identifications, they can also be encoded separately, such as: camera, screen or components, etc. The remote device information is structured information and can be directly encoded to obtain the corresponding feature vector, which is named by analogy. 、/> 、/> …….
其中，需要注意的是=0或≥1，/>=0或≥1，=0或≥1……。Among them, it should be noted that =0 or ≥1,/> =0 or ≥1, =0 or ≥1…….
更为具体的，所述步骤2具体包括：More specifically, the step 2 specifically includes:
使用特征融合算法，将所述背景信息特征和设备信息特征进行特征融合，通过简单的矩阵排列计算，得到第一环境信息特征。Using feature fusion algorithm , the background information feature and the device information feature are feature-fused, and the first environment information feature is obtained through simple matrix arrangement calculation.
本发明优选的远程设备为位置坐标和设备ID，所对应的设备编码为位置信息特征和设备ID信息特征。在进行特征融合时，将高疏离度、低密度的设备编码放置于高位；低疏离度、高密度的设备编码放置于低位。The preferred remote device of the present invention is the location coordinates and device ID, and the corresponding device code is the location information feature and the device ID information feature. When performing feature fusion, the device code with high alienation and low density is placed in the high position; the device code with low alienation and high density is placed in the low position.
所述高疏离度、低密度的设备编码指的是位置信息和设备ID表示的向量中元素更少的一项，所对应的位置信息特征或设备ID信息特征中元素更少的一项；所述低疏离度、高密度的设备编码指的是位置信息和设备ID表示的向量中元素更多的一项，所对应的位置信息特征或设备ID信息特征中元素更高的一项。The high-isolation, low-density device code refers to an item with fewer elements in the vector represented by the location information and the device ID, and the corresponding item has fewer elements in the location information feature or the device ID information feature; the low-isolation, high-density device code refers to an item with more elements in the vector represented by the location information and the device ID, and the corresponding item has a higher element in the location information feature or the device ID information feature.
例如一个设备ID可能有很多种，那么位置信息也有很多种，但是对位置信息做一定的区域划分分析，所述位置信息就没有所述设备ID多，就认为所述位置信息是一个低密度的，也就是高疏离度的。同理，所述设备ID是高密度的、低疏离度的。因此，在进行特征融合算法计算时，所述设备ID位于矩阵的上位，所述位置信息位于矩阵的下位进行计算。如果说位置信息没有做区分，或者根据客户的要求，不需要做区分，所述设备ID的数量是有限的情况下，那么设备ID就属于高疏离度。For example, there may be many types of device IDs, and there are also many types of location information. However, if a certain area division analysis is performed on the location information, the location information is not as much as the device ID, and the location information is considered to be low-density, that is, high-isolation. Similarly, the device ID is high-density and low-isolation. Therefore, when performing feature fusion algorithm calculations, the device ID is located in the upper position of the matrix, and the location information is located in the lower position of the matrix for calculation. If the location information is not differentiated, or according to customer requirements, no differentiation is required, and the number of device IDs is limited, then the device ID is high-isolation.
所述高位指设备信息特征和背景信息特征在特征融合时所形成的矩阵的前端或上端部分；而所述低位指的是设备信息特征和背景信息特征在特征融合时所形成的矩阵的后端或下端部分。The high bits refer to the front end or upper part of the matrix formed by the device information features and the background information features when the features are fused; and the low bits refer to the back end or lower part of the matrix formed by the device information features and the background information features when the features are fused.
下面结合具体的实施例进行描述：首先将采集到的远程设备的结构化信息进行编码，得到设备信息特征 = [4,0,0,0,0,0,0,0]；/> = [1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]；并且同时将所述远程设备采集到的远程图像的背景环境进行归一化编码，得到背景信息特征/> = [0.056498,0.056654,0.060541,0.002156,...,0.002165,0.034123]。之后将所述设备信息特征和所述背景信息特征进行特征融合，得到第一环境信息特征Feat = [4,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.056498,0.056654,0.060541,0.002156...,0.002165,0.034123]。The following is a description of a specific embodiment: First, the structured information of the remote device collected is encoded to obtain the device information feature = [4,0,0,0,0,0,0,0]; /> = [1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]; and at the same time, the background environment of the remote image collected by the remote device is normalized and encoded to obtain the background information feature/> = [0.056498,0.056654,0.060541,0.002156,...,0.002165,0.034123]. Then, the device information feature and the background information feature are fused to obtain the first environment information feature Feat = [4,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.056498,0.056654,0.060541,0.002156...,0.002165,0.034123].
将背景信息特征和设备信息特征融合得到的所述第一环境信息特征存储进环境信息特征库，方便存储所述第一环境信息特征进行后续步骤3的检索以及步骤4的输出结论或步骤5的更新优化。The first environmental information feature obtained by fusing the background information feature and the device information feature is stored in an environmental information feature library, so as to facilitate the storage of the first environmental information feature for subsequent retrieval of step 3 and output conclusion of step 4 or update optimization of step 5.
更为具体的，如图4所示，所述步骤3具体包括：More specifically, as shown in FIG4 , step 3 specifically includes:
步骤301、获取新的远程图像的信息特征并进行特征融合，得到第二环境信息特征。当再次收到远程视频或截图时，根据步骤1~步骤2所述的方法进行图像编码和设备编码，将对应得到的背景信息特征和设备信息特征进行特征融合，得到远程视频图像的第二环境信息特征。Step 301, obtain the information features of the new remote image and perform feature fusion to obtain the second environment information features. When the remote video or screenshot is received again, image encoding and device encoding are performed according to the method described in steps 1 to 2, and the corresponding background information features and device information features are feature fused to obtain the second environment information features of the remote video image.
步骤302、根据所述第二环境信息特征在所述环境信息特征库中检索相似的第一环境信息特征。所述检索指根据所述第二环境信息特征与所述第一环境信息特征求内积，即将所述第一环境信息特征的矩阵与所述第二环境信息特征矩阵点乘得到环境信息特征内积绝对值。所述环境信息特征内积绝对值表示相似度，当所述环境信息特征内积绝对值小于等于第一阈值时，表示所述第二环境信息特征与所述第一环境信息特征相似；当所述环境信息特征内积绝对值大于第一阈值时，表示所述第二环境信息特征与所述第一环境信息特征不相似。Step 302: Retrieve similar first environmental information features from the environmental information feature library based on the second environmental information feature. The retrieval refers to finding the inner product of the second environmental information feature and the first environmental information feature, that is, multiplying the matrix of the first environmental information feature by the matrix of the second environmental information feature to obtain the absolute value of the inner product of the environmental information feature. The absolute value of the inner product of the environmental information feature represents the similarity. When the absolute value of the inner product of the environmental information feature is less than or equal to a first threshold, it indicates that the second environmental information feature is similar to the first environmental information feature; when the absolute value of the inner product of the environmental information feature is greater than the first threshold, it indicates that the second environmental information feature is not similar to the first environmental information feature.
所述第一阈值是预设值，可以根据实际不同的应用场景精度进行修改。但是需要注意的是，一旦设定完成，在整个方法流程中将不可再修改。The first threshold is a preset value and can be modified according to the accuracy of different actual application scenarios. However, it should be noted that once the setting is completed, it cannot be modified in the entire method process.
需要注意的是，所述环境信息特征库是被第二环境信息特征不断修正的环境信息特征库。即第一次检索的环境信息特征库由第一环境信息特征组成，根据第二环境信息特征和第一环境信息特征的检索结果来修正所述环境信息特征库，并将此时修正过的环境信息特征库中的环境信息特征重新命名为第一环境信息特征。而第二次检索的环境信息特征库是被第二环境信息特征修正的环境信息特征组成。依次类推，所述环境信息特征库由上一次检索的第二环境信息特征修正得到。所述环境信息特征库在使用中被不断的更新优化。It should be noted that the environmental information feature library is an environmental information feature library that is continuously corrected by the second environmental information feature. That is, the environmental information feature library retrieved for the first time is composed of the first environmental information feature, and the environmental information feature library is corrected according to the search results of the second environmental information feature and the first environmental information feature, and the environmental information feature in the corrected environmental information feature library is renamed as the first environmental information feature. The environmental information feature library retrieved for the second time is composed of environmental information features corrected by the second environmental information feature. And so on, the environmental information feature library is corrected by the second environmental information feature retrieved last time. The environmental information feature library is constantly updated and optimized during use.
更为具体的，所述步骤4具体为：More specifically, the step 4 is as follows:
根据所述步骤3中在所述环境信息特征库中的检索结果，如果所述环境信息特征库中的第一环境信息特征有与所述第二环境信息特征的环境信息特征内积绝对值小于等于第一阈值，判定有风险，输出相似环境信息提示或告警的结论。如果所述环境信息特征库中的第一环境信息特征有与所述第二环境信息特征的环境信息特征内积绝对值大于第一阈值，判定为无风险，输出安全的结论。According to the search result in the environmental information feature library in step 3, if the absolute value of the inner product of the first environmental information feature in the environmental information feature library with the environmental information feature of the second environmental information feature is less than or equal to the first threshold, it is determined that there is a risk, and a conclusion of similar environmental information prompt or alarm is output. If the absolute value of the inner product of the first environmental information feature in the environmental information feature library with the environmental information feature of the second environmental information feature is greater than the first threshold, it is determined that there is no risk, and a conclusion of safety is output.
所述相似环境信息提示或告警的结论以及安全的结论都只是一个简单的提示和/或结果显示，后续还需要进行人工或者计算机的大量排查，在此不做具体的限制。The conclusion of the similar environmental information prompt or warning and the safety conclusion are just a simple prompt and/or result display, and a large amount of manual or computer investigation is required later, and no specific restrictions are made here.
更为具体的，所述步骤5具体为：More specifically, the step 5 is as follows:
根据所述步骤3中在所述环境信息特征库中的检索结果，更新优化所述环境信息特征库。当所述环境信息特征库中的第一环境信息特征与所述第二环境信息特征的环境信息特征内积绝对值小于等于第一阈值，通过所述第二环境特征向量优化所述环境信息特征库；将所述第一环境信息特征和所述第二环境信息特征进行结合，即将所述第一环境信息特征向量和第二环境信息特征向量加权取平均值，得到第三环境信息特征，将所述第三环境信息特征替换所述第一环境信息特征，输入所述环境信息特征库中。当所述环境信息特征库中的第一环境信息特征与所述第二环境信息特征的环境信息特征内积绝对值大于第一阈值，则将所述第二环境信息特征作为补充，直接输入所述环境信息特征库中。在不断的检索使用过程中，所述环境信息特征库被不断的更新优化，而检索的结果也会更为准确。According to the search result in the environmental information feature library in step 3, the environmental information feature library is updated and optimized. When the absolute value of the inner product of the environmental information features of the first environmental information feature and the second environmental information feature in the environmental information feature library is less than or equal to the first threshold, the environmental information feature library is optimized by the second environmental feature vector; the first environmental information feature and the second environmental information feature are combined, that is, the first environmental information feature vector and the second environmental information feature vector are weighted and averaged to obtain a third environmental information feature, and the third environmental information feature replaces the first environmental information feature and is input into the environmental information feature library. When the absolute value of the inner product of the environmental information features of the first environmental information feature and the second environmental information feature in the environmental information feature library is greater than the first threshold, the second environmental information feature is used as a supplement and directly input into the environmental information feature library. In the process of continuous retrieval and use, the environmental information feature library is continuously updated and optimized, and the retrieval results will be more accurate.
基于信息融合特征建立环境信息档案的系统，适用于所述基于信息融合特征建立环境信息档案的方法，如图5所示，包括编码单元、特征融合单元、结果判定单元和特征修正单元。所述编码单元将远程设备以及所述远程设备采集到的远程图像进行编码，得到对应的信息特征。所述编码单元还包括背景编码模块和设备编码模块；所述背景编码模块将所述远程图像中的背景提取出来并进行归一化编码，得到背景信息特征；所述设备编码模块将所述远程设备的结构化信息进行编码，得到设备信息特征，并依次命名为/>，……。The system for establishing an environmental information archive based on information fusion features is applicable to the method for establishing an environmental information archive based on information fusion features, as shown in FIG5 , and includes a coding unit, a feature fusion unit, a result determination unit, and a feature correction unit. The coding unit encodes the remote device and the remote image collected by the remote device to obtain corresponding information features. The coding unit also includes a background coding module and a device coding module; the background coding module extracts the background from the remote image and performs normalized coding to obtain background information features. ; The device encoding module encodes the structured information of the remote device to obtain device information features, which are named in turn/> , …….
所述特征融合单元将所述编码单元得到的各信息特征进行融合，得到第一环境信息特征，并存储在所述环境信息特征库中。The feature fusion unit fuses the information features obtained by the encoding unit to obtain a first environmental information feature, and stores the first environmental information feature in the environmental information feature library.
所述结果判定单元将新输入远程图像通过特征融合得到的第二环境信息特征在所述环境信息特征库中与第一环境信息特征求内积得到环境信息特征内积绝对值。当所述环境信息特征库中有第一环境信息特征与所述第二环境信息特征的环境信息特征内积绝对值小于等于第一阈值时，则判定有风险，输出提示或预警的结论；当所述环境信息特征库中有第一环境信息特征与所述第二环境信息特征的环境信息特征内积绝对值大于第一阈值时，判定为无风险，则输出安全的结论。The result determination unit calculates the inner product of the second environmental information feature obtained by feature fusion of the newly input remote image with the first environmental information feature in the environmental information feature library to obtain the absolute value of the inner product of the environmental information feature. When the absolute value of the inner product of the environmental information feature of the first environmental information feature and the second environmental information feature in the environmental information feature library is less than or equal to the first threshold, it is determined that there is a risk, and a prompt or warning conclusion is output; when the absolute value of the inner product of the environmental information feature of the first environmental information feature and the second environmental information feature in the environmental information feature library is greater than the first threshold, it is determined that there is no risk, and a safe conclusion is output.
所述特征修正单元对环境信息特征库中存储的环境信息特征进行优化更新。当所述环境信息特征库中有第一环境信息特征与所述第二环境信息特征的环境信息特征内积绝对值小于等于第一阈值时，所述特征修正单元将所述第一环境信息特征和第二环境信息特征进行加权取平均值，得到第三环境信息特征；所述第三环境信息特征替换所述第一环境信息特征，存储在所述环境信息特征库。当所述环境信息特征库中有第一环境信息特征与所述第二环境信息特征的环境信息特征内积绝对值大于第一阈值时，将所述第二环境信息特征作为补充，直接存储在所述环境信息特征库。The feature correction unit optimizes and updates the environmental information features stored in the environmental information feature library. When the absolute value of the inner product of the environmental information features of the first environmental information feature and the second environmental information feature in the environmental information feature library is less than or equal to a first threshold, the feature correction unit weights and averages the first environmental information feature and the second environmental information feature to obtain a third environmental information feature; the third environmental information feature replaces the first environmental information feature and is stored in the environmental information feature library. When the absolute value of the inner product of the environmental information features of the first environmental information feature and the second environmental information feature in the environmental information feature library is greater than a first threshold, the second environmental information feature is used as a supplement and directly stored in the environmental information feature library.
基于信息融合特征建立环境信息档案的设备，适用于所述基于信息融合特征建立环境信息档案的方法和系统，如图6所示，包括处理器100，存储器110和显示器120。所述处理器100，存储器110和显示器120通过通信总线130和/或其他形式的连接机构互连并相互通讯。The device for establishing an environmental information archive based on information fusion features is applicable to the method and system for establishing an environmental information archive based on information fusion features, as shown in FIG6 , and includes a processor 100, a memory 110, and a display 120. The processor 100, the memory 110, and the display 120 are interconnected and communicate with each other through a communication bus 130 and/or other forms of connection mechanisms.
其中，所述存储器110包括一个或多个，包括但不限于随机存取存储器(RandomAccess Memory，简称RAM)，只读存储器(Read Only Memory，简称ROM)，可编程只读存储器(Programmable Read-Only Memory，简称PROM)，可擦除只读存储器(ErasableProgrammable Read-Only Memory，简称EPROM)，电可擦除只读存储器(Electric ErasableProgrammable Read-Only Memory，简称EEPROM)等。The memory 110 includes one or more, including but not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.
所述处理器100可对存储器110进行访问，读和/或写其中的数据。所述处理器100包括一个或多个，可以是一种集成电路芯片，具有信号的处理能力。所述处理器100包括但不限于通用处理器，包括中央处理器(Central Processing Unit，简称CPU)、微控制单元(Micro Controller Unit，简称MCU)、网络处理器(Network Processor，简称NP)或者其他常规处理器；还可以是专用处理器，包括数字信号处理器(Digital Signal Processor，简称DSP)、专用集成电路(Application Specific Integrated Circuits，简称ASIC)、现场可编程门阵列(Field Programmable Gate Array，简称FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。The processor 100 can access the memory 110, read and/or write data therein. The processor 100 includes one or more, and can be an integrated circuit chip with signal processing capabilities. The processor 100 includes but is not limited to a general-purpose processor, including a central processing unit (CPU), a microcontroller unit (MCU), a network processor (NP) or other conventional processors; it can also be a special-purpose processor, including a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
所述显示器120可以包括一个或多个，包括但不限于阴极射线管显示器（CathodeRay Tube，简称CRT），等离子显示器（Plasma Display Panel，简称PDP），液晶显示器（Liquid Crystal Display，简称LCD）等。The display 120 may include one or more, including but not limited to a cathode ray tube display (CRT), a plasma display panel (PDP), a liquid crystal display (LCD), etc.
在所述存储器110中可以存储一个或多个计算机程序指令，所述处理器100可以读取并运行这些计算机程序指令，以实现本发明所述的基于信息融合特征修正环境信息档案的方法，之后在所述显示器120上显示最后判定的结果。One or more computer program instructions may be stored in the memory 110 , and the processor 100 may read and execute these computer program instructions to implement the method of correcting environmental information files based on information fusion features described in the present invention, and then display the final judgment result on the display 120 .
本发明基于信息融合特征建立环境信息档案的方法、系统及设备，通过将远程图像的背景信息特征和设备信息特征融合成一个环境信息特征，实现快速、全面分析场景信息，建立档案、检索和比对的效果。The present invention provides a method, system and device for establishing an environmental information archive based on information fusion features, which fuses the background information features and device information features of a remote image into one environmental information feature, thereby achieving the effects of rapid and comprehensive analysis of scene information, archive establishment, retrieval and comparison.
以上内容是对本发明创造的优选的实施例的说明，可以帮助本领域技术人员更充分地理解本发明创造的技术方案。但是，这些实施例仅仅是举例说明，不能认定本发明创造的具体实施方式仅限于这些实施例的说明。对本发明创造所属技术领域的普通技术人员来说，在不脱离本发明创造构思的前提下，还可以做出若干简单推演和变换，都应当视为属于本发明创造的保护范围。The above content is an explanation of the preferred embodiments of the present invention, which can help those skilled in the art to more fully understand the technical solution of the present invention. However, these embodiments are merely illustrative, and it cannot be determined that the specific implementation methods of the present invention are limited to the description of these embodiments. For ordinary technicians in the technical field to which the present invention belongs, without departing from the concept of the present invention, several simple deductions and transformations can be made, which should be regarded as belonging to the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410106091.0ACN118069870B (en) | 2024-01-25 | 2024-01-25 | Method, system and equipment for establishing environment information file based on information fusion characteristics |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410106091.0ACN118069870B (en) | 2024-01-25 | 2024-01-25 | Method, system and equipment for establishing environment information file based on information fusion characteristics |
| Publication Number | Publication Date |
|---|---|
| CN118069870Atrue CN118069870A (en) | 2024-05-24 |
| CN118069870B CN118069870B (en) | 2024-07-23 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410106091.0AActiveCN118069870B (en) | 2024-01-25 | 2024-01-25 | Method, system and equipment for establishing environment information file based on information fusion characteristics |
| Country | Link |
|---|---|
| CN (1) | CN118069870B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104134222A (en)* | 2014-07-09 | 2014-11-05 | 郑州大学 | Traffic flow monitoring image detecting and tracking system and method based on multi-feature fusion |
| CN111914920A (en)* | 2020-07-24 | 2020-11-10 | 山东工商学院 | Sparse coding-based similarity image retrieval method and system |
| CN111914921A (en)* | 2020-07-24 | 2020-11-10 | 山东工商学院 | A method and system for similarity image retrieval based on multi-feature fusion |
| CN113129234A (en)* | 2021-04-20 | 2021-07-16 | 河南科技学院 | Incomplete image fine repairing method based on intra-field and extra-field feature fusion |
| CN113763267A (en)* | 2021-08-26 | 2021-12-07 | 长春理工大学 | An Image Restoration Method in Strong Scattering Environment Based on NSCT Image Fusion |
| CN113971222A (en)* | 2021-10-28 | 2022-01-25 | 重庆紫光华山智安科技有限公司 | A method and system for multimodal composite coded image retrieval |
| WO2022126588A1 (en)* | 2020-12-18 | 2022-06-23 | 深圳先进技术研究院 | Pet-mri image denoising method and apparatus based on dual-encoding fusion network model |
| CN115223022A (en)* | 2022-09-15 | 2022-10-21 | 平安银行股份有限公司 | Image processing method, device, storage medium and equipment |
| CN116129229A (en)* | 2022-12-08 | 2023-05-16 | 炫彩互动网络科技有限公司 | Image fusion rendering method and system for VR equipment |
| US20230206298A1 (en)* | 2021-12-29 | 2023-06-29 | Samsung Electronics Co., Ltd. | Method and electronic device for recognizing product |
| CN116912143A (en)* | 2023-07-20 | 2023-10-20 | 上海蜜度信息技术有限公司 | Image synthesis method, equipment and computer-readable medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104134222A (en)* | 2014-07-09 | 2014-11-05 | 郑州大学 | Traffic flow monitoring image detecting and tracking system and method based on multi-feature fusion |
| CN111914920A (en)* | 2020-07-24 | 2020-11-10 | 山东工商学院 | Sparse coding-based similarity image retrieval method and system |
| CN111914921A (en)* | 2020-07-24 | 2020-11-10 | 山东工商学院 | A method and system for similarity image retrieval based on multi-feature fusion |
| WO2022126588A1 (en)* | 2020-12-18 | 2022-06-23 | 深圳先进技术研究院 | Pet-mri image denoising method and apparatus based on dual-encoding fusion network model |
| CN113129234A (en)* | 2021-04-20 | 2021-07-16 | 河南科技学院 | Incomplete image fine repairing method based on intra-field and extra-field feature fusion |
| CN113763267A (en)* | 2021-08-26 | 2021-12-07 | 长春理工大学 | An Image Restoration Method in Strong Scattering Environment Based on NSCT Image Fusion |
| CN113971222A (en)* | 2021-10-28 | 2022-01-25 | 重庆紫光华山智安科技有限公司 | A method and system for multimodal composite coded image retrieval |
| US20230206298A1 (en)* | 2021-12-29 | 2023-06-29 | Samsung Electronics Co., Ltd. | Method and electronic device for recognizing product |
| CN115223022A (en)* | 2022-09-15 | 2022-10-21 | 平安银行股份有限公司 | Image processing method, device, storage medium and equipment |
| CN116129229A (en)* | 2022-12-08 | 2023-05-16 | 炫彩互动网络科技有限公司 | Image fusion rendering method and system for VR equipment |
| CN116912143A (en)* | 2023-07-20 | 2023-10-20 | 上海蜜度信息技术有限公司 | Image synthesis method, equipment and computer-readable medium |
| Title |
|---|
| ZIRUI WANG; YUNMENG LIU; RUI ZHU; WENMING YANG; QINGMIN LIAO: "Lightweight Single Image Super-Resolution With Similar Feature Fusion Block", IEEE ACCESS, 11 March 2022 (2022-03-11), pages 30974* |
| 杨艳春；王可；闫岩: "基于FRGF和改进PCNN的红外可见光图像融合", 激光与红外, 31 October 2023 (2023-10-31), pages 1593 - 1601* |
| 濮永仙;: "基于图像内容检索的作物病害识别数据库系统研究", 计算机与现代化, no. 04, 15 April 2015 (2015-04-15), pages 54 - 60* |
| Publication number | Publication date |
|---|---|
| CN118069870B (en) | 2024-07-23 |
| Publication | Publication Date | Title |
|---|---|---|
| CN113326764B (en) | Method and device for training image recognition model and image recognition | |
| CN112101437B (en) | Fine granularity classification model processing method based on image detection and related equipment thereof | |
| CN116580257A (en) | Feature fusion model training and sample retrieval method, device and computer equipment | |
| JP7393472B2 (en) | Display scene recognition method, device, electronic device, storage medium and computer program | |
| CN111539942A (en) | Detection method of face depth forgery based on multi-scale depth feature fusion | |
| CN112863683A (en) | Medical record quality control method and device based on artificial intelligence, computer equipment and storage medium | |
| CN114373092B (en) | A progressive training approach for fine-grained visual classification based on jigsaw puzzle permutation learning | |
| CN113139490B (en) | Image feature matching method and device, computer equipment and storage medium | |
| CN113657411B (en) | Training method of neural network model, image feature extraction method and related devices | |
| CN115578735B (en) | Text detection method and training method and device of text detection model | |
| CN109246446A (en) | Compare the method, apparatus and equipment of video content similitude | |
| CN117671587A (en) | A power equipment defect detection method and system based on self-supervised learning | |
| CN116238124A (en) | Method and system for processing collected data for injection molding production of syringes | |
| CN109165572B (en) | Method and apparatus for generating information | |
| CN118069870B (en) | Method, system and equipment for establishing environment information file based on information fusion characteristics | |
| CN116486195A (en) | Video recommendation model training method, video recommendation method and device | |
| CN119692420A (en) | Method and system for constructing, training and evaluating large visual models of power equipment | |
| CN114782956A (en) | Text recognition method and device, electronic equipment and storage medium | |
| CN116186549B (en) | Model training method, device, equipment and medium | |
| CN112990046A (en) | Difference information acquisition method, related device and computer program product | |
| CN115496916B (en) | Training method of image recognition model, image recognition method and related device | |
| CN115797291B (en) | Loop terminal identification method, loop terminal identification device, computer equipment and storage medium | |
| US20220383626A1 (en) | Image processing method, model training method, relevant devices and electronic device | |
| CN116189007A (en) | Fire point determining method, fire point determining device, electronic equipment and computer readable storage medium | |
| CN113705439B (en) | Pedestrian attribute recognition method based on weak supervision and metric learning |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | Effective date of registration:20250124 Address after:No. 11359, 1st Floor, Building 11-3, No. 11 Zaojia Street South Lane, Fengtai District, Beijing 100070 Patentee after:Beijing Feihu Yundong Technology Co.,Ltd. Country or region after:China Address before:Room 1050, 10th Floor, No. 11 Zhongguancun Street, Haidian District, Beijing, 100190 Patentee before:Feihu interactive technology (Beijing) Co.,Ltd. Country or region before:China | |
| TR01 | Transfer of patent right |