技术领域Technical Field
本发明涉及电力安监技术领域，具体是指基于深度图像的电力生产人员安全监测方法。The present invention relates to the technical field of electric power safety supervision, and in particular to a method for safety monitoring of electric power production personnel based on depth images.
背景技术Background Art
在电网企业中基建、运检等生产业务场景下，工作人员为了完成建设任务或及时排查隐患需要到线路及变电站执行，但是目前电力生产现场仍存在安全监管水平较低、操作粗放的现象，同时有事后管理、不量化、随机性等问题，导致基建、运检等生产业务中缺乏对一线人员安全的有效管控，客观上存在超范围作业、误入带电间隔等问题。In power grid companies' infrastructure construction, operation and inspection and other production business scenarios, workers need to go to the lines and substations to complete construction tasks or promptly detect hidden dangers. However, the current power production site still has a low level of safety supervision and extensive operations. At the same time, there are problems such as post-management, non-quantification, and randomness. As a result, there is a lack of effective control over the safety of front-line personnel in infrastructure construction, operation and inspection and other production businesses, and objectively there are problems such as operating beyond the scope and mistakenly entering the energized interval.
目前，电力能源企业对一线生产作业人员的位置信息仍然主要依靠后台监控人员通过视觉判定，少部分企业通过GPS/BDS定位系统、FRID与摄像联动定位等方法进行人员位置信息的采集。但是以上技术存在一定不足：视觉定位由于无法测出深度，难以准确判断人员与设备间距离；GPS/BDS的定位存在米级误差，且涉及便携式设备的资金投入；RFID中天线通过反馈外界微波激励进行距离报警，然而激励信号的强弱、发射角度都会影响RFID的反馈，从而容易导致误判、漏判。亟待结合人工智能技术提出一种精度高、响应快的新型电力生产人员安全监测方法，迎接大量基建、运检需求为人员安全管控水平带来的更高挑战。At present, electric power enterprises still rely mainly on visual judgment by backstage monitoring personnel for the location information of front-line production workers. A small number of enterprises collect personnel location information through GPS/BDS positioning systems, FRID and camera linkage positioning and other methods. However, the above technologies have certain shortcomings: visual positioning cannot measure depth, so it is difficult to accurately judge the distance between personnel and equipment; GPS/BDS positioning has meter-level errors and involves capital investment in portable equipment; RFID antennas use feedback from external microwave excitation to generate distance alarms, but the strength and emission angle of the excitation signal will affect the feedback of RFID, which can easily lead to misjudgment and missed judgment. It is urgent to combine artificial intelligence technology to propose a new type of power production personnel safety monitoring method with high accuracy and fast response to meet the higher challenges brought by a large number of infrastructure and operation and inspection needs to the level of personnel safety management and control.
发明内容Summary of the invention
本发明提供一种基于深度图像的电力生产人员安全监测方法，为提升我国电网企业的电力基建、运检等生产业务场景下工作人员的安全管控水平提供技术支撑。The present invention provides a method for safety monitoring of power production personnel based on depth images, which provides technical support for improving the safety management and control level of personnel in production business scenarios such as power infrastructure construction, operation and inspection of my country's power grid enterprises.
本发明通过以下S1到S6步骤所述技术方案实现上述目的：The present invention achieves the above-mentioned purpose through the technical solutions described in the following steps S1 to S6:
S1、利用摄像设备，对准变电设备运维位置，采集运检人员及电力设备图像数据。S1. Use video equipment to focus on the operation and maintenance location of substation equipment and collect image data of operation and maintenance personnel and power equipment.
S2、提取现场摄像设备采集到的运检人员及电力设备图像数据，进行基于深度神经网络的三维深度信息提取，可求得运检人员及电力设备图像的三维深度信息。S2. Extract the image data of the operation and maintenance personnel and the power equipment collected by the on-site camera equipment, and perform three-dimensional depth information extraction based on a deep neural network to obtain the three-dimensional depth information of the operation and maintenance personnel and the power equipment images.
S3、提取现场摄像设备采集到的运检人员及电力设备图像数据，进行基于深度神经网络的运检人员与电力设备检测识别，计算获得运检人员与电力设备的三维边框。其中，深度神经网络中的角度预测采用一个3×3的可变形卷积模块和一个3×3的卷积模块结合，增强对旋转目标的适应性。可变形卷积通过预测每个特征点对应的所有卷积点的位置偏移，使卷积核可以自由在特征图上提取任意特征，对异构物体、旋转目标等复杂检测结果具备更好的适应性。考虑到角度对目标检测精度的影响权重较大和线性角度回归与检测框交并比不统一的问题，提出一种基于目标框长宽比的角度预测损失函数：S3. Extract the image data of the maintenance personnel and power equipment collected by the on-site camera equipment, perform maintenance personnel and power equipment detection and recognition based on the deep neural network, and calculate the three-dimensional bounding box of the maintenance personnel and the power equipment. Among them, the angle prediction in the deep neural network adopts a 3×3 deformable convolution module and a 3×3 convolution module to enhance the adaptability to rotating targets. Deformable convolution predicts the position offset of all convolution points corresponding to each feature point, so that the convolution kernel can freely extract any feature on the feature map, and has better adaptability to complex detection results such as heterogeneous objects and rotating targets. Taking into account the large influence of angle on target detection accuracy and the problem of inconsistent intersection and union ratio between linear angle regression and detection frame, an angle prediction loss function based on the aspect ratio of the target frame is proposed:
其中，θi为第i个目标的角度预测，为第i个目标的角度预测，ri为该目标框的长宽比ri≥1)，b和γ为超参数，用于调节其影响。Among them, θi is the angle prediction of the i-th target, is the angle prediction of the i-th target,ri is the aspect ratio of the target box (ri ≥ 1), b and γ are hyperparameters used to adjust their influence.
S4、在S3计算获得运检人员与电力设备的三维边框的基础上，实时计算运检人员三维边框与被检测电力设备三维边框的最小欧式距离a，计算方法如下式。并与安全阈值D’对比后，判断是否存在误入带电间隔、超过安全距离等情况，如存在，转入S6。S4. Based on the three-dimensional bounding box of the operator and the power equipment calculated in S3, the minimum Euclidean distance a between the three-dimensional bounding box of the operator and the three-dimensional bounding box of the power equipment under inspection is calculated in real time. The calculation method is as follows. After comparing with the safety threshold D' , it is determined whether there is a situation of mistakenly entering the live interval or exceeding the safety distance. If so, it is transferred to S6.
S5、提取现场摄像设备采集到的运检人员及电力设备图像数据，进行基于深度神经网络的运检人员骨骼关节检测，计算运检人员是否存在异常行为，如存在，转入S6。S5. Extract the image data of the operation and maintenance personnel and power equipment collected by the on-site camera equipment, perform skeletal joint detection of the operation and maintenance personnel based on the deep neural network, and calculate whether the operation and maintenance personnel have abnormal behavior. If so, proceed to S6.
根据人员骨骼关节的结构特点和相关性，提出一种基于中心点距离的关联性人员骨骼关节与S3中所述运检人员检测识别结果的配对算法。对于同一人员有确定关联性的人员骨骼关节与人员检测识别结果CT、CS，将网络检测结果划分为集合T＝{t1,t2,…,tm}与S＝{s1,s2,…,sn}，其中，ti为第i个CT类设备中心点，sj为第j个CS类中心点。依次找出集合T和S中距离最近的一组点(ti,sj)，并将其从集合中取出，作为一对目标保留，直至集合T或S没有元素，或T和S中距离最近的点的距离超过设定的阈值超参数。According to the structural characteristics and correlation of the personnel skeleton joints, a pairing algorithm based on the center point distance between the associated personnel skeleton joints and the inspection personnel detection and recognition results described in S3 is proposed. For the personnel skeleton joints and personnel detection and recognition resultsCT andCS that have a certain correlation for the same person, the network detection results are divided into sets T = {t1 , t2 , ..., tm } and S = {s1 , s2 , ..., sn }, whereti is the center point of the ithCT class equipment and sj is the center point of the jthCS class. The closest set of points (ti , sj ) in sets T and S are found in turn, and they are taken out of the set and retained as a pair of targets until there are no elements in sets T or S, or the distance between the closest points in T and S exceeds the set threshold hyperparameter.
S6、根据S4与S5的预警情况发出警示信号，并将警示信息转化为文本数据发送至电力企业的数据后台。S6: Issue a warning signal based on the warning conditions of S4 and S5, and convert the warning information into text data and send it to the data background of the power company.
本发明具有如下优点：本发明属于电力安监技术领域，涉及基于深度图像的电力生产人员安全监测方法。The present invention has the following advantages: the present invention belongs to the technical field of electric power safety supervision, and relates to a method for safety monitoring of electric power production personnel based on depth images.
基于电力企业的存量摄像设备采集电力生产业务场景下人员、设备、环境的可见光影像数据；通过深度学习算法识别定位二维图像中被测对象的空间坐标；最后通过人体骨骼节点分析与安全距离分析，判断检修人员是否超出安全距离并及时预警或其他异常行为，有效提升我国电网企业对电力基建、运检等生产业务场景下工作人员的安全管控水平。Based on the existing camera equipment of power companies, visible light image data of personnel, equipment, and environment in power production business scenarios are collected; the spatial coordinates of the measured objects in the two-dimensional image are identified and located through deep learning algorithms; finally, through human skeleton node analysis and safety distance analysis, it is determined whether the maintenance personnel have exceeded the safety distance and timely warnings or other abnormal behaviors are issued, effectively improving the safety management level of my country's power grid companies for workers in production business scenarios such as power infrastructure, operation and inspection.
本发明提出一种基于深度图像的电力生产人员安全监测方法，将从两个方面提升我国电网企业对电力基建、运检等生产业务场景下工作人员的安全管控水平：This paper proposes a method for safety monitoring of power production personnel based on depth images, which will improve the safety management and control level of personnel in power infrastructure, operation and inspection and other production business scenarios of my country's power grid enterprises from two aspects:
其一，实现人与设备三维空间信息的动态监控。本发明提出采用摄像设备对人员及变电设备等检修位置进行监测记录。基于深度神经网络训练三维深度信息提取模型与目标检测模型，识别RGB图像中的人和设备，获取包含检修人员与被检设备的深度图，得到目标的三维信息。First, dynamic monitoring of the three-dimensional spatial information of people and equipment is achieved. The present invention proposes to use a camera to monitor and record the maintenance locations of personnel and substation equipment. Based on the deep neural network, a three-dimensional depth information extraction model and a target detection model are trained to identify people and equipment in the RGB image, obtain a depth map containing maintenance personnel and the inspected equipment, and obtain the three-dimensional information of the target.
其二，实现检修人员异常行为的诊断。本发明提出一种人员骨骼关节的方向自适应检测方法以及基于中心点距离的关联人员骨骼关节与人员检测识别结果的配对方法，有助于提升电力基建、运检等生产业务场景下工作人员的安全管控水平。Second, the diagnosis of abnormal behavior of maintenance personnel is realized. The present invention proposes a method for adaptively detecting the direction of personnel skeletal joints and a method for pairing the associated personnel skeletal joints with personnel detection and recognition results based on the center point distance, which is helpful to improve the safety management and control level of personnel in production business scenarios such as power infrastructure and operation and inspection.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
图1为本发明实施例的基于深度图像的电力生产人员安全监测方法示意图。FIG1 is a schematic diagram of a method for safety monitoring of power production personnel based on depth images according to an embodiment of the present invention.
具体实施方式DETAILED DESCRIPTION
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
以220kV变电站基建场景的人员安全监测为例，本发明通过以下S1到S6步骤所述技术方案实现上述目的：Taking the personnel safety monitoring of the 220kV substation infrastructure scenario as an example, the present invention achieves the above purpose through the technical solutions described in the following steps S1 to S6:
S1、利用摄像设备，对准变电设备运维位置，采集运检人员及电力设备图像数据RGB图像。S1. Use the camera to aim at the operation and maintenance location of the substation equipment and collect RGB images of the operation and maintenance personnel and power equipment image data.
S2、提取现场摄像设备采集到的运检人员及电力设备图像数据RGB图像，进行基于深度神经网络的三维深度信息提取，可求得运检人员及电力设备图像的三维深度信息。S2. Extract the RGB image of the image data of the operation and maintenance personnel and the power equipment collected by the on-site camera equipment, and perform three-dimensional depth information extraction based on a deep neural network to obtain the three-dimensional depth information of the operation and maintenance personnel and the power equipment images.
S3、提取现场摄像设备采集到的运检人员及电力设备图像数据RGB图像，进行基于深度神经网络的运检人员与电力设备检测识别，计算获得运检人员与电力设备的三维边框。其中，深度神经网络中的角度预测采用一个3×3的可变形卷积模块和一个3×3的卷积模块结合，增强对旋转目标的适应性。可变形卷积通过预测每个特征点对应的所有卷积点的位置偏移，使卷积核可以自由在特征图上提取任意特征，对异构物体、旋转目标等复杂检测结果具备更好的适应性。考虑到角度对目标检测精度的影响权重较大和线性角度回归与检测框交并比不统一的问题，提出一种基于目标框长宽比的角度预测损失函数：S3. Extract the RGB images of the image data of the inspection personnel and power equipment collected by the on-site camera equipment, perform inspection and recognition of the inspection personnel and power equipment based on the deep neural network, and calculate the three-dimensional bounding boxes of the inspection personnel and the power equipment. Among them, the angle prediction in the deep neural network adopts a 3×3 deformable convolution module and a 3×3 convolution module to enhance the adaptability to rotated targets. The deformable convolution predicts the position offset of all convolution points corresponding to each feature point, so that the convolution kernel can freely extract any feature on the feature map, and has better adaptability to complex detection results such as heterogeneous objects and rotating targets. Taking into account the large weight of the influence of angle on the accuracy of target detection and the problem of inconsistent intersection and union ratio of linear angle regression and detection frame, an angle prediction loss function based on the aspect ratio of the target frame is proposed:
其中，θi为第i个目标的角度预测，为第i个目标的角度预测，ri为该目标框的长宽比ri≥1)，b和γ为超参数，用于调节其影响。Among them, θi is the angle prediction of the i-th target, is the angle prediction of the i-th target,ri is the aspect ratio of the target box (ri ≥ 1), b and γ are hyperparameters used to adjust their influence.
S4、在S3计算获得运检人员与电力设备的三维边框的基础上，实时计算运检人员三维边框与被检测电力设备三维边框的最小欧式距离a，计算方法如下式。并与安全阈值D’对比后，判断是否存在误入带电间隔、超过安全距离等情况，如存在，转入S6。S4. Based on the three-dimensional bounding box of the operator and the power equipment calculated in S3, the minimum Euclidean distance a between the three-dimensional bounding box of the operator and the three-dimensional bounding box of the power equipment under inspection is calculated in real time. The calculation method is as follows. After comparing with the safety threshold D' , it is determined whether there is a situation of mistakenly entering the live interval or exceeding the safety distance. If so, it is transferred to S6.
S5、提取现场摄像设备采集到的运检人员及电力设备图像数据RGB图像，进行基于深度神经网络的运检人员骨骼关节检测，计算运检人员是否存在异常行为，如存在，转入S6。S5. Extract the RGB images of the image data of the operation and maintenance personnel and the power equipment collected by the on-site camera equipment, perform skeletal joint detection of the operation and maintenance personnel based on the deep neural network, and calculate whether the operation and maintenance personnel have abnormal behavior. If so, proceed to S6.
根据人员骨骼关节的结构特点和相关性，提出一种基于中心点距离的关联性人员骨骼关节与S3中所述运检人员检测识别结果的配对算法。对于同一人员有确定关联性的人员骨骼关节与人员检测识别结果CT、CS，将网络检测结果划分为集合T＝{t1,t2,…,tm}与S＝{s1,s2,…,sn}，其中，ti为第i个CT类设备中心点，sj为第j个CS类中心点。依次找出集合T和S中距离最近的一组点(ti,sj)，并将其从集合中取出，作为一对目标保留，直至集合T或S没有元素，或T和S中距离最近的点的距离超过设定的阈值超参数。According to the structural characteristics and correlation of the personnel skeleton joints, a pairing algorithm based on the center point distance between the associated personnel skeleton joints and the inspection personnel detection and recognition results described in S3 is proposed. For the personnel skeleton joints and personnel detection and recognition resultsCT andCS that have a certain correlation for the same person, the network detection results are divided into sets T = {t1 , t2 , ..., tm } and S = {s1 , s2 , ..., sn }, whereti is the center point of the ithCT class equipment and sj is the center point of the jthCS class. The closest set of points (ti , sj ) in sets T and S are found in turn, and they are taken out of the set and retained as a pair of targets until there are no elements in sets T or S, or the distance between the closest points in T and S exceeds the set threshold hyperparameter.
S6、根据S4与S5的预警情况发出警示信号，并将警示信息转化为文本数据发送至电力企业的数据后台。S6: Issue a warning signal based on the warning conditions of S4 and S5, and convert the warning information into text data and send it to the data background of the power company.
以上所举实施例为本发明的较佳实施方式，仅用来方便说明本发明，并非对本发明作任何形式上的限制，任何所属技术领域中具有通常知识者，若在不脱离本发明所提技术特征的范围内，利用本发明所揭示技术内容所做出局部更动或修饰的等效实施例，并且未脱离本发明的技术特征内容，均仍属于本发明技术特征的范围内。The above embodiments are preferred implementation modes of the present invention and are only used to facilitate the explanation of the present invention. They are not intended to limit the present invention in any form. Any person with ordinary knowledge in the relevant technical field, if they do not depart from the scope of the technical features of the present invention, can make equivalent embodiments by partial changes or modifications made by the technical contents disclosed in the present invention, and they still fall within the scope of the technical features of the present invention without departing from the technical features of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111513440.3ACN114445850B (en) | 2021-12-10 | 2021-12-10 | Depth image-based safety monitoring method for power producer |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111513440.3ACN114445850B (en) | 2021-12-10 | 2021-12-10 | Depth image-based safety monitoring method for power producer |
| Publication Number | Publication Date |
|---|---|
| CN114445850A CN114445850A (en) | 2022-05-06 |
| CN114445850Btrue CN114445850B (en) | 2024-08-20 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111513440.3AActiveCN114445850B (en) | 2021-12-10 | 2021-12-10 | Depth image-based safety monitoring method for power producer |
| Country | Link |
|---|---|
| CN (1) | CN114445850B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116883931A (en)* | 2023-06-16 | 2023-10-13 | 国网山东省电力公司济宁市任城区供电公司 | Electric power operation site safety monitoring method and system based on image recognition |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111753712A (en)* | 2020-06-22 | 2020-10-09 | 中国电力科学研究院有限公司 | Power production personnel safety monitoring method, system and equipment |
| CN112766372A (en)* | 2021-01-19 | 2021-05-07 | 中国电力科学研究院有限公司 | Power equipment state monitoring and early warning method and system based on microclimate perception |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109784333B (en)* | 2019-01-22 | 2021-09-28 | 中国科学院自动化研究所 | Three-dimensional target detection method and system based on point cloud weighted channel characteristics |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111753712A (en)* | 2020-06-22 | 2020-10-09 | 中国电力科学研究院有限公司 | Power production personnel safety monitoring method, system and equipment |
| CN112766372A (en)* | 2021-01-19 | 2021-05-07 | 中国电力科学研究院有限公司 | Power equipment state monitoring and early warning method and system based on microclimate perception |
| Publication number | Publication date |
|---|---|
| CN114445850A (en) | 2022-05-06 |
| Publication | Publication Date | Title |
|---|---|---|
| Yang et al. | Deep learning‐based bolt loosening detection for wind turbine towers | |
| CN112183264B (en) | A Judgment Method for People Staying Under Crane Arm Based on Spatial Relationship Learning | |
| CN110910440B (en) | Power transmission line length determination method and system based on power image data | |
| CN115965579B (en) | A method and system for identifying and locating three-dimensional defects in substation inspections | |
| CN111967323B (en) | Electric power live working safety detection method based on deep learning algorithm | |
| CN118887036A (en) | Substation operation and maintenance personnel positioning and safety warning system | |
| CN116862712A (en) | Electric power construction potential safety risk detection method and system based on thunder fusion | |
| CN113569801A (en) | Distribution construction site live equipment and live area identification method and device thereof | |
| CN106920241A (en) | A kind of steam water-level monitoring method, apparatus and system | |
| CN117114420B (en) | Image recognition-based industrial and trade safety accident risk management and control system and method | |
| CN103500330A (en) | Semi-supervised human detection method based on multi-sensor and multi-feature fusion | |
| CN112734637A (en) | Thermal infrared image processing method and system for monitoring temperature of lead | |
| Ronoh et al. | Human-wildlife conflict early warning system using the internet of things and short message service | |
| CN118658279A (en) | Operation warning method, device and medium | |
| CN114445850B (en) | Depth image-based safety monitoring method for power producer | |
| CN118691982A (en) | Urban pipeline repair monitoring method and system based on remote sensing | |
| Chang et al. | Safety risk assessment of electric power operation site based on variable precision rough set | |
| CN116863196A (en) | Distance measurement and alarm method and system for electric power operation training and production safety supervision | |
| Ma et al. | Scene understanding method utilizing global visual and spatial interaction features for safety production | |
| CN116500631A (en) | Target ranging method and device based on three-dimensional point cloud and vision fusion | |
| CN119620759A (en) | A tower crane installation path planning method, system, terminal and medium | |
| CN119206626A (en) | Abnormal alarm method, device, equipment and storage medium for traffic infrastructure | |
| CN119250537A (en) | A method for line fault diagnosis based on satellite and meteorological information | |
| Ramlal et al. | Toward automated utility pole condition monitoring: A deep learning approach | |
| CN118608119A (en) | A small target detection and trajectory prediction method for substation patrol scenes |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |