CN114757935A - A multi-target automatic temperature measurement and early warning method for steel cold rolling annealing furnace components - Google Patents
A multi-target automatic temperature measurement and early warning method for steel cold rolling annealing furnace componentsDownload PDFInfo
- Publication number
- CN114757935A CN114757935ACN202210482010.8ACN202210482010ACN114757935ACN 114757935 ACN114757935 ACN 114757935ACN 202210482010 ACN202210482010 ACN 202210482010ACN 114757935 ACN114757935 ACN 114757935A
- Authority
- CN
- China
- Prior art keywords
- image
- component
- annealing furnace
- components
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30136—Metal
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Quality & Reliability (AREA)
- Radiation Pyrometers (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及金属冶炼技术领域，具体为一种钢铁冷轧退火炉元器件多目标自动测温与预警方法。The invention relates to the technical field of metal smelting, in particular to a multi-target automatic temperature measurement and early warning method for components of an iron and steel cold rolling annealing furnace.
背景技术Background technique
由于在网络上没有开源的钢铁企业退火炉关键元器件图像，需要通过人工拍摄采集获得图像作为模型训练的基础数据集，并且需要进行数据集的人工标注工作，再通过数据增强的方法对数据集进行扩充。由于需要增强模型的泛化性与精确程度，所以需要采集多种角度、不同光照强度下的元器件图像；通过前期的数据采集工作获得的元器件图像数据集，相较于一些网络上开源的机器视觉训练数据集来说，关键目标的特征相对不那么明显。在使用自主构建的符合真实场景的训练数据集完成目标检测算法与图像语义分割算法的训练过程中，如何选择在精度、速度等方面都具备优势的基础模型或网络，构建鲁棒的目标检测与图像语义分割模型；Since there is no open-source image of key components of the annealing furnace of iron and steel enterprises on the Internet, it is necessary to obtain images through manual shooting and collection as the basic data set for model training, and manual labeling of the data set is required, and then the data set is analyzed by data enhancement methods. to expand. Due to the need to enhance the generalization and accuracy of the model, it is necessary to collect component images from various angles and under different light intensities; the component image data sets obtained through the previous data collection work are compared with some open source ones on the Internet. For machine vision training datasets, the features of key objects are relatively inconspicuous. How to choose a basic model or network with advantages in accuracy, speed, etc. to build a robust target detection and Image semantic segmentation model;
结合现场实际工况，需要使用红外热成像摄像头采集的退火炉关键元器件的图像信息，并且基于实时目标检测与图像语义分割等多源图像解析技术，计算各元器件的实时温度，并实现温度异常检测功能。如何将目标检测与图像语义分割输出的目标与元器件轮廓结果与红外热成像摄像头采集图像相结合，以获得各个元器件的像素区域，计算对应元器件的温度；鉴于此，我们提出了一种钢铁冷轧退火炉元器件多目标自动测温与预警方法。Combined with the actual working conditions on site, it is necessary to use the image information of the key components of the annealing furnace collected by the infrared thermal imaging camera, and based on multi-source image analysis technologies such as real-time target detection and image semantic segmentation, the real-time temperature of each component is calculated, and the temperature is realized. Anomaly detection function. How to combine the target and component contour results output by target detection and image semantic segmentation with the image captured by the infrared thermal imaging camera to obtain the pixel area of each component and calculate the temperature of the corresponding component; in view of this, we propose a method Multi-target automatic temperature measurement and early warning method for steel cold rolling annealing furnace components.
发明内容SUMMARY OF THE INVENTION
针对现有技术的不足，本发明提供了一种钢铁冷轧退火炉元器件多目标自动测温与预警方法，解决了上述背景技术提到的问题。In view of the deficiencies of the prior art, the present invention provides a multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace, which solves the problems mentioned in the above background technology.
为实现以上目的，本发明通过以下技术方案予以实现：一种钢铁冷轧退火炉元器件多目标自动测温与预警方法，该自动测温与预警方法包括以下步骤：In order to achieve the above purpose, the present invention is achieved through the following technical solutions: a multi-target automatic temperature measurement and early warning method for components of an iron and steel cold rolling annealing furnace, and the automatic temperature measurement and early warning method comprises the following steps:
S1：采集退火炉关键元器件图像数据，通过多角度拍摄待检测的元器件获取更为有效的特征信息；基于labelme开源标注工具研究确定适用于本发明的样本标注方法，制定电机、减速机、轴承的元器件标注的准则，进行数据集目标识别与语义分割任务所需特征的精细化标注；S1: Collect the image data of the key components of the annealing furnace, and obtain more effective feature information by shooting the components to be detected from multiple angles; based on the research on the labelme open source labeling tool, the sample labeling method suitable for the present invention is determined, and the motor, reducer, Criteria for component labeling of bearings, fine labeling of features required for data set target recognition and semantic segmentation tasks;
S2：退火炉元器件目标检测算法开发过程，基于YOLOv3模型开发退火炉元器件目标检测算法；S2: The development process of the target detection algorithm for annealing furnace components, and the target detection algorithm for annealing furnace components is developed based on the YOLOv3 model;
S3：退火炉元器件图像语义分割算法开发的过程，使用FCN模型对图像进行像素级的分类，进而对元器件图像进行语义级别的图像分割；S3: The process of developing the semantic segmentation algorithm for the component image of the annealing furnace, using the FCN model to classify the image at the pixel level, and then perform semantic-level image segmentation on the component image;
S4：在炉辊一侧的通道部署滑轨，将红外热成像摄像头在滑轨上匀速移动拍摄得到每个炉辊的红外热图像。根据红外热图像提供的每个像素的温度，结合通过目标检测与图像分割算法获得的各个元器件的像素区域，通过平均加权算法计算各个元器件的实时温度；S4: Deploy the slide rail in the channel on one side of the furnace roll, and move the infrared thermal imaging camera on the slide rail at a constant speed to capture the infrared thermal image of each furnace roll. According to the temperature of each pixel provided by the infrared thermal image, combined with the pixel area of each component obtained by the target detection and image segmentation algorithm, the real-time temperature of each component is calculated by the average weighting algorithm;
S5：通过分析各个元器件的历史温度，设计温度异常检测算法；分析统计部件的历史温度，计算过去一段时间内的最高、最低和平均温度，从而计算出当前时刻该元器件的正常温度范围，继而判断该元器件当前的温度是否处于正常温度范围内。如若不然，则判定该元器件当前温度异常，发出报警。S5: Design a temperature abnormality detection algorithm by analyzing the historical temperature of each component; analyze and count the historical temperature of the component, calculate the highest, lowest and average temperature in the past period of time, so as to calculate the normal temperature range of the component at the current moment, Then it is judged whether the current temperature of the component is within the normal temperature range. If not, it is determined that the current temperature of the component is abnormal and an alarm is issued.
优选的，步骤S1数据采集及预处理的过程还包括：Preferably, the process of data collection and preprocessing in step S1 further includes:
S11、通过人工在多角度、不同明暗状态下拍摄退火炉元器件照片，覆盖所有关键检测元器件，模拟多种实装后红外摄像头的拍摄角度；S11. By artificially taking photos of the annealing furnace components at multiple angles and in different light and dark states, covering all key detection components, and simulating the shooting angles of various infrared cameras after installation;
S12、通过业务调研分析，制定各个关键特征如目标检测过程中的关键目标、图像语义分割过程中的元件覆盖区域的标注准则；S12, through business research and analysis, formulate various key features, such as key targets in the target detection process, and labeling criteria for the component coverage area in the image semantic segmentation process;
S13、根据业务调研分析及退火炉元器件目标特征，对元器件图像数据集采用翻转、随机修剪、平移变换、尺度变化、噪声扰动的数据增广技术，对数据集进行预处理。S13. According to the business research and analysis and the target characteristics of the components of the annealing furnace, the data augmentation techniques of flipping, random pruning, translation transformation, scale change, and noise disturbance are used for the component image data set to preprocess the data set.
优选的，步骤S2退火炉元器件目标检测算法开发过程还包括：Preferably, the development process of the target detection algorithm for the components of the annealing furnace in step S2 further includes:
S21、根据退火炉元器件目标识别需要的目标数、目标尺寸及定位要求的特性，综合实际业务中对模型推理速度与结果精确度的要求，进行目标检测模型或网络的选取。S21. According to the characteristics of the target number, target size and positioning requirements required for target identification of the components of the annealing furnace, and synthesizing the requirements for model inference speed and result accuracy in actual business, the target detection model or network is selected.
优选的，步骤S2进一步还包括：Preferably, step S2 further includes:
S22、YOLOv3模型仅使用卷积层，采用Darknet-53特征提取网络，它包含53个卷积层，每个后面跟随着批标准化层和leaky ReLU层；The S22 and YOLOv3 models only use convolutional layers and use the Darknet-53 feature extraction network, which contains 53 convolutional layers, each followed by a batch normalization layer and a leaky ReLU layer;
S23、YOLOv3算法同时在Darknet-53网络中加入了残差模块。The S23 and YOLOv3 algorithms also add a residual module to the Darknet-53 network.
优选的，步骤S3退火炉元器件图像语义分割算法开发的过程还包括：Preferably, the process of developing the semantic segmentation algorithm for the component image of the annealing furnace in step S3 further includes:
S31、根据对退火炉元器件的多源图像解析任务要求，综合实际业务中对模型推理速度与结果精确度的要求，进行图像语义分割模型的选取，对比试验结果，选定FCN模型作为退火炉元器件图像语义分割任务的基础模型。S31. According to the multi-source image analysis task requirements for the components of the annealing furnace, and synthesizing the requirements on the model inference speed and result accuracy in the actual business, select the image semantic segmentation model, compare the test results, and select the FCN model as the annealing furnace. A basic model for the task of semantic segmentation of component images.
优选的，步骤S3进一步的还包括：Preferably, step S3 further includes:
S32、在FCN模型中，输入模型的图像经过多个卷积层和一个最大池化层，变为特征图pool1，宽高变为1/2；特征图pool1再经过经过多个卷积层和一个最大池化层特征图pool2，宽高变为1/4；生成特征图pool5，其宽高变为1/32。而网络的第6层和第7层分别是一个长度为4096的一维向量，第8层是长度为1000的一维向量，分别对应1000个不同类别的概率；S32. In the FCN model, the image of the input model goes through multiple convolutional layers and a maximum pooling layer to become the feature map pool1, and the width and height become 1/2; the feature map pool1 goes through multiple convolutional layers and A maximum pooling layer feature map pool2, whose width and height become 1/4; generate feature map pool5, whose width and height become 1/32. The sixth and seventh layers of the network are respectively a one-dimensional vector with a length of 4096, and the eighth layer is a one-dimensional vector with a length of 1000, respectively corresponding to the probabilities of 1000 different categories;
S33、生成热图之后，需要对其进行上采样过程，把图像进行多次放大，直到还原为原图像的大小。S33. After the heat map is generated, an upsampling process needs to be performed on the heat map, and the image is enlarged for many times until it is restored to the size of the original image.
优选的，步骤S4元器件实时温度检测的过程还包括：Preferably, the process of real-time temperature detection of components in step S4 further includes:
S41、综合目标检测方案与图像语义分割算法，实现退火炉元器件多元图像解析方案，以目标检测算法实现退火炉元器件的检测定位，以图像语义分割算法确认每个元器件的准确轮廓；S41, synthesizing the target detection scheme and the image semantic segmentation algorithm, to realize the multi-image analysis scheme of the annealing furnace components, to realize the detection and positioning of the annealing furnace components with the target detection algorithm, and to confirm the accurate contour of each component with the image semantic segmentation algorithm;
S42、将经过多元图像解析中的目标检测与实例分割算法生成的实例分割图像与红外热成像摄像头传回的图像进行图像矩阵对齐。S42. Perform image matrix alignment on the instance segmented image generated by the target detection and instance segmentation algorithm in the multivariate image analysis and the image returned by the infrared thermal imaging camera.
优选的，步骤S4进一步的包括：Preferably, step S4 further includes:
S43红外热成像摄像头拍摄红外热图像，摄像头内部算法会对图像进行计算处理，输出每个像素点的温度T_p，其中p表示像素点，目标检测与图像分割算法会对图像进行目标检测与图像分割，从而获得各个元器件的像素区域obji，表示第i个元器件的像素区域，然后通过平均加权算法，求出各个部件的实时温度，计算公式如下：The S43 infrared thermal imaging camera captures infrared thermal images, and the internal algorithm of the camera will calculate and process the image, and output the temperature T_p of each pixel point, where p represents the pixel point, and the target detection and image segmentation algorithm will perform target detection and image segmentation on the image. , so as to obtain the pixel area obji of each component, which represents the pixel area of the i-th component, and then calculate the real-time temperature of each component through the average weighting algorithm. The calculation formula is as follows:
式中：mi表示第i个元器件区域内的像素个数。In the formula: mi represents the number of pixels in the i-th component area.
优选的，步骤S5元器件实时温度状态判定的过程还包括：Preferably, the process of determining the real-time temperature state of the components in step S5 also includes:
S51、通过S4步骤中计算得到每个元器件的实时温度并记录，通过统计分析每个元器件在过去t时刻的温度Cit，计算出过去一段时间内的最高、最低和平均温度，分别记为Maxi、Mini和Averi，从而计算出当前时刻该元器件的正常温度范围。S51. Obtain and record the real-time temperature of each component through the calculation in step S4, and calculate the highest, lowest and average temperatures in the past period of time by statistically analyzing the temperature Cit of each component at time t in the past, and record them respectively. are Maxi , Mini and Averi , so as to calculate the normal temperature range of the component at the current moment.
S52、通过实时检测每个元器件的温度与计算得到的正常温度范围比较，判断元器件当前温度是否正常，对于异常状态需要发出报警。S52 , by comparing the temperature of each component in real time with the calculated normal temperature range, it is judged whether the current temperature of the component is normal, and an alarm needs to be issued for the abnormal state.
本发明提供了一种钢铁冷轧退火炉元器件多目标自动测温与预警方法。具备以下有益效果：The invention provides a multi-target automatic temperature measurement and early warning method for components of an iron and steel cold rolling annealing furnace. Has the following beneficial effects:
该钢铁冷轧退火炉元器件多目标自动测温与预警方法，基于红外热成像摄像头和人工智能技术相结合的自动温度监测系统，并采用基于机器视觉技术的目标检测和图像语义分割算法，获得实时设备元器件图像。在获取现场实测数据的基础上，基于红外热成像摄像头获取关键元器件的实时温度数据，并开发温度异常检测算法，从而判断设备的运行状态并及时对设备异常进行预警。The multi-target automatic temperature measurement and early warning method for steel cold rolling annealing furnace components is based on an automatic temperature monitoring system that combines infrared thermal imaging cameras and artificial intelligence technology, and adopts target detection and image semantic segmentation algorithms based on machine vision technology. Real-time device component images. On the basis of acquiring on-site measured data, real-time temperature data of key components is acquired based on infrared thermal imaging cameras, and a temperature anomaly detection algorithm is developed to judge the operating status of the equipment and give early warning of equipment anomalies in a timely manner.
附图说明Description of drawings
图1为本发明目标检测YOLOv3模型网络结构图；Fig. 1 is the target detection YOLOv3 model network structure diagram of the present invention;
图2为本发明图像语义分割FCN模型网络结构图；Fig. 2 is the image semantic segmentation FCN model network structure diagram of the present invention;
图3为本发明图像解析效果与红外热成像摄像头实拍图；FIG. 3 is an image analysis effect of the present invention and a real shot of an infrared thermal imaging camera;
图4为本发明元器件位置与温度推断流程示意图。FIG. 4 is a schematic diagram of a flow chart of component position and temperature estimation according to the present invention.
具体实施方式Detailed ways
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
请参阅图1-图4，本发明提供一种技术方案：一种钢铁冷轧退火炉元器件多目标自动测温与预警方法，该自动测温与预警方法包括以下步骤：Please refer to FIG. 1-FIG. 4, the present invention provides a technical solution: a multi-target automatic temperature measurement and early warning method for steel cold rolling annealing furnace components, and the automatic temperature measurement and early warning method includes the following steps:
S1：采集退火炉关键元器件图像数据，通过多角度拍摄待检测的元器件获取更为有效的特征信息；基于labelme开源标注工具研究确定适用于本发明的样本标注方法，制定电机、减速机、轴承的元器件标注的准则，进行数据集目标识别与语义分割任务所需特征的精细化标注；对数据集采用翻转、随机修剪、平移变换、尺度变化、噪声扰动的数据增广技术进行预处理，以提高模型的泛化能力。S1: Collect the image data of the key components of the annealing furnace, and obtain more effective feature information by shooting the components to be detected from multiple angles; based on the research on the labelme open source labeling tool, the sample labeling method suitable for the present invention is determined, and the motor, reducer, Criteria for the labeling of bearing components, fine labeling of features required for data set target recognition and semantic segmentation tasks; data augmentation techniques such as flipping, random pruning, translation transformation, scale change, and noise disturbance are used to preprocess the data set. , to improve the generalization ability of the model.
具体的，步骤S1包括：Specifically, step S1 includes:
S11、通过人工在多角度、不同明暗状态下拍摄退火炉元器件照片，覆盖所有关键检测元器件，模拟多种实装后红外摄像头的拍摄角度；S11. By artificially taking photos of the annealing furnace components at multiple angles and in different light and dark states, covering all key detection components, and simulating the shooting angles of various infrared cameras after installation;
S12、通过业务调研分析，制定各个关键特征如目标检测过程中的关键目标、图像语义分割过程中的元件覆盖区域的标注准则；使用labelme软件对千余张退火炉元器件图像进行标注；S12. Through business research and analysis, formulate key features such as key targets in the target detection process, and labeling criteria for the component coverage area in the image semantic segmentation process; use labelme software to label more than a thousand annealing furnace component images;
S13、根据业务调研分析及退火炉元器件目标特征，对元器件图像数据集采用翻转、随机修剪、平移变换、尺度变化、噪声扰动的数据增广技术，对数据集进行预处理，以在模型训练过程中提高模型的泛化能力。S13. According to the business research and analysis and the target characteristics of the components of the annealing furnace, the data augmentation techniques of flipping, random pruning, translation transformation, scale change, and noise disturbance are used for the component image data set, and the data set is preprocessed so as to be used in the model. Improve the generalization ability of the model during training.
S2：根据退火炉关键元器件数据集目标特征，基于YOLOv3模型开发退火炉元器件目标检测算法；YOLOv3是兼具模型推理速度与精确度的小物体目标检测算法；算法使用一个单独神经网络作用在图像上，将图像划分多个区域并且预测边界框和每个区域的概率。S2: According to the target characteristics of the key components of the annealing furnace, the target detection algorithm of the annealing furnace components is developed based on the YOLOv3 model; YOLOv3 is a small object target detection algorithm with both model inference speed and accuracy; the algorithm uses a single neural network to act on On the image, divide the image into regions and predict bounding boxes and probabilities for each region.
具体的，步骤S2包括：Specifically, step S2 includes:
S21、根据退火炉元器件目标识别需要的目标数、目标尺寸及定位要求的特性，综合实际业务中对模型推理速度与结果精确度的要求，进行目标检测模型或网络的选取；对比试验结果，最终选定在保持YOLO家族算法速度优势的同时，对小目标识别精度进一步提升的YOLOv3模型作为基础模型；YOLO创造性的提出了one-stage，也就是将物体分类和物体定位在一个步骤中完成，并直接在输出层回归边框所在位置和边框所属类别，从而实现one-stage；通过这种方式，可实现45帧/秒的运算速度，可以满足实时性要求；S21. According to the characteristics of the target number, target size and positioning requirements required for target identification of the components of the annealing furnace, and synthesizing the requirements for model inference speed and result accuracy in the actual business, select a target detection model or network; compare the test results, Finally, the YOLOv3 model, which further improves the accuracy of small target recognition while maintaining the speed advantage of the YOLO family algorithm, was selected as the basic model; YOLO creatively proposed one-stage, that is, object classification and object positioning are completed in one step, And directly return the position of the border and the category of the border in the output layer, so as to realize one-stage; in this way, the operation speed of 45 frames per second can be achieved, which can meet the real-time requirements;
S22、YOLOv3模型仅使用卷积层，采用Darknet-53特征提取网络，它包含53个卷积层，每个后面跟随着批标准化层和leaky ReLU层，没有池化层，使用步幅为2的卷积层替代池化层进行特征图的下采样过程，这样可以有效阻止由于池化层导致的低层级特征的损失；The S22 and YOLOv3 models only use convolutional layers and use the Darknet-53 feature extraction network, which contains 53 convolutional layers, each followed by a batch normalization layer and a leaky ReLU layer, without a pooling layer, using a stride of 2 The convolutional layer replaces the pooling layer for the downsampling process of the feature map, which can effectively prevent the loss of low-level features caused by the pooling layer;
S23、YOLOv3算法同时在Darknet-53网络中加入了残差模块，这样有利于解决深层次网络的梯度问题，每个残差模块由两个卷积层和一个直接链接组成。如果输入416*416*3的图像，通过Darknet-53网络将得到三种不同尺度的预测结果，每个尺度都对应N个通道，包含着预测的信息。YOLOv3共有13*13*3+26*26*3+52*52*3个预测；每个预测对应85维，分别是4(坐标值)、1(置信度分数)、80(coco类别数)；YOLOv3模型网络结构如图1所示。The S23 and YOLOv3 algorithms also add a residual module to the Darknet-53 network, which is beneficial to solve the gradient problem of the deep network. Each residual module consists of two convolutional layers and a direct link. If you input a 416*416*3 image, you will get three different scales of prediction results through the Darknet-53 network, each scale corresponds to N channels and contains the predicted information. YOLOv3 has a total of 13*13*3+26*26*3+52*52*3 predictions; each prediction corresponds to 85 dimensions, which are 4 (coordinate value), 1 (confidence score), 80 (number of coco categories) ; The network structure of the YOLOv3 model is shown in Figure 1.
S3：根据退火炉关键元器件数据集目标特征，使用FCN模型对图像进行像素级的分类，进而对元器件图像进行语义级别的图像分割。FCN模型采用反卷积层对最后一个卷积层的特征图进行上采样，使它恢复到输入图像相同的尺寸，从而可以对每个像素都产生一个预测，同时保留了原始输入图像中的空间信息。再通过上采样过程，对上采样后的特征图进行逐像素分类。最后逐个像素计算softmax分类的损失，这个过程相当于每个像素对应一个训练样本，最终实现对每个像素的实例分割。S3: According to the target features of the key component data set of the annealing furnace, the FCN model is used to classify the image at the pixel level, and then the component image is segmented at the semantic level. The FCN model uses a deconvolutional layer to upsample the feature map of the last convolutional layer to bring it back to the same size as the input image, allowing a prediction for each pixel while preserving the space in the original input image information. Then, through the upsampling process, the upsampled feature maps are classified pixel by pixel. Finally, the loss of softmax classification is calculated pixel by pixel. This process is equivalent to each pixel corresponding to a training sample, and finally the instance segmentation of each pixel is realized.
具体的，步骤S3包括：Specifically, step S3 includes:
S31、根据对退火炉元器件的多源图像解析任务要求，综合实际业务中对模型推理速度与结果精确度的要求，进行图像语义分割模型的选取。对比试验结果，选定FCN模型作为退火炉元器件图像语义分割任务的基础模型。对于一般的分类CNN网络，网络的最后通常需要加入一些全连接层，经过softmax后就可以获得类别概率信息；FCN模型将网络后向的全连接曾都替换为卷积层，这样就可以获得一张二维的特征图，最后经过softmax层获得每个像素点的分类信息，从而获得了二维的概率信息，由此解决了语义分割问题；S31. According to the multi-source image analysis task requirements for the components of the annealing furnace, and synthesizing the requirements for model inference speed and result accuracy in actual business, select an image semantic segmentation model. Compared with the test results, the FCN model is selected as the basic model for the semantic segmentation task of component images in the annealing furnace. For a general classification CNN network, some fully connected layers usually need to be added at the end of the network, and the category probability information can be obtained after softmax; the FCN model replaces the full connection of the network backwards with convolutional layers, so that a convolutional layer can be obtained. A two-dimensional feature map is obtained, and finally the classification information of each pixel is obtained through the softmax layer, thereby obtaining two-dimensional probability information, thus solving the problem of semantic segmentation;
S32、在FCN模型中，输入模型的图像经过多个卷积层和一个最大池化层，变为特征图pool1，宽高变为1/2；特征图pool1再经过经过多个卷积层和一个最大池化层特征图pool2，宽高变为1/4；利用相同处理过程，最终生成特征图pool5，其宽高变为1/32。而网络的第6层和第7层分别是一个长度为4096的一维向量，第8层是长度为1000的一维向量，分别对应1000个不同类别的概率。FCN将这3层表示为卷积层，卷积核的大小(通道数，宽，高)分别为(4096，1，1)、(4096，1，1)、(1000，1，1)，并使用的之前CNN已经训练好的权值和偏置，但是不一样的在于权值和偏置是有自己的范围，属于自己的一个卷积核。因此FCN网络中所有的层都是卷积层，故称为全卷积网络。经过多次卷积和池化以后，得到的图像越来越小，分辨率越来越低。最后一层后所产生高维特征图称为热图。图像语义分割FCN模型网络结构如图2所示；S32. In the FCN model, the image of the input model goes through multiple convolutional layers and a maximum pooling layer to become the feature map pool1, and the width and height become 1/2; the feature map pool1 goes through multiple convolutional layers and A maximum pooling layer feature map pool2, whose width and height become 1/4; using the same processing process, the feature map pool5 is finally generated, whose width and height become 1/32. The sixth and seventh layers of the network are respectively a one-dimensional vector with a length of 4096, and the eighth layer is a one-dimensional vector with a length of 1000, corresponding to the probabilities of 1000 different categories. FCN represents these three layers as convolutional layers, and the size of the convolution kernel (number of channels, width, height) is (4096, 1, 1), (4096, 1, 1), (1000, 1, 1), respectively, The weights and biases that CNN have been trained before are used, but the difference is that the weights and biases have their own ranges and belong to a convolution kernel of their own. Therefore, all layers in the FCN network are convolutional layers, so it is called a fully convolutional network. After multiple convolutions and pooling, the resulting images are getting smaller and smaller, and the resolution is getting lower and lower. The high-dimensional feature map generated after the last layer is called a heatmap. The network structure of the FCN model for image semantic segmentation is shown in Figure 2;
S33、生成热图之后，需要对其进行上采样过程，即把图像进行多次放大，直到还原为原图像的大小。最后的输出是经过上采样的热图生成的原图大小的图像，图像中已对每个像素进行分类预测，并且通过该过程实现像素级的语义分割。S33 , after the heat map is generated, an upsampling process needs to be performed on the heat map, that is, the image is enlarged for many times until it is restored to the size of the original image. The final output is the original size image generated by the upsampled heatmap, in which each pixel has been classified and predicted, and pixel-level semantic segmentation is achieved through this process.
S4：在炉辊一侧的通道部署滑轨，将红外热成像摄像头在滑轨上匀速移动拍摄得到每个炉辊的红外热图像。根据红外热图像提供的每个像素的温度，结合通过目标检测与图像分割算法获得的各个元器件的像素区域，通过平均加权算法计算各个元器件的实时温度。S4: Deploy the slide rail in the channel on one side of the furnace roll, and move the infrared thermal imaging camera on the slide rail at a constant speed to capture the infrared thermal image of each furnace roll. According to the temperature of each pixel provided by the infrared thermal image, combined with the pixel area of each component obtained by the target detection and image segmentation algorithm, the real-time temperature of each component is calculated by the average weighting algorithm.
具体的，步骤S4包括：Specifically, step S4 includes:
S41、综合目标检测方案与图像语义分割算法，实现退火炉元器件多元图像解析方案，以目标检测算法实现退火炉元器件的检测定位，以图像语义分割算法确认每个元器件的准确轮廓。将红外摄像头传入的图像输入图像语义分割网络中，得到输入图像的语义分割图形；再将语义分割图形与元器件目标检测所得的定位结合，选取目标检测位置与语义分割区域在缩放后的交集来优化元器件识别结果所选范围，得到单个元器件的实例分割图像；S41 , synthesizing the target detection scheme and the image semantic segmentation algorithm, realizes a multi-image analysis scheme for the annealing furnace components, realizes the detection and positioning of the annealing furnace components with the target detection algorithm, and confirms the accurate contour of each component with the image semantic segmentation algorithm. Input the image input from the infrared camera into the image semantic segmentation network, and obtain the semantic segmentation graph of the input image; then combine the semantic segmentation graph with the positioning obtained by the component target detection, and select the intersection of the target detection position and the semantic segmentation area after scaling. to optimize the selected range of the component recognition results, and obtain the instance segmentation image of a single component;
S42、将经过多元图像解析中的目标检测与实例分割算法生成的实例分割图像与红外热成像摄像头传回的图像进行图像矩阵对齐。图像解析效果图与红外热成像摄像头实拍图如图3所示；同时精确识别红外热成像摄像头图像中每个元器件的位置与轮廓所在的像素区域，再通过平均加权算法计算各个元器件的实时温度。元器件位置与其温度推断流程如图4所示；S42. Perform image matrix alignment on the instance segmented image generated by the target detection and instance segmentation algorithm in the multivariate image analysis and the image returned by the infrared thermal imaging camera. The image analysis effect diagram and the real shot of the infrared thermal imaging camera are shown in Figure 3; at the same time, the position of each component in the infrared thermal imaging camera image and the pixel area where the outline is located are accurately identified, and then the average weighting algorithm is used to calculate the real-time temperature. The component position and its temperature inference process are shown in Figure 4;
S43、红外热成像摄像头拍摄红外热图像，摄像头内部算法会对图像进行计算处理，输出每个像素点的温度T_p，其中p表示像素点。目标检测与图像分割算法会对图像进行目标检测与图像分割，从而获得各个元器件的像素区域obji，表示第i个元器件的像素区域，然后通过平均加权算法，求出各个部件的实时温度。计算公式如下：S43 , the infrared thermal imaging camera captures an infrared thermal image, and an internal algorithm of the camera will calculate and process the image, and output the temperature T_p of each pixel point, where p represents a pixel point. The target detection and image segmentation algorithm will perform target detection and image segmentation on the image, so as to obtain the pixel area obji of each component, which represents the pixel area of the i-th component, and then obtain the real-time temperature of each component through the average weighting algorithm . Calculated as follows:
式中：mi表示第i个元器件区域内的像素个数。In the formula: mi represents the number of pixels in the i-th component area.
S5：通过分析各个元器件的历史温度，设计温度异常检测算法。分析统计部件的历史温度，计算过去一段时间内的最高、最低和平均温度，从而计算出当前时刻该元器件的正常温度范围，继而判断该元器件当前的温度是否处于正常温度范围内。如若不然，则判定该元器件当前温度异常，发出报警。S5: Design a temperature anomaly detection algorithm by analyzing the historical temperature of each component. Analyze and count the historical temperature of the component, calculate the highest, lowest and average temperature in the past period of time, so as to calculate the normal temperature range of the component at the current moment, and then determine whether the current temperature of the component is within the normal temperature range. If not, it is determined that the current temperature of the component is abnormal and an alarm is issued.
具体的，步骤S5包括：Specifically, step S5 includes:
S51、通过S4步骤中计算得到每个元器件的实时温度并记录。通过统计分析每个元器件在过去t时刻的温度Cit，计算出过去一段时间内的最高、最低和平均温度，分别记为Maxi、Mini和Averi，从而计算出当前时刻该元器件的正常温度范围；S51. Obtain and record the real-time temperature of each component through the calculation in step S4. Through statistical analysis of the temperature Cit of each component at time t in the past, the highest, lowest and average temperatures in the past period of time are calculated, which are recorded as Maxi , Minii and Averi respectively, so as to calculate the current moment of the component. normal temperature range;
S52、通过实时检测每个元器件的温度与计算得到的正常温度范围比较，判断元器件当前温度是否正常。对于异常状态需要发出报警。S52. Determine whether the current temperature of the components is normal by comparing the temperature of each component in real time with the calculated normal temperature range. Alarms need to be issued for abnormal conditions.
尽管已经示出和描述了本发明的实施例，对于本领域的普通技术人员而言，可以理解在不脱离本发明的原理和精神的情况下可以对这些实施例进行多种变化、修改、替换和变型，本发明的范围由所附权利要求及其等同物限定。Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, and substitutions can be made in these embodiments without departing from the principle and spirit of the invention and modifications, the scope of the present invention is defined by the appended claims and their equivalents.
Claims (9)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210482010.8ACN114757935B (en) | 2022-05-05 | 2022-05-05 | A multi-target automatic temperature measurement and early warning method for components of steel cold rolling annealing furnace |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210482010.8ACN114757935B (en) | 2022-05-05 | 2022-05-05 | A multi-target automatic temperature measurement and early warning method for components of steel cold rolling annealing furnace |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114757935Atrue CN114757935A (en) | 2022-07-15 |
| CN114757935B CN114757935B (en) | 2025-04-22 |
Family
ID=82332191
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210482010.8AActiveCN114757935B (en) | 2022-05-05 | 2022-05-05 | A multi-target automatic temperature measurement and early warning method for components of steel cold rolling annealing furnace |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114757935B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115578612A (en)* | 2022-10-11 | 2023-01-06 | 浙江大学 | Recognition method and device for blast furnace top material distribution stage based on marker target detection |
| CN115896817A (en)* | 2023-03-08 | 2023-04-04 | 福建德尔科技股份有限公司 | Production method and system of fluorine-nitrogen mixed gas |
| CN119289722A (en)* | 2024-12-16 | 2025-01-10 | 江西铜业技术研究院有限公司 | An intelligent management system and method for smelting equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN208577803U (en)* | 2018-05-28 | 2019-03-05 | 湖南红太阳光电科技有限公司 | A kind of anti-light decline stove equipment of the high production capacity crystal-silicon battery slice of double track |
| CN111368687A (en)* | 2020-02-28 | 2020-07-03 | 成都市微泊科技有限公司 | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation |
| AU2020103901A4 (en)* | 2020-12-04 | 2021-02-11 | Chongqing Normal University | Image Semantic Segmentation Method Based on Deep Full Convolutional Network and Conditional Random Field |
- 2022
- 2022-05-05CNCN202210482010.8Apatent/CN114757935B/enactiveActive
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN208577803U (en)* | 2018-05-28 | 2019-03-05 | 湖南红太阳光电科技有限公司 | A kind of anti-light decline stove equipment of the high production capacity crystal-silicon battery slice of double track |
| CN111368687A (en)* | 2020-02-28 | 2020-07-03 | 成都市微泊科技有限公司 | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation |
| AU2020103901A4 (en)* | 2020-12-04 | 2021-02-11 | Chongqing Normal University | Image Semantic Segmentation Method Based on Deep Full Convolutional Network and Conditional Random Field |
Non-Patent Citations (1)
| Title |
|---|
| 李琦;刘伟;赵建敏;: "基于双目视觉及Mask RCNN的牛体尺无接触测量", 黑龙江畜牧兽医, no. 12, 20 June 2020 (2020-06-20)* |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115578612A (en)* | 2022-10-11 | 2023-01-06 | 浙江大学 | Recognition method and device for blast furnace top material distribution stage based on marker target detection |
| CN115578612B (en)* | 2022-10-11 | 2023-08-04 | 浙江大学 | Blast furnace top distribution stage identification method and device based on marker target detection |
| CN115896817A (en)* | 2023-03-08 | 2023-04-04 | 福建德尔科技股份有限公司 | Production method and system of fluorine-nitrogen mixed gas |
| CN115896817B (en)* | 2023-03-08 | 2023-05-19 | 福建德尔科技股份有限公司 | Production method and system of fluorine-nitrogen mixed gas |
| CN119289722A (en)* | 2024-12-16 | 2025-01-10 | 江西铜业技术研究院有限公司 | An intelligent management system and method for smelting equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114757935B (en) | 2025-04-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114757935A (en) | A multi-target automatic temperature measurement and early warning method for steel cold rolling annealing furnace components | |
| CN114627286B (en) | Detection method of weighbridge intrusion based on PSPNet and improved YOLOv4 | |
| CN112861631B (en) | Wagon balance human body intrusion detection method based on Mask Rcnn and SSD | |
| CN115439458A (en) | Industrial image defect target detection algorithm based on depth map attention | |
| Xu et al. | Real-time monitoring of concrete crack based on deep learning algorithms and image processing techniques | |
| Li et al. | Deep learning-based defects detection of certain aero-engine blades and vanes with DDSC-YOLOv5s | |
| Sengar et al. | Motion detection using block based bi-directional optical flow method | |
| Oh et al. | Adaptive SVM-based real-time quality assessment for primer-sealer dispensing process of sunroof assembly line | |
| CN114155474A (en) | Damage identification technology based on video semantic segmentation algorithm | |
| CN109934170A (en) | A Computer Vision-Based Mine Resource Statistical Method | |
| Hou et al. | A self-supervised CNN for particle inspection on optical element | |
| CN117058095A (en) | Quality judging method and system for multi-angle infrared image of transformer | |
| Xu et al. | Defect detection for PV Modules based on the improved YOLOv5s | |
| Nooralishahi et al. | PHM-IRNET: Self-training thermal segmentation approach for thermographic inspection of industrial components | |
| Anitha et al. | A survey on crack detection algorithms for concrete structures | |
| CN119515858A (en) | A rail surface defect detection method, device, medium and equipment | |
| CN118537708A (en) | Hole detection image damage identification model based on improved convolutional neural network and application system | |
| CN112329743A (en) | Abnormal body temperature monitoring method, device and medium in epidemic situation environment | |
| CN118095314A (en) | Magnetizing tag detection method based on deep learning | |
| CN117237369A (en) | Blast furnace iron notch opening depth measurement method based on computer vision | |
| Kapoor | A video surveillance detection of moving object using deep learning | |
| CN117745945A (en) | AI-based boiler furnace water wall tube wall image acquisition and processing system | |
| Deng et al. | Crack analysis of tall concrete wind towers using an ad-hoc deep multiscale encoder–decoder with depth separable convolutions under severely imbalanced data | |
| CN115345021A (en) | A Measuring System of Drag Coefficient of Marine Biological Flow | |
| Tian et al. | An intelligent method for leakage segmentation and area quantification to evaluate the safety performance of tunnels |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |