CN112733588A - Machine running state detection method and device and electronic equipment - Google Patents
Machine running state detection method and device and electronic equipmentDownload PDFInfo
- Publication number
- CN112733588A CN112733588ACN202010809719.5ACN202010809719ACN112733588ACN 112733588 ACN112733588 ACN 112733588ACN 202010809719 ACN202010809719 ACN 202010809719ACN 112733588 ACN112733588 ACN 112733588A
- Authority
- CN
- China
- Prior art keywords
- result
- target machine
- state
- image
- running
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
- G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及故障检测技术领域，尤其是涉及一种机器运行状态检测方法、装置及电子设备。The present invention relates to the technical field of fault detection, and in particular, to a method, device and electronic equipment for detecting the running state of a machine.
背景技术Background technique
随着科技的发展，机器的应用场景越来越广泛，为了保证机器的正常运行，对于机器运行状态的监测十分必要，以便在机器出现故障时及时维修，保证生产效率的同时，避免重大事故的发生。以煤矿开采现场的采煤机为例，采煤机在采煤过程中，可能会出现非正常采煤的状态，诸如采煤机处于空转状态或空跑状态，目前的采煤机运行状态检测方式，主要采用人工监督采煤机运行状态，或者采集采煤机图像，根据图像判断采煤机的运行状态。然而，人工监督采煤机运行状态的方式耗费人力成本，仅仅依靠采煤机图像判断采煤机状态可能存在准确率较低的问题。因此，现有的机器故障检测方式还存在耗费人力成本或状态检测准确率较低的问题。With the development of science and technology, the application scenarios of machines are becoming more and more extensive. In order to ensure the normal operation of the machine, it is necessary to monitor the running status of the machine, so that the machine can be repaired in time when it breaks down, so as to ensure production efficiency and avoid major accidents. occur. Taking the shearer at the coal mining site as an example, in the process of coal mining, the shearer may appear abnormal coal mining state, such as the shearer is in an idling state or an idle state, and the current shearer operating state detection The main method is to manually supervise the operation state of the shearer, or collect the image of the shearer, and judge the operation state of the shearer according to the image. However, the method of manually supervising the operation status of the shearer consumes labor costs, and there may be a problem of low accuracy in judging the status of the shearer only by relying on the image of the shearer. Therefore, the existing machine fault detection methods also have problems of labor-intensive cost or low accuracy of state detection.
发明内容SUMMARY OF THE INVENTION
有鉴于此，本发明的目的在于提供一种机器运行状态检测方法、装置及电子设备，能够节省人力成本，同时提升目标机器状态检测的准确率。In view of this, the purpose of the present invention is to provide a machine running state detection method, device and electronic device, which can save labor cost and improve the accuracy of target machine state detection.
为了实现上述目的，本发明实施例采用的技术方案如下：In order to achieve the above purpose, the technical solutions adopted in the embodiments of the present invention are as follows:
第一方面，本发明实施例提供了一种机器运行状态检测方法，包括：获取目标机器运行过程中的音频数据，基于所述音频数据确定所述目标机器的运行状态，得到第一结果；获取所述目标机器的运行图像，基于所述运行图像识别所述目标机器的运行状态，得到第二结果；对所述第一结果和所述第二结果进行数据融合，得到所述目标机器的状态检测结果。In a first aspect, an embodiment of the present invention provides a method for detecting a machine running state, including: acquiring audio data during a running process of a target machine, determining the running state of the target machine based on the audio data, and obtaining a first result; acquiring The operating image of the target machine, identifying the operating state of the target machine based on the operating image, and obtaining a second result; performing data fusion on the first result and the second result to obtain the state of the target machine Test results.
进一步，本发明实施例提供了第一方面的第一种可能的实施方式，其中，所述对所述第一结果和所述第二结果进行数据融合，得到所述目标机器的状态检测结果的步骤，包括：基于D-S证据理论算法对所述第一结果和所述第二结果进行融合计算，得到所述目标机器的状态检测结果。Further, an embodiment of the present invention provides a first possible implementation manner of the first aspect, wherein the data fusion of the first result and the second result is performed to obtain the state detection result of the target machine. The step includes: performing fusion calculation on the first result and the second result based on the D-S evidence theory algorithm to obtain the state detection result of the target machine.
进一步，本发明实施例提供了第一方面的第二种可能的实施方式，其中，所述基于D-S证据理论算法对所述第一结果和所述第二结果进行融合计算，得到所述目标机器的状态检测结果的步骤，包括：基于所述第一结果、所述第二结果及融合计算算式确定所述目标机器的状态检测结果。Further, the embodiment of the present invention provides a second possible implementation manner of the first aspect, wherein the D-S evidence theory-based algorithm performs fusion calculation on the first result and the second result to obtain the target machine The step of determining the state detection result of the target machine includes: determining the state detection result of the target machine based on the first result, the second result and the fusion calculation formula.
进一步，本发明实施例提供了第一方面的第三种可能的实施方式，其中，所述目标机器包括采煤机，所述运行状态包括正常运行状态、停机状态和非正常运行状态；所述融合计算算式为：Further, an embodiment of the present invention provides a third possible implementation manner of the first aspect, wherein the target machine includes a shearer, and the operating state includes a normal operating state, a shutdown state, and an abnormal operating state; the The fusion calculation formula is:
其中，m(A)为所述状态检测结果，m1(A1)为所述第一结果， m2(A2)为所述第二结果，所述第一结果和所述第二结果包括所述采煤机处于各所述运行状态下的概率值，A1为基于所述音频数据确定所述目标机器的运行状态的事件，A2为基于所述运行图像识别所述目标机器的运行状态的事件，k为冲突因子。Wherein, m(A) is the state detection result, m1 (A1 ) is the first result, m2 (A2 ) is the second result, the first result and the second result Including the probability value that the shearer is in each of the operating states, A1 is an event that determines the operating state of the target machine based on the audio data, and A2 is an event that identifies the target machine based on the operating image. Events in the running state, k is the conflict factor.
进一步，本发明实施例提供了第一方面的第四种可能的实施方式，其中，所述获取目标机器运行过程中的音频数据，基于所述音频数据确定所述目标机器的运行状态，得到第一结果的步骤，包括：采集目标机器运行过程中的音频数据，并基于所述音频数据确定所述目标机器运行产生的频谱图；将所述频谱图输入预先训练得到的第一神经网络模型中，得到所述目标机器的第一结果；其中，所述第一神经网络模型是基于标注有运行状态的频谱图样本训练得到的。Further, an embodiment of the present invention provides a fourth possible implementation manner of the first aspect, wherein the acquiring audio data during the running of the target machine, determining the running state of the target machine based on the audio data, and obtaining the first The step of a result includes: collecting audio data during the operation of the target machine, and determining a spectrogram generated by the operation of the target machine based on the audio data; inputting the spectrogram into a first neural network model obtained by pre-training , and obtain the first result of the target machine; wherein, the first neural network model is obtained by training based on the spectrogram samples marked with the running state.
进一步，本发明实施例提供了第一方面的第五种可能的实施方式，其中，所述基于所述音频数据确定所述目标机器运行产生的频谱图的步骤，包括：将所述音频数据分割为预设时长的音频片段；对各所述音频片段进行去噪处理，并利用音频数据处理库将各所述音频片段转换为对应的频谱图。Further, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, wherein the step of determining a spectrogram generated by the operation of the target machine based on the audio data includes: dividing the audio data is an audio clip with a preset duration; performs denoising processing on each audio clip, and converts each audio clip into a corresponding spectrogram by using an audio data processing library.
进一步，本发明实施例提供了第一方面的第六种可能的实施方式，其中，所述基于所述运行图像识别所述目标机器的运行状态，得到第二结果的步骤，包括：将所述运行图像输入预先训练得到的第二神经网络模型中，得到所述目标机器的第二结果；其中，所述第二神经网络模型是基于标注有运行状态的运行图像样本训练得到的；或者，根据所述运行图像获取与所述目标机器运行状态相关的参数信息，基于所述参数信息确定所述目标机器的运行状态，得到第二结果。Further, an embodiment of the present invention provides a sixth possible implementation manner of the first aspect, wherein the step of recognizing the operating state of the target machine based on the operating image to obtain a second result includes: The second result of the target machine is obtained by inputting the operating image into the second neural network model obtained by pre-training; wherein, the second neural network model is obtained by training based on the operating image samples marked with the operating state; or, according to The operating image acquires parameter information related to the operating state of the target machine, determines the operating state of the target machine based on the parameter information, and obtains a second result.
第二方面，本发明实施例还提供了一种机器运行状态检测装置，包括：状态确定模块，用于获取目标机器的音频数据，基于所述音频数据确定所述目标机器的运行状态，得到第一结果；状态识别模块，用于获取所述目标机器的运行图像，基于所述运行图像识别所述目标机器的运行状态，得到第二结果；数据融合模块，用于对所述第一结果和所述第二结果进行数据融合，得到所述目标机器的状态检测结果。In a second aspect, an embodiment of the present invention further provides a device for detecting a running state of a machine, including: a state determination module configured to acquire audio data of a target machine, determine the running state of the target machine based on the audio data, and obtain the first a result; a state identification module, configured to acquire the running image of the target machine, identify the running state of the target machine based on the running image, and obtain a second result; a data fusion module, configured to compare the first result and the Data fusion is performed on the second result to obtain the state detection result of the target machine.
第三方面，本发明实施例提供了一种电子设备，包括：声音采集装置、图像采集装置、处理器和存储装置；所述图像采集装置用于获取目标机器运行过程中的音频数据；所述声音采集装置用于获取目标机器的运行图像；所述存储装置上存储有计算机程序，所述计算机程序在被所述处理器运行时执行如第一方面任一项所述的方法。In a third aspect, an embodiment of the present invention provides an electronic device, including: a sound acquisition device, an image acquisition device, a processor, and a storage device; the image acquisition device is used to acquire audio data during the operation of a target machine; the The sound collection device is used for acquiring the running image of the target machine; a computer program is stored on the storage device, and the computer program executes the method according to any one of the first aspect when it is run by the processor.
第四方面，本发明实施例提供了一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，其特征在于，所述计算机程序被处理器运行时执行上述第一方面任一项所述的方法的步骤。In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, wherein the computer program executes any of the above-mentioned first aspect when the computer program is run by a processor. A step of the method.
本发明实施例提供了一种机器运行状态检测方法、装置及电子设备，在该方法中，首先获取目标机器运行过程中的音频数据，基于音频数据确定目标机器的运行状态，得到第一结果；然后获取目标机器的运行图像，基于运行图像识别目标机器的运行状态，得到第二结果；最后对第一结果和第二结果进行数据融合，得到目标机器的状态检测结果。通过对基于目标机器运行的音频数据得到的第一结果，及基于目标机器的运行图像得到的第二结果进行数据融合，可以自动得到目标机器的状态检测结果，节省了人力成本，通过基于目标机器运行的声音和图像综合判断运行状态，提升了目标机器状态检测的准确率。Embodiments of the present invention provide a method, a device, and an electronic device for detecting a running state of a machine. In the method, audio data during the running process of the target machine is first obtained, and the running state of the target machine is determined based on the audio data, and a first result is obtained; Then, the running image of the target machine is obtained, and the running state of the target machine is recognized based on the running image to obtain the second result; finally, the data fusion of the first result and the second result is performed to obtain the state detection result of the target machine. By data fusion of the first result obtained based on the audio data running on the target machine and the second result obtained based on the running image of the target machine, the state detection result of the target machine can be automatically obtained, saving labor costs. The running sound and image comprehensively judge the running status, which improves the accuracy of the target machine status detection.
本发明实施例的其他特征和优点将在随后的说明书中阐述，或者，部分特征和优点可以从说明书推知或毫无疑义地确定，或者通过实施本发明实施例的上述技术即可得知。Other features and advantages of embodiments of the present invention will be set forth in the description that follows, or some of the features and advantages may be inferred or unequivocally determined from the description, or may be learned by implementing the above-described techniques of embodiments of the present invention.
为使本发明的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below, and are described in detail as follows in conjunction with the accompanying drawings.
附图说明Description of drawings
为了更清楚地说明本发明具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to illustrate the specific embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the specific embodiments or the prior art. Obviously, the accompanying drawings in the following description The drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.
图1示出了本发明实施例所提供的一种机器运行状态检测方法流程图；1 shows a flowchart of a method for detecting a machine running state provided by an embodiment of the present invention;
图2示出了本发明实施例所提供的一种D-S证据理论的置信区间示意图；FIG. 2 shows a schematic diagram of a confidence interval of a D-S evidence theory provided by an embodiment of the present invention;
图3示出了本发明实施例所提供的一种采煤机正常割煤频谱图；Fig. 3 shows the frequency spectrum diagram of a shearer normally cutting coal provided by an embodiment of the present invention;
图4示出了本发明实施例所提供的一种采煤机空跑刀频谱图；Fig. 4 shows the frequency spectrum diagram of a shearer empty running cutter provided by an embodiment of the present invention;
图5示出了本发明实施例所提供的一种采煤机割底板频谱图；5 shows a spectrum diagram of a shearer cutting floor provided by an embodiment of the present invention;
图6示出了本发明实施例所提供的一种采煤机运行状态检测流程图；FIG. 6 shows a flow chart for detecting the operating state of a shearer provided by an embodiment of the present invention;
图7示出了本发明实施例所提供的一种机器运行状态检测装置结构示意图；FIG. 7 shows a schematic structural diagram of a device for detecting a machine running state provided by an embodiment of the present invention;
图8示出了本发明实施例所提供的一种电子设备的结构示意图。FIG. 8 shows a schematic structural diagram of an electronic device provided by an embodiment of the present invention.
具体实施方式Detailed ways
以下由特定的具体实施例说明本发明的实施方式，熟悉此技术的人士可由本说明书所揭露的内容轻易地了解本发明的其他优点及功效，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The embodiments of the present invention are described below by specific specific embodiments. Those who are familiar with the technology can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the present invention. , not all examples. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
为使本发明的上述目的、特征和优点能够更为明显易懂，下面结合附图对本发明的具体实施例做详细的说明。In order to make the above objects, features and advantages of the present invention more clearly understood, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
实施例一：Example 1:
本实施例提供了一种机器运行状态检测方法，该方法可以应用于电子设备，该电子设备可以包括声音采集装置和图像采集装置，参见图1所示的机器运行状态检测方法流程图，该方法主要包括以下步骤 S102～步骤S106：This embodiment provides a method for detecting a machine running state, which can be applied to an electronic device. The electronic device may include a sound acquisition device and an image acquisition device. Referring to the flowchart of the method for detecting a machine running state shown in FIG. 1 , the method It mainly includes the following steps S102 to S106:
步骤S102，获取目标机器运行过程中的音频数据，基于音频数据确定目标机器的运行状态，得到第一结果。Step S102: Acquire audio data in the running process of the target machine, determine the running state of the target machine based on the audio data, and obtain a first result.
通过声音采集装置(诸如录音器或者带有录音功能的高清防爆摄像头)实时或以预设时间间隔采集目标机器运行过程中的声音，得到目标机器运行的音频数据，由于机器在不同的运行状态下的声音不同，产生的音频数据也不相同，根据目标机器运行过程中的音频数据，可以判断出目标机器的运行状态。Collect the sound of the target machine in real time or at preset time intervals through a sound collection device (such as a recorder or a high-definition explosion-proof camera with a recording function), and obtain the audio data of the target machine's operation. Since the machine is in different operating states According to the audio data in the running process of the target machine, the running state of the target machine can be judged.
步骤S104，获取目标机器的运行图像，基于运行图像识别目标机器的运行状态，得到第二结果。In step S104, an operating image of the target machine is acquired, the operating state of the target machine is recognized based on the operating image, and a second result is obtained.
在采集上述音频数据的同时，通过图像采集装置实时或以预设时间间隔采集目标机器运行过程中的图像，或者，通过图像采集装置采集目标机器运行过程中的视频数据，得到目标机器的运行图像，根据目标机器的运行图像，可以识别出目标机器的运行状态，诸如根据采煤机运行图像中煤块的位置变化情况，可以确定采煤机的运行状态。While collecting the above-mentioned audio data, the image during the operation of the target machine is collected in real time or at preset time intervals through the image collection device, or the video data during the operation of the target machine is collected through the image collection device to obtain the operation image of the target machine , according to the operation image of the target machine, the operation state of the target machine can be identified, such as the operation state of the shearer can be determined according to the position change of the coal block in the operation image of the shearer.
步骤S106，对第一结果和第二结果进行数据融合，得到目标机器的状态检测结果。Step S106, perform data fusion on the first result and the second result to obtain a state detection result of the target machine.
将上述得到的第一结果和第二结果(该第一结果及第二结果为同一时间段内对目标机器进行检测得到的结果)进行数据融合，诸如可以将上述第一结果与上述第二结果进行加权平均计算，综合考虑基于目标机器的声音判断得到的第一结果和基于目标机器的图像判断得到的第二结果，得到目标机器的状态检测结果。Data fusion is performed on the first result obtained above and the second result (the first result and the second result are the results obtained by detecting the target machine in the same time period), such as the above first result and the above second result. A weighted average calculation is performed, and the first result obtained based on the sound judgment of the target machine and the second result obtained based on the image judgment of the target machine are comprehensively considered to obtain the state detection result of the target machine.
本实施例提供的上述机器运行状态检测方法，通过对基于目标机器运行的音频数据得到的第一结果，及基于目标机器的运行图像得到的第二结果进行数据融合，可以自动得到目标机器的状态检测结果，节省了人力成本，通过基于目标机器运行的声音和图像综合判断运行状态，提升了目标机器状态检测的准确率。The above-mentioned machine operating state detection method provided in this embodiment can automatically obtain the state of the target machine by performing data fusion on the first result obtained based on the audio data of the target machine operation and the second result obtained based on the operating image of the target machine. The detection result saves labor costs, and improves the accuracy of the state detection of the target machine by comprehensively judging the running state based on the sound and image of the target machine running.
为了准确得到目标机器的状态检测结果，本实施例提供了对第一结果和第二结果进行数据融合，得到目标机器的状态检测结果的具体实施方式：基于D-S证据理论算法对第一结果和第二结果进行融合计算，得到目标机器的状态检测结果。采用D-S证据理论算法对第一结果和第二结果进行融合判断，计算所得的结果为最终的融合结果。其中，D-S证据理论算法的基本原理为：随机变量X的可能取值构成了基本框架U，即所有取值构成的集合。其中各值互不兼容、相互独立，可以称U为目标X的识别框架。U的幂集2U即可求得U中元素个数为N的幂级数为2N。设U为识别框架，U中元素个数为N，其幂集为2U。对于2U中的任何子集A，称为命题A。基本概率分配函数m 在2U上定义，取值[0,1]，m(A)表示证据对命题A成立的信任度，满足以下规则：
设U为识别框架，U中元素个数为N，其幂集为2U。在2U上定义信度函数Bel，取值[0,1]；在2U上定义似然函数Pl，取值[0,1]。对2U中的任何命题A，满足：
当命题A有多个相互独立的证据源支持时(诸如基于音频数据识别目标机器的运行状态、基于运行图像识别目标机器的运行状态)，证据理论用组合规则来计算基本概率分配值BPA。对于n个证据的 BPA函数，组合规则为：When proposition A is supported by multiple independent evidence sources (such as recognizing the operating state of the target machine based on audio data, and recognizing the operating state of the target machine based on operating images), evidence theory uses combination rules to calculate the basic probability assignment value BPA. For the BPA function of n evidences, the combination rule is:
其中，
设上述命题A为目标机器的状态检测结果(该命题A存在两个相互独立的证据源：基于音频数据识别得到的目标机器的运行状态 m1(A1)及基于运行图像识别得到的目标机器的运行状态m2(A2))，上述目标机器包括采煤机，运行状态包括正常运行状态、停机状态和非正常运行状态；基于第一结果、第二结果及融合计算算式确定目标机器的状态检测结果。上述融合计算算式为：Let the above proposition A be the state detection result of the target machine (there are two mutually independent evidence sources for this proposition A: the operating state m1 (A1 ) of the target machine recognized based on the audio data and the target machine recognized based on the operating image. the operating state m2 (A2 )), the above-mentioned target machine includes a shearer, and the operating state includes a normal operating state, a shutdown state and an abnormal operating state; based on the first result, the second result and the fusion calculation formula, determine the Status check result. The above fusion calculation formula is:
其中，m(A)为状态检测结果，m1(A1)为第一结果，m2(A2)为第二结果，第一结果和第二结果包括采煤机处于各运行状态下的概率值，A1为基于音频数据确定目标机器的运行状态的事件，A2为基于运行图像识别目标机器的运行状态的事件，k为冲突因子，
为了提升上述第一结果的准确性，本实施例提供了获取目标机器运行过程中的音频数据，基于音频数据确定目标机器的运行状态，得到第一结果的实施方式，具体可参照如下步骤(1)～步骤(2)执行：In order to improve the accuracy of the above-mentioned first result, this embodiment provides an implementation manner of obtaining the audio data during the operation of the target machine, determining the operating state of the target machine based on the audio data, and obtaining the first result. For details, refer to the following steps (1 ) to step (2) execute:
步骤(1)：采集目标机器运行过程中的音频数据，并基于音频数据确定目标机器运行产生的频谱图。Step (1): collect audio data during the operation of the target machine, and determine a spectrogram generated by the operation of the target machine based on the audio data.
在目标机器旁边安装声音采集装置，诸如麦克风等录音装置或者具有声音采集功能的摄像头，通过声音采集装置实时采集目标机器运行过程中清晰的机器声音，得到目标机器的音频数据。将音频数据分割为预设时长的音频片段，对各音频片段进行去噪处理，并利用音频数据处理库将各音频片段转换为对应的频谱图。将采集到的音频数据平均分割成相同时间段(预设时长，诸如可以是4～10s之间的任意数值)的多个音频片段，对各个音频片段进行去噪处理，利用音频数据处理库(诸如可以是Librosa音频处理库)将去噪后的各个音频片段转换成频谱图。Install a sound collection device next to the target machine, such as a recording device such as a microphone or a camera with sound collection function, and collect the clear machine sound during the operation of the target machine in real time through the sound collection device to obtain the audio data of the target machine. The audio data is divided into audio segments of preset duration, each audio segment is denoised, and each audio segment is converted into a corresponding spectrogram using an audio data processing library. Divide the collected audio data into multiple audio clips of the same time period (the preset duration, such as any value between 4 and 10s), perform denoising processing on each audio clip, and use the audio data processing library ( Such as the Librosa audio processing library) converts the denoised individual audio segments into spectrograms.
步骤(2)：将频谱图输入预先训练得到的第一神经网络模型中，得到目标机器的第一结果。Step (2): Input the spectrogram into the first neural network model obtained by pre-training to obtain the first result of the target machine.
上述神经网络模型可以是卷积神经网络，由于目标机器在不同的工作状态下产生的频谱图的在频率、强弱等方面均存在差异，将上述得到的频谱图输入预先训练得到的第一神经网络模型中，可以得到输入的频谱图对应的状态识别结果，记为第一结果。在实际应用中，上述目标机器可以是采煤机，上述运行状态包括正常运行状态、停机状态和非正常运行状态，其中，上述非正常运行状态包括跑空刀状态和割底板状态，上述第一结果可以以数组的形式呈现，该数组中包括频谱图为各个运行状态的概率，诸如，上述第一结果可以是[0.9，0.02，0.08]，其中，0.9是采煤机为正常运行状态的概率，0.02是采煤机为停机状态的概率，0.08是采煤机为非正常运行状态的概率。The above neural network model can be a convolutional neural network. Since the spectrograms generated by the target machine in different working states are different in terms of frequency, strength, etc., the spectrograms obtained above are input into the first neural network obtained by pre-training. In the network model, the state identification result corresponding to the input spectrogram can be obtained, which is recorded as the first result. In practical applications, the above-mentioned target machine may be a shearer, and the above-mentioned operating states include a normal operating state, a shutdown state and an abnormal operating state, wherein the above-mentioned abnormal operating states include a running-away knife state and a cutting floor state. The results can be presented in the form of an array including the probability that the spectrogram is in each operating state, for example, the above first result can be [0.9, 0.02, 0.08], where 0.9 is the probability that the shearer is in a normal operating state , 0.02 is the probability that the shearer is in a shutdown state, and 0.08 is the probability that the shearer is in an abnormal operating state.
上述第一神经网络模型是基于标注有运行状态的频谱图样本训练得到的，将标注有不同运行状态的多张频谱图输入第一神经网络模型中进行训练，直至达到预设的迭代次数，得到训练后的第一神经网络模型。为了提升第一神经网络模型识别的准确性，上述频谱图样本可以包括采煤机在各个运行状态下的频谱图，参见如图3所示的采煤机正常割煤频谱图，图3中示出了采煤机在正常运行状态下的频谱图，参见如图4所示的采煤机空跑刀频谱图，图4中示出了采煤机在非正常状态跑空刀时频谱图，参见如图5所示的采煤机割底板频谱图，图 5中示出了采煤机在非正常状态割底板时的频谱图，从图3至图5中可以看出，采煤机在各个运行状态下的频谱图中的频率和强度均不相同，第一神经网络模型根据频谱图中的声音震动频率及震动强度，可以确定频谱图对应的运行状态。The above-mentioned first neural network model is obtained by training based on the spectrogram samples marked with operating states, and multiple spectrograms marked with different operating states are input into the first neural network model for training until a preset number of iterations is reached, and the result is obtained. The first neural network model after training. In order to improve the recognition accuracy of the first neural network model, the above-mentioned spectrogram samples may include the spectrograms of the shearer in various operating states. Please refer to the spectrogram of the shearer normally cutting coal shown in FIG. 3 . The spectrogram of the shearer in the normal operating state is shown. See the spectrogram of the shearer running the knife in the air as shown in Figure 4. Figure 4 shows the frequency spectrum when the shearer runs the knife in the abnormal state. Referring to the frequency spectrum of the shearer cutting the floor as shown in Figure 5, Figure 5 shows the frequency spectrum of the shearer cutting the floor in an abnormal state. It can be seen from Figures 3 to 5 that the shearer is in The frequencies and intensities in the spectrogram in each operating state are different, and the first neural network model can determine the operating state corresponding to the spectrogram according to the sound vibration frequency and vibration intensity in the spectrogram.
上述频谱图样本中的各个频谱图是基于预设时长的音频片段转换得到的，且各个音频片段的长度相同，当音频片段的长度不一致时，在后期的频谱图识别中容易导致识别准确率较低，因此在神经网络模型的训练及识别过程中使用的频谱图均是基于相同长度的音频片段转换得到的，该音频片段的长度诸如可以是4s。The spectrograms in the above spectrogram samples are converted based on audio clips of preset duration, and the lengths of the audio clips are the same. Therefore, the spectrograms used in the training and identification of the neural network model are all converted based on audio clips of the same length, and the length of the audio clips may be, for example, 4s.
为了提升上述第二结果的准确性，本实施例提供了基于运行图像识别目标机器的运行状态，得到第二结果的两种实施方式，具体可参照如下实施方式一和实施方式二执行：In order to improve the accuracy of the above-mentioned second result, this embodiment provides two implementations for recognizing the running state of the target machine based on the running image to obtain the second result, which can be implemented with reference to the following implementations one and two:
实施方式一：将运行图像输入预先训练得到的第二神经网络模型中，得到目标机器的第二结果。利用图像采集装置采集目标机器的运行图像，上述图像采集装置可以设置于目标机器的正上方或正前方，以拍摄目标机器的工作过程。由于目标机器在不同运行状态下零部件的位置不同，以及在不同运行状态下目标机器上的材料位置或数量不同，将目标机器的运行图像输入预先训练得到的第二神经网络模型中，可以识别得到目标机器的运行状态，记为第二结果。该第二结果也可以是以数组的形式呈现，该数组中包括输入第二神经网络模型的运行图像为各个运行状态的概率值。Embodiment 1: Input the running image into the second neural network model obtained by pre-training, and obtain the second result of the target machine. The operation image of the target machine is collected by using an image collection device, and the above-mentioned image collection device can be arranged directly above or in front of the target machine to photograph the working process of the target machine. Since the positions of the parts and components of the target machine are different in different operating states, and the positions or quantities of materials on the target machine are different in different operating states, the operating images of the target machine are input into the pre-trained second neural network model, which can identify The running state of the target machine is obtained, which is recorded as the second result. The second result may also be presented in the form of an array, where the array includes probability values that the operating image input to the second neural network model is each operating state.
第二神经网络模型可以识别到运行图像中的两个滚筒及落煤图像的坐标位置，并根据两个滚筒及落煤图像的坐标位置得到第二结果。诸如，当第二神经网络模型识别到运行图像中采煤机的两个滚筒未处于同一水平线，且距离滚筒预设距离范围内的图像包含落煤图像时，得到的第二结果中正常运行状态对应的概率值最大，即采煤机处于正常运行状态；由于采煤机在运行时落煤位置会产生变化，因此，输入第二神经网络模型中的运行图像可以是一张运行图像也可以是多张连续帧的运行图像，如果第二神经网络识别到在相邻第一预设数量的帧图像中，存在采煤机的两个滚筒未处于同一水平线，且距离滚筒预设距离范围内的图像包含落煤图像的运行图像，确定采煤机处于正常运行状态；当第二神经网络模型识别到在相邻第二预设数量的运行图像中采煤机的两个滚筒处于同一水平线保持位置不变时，得到的第二结果中停机状态对应的概率值最大，即采煤机处于停机状态；当第二神经网络模型识别到在相邻第三预设数量的运行图像中采煤机的两个滚筒处于同一水平线且发生位置变化时，得到的第二结果中非正常运行状态对应的概率值最大，即采煤机处于非正常运行状态。The second neural network model can identify the coordinate positions of the two rollers and the coal falling image in the running image, and obtain a second result according to the coordinate positions of the two rollers and the coal falling image. For example, when the second neural network model recognizes that the two drums of the shearer are not on the same horizontal line in the operating image, and the images within a preset distance from the drums include images of falling coal, the second result obtained is in the normal operating state. The corresponding probability value is the largest, that is, the shearer is in a normal operation state; since the coal drop position of the shearer will change during operation, the operation image input into the second neural network model can be an operation image or a Running images of multiple consecutive frames, if the second neural network recognizes that in the adjacent first preset number of frame images, there are two drums of the shearer that are not on the same horizontal line and are within a preset distance from the drums. The image contains the running image of the coal falling image, and it is determined that the shearer is in a normal operation state; when the second neural network model recognizes that the two drums of the shearer are in the same horizontal line holding position in the adjacent second preset number of running images When unchanged, the probability value corresponding to the shutdown state in the obtained second result is the largest, that is, the shearer is in the shutdown state; when the second neural network model recognizes that the shearer is in the adjacent third preset number of running images. When the two drums are on the same horizontal line and the position changes, the probability value corresponding to the abnormal operation state in the second result obtained is the largest, that is, the shearer is in the abnormal operation state.
其中，上述第二神经网络模型是基于标注有运行状态的运行图像样本训练得到的，为了提升第二神经网络模型的准确性，上述运行图像样本包括各个运行状态下的运行图像样本。通过对目标机器的运行图像样本进行标注，在运行图像中标注零部件的位置或者目标机器上材料的位置或数量，并标注该运行图像对应的运行状态，并将标注好的多张运行图像样本输入第二神经网络模型中，基于多张运行图像样本对第二神经网络模型进行训练，直至达到预设的迭代次数，得到训练后的第二神经网络模型。Wherein, the above-mentioned second neural network model is obtained by training based on running image samples marked with running states. In order to improve the accuracy of the second neural network model, the above-mentioned running image samples include running image samples in each running state. By annotating the running image samples of the target machine, the positions of parts or the positions or quantities of materials on the target machine are marked in the running images, and the corresponding running states of the running images are marked, and the marked multiple running image samples are marked. The second neural network model is input into the second neural network model, and the second neural network model is trained based on the plurality of running image samples until a preset number of iterations is reached, and the trained second neural network model is obtained.
实施方式二：根据运行图像获取与目标机器运行状态相关的参数信息，基于参数信息确定目标机器的运行状态，得到第二结果。在该实施方式中，上述运行图像包括多张连续帧运行图像，上述目标机器为采煤机，上述与目标机器运行状态相关的参数信息包括采煤机的两个滚筒的位置信息及距离该滚筒预设距离范围内的图像。上述采煤机的运行状态包括正常运行状态、停机状态、空跑状态及空转状态。根据运行图像中采煤机的两个滚筒的位置信息，及距离滚筒预设距离范围内的图像中是否有落煤图像，可以得到目标机器的运行状态。Embodiment 2: Acquire parameter information related to the running state of the target machine according to the running image, determine the running state of the target machine based on the parameter information, and obtain the second result. In this embodiment, the above-mentioned operation image includes a plurality of continuous frame operation images, the above-mentioned target machine is a shearer, and the above-mentioned parameter information related to the operation state of the target machine includes the position information of the two drums of the shearer and the distance between the drums. Images within a preset distance range. The operating states of the above shearer include normal operating state, shutdown state, idling state and idling state. According to the position information of the two drums of the shearer in the running image, and whether there is an image of falling coal in the image within a preset distance from the drum, the running state of the target machine can be obtained.
具体的，可以识别采煤机的运行图像中两个滚筒的位置，得到两个滚筒的位置信息，根据多张连续帧图像中两个滚筒的位置信息及距离上滚筒预设距离范围内的图像是否包含落煤图像等信息，确定采煤机的运行状态。Specifically, the positions of the two drums in the operating image of the shearer can be identified, and the position information of the two drums can be obtained. Whether it contains information such as coal falling images to determine the operating status of the shearer.
当第一帧图像中采煤机的两个滚筒未处于同一水平线，第二帧图像中采煤机的上滚筒的位置与第一帧图像中的采煤机的上滚筒的位置不相同，且第二帧图像中距离上滚筒预设距离范围内的图像包含落煤图像时，确定采煤机处于正常运行状态。在实际应用中，可以通过卷积神经网络方法识别运行图像中每一个滚筒分别对应的位置信息，其中位置信息包括与每一个滚筒位置对应的中心坐标、高度和宽度。当确定两个滚筒的中心坐标中的纵坐标之间的绝对差值大于高度的预设百分比数值后，确定两个滚筒未处于同一水平线，否则确定两个滚筒处于同一水平线。还利用预构建的卷积神经网络模型识别距离上滚筒预设距离范围内的图像是否包含落煤图像，其中，预构建的卷积神经网络模型为采用第一样本和第二样本共同训练后获取的卷积神经网络模型，第一样本为在上滚筒预设距离范围内包含落煤图像的图像样本；第二样本为在上滚筒预设距离范围内未包含落煤图像的样本，且在第一样本和第二样本中，均以预设标识对上滚筒预设距离范围内的图像进行标识。When the two drums of the shearer are not on the same horizontal line in the first frame image, the position of the upper drum of the shearer in the second frame image is different from the position of the upper drum of the shearer in the first frame image, and When the image within the preset distance range from the upper drum in the second frame of image contains the coal falling image, it is determined that the shearer is in a normal operation state. In practical applications, the convolutional neural network method can be used to identify the position information corresponding to each drum in the running image, where the position information includes the center coordinates, height and width corresponding to the position of each drum. When it is determined that the absolute difference between the ordinates in the center coordinates of the two drums is greater than the preset percentage value of the height, it is determined that the two drums are not on the same horizontal line, otherwise it is determined that the two drums are on the same horizontal line. A pre-built convolutional neural network model is also used to identify whether the images within the preset distance range of the upper drum include images of falling coal, wherein the pre-built convolutional neural network model is jointly trained by using the first sample and the second sample. In the acquired convolutional neural network model, the first sample is an image sample that includes a coal falling image within a preset distance range of the upper drum; the second sample is a sample that does not include a coal falling image within the preset distance range of the upper drum, and In both the first sample and the second sample, images within a preset distance range of the upper drum are identified with preset identifications.
当第一帧图像中采煤机的两个滚筒未处于同一水平线，第二帧图像中采煤机的上滚筒的位置与第一帧图像中的采煤机的上滚筒的位置不相同，且第二帧图像中距离上滚筒预设距离范围内的图像没有包含落煤图像时，记录第一时刻，且以第一时刻为起始，在第一预设时间内以逐帧形式识别第二帧图像之后的图像中，距离上滚筒预设距离范围内的图像包含落煤图像时，确定采煤机处于正常运行状态。或者，当第一帧图像中采煤机的两个滚筒未处于同一水平线，第二帧图像中采煤机的上滚筒的位置与第一帧图像中的采煤机的上滚筒的位置相同，且第二帧图像中所包含的距离上滚筒预设距离范围内的图像包含落煤图像时，确定采煤机处于正常运行状态。When the two drums of the shearer are not on the same horizontal line in the first frame image, the position of the upper drum of the shearer in the second frame image is different from the position of the upper drum of the shearer in the first frame image, and When the images within the preset distance from the upper drum in the second frame of images do not contain coal falling images, the first moment is recorded, and starting from the first moment, the second frame is identified within the first preset time frame by frame. In the image after the frame image, when the image within the preset distance range from the upper drum contains the coal falling image, it is determined that the shearer is in a normal operation state. Or, when the two drums of the shearer in the first frame image are not on the same horizontal line, the position of the upper drum of the shearer in the second frame image is the same as the position of the upper drum of the shearer in the first frame image, And when the images included in the second frame of images within the preset distance range of the upper drum include images of falling coal, it is determined that the shearer is in a normal operation state.
当根据第一帧图像中采煤机的两个滚筒的位置信息确定两个滚筒处于同一水平线时，记录第二时刻，且以第二时刻为起始，第二预设时间内以逐帧的形式识别第2+i帧图像中采煤机的两个滚筒位置未处于同一水平线时，识别第2+i帧图像中采煤机的上滚筒位置和第1+i 帧图像中上滚筒的位置是否相同，间接确定采煤机处于正常运行状态或者处于空跑状态，其中，i为正整数，i依次递进取值，初始取值为 1。When it is determined according to the position information of the two drums of the shearer in the first frame of images that the two drums are on the same horizontal line, the second moment is recorded, and starting from the second moment, the second preset time is frame-by-frame Formal Recognition When the positions of the two drums of the shearer in the 2+i frame image are not on the same horizontal line, identify the position of the upper drum of the shearer in the 2+i frame image and the position of the upper drum in the 1+i frame image Whether it is the same or not, it indirectly determines whether the shearer is in a normal operation state or in an idle state, where i is a positive integer, i takes values progressively in turn, and the initial value is 1.
当第一帧图像中采煤机的两个滚筒未处于同一水平线，第二帧图像中采煤机的上滚筒的位置与第一帧图像中的采煤机的上滚筒的位置不相同，以逐帧的形式识别第一时刻之后的第2+i帧图像中距离上滚筒预设距离范围内的图像没有包含落煤图像，且逐帧识别的时间超出第一预设时间时，确定采煤机处于空跑状态，或者，当相邻两帧图像中第一帧图像中采煤机的两个滚筒的位置信息确定两个滚筒处于同一水平线，以逐帧的形式识别第二时刻之后的图像中采煤机的两个滚筒位置处于同一水平线，逐帧识别的时间超过第二预设时间，且确定第二预设时间内最后一帧图像中采煤机的两个滚筒所在位置与前一帧图像中采煤机的两个滚筒所在位置不相同时，确定采煤机处于空跑状态。When the two drums of the shearer are not on the same horizontal line in the first frame image, the position of the upper drum of the shearer in the second frame image is different from the position of the upper drum of the shearer in the first frame image, so that In the frame-by-frame form identification, in the 2+i frame images after the first moment, the images within the preset distance range from the upper drum do not contain coal falling images, and when the frame-by-frame identification time exceeds the first preset time, it is determined that coal mining is carried out. The machine is in an idle state, or, when the position information of the two drums of the shearer in the first frame of the adjacent two frames of images determines that the two drums are on the same horizontal line, the images after the second moment are identified frame by frame. The positions of the two drums of the middle shearer are on the same horizontal line, the frame-by-frame recognition time exceeds the second preset time, and it is determined that the positions of the two drums of the shearer in the last frame of the second preset time are the same as the previous one. When the positions of the two drums of the shearer in the frame image are not the same, it is determined that the shearer is in an empty running state.
上述参数信息还包括刮板输送机上的煤块图像。当根据相邻两帧图像中采煤机的两个滚筒的位置信息，确定第一帧图像中对应的采煤机的两个滚筒未处于同一水平线，相邻两帧图像中第二帧图像中对应的采煤机的上滚筒的位置与第一帧图像中的采煤机的上滚筒的位置相同，且识别第2+i帧图像中距离上滚筒预设距离范围内的图像中没有落煤图像时，若根据第2+i帧图像中包含的刮板输送机上的煤块图像确定刮板输送机上的煤块未发生移动时，确定采煤机处于非正常停机状态。或者，若根据第2+i帧图像中包含的刮板输送机上的煤块图像确定刮板输送机上的煤块发生移动，记录第三时刻；以第三时刻开始，在第三预设时间内以逐帧的形式识别包含刮板输送机上的煤块图像的图像中，若确定刮板输送机上的煤块未发生移动时，确定采煤机处于非正常停机状态；或者，若确定以第三时刻开始，逐帧识别的时间超出第三预设时间，且确定第2+j帧图像中包含的刮板输送机上的煤块发生移动时，确定采煤机处于空转状态，其中j为大于或者等于 i的正整数，j依次递进取值，初始取值为1。The above parameter information also includes images of coal lumps on the scraper conveyor. When it is determined according to the position information of the two drums of the shearer in the two adjacent frames of images that the two drums of the corresponding shearer in the first frame are not on the same horizontal line, the second frame of the adjacent two frames The position of the upper drum of the corresponding shearer is the same as the position of the upper drum of the shearer in the first frame image, and it is recognized that there is no coal falling in the image within the preset distance range from the upper drum in the second+i frame image. In the image, if it is determined that the coal block on the scraper conveyor does not move according to the coal block image on the scraper conveyor included in the 2+i frame image, it is determined that the shearer is in an abnormal shutdown state. Or, if it is determined that the coal block on the scraper conveyor moves according to the image of the coal block on the scraper conveyor included in the 2+i frame image, record the third time; starting from the third time, within the third preset time In the image containing the image of the coal block on the scraper conveyor in the form of frame-by-frame identification, if it is determined that the coal block on the scraper conveyor has not moved, it is determined that the shearer is in an abnormal shutdown state; or, if it is determined that the third Starting from the moment, the frame-by-frame recognition time exceeds the third preset time, and when it is determined that the coal block on the scraper conveyor included in the 2+j frame image moves, it is determined that the shearer is in an idling state, where j is greater than or A positive integer equal to i, j takes values sequentially, and the initial value is 1.
上述参数信息还包括刮板输送机上的煤块图像，当第一帧图像中采煤机的两个滚筒未处于同一水平线，以逐帧的形式识别第二时刻之后的图像中采煤机的两个滚筒位置处于同一水平线，逐帧识别的时间超过第二预设时间，且确定第二预设时间内最后一帧图像中采煤机的两个滚筒所在位置与前一帧图像中采煤机的两个滚筒所在位置相同时；如果当根据第二预设时间内最后一帧图像包含的刮板输送机上的煤块图像，以及第二预设时间内倒数第二帧图像包含的刮板输送机上的煤块图像，确定第二预设时间内最后一帧图像包含的刮板输送机上的煤块未发生移动时，确定采煤机处于正常停机状态。如果根据第二预设时间内最后一帧图像包含的刮板输送机上的煤块图像，以及第二预设时间内倒数第二帧图像包含的刮板输送机上的煤块图像，确定第二预设时间内最后一帧图像包含的刮板输送机上的煤块发生移动时，记录第四时刻，以逐帧的形式识别第四时刻之后的图像中所包含的刮板输送机上的煤没有移动，且逐帧识别的时间超出第三预设时间时，确定采煤机处于空转状态。The above parameter information also includes the coal block image on the scraper conveyor. When the two drums of the shearer are not on the same horizontal line in the first frame image, the two rollers of the shearer in the image after the second moment are identified frame by frame. The positions of the two drums are on the same horizontal line, the frame-by-frame identification time exceeds the second preset time, and it is determined that the positions of the two drums of the shearer in the last frame of the image within the second preset time are the same as those of the shearer in the previous frame. When the positions of the two rollers of the The coal block image on the machine is determined, and when it is determined that the coal block on the scraper conveyor included in the last frame image within the second preset time has not moved, it is determined that the shearer is in a normal shutdown state. If the image of coal lumps on the scraper conveyor included in the last frame of image within the second preset time and the image of coal lumps on the scraper conveyor included in the penultimate frame image within the second preset time, determine the second preset When the coal block on the scraper conveyor included in the last frame of the image in the set time moves, record the fourth moment, and recognize that the coal on the scraper conveyor included in the image after the fourth moment has not moved in the form of frame by frame. And when the frame-by-frame identification time exceeds the third preset time, it is determined that the shearer is in an idling state.
本实施例提供的上述机器运行状态检测方法，通过实时采集现场机器运作的声音并识别，以及实时采集机器现场的声音并识别，能够实时检测目标机器的运行状态，以便在机器出现故障时能够及时通知相关人员进行设备维护检修，降低了人力成本，同时避免了因生产现场工作人员经验不足造成无法及时发现机器异常的问题，提升了机器运行状态检测的准确性。The above-mentioned machine operating state detection method provided in this embodiment can detect the operating state of the target machine in real time by collecting and recognizing the sound of on-site machine operation in real time, and collecting and recognizing the sound of the machine site in real time, so as to be able to timely detect the operating state of the target machine when the machine fails. Relevant personnel are notified to carry out equipment maintenance and repair, which reduces labor costs, and at the same time avoids the problem that machine abnormalities cannot be detected in time due to the lack of experience of production site staff, and improves the accuracy of machine running status detection.
实施例二：Embodiment 2:
对于实施例二中所提供的机器运行状态检测方法，本发明实施例提供了应用上述机器运行状态检测方法对采煤机进行故障检测的示例，参见如图6所示的采煤机运行状态检测流程图，具体可参照如下步骤S502～步骤S510执行：For the machine running state detection method provided in the second embodiment, the embodiment of the present invention provides an example of applying the above-mentioned machine running state detection method to detect the fault of the shearer, see the operation state detection of the shearer as shown in FIG. 6 . The flow chart can be specifically executed with reference to the following steps S502 to S510:
步骤S502：获取采煤机采煤过程中的工作图像，基于该工作图像识别与采煤机运行状态相关的参数信息。Step S502: Acquire a working image of the shearer during the coal mining process, and identify parameter information related to the operating state of the shearer based on the working image.
步骤S504：根据与采煤机运行状态相关的参数信息确定采煤机的运行状态，得到第一结果。Step S504: Determine the operation state of the shearer according to the parameter information related to the operation state of the shearer, and obtain the first result.
步骤S506：获取采煤机采煤过程中的音频数据，根据声音识别算法识别采煤机的工作状态，得到第二结果。Step S506: Acquire audio data during the coal mining process of the shearer, identify the working state of the shearer according to a voice recognition algorithm, and obtain a second result.
步骤S508：根据预设的数据融合算法对上述第一结果和第二结果进行数据融合，得到采煤机的最终运行状态。Step S508: Perform data fusion on the first result and the second result according to a preset data fusion algorithm to obtain the final operating state of the shearer.
上述预设的数据融合算法可以是加权平均算法，为上述第一结果和第二结果赋予相应的权重，也可以是基于D-S证据理论算法对第一结果和第二结果进行融合计算，得到采煤机的最终运行状态。The above-mentioned preset data fusion algorithm may be a weighted average algorithm, assigning corresponding weights to the above-mentioned first and second results, or may be based on the D-S evidence theory algorithm to perform fusion calculation on the first and second results to obtain coal mining. the final operating state of the machine.
步骤S510：当采煤机的最终运行状态为异常运行状态时，发出告警信号，以提示工作人员进行故障维修。Step S510 : when the final operating state of the shearer is an abnormal operating state, an alarm signal is sent to prompt the staff to perform fault maintenance.
上述异常运行状态包括采煤机的跑空刀及割底板等异常运行状态。通过对采煤机进行安全预警监测，当发现采煤机的运行状态为异常运行状态时，发出告警信号，可以自动及时发现采煤机的运行故障，保证生产效率。The above-mentioned abnormal operation states include abnormal operation states such as the runaway knife and the cutting floor of the shearer. Through the safety early warning monitoring of the shearer, when the operating state of the shearer is found to be abnormal, an alarm signal will be issued, and the operation failure of the shearer can be automatically detected in time to ensure production efficiency.
实施例三：Embodiment three:
对于实施例二中所提供的机器运行状态检测方法，本发明实施例提供了一种机器运行状态检测装置，参见图7所示的一种机器运行状态检测装置结构示意图，该装置包括以下模块：For the machine operating state detection method provided in the second embodiment, an embodiment of the present invention provides a machine operating state detection device. Referring to the schematic structural diagram of a machine operating state detection device shown in FIG. 7 , the device includes the following modules:
状态确定模块61，用于获取目标机器的音频数据，基于音频数据确定目标机器的运行状态，得到第一结果。The state determination module 61 is configured to acquire audio data of the target machine, determine the running state of the target machine based on the audio data, and obtain the first result.
状态识别模块62，用于获取目标机器的运行图像，基于运行图像识别目标机器的运行状态，得到第二结果。The state recognition module 62 is configured to acquire the operating image of the target machine, recognize the operating state of the target machine based on the operating image, and obtain the second result.
数据融合模块63，用于对第一结果和第二结果进行数据融合，得到目标机器的状态检测结果。The data fusion module 63 is used for data fusion of the first result and the second result to obtain the state detection result of the target machine.
本实施例提供的上述机器运行状态检测装置，通过对基于目标机器运行的音频数据得到的第一结果，及基于目标机器的运行图像得到的第二结果进行数据融合，可以自动得到目标机器的状态检测结果，节省了人力成本，通过基于目标机器运行的声音和图像综合判断运行状态，提升了目标机器状态检测的准确率。The above-mentioned machine operating state detection device provided in this embodiment can automatically obtain the state of the target machine by performing data fusion on the first result obtained based on the audio data of the target machine operation and the second result obtained based on the operating image of the target machine. The detection result saves labor costs, and improves the accuracy of the state detection of the target machine by comprehensively judging the running state based on the sound and image of the target machine running.
在一种实施方式中，上述数据融合模块63，进一步用于基于D-S 证据理论算法对第一结果和第二结果进行融合计算，得到目标机器的状态检测结果。In an embodiment, the above-mentioned data fusion module 63 is further configured to perform fusion calculation on the first result and the second result based on the D-S evidence theory algorithm to obtain the state detection result of the target machine.
在一种实施方式中，上述数据融合模块63，进一步用于基于第一结果、第二结果及融合计算算式确定目标机器的状态检测结果。In an embodiment, the above-mentioned data fusion module 63 is further configured to determine the state detection result of the target machine based on the first result, the second result and the fusion calculation formula.
在一种实施方式中，上述目标机器包括采煤机，运行状态包括正常运行状态、停机状态和非正常运行状态；融合计算算式为：In one embodiment, the above-mentioned target machine includes a shearer, and the operating state includes a normal operating state, a shutdown state and an abnormal operating state; the fusion calculation formula is:
其中，m(A)为状态检测结果，m1(A1)为第一结果，m2(A2)为第二结果，第一结果和第二结果包括采煤机处于各运行状态下的概率值，A1为基于音频数据确定目标机器的运行状态的事件，A2为基于运行图像识别目标机器的运行状态的事件，k为冲突因子。Among them, m(A) is the state detection result, m1 (A1 ) is the first result, and m2 (A2 ) is the second result. The first result and the second result include the shearer in each operating state. Probability value, A1 is the event of determining the running stateof the target machine based on audio data, A2 is the event of identifying the running state of the target machine based on the running image, andk is the conflict factor.
在一种实施方式中，上述状态确定模块61，进一步用于采集目标机器运行过程中的音频数据，并基于音频数据确定目标机器运行产生的频谱图；将频谱图输入预先训练得到的第一神经网络模型中，得到目标机器的第一结果；其中，第一神经网络模型是基于标注有运行状态的频谱图样本训练得到的。In one embodiment, the above-mentioned state determination module 61 is further configured to collect audio data during the operation of the target machine, and determine the spectrogram generated by the operation of the target machine based on the audio data; input the spectrogram into the first neural network obtained by pre-training In the network model, the first result of the target machine is obtained; wherein, the first neural network model is obtained by training based on the spectrogram samples marked with the running state.
在一种实施方式中，上述将音频数据分割为预设时长的音频片段；对各音频片段进行去噪处理，并利用音频数据处理库将各音频片段转换为对应的频谱图。In one embodiment, the audio data is divided into audio segments of preset duration; denoising is performed on each audio segment, and an audio data processing library is used to convert each audio segment into a corresponding spectrogram.
在一种实施方式中，上述状态识别模块62，进一步用于将运行图像输入预先训练得到的第二神经网络模型中，得到目标机器的第二结果；其中，第二神经网络模型是基于标注有运行状态的运行图像样本训练得到的；或者，根据运行图像获取与目标机器运行状态相关的参数信息，基于参数信息确定目标机器的运行状态，得到第二结果。In one embodiment, the above-mentioned state identification module 62 is further configured to input the operating image into the second neural network model obtained by pre-training to obtain the second result of the target machine; wherein, the second neural network model is based on the Or, obtain parameter information related to the running state of the target machine according to the running image, determine the running state of the target machine based on the parameter information, and obtain the second result.
本实施例提供的上述机器运行状态检测装置，通过实时采集现场机器运作的声音并识别，以及实时采集机器现场的声音并识别，能够实时检测目标机器的运行状态，以便在机器出现故障时能够及时通知相关人员进行设备维护检修，降低了人力成本，同时避免了因生产现场工作人员经验不足造成无法及时发现机器异常的问题，提升了机器运行状态检测的准确性。The above-mentioned machine operating state detection device provided in this embodiment can detect the operating state of the target machine in real time by collecting and recognizing the sound of on-site machine operation in real time, and collecting and recognizing the sound of the machine site in real time, so as to be able to timely detect the operating state of the target machine when the machine fails Relevant personnel are notified to carry out equipment maintenance and repair, which reduces labor costs, and at the same time avoids the problem that machine abnormalities cannot be detected in time due to the lack of experience of production site staff, and improves the accuracy of machine running status detection.
本实施例所提供的装置，其实现原理及产生的技术效果和前述实施例相同，为简要描述，装置实施例部分未提及之处，可参考前述方法实施例中相应内容。The implementation principle and the technical effects of the device provided in this embodiment are the same as those in the foregoing embodiments. For brief description, for the parts not mentioned in the device embodiment, reference may be made to the corresponding content in the foregoing method embodiments.
实施例四：Embodiment 4:
本发明实施例提供了一种电子设备，如图8所示的电子设备结构示意图，电子设备包括声音采集装置(图中未示出)、图像采集装置 (图中未示出)、处理器71、存储器72，图像采集装置用于获取目标机器运行过程中的音频数据，声音采集装置用于获取目标机器的运行图像，所述存储器中存储有可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述实施例提供的方法的步骤。An embodiment of the present invention provides an electronic device, as shown in FIG. 8 , a schematic structural diagram of the electronic device. The electronic device includes a sound acquisition device (not shown in the figure), an image acquisition device (not shown in the figure), and a
参见图8，电子设备还包括：总线74和通信接口73，处理器71、通信接口73和存储器72通过总线74连接。处理器71用于执行存储器72中存储的可执行模块，例如计算机程序。Referring to FIG. 8 , the electronic device further includes: a bus 74 and a
其中，存储器72可能包含高速随机存取存储器(RAM，Random Access Memory)，也可能还包括非易失性存储器(non-volatile memory)，例如至少一个磁盘存储器。通过至少一个通信接口73(可以是有线或者无线)实现该系统网元与至少一个其他网元之间的通信连接，可以使用互联网，广域网，本地网，城域网等。The
总线74可以是ISA(Industry Standard Architecture，工业标准体系结构)总线、PCI(Peripheral Component Interconnect，外设部件互连标准)总线或EISA(ExtendedIndustry Standard Architecture，扩展工业标准结构)总线等。所述总线可以分为地址总线、数据总线、控制总线等。为便于表示，图8中仅用一个双向箭头表示，但并不表示仅有一根总线或一种类型的总线。The bus 74 may be an ISA (Industry Standard Architecture, industry standard architecture) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus, or an EISA (Extended Industry Standard Architecture, extended industry standard architecture) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one bidirectional arrow is used in FIG. 8, but it does not mean that there is only one bus or one type of bus.
其中，存储器72用于存储程序，所述处理器71在接收到执行指令后，执行所述程序，前述本发明实施例任一实施例揭示的流过程定义的装置所执行的方法可以应用于处理器71中，或者由处理器71实现。The
处理器71可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器71中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器71可以是通用处理器，包括中央处理器(Central Processing Unit，简称CPU)、网络处理器(Network Processor，简称NP)等。还可以是数字信号处理器(Digital SignalProcessing，简称DSP)、专用集成电路(Application Specific Integrated Circuit，简称ASIC)、现成可编程门阵列 (Field-Programmable Gate Array，简称FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本发明实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本发明实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器72，处理器71读取存储器72中的信息，结合其硬件完成上述方法的步骤。The
实施例五：Embodiment 5:
本发明实施例提供了一种计算机可读介质，其中，所述计算机可读介质存储有计算机可执行指令，所述计算机可执行指令在被处理器调用和执行时，所述计算机可执行指令促使所述处理器实现上述实施例所述的方法。An embodiment of the present invention provides a computer-readable medium, wherein the computer-readable medium stores computer-executable instructions, and when the computer-executable instructions are invoked and executed by a processor, the computer-executable instructions cause the The processor implements the methods described in the above embodiments.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统具体工作过程，可以参考前述实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the system described above, reference may be made to the corresponding process in the foregoing embodiments, and details are not repeated here.
本发明实施例所提供的机器运行状态检测方法、装置及电子设备的计算机程序产品，包括存储了程序代码的计算机可读存储介质，所述程序代码包括的指令可用于执行前面方法实施例中所述的方法，具体实现可参见方法实施例，在此不再赘述。The computer program product of the method, apparatus, and electronic device for detecting a machine running state provided by the embodiments of the present invention includes a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the methods described in the foregoing method embodiments. For the specific implementation, reference may be made to the method embodiments, which will not be repeated here.
另外，在本发明实施例的描述中，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通。对于本领域的普通技术人员而言，可以具体情况理解上述术语在本发明中的具体含义。In addition, in the description of the embodiments of the present invention, unless otherwise expressly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection , or integrally connected; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or the internal communication between the two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood in specific situations.
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备 (可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器 (RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .
在本发明的描述中，需要说明的是，术语“中心”、“上”、“下”、“左”、“右”、“竖直”、“水平”、“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本发明和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本发明的限制。此外，术语“第一”、“第二”、“第三”仅用于描述目的，而不能理解为指示或暗示相对重要性。In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. The indicated orientation or positional relationship is based on the orientation or positional relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the indicated device or element must have a specific orientation or a specific orientation. construction and operation, and therefore should not be construed as limiting the invention. Furthermore, the terms "first", "second", and "third" are used for descriptive purposes only and should not be construed to indicate or imply relative importance.
最后应说明的是：以上所述实施例，仅为本发明的具体实施方式，用以说明本发明的技术方案，而非对其限制，本发明的保护范围并不局限于此，尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本发明实施例技术方案的精神和范围，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以所述权利要求的保护范围为准。Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present invention, and are used to illustrate the technical solutions of the present invention, but not to limit them. The protection scope of the present invention is not limited thereto, although referring to the foregoing The embodiment has been described in detail the present invention, those of ordinary skill in the art should understand: any person skilled in the art who is familiar with the technical field within the technical scope disclosed by the present invention can still modify the technical solutions described in the foregoing embodiments. Or can easily think of changes, or equivalently replace some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be covered in the present invention. within the scope of protection. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010809719.5ACN112733588A (en) | 2020-08-13 | 2020-08-13 | Machine running state detection method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010809719.5ACN112733588A (en) | 2020-08-13 | 2020-08-13 | Machine running state detection method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112733588Atrue CN112733588A (en) | 2021-04-30 |
Family
ID=75597144
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010809719.5APendingCN112733588A (en) | 2020-08-13 | 2020-08-13 | Machine running state detection method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112733588A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113376457A (en)* | 2021-05-10 | 2021-09-10 | 首钢京唐钢铁联合有限责任公司 | Equipment operation state detection method, system, device, equipment and medium |
| CN113641544A (en)* | 2021-08-23 | 2021-11-12 | 北京百度网讯科技有限公司 | Method, apparatus, device, medium and product for detecting application status |
| CN113743355A (en)* | 2021-09-15 | 2021-12-03 | 中国南方电网有限责任公司超高压输电公司大理局 | Method, device and system for checking state of switch device and computer equipment |
| CN113887446A (en)* | 2021-10-08 | 2022-01-04 | 黑龙江雨谷科技有限公司 | Audio and video combined pedestrian accidental falling monitoring method based on neural network |
| CN114964467A (en)* | 2022-03-21 | 2022-08-30 | 珠海格力电器股份有限公司 | Equipment detection method and device, sound collection equipment and storage medium |
| CN115264749A (en)* | 2022-06-29 | 2022-11-01 | 青岛海尔空调器有限总公司 | Air conditioner fault detection method and device, air conditioner and electronic equipment |
| CN116030086A (en)* | 2023-01-04 | 2023-04-28 | 中山大学 | A target detection method and device based on information fusion |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102528225A (en)* | 2012-02-09 | 2012-07-04 | 上海市特种设备监督检验技术研究院 | Sound signal transduction and prediction method of GTAW (gas tungsten arc welding) welding fusion penetration state |
| CN106826846A (en)* | 2017-01-06 | 2017-06-13 | 南京赫曼机器人自动化有限公司 | The intellect service robot and method driven based on abnormal sound and image event |
| CN107272607A (en)* | 2017-05-11 | 2017-10-20 | 上海斐讯数据通信技术有限公司 | A kind of intelligent home control system and method |
| CN108830457A (en)* | 2018-05-14 | 2018-11-16 | 国网辽宁省电力有限公司 | Operation of power networks monitoring and control panorama management system based on artificial intelligence |
| CN109669087A (en)* | 2019-01-31 | 2019-04-23 | 国网河南省电力公司 | A kind of method for diagnosing fault of power transformer based on Multi-source Information Fusion |
| CN109903392A (en)* | 2017-12-11 | 2019-06-18 | 北京京东尚科信息技术有限公司 | Augmented reality method and device |
| CN110206748A (en)* | 2019-06-12 | 2019-09-06 | 精英数智科技股份有限公司 | A kind of methods, devices and systems of monitoring lock between air and electricity experiment |
| CN110400448A (en)* | 2019-07-29 | 2019-11-01 | 精英数智科技股份有限公司 | A kind of coalcutter operating status judgment method, system and storage medium |
| CN110556129A (en)* | 2019-09-09 | 2019-12-10 | 北京大学深圳研究生院 | Bimodal emotion recognition model training method and bimodal emotion recognition method |
| CN110659676A (en)* | 2019-09-09 | 2020-01-07 | 卓尔智联(武汉)研究院有限公司 | Information processing method, device and storage medium |
| CN110867196A (en)* | 2019-12-03 | 2020-03-06 | 桂林理工大学 | A machine and equipment condition monitoring system based on deep learning and voice recognition |
| CN111199257A (en)* | 2020-01-10 | 2020-05-26 | 中国铁道科学研究院集团有限公司电子计算技术研究所 | Fault diagnosis method and device for high-speed railway running equipment |
| CN111362089A (en)* | 2020-04-30 | 2020-07-03 | 郑州大学 | A method and system for artificial intelligence to identify the entry of an electric vehicle into a car |
- 2020
- 2020-08-13CNCN202010809719.5Apatent/CN112733588A/enactivePending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102528225A (en)* | 2012-02-09 | 2012-07-04 | 上海市特种设备监督检验技术研究院 | Sound signal transduction and prediction method of GTAW (gas tungsten arc welding) welding fusion penetration state |
| CN106826846A (en)* | 2017-01-06 | 2017-06-13 | 南京赫曼机器人自动化有限公司 | The intellect service robot and method driven based on abnormal sound and image event |
| CN107272607A (en)* | 2017-05-11 | 2017-10-20 | 上海斐讯数据通信技术有限公司 | A kind of intelligent home control system and method |
| CN109903392A (en)* | 2017-12-11 | 2019-06-18 | 北京京东尚科信息技术有限公司 | Augmented reality method and device |
| CN108830457A (en)* | 2018-05-14 | 2018-11-16 | 国网辽宁省电力有限公司 | Operation of power networks monitoring and control panorama management system based on artificial intelligence |
| CN109669087A (en)* | 2019-01-31 | 2019-04-23 | 国网河南省电力公司 | A kind of method for diagnosing fault of power transformer based on Multi-source Information Fusion |
| CN110206748A (en)* | 2019-06-12 | 2019-09-06 | 精英数智科技股份有限公司 | A kind of methods, devices and systems of monitoring lock between air and electricity experiment |
| CN110400448A (en)* | 2019-07-29 | 2019-11-01 | 精英数智科技股份有限公司 | A kind of coalcutter operating status judgment method, system and storage medium |
| CN110556129A (en)* | 2019-09-09 | 2019-12-10 | 北京大学深圳研究生院 | Bimodal emotion recognition model training method and bimodal emotion recognition method |
| CN110659676A (en)* | 2019-09-09 | 2020-01-07 | 卓尔智联(武汉)研究院有限公司 | Information processing method, device and storage medium |
| CN110867196A (en)* | 2019-12-03 | 2020-03-06 | 桂林理工大学 | A machine and equipment condition monitoring system based on deep learning and voice recognition |
| CN111199257A (en)* | 2020-01-10 | 2020-05-26 | 中国铁道科学研究院集团有限公司电子计算技术研究所 | Fault diagnosis method and device for high-speed railway running equipment |
| CN111362089A (en)* | 2020-04-30 | 2020-07-03 | 郑州大学 | A method and system for artificial intelligence to identify the entry of an electric vehicle into a car |
Non-Patent Citations (1)
| Title |
|---|
| 董国芳: "《面向数字矿山的铅锌矿井下风险动态模糊评估研究》", 31 May 2020, 天津：天津大学出版社, pages: 69 - 71* |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113376457A (en)* | 2021-05-10 | 2021-09-10 | 首钢京唐钢铁联合有限责任公司 | Equipment operation state detection method, system, device, equipment and medium |
| CN113641544A (en)* | 2021-08-23 | 2021-11-12 | 北京百度网讯科技有限公司 | Method, apparatus, device, medium and product for detecting application status |
| CN113641544B (en)* | 2021-08-23 | 2024-04-26 | 北京百度网讯科技有限公司 | Method, apparatus, device, medium and product for detecting application state |
| CN113743355A (en)* | 2021-09-15 | 2021-12-03 | 中国南方电网有限责任公司超高压输电公司大理局 | Method, device and system for checking state of switch device and computer equipment |
| CN113743355B (en)* | 2021-09-15 | 2024-01-09 | 中国南方电网有限责任公司超高压输电公司大理局 | Switch device state checking method, device, system and computer equipment |
| CN113887446A (en)* | 2021-10-08 | 2022-01-04 | 黑龙江雨谷科技有限公司 | Audio and video combined pedestrian accidental falling monitoring method based on neural network |
| CN114964467A (en)* | 2022-03-21 | 2022-08-30 | 珠海格力电器股份有限公司 | Equipment detection method and device, sound collection equipment and storage medium |
| CN114964467B (en)* | 2022-03-21 | 2023-01-13 | 珠海格力电器股份有限公司 | Equipment detection method and device, sound collection equipment and storage medium |
| CN115264749A (en)* | 2022-06-29 | 2022-11-01 | 青岛海尔空调器有限总公司 | Air conditioner fault detection method and device, air conditioner and electronic equipment |
| CN116030086A (en)* | 2023-01-04 | 2023-04-28 | 中山大学 | A target detection method and device based on information fusion |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112733588A (en) | Machine running state detection method and device and electronic equipment | |
| CN114463594B (en) | Integrated identification method and equipment for power generation equipment anomalies based on multimodal deep learning | |
| CN112712113B (en) | An indicator-based alarm method, device and computer system | |
| CN110221145B (en) | Power equipment fault diagnosis method and device and terminal equipment | |
| CN111766253A (en) | Solder paste printing quality detection method, data processing device and computer storage medium | |
| CN110609780B (en) | Data monitoring method and device, electronic equipment and storage medium | |
| CN119046848B (en) | Data processing method of digital intelligent terminal, electronic equipment and digital intelligent terminal | |
| CN111914656A (en) | Personnel behavior detection method, device, electronic device and storage medium | |
| CN114034972B (en) | Intelligent cable fault determination method and device based on image data | |
| CN116896166A (en) | Digital twin panoramic management and control system and method for urban cable | |
| CN112751910A (en) | Information collection method and device | |
| CN118427769A (en) | Belt conveyor fault diagnosis method, system, terminal and storage medium | |
| CN118245838A (en) | Hanging basket walking monitoring control early warning system based on digital twin technology | |
| CN119851214A (en) | Image data management method and system based on engineering supervision platform | |
| CN114742992A (en) | Video abnormity detection method and device and electronic equipment | |
| CN113535697B (en) | Climbing frame data cleaning method, climbing frame control device and storage medium | |
| CN113850773A (en) | Detection method, device, equipment and computer readable storage medium | |
| CN118799971A (en) | Inspection report generation method, device, equipment and storage medium | |
| CN113869116A (en) | Construction machinery early warning method, device and medium for overhead transmission line | |
| CN112364918A (en) | Abnormality recognition method, terminal, and computer-readable storage medium | |
| CN117129815B (en) | Comprehensive detection method and system for multi-degradation insulator based on Internet of things | |
| CN118053125A (en) | Project progress visualization image supervision method, device, equipment and medium | |
| CN118038353A (en) | Cable monitoring abnormality identification system and method based on image | |
| CN108880881A (en) | The method and apparatus of monitoring resource under a kind of cloud environment | |
| CN116216239A (en) | Material spreading detection method and device for belt conveyor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20210430 | |
| RJ01 | Rejection of invention patent application after publication |