CN113514544A - Mobile robot pavement material identification method based on sound characteristics - Google Patents
Mobile robot pavement material identification method based on sound characteristicsDownload PDFInfo
- Publication number
- CN113514544A CN113514544ACN202011597385.6ACN202011597385ACN113514544ACN 113514544 ACN113514544 ACN 113514544ACN 202011597385 ACN202011597385 ACN 202011597385ACN 113514544 ACN113514544 ACN 113514544A
- Authority
- CN
- China
- Prior art keywords
- sound data
- road surface
- surface material
- pavement
- mel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000000463materialSubstances0.000titleclaimsabstractdescription156
- 238000000034methodMethods0.000titleclaimsabstractdescription29
- 238000007781pre-processingMethods0.000claimsabstractdescription10
- 238000000605extractionMethods0.000claimsabstractdescription9
- 238000003062neural network modelMethods0.000claimsabstractdescription6
- 238000013527convolutional neural networkMethods0.000claimsabstractdescription5
- 230000004913activationEffects0.000claimsdescription36
- 230000006870functionEffects0.000claimsdescription24
- 238000001914filtrationMethods0.000claimsdescription16
- 238000010606normalizationMethods0.000claimsdescription12
- 238000011176poolingMethods0.000claimsdescription12
- 238000001228spectrumMethods0.000claimsdescription12
- 238000009432framingMethods0.000claimsdescription11
- 210000004205output neuronAnatomy0.000claimsdescription9
- 238000005070samplingMethods0.000claimsdescription9
- 239000013598vectorSubstances0.000claimsdescription9
- 238000001514detection methodMethods0.000claimsdescription7
- 230000009467reductionEffects0.000claimsdescription7
- 238000013528artificial neural networkMethods0.000claimsdescription6
- 230000009466transformationEffects0.000claimsdescription6
- 239000010426asphaltSubstances0.000claimsdescription5
- 239000004568cementSubstances0.000claimsdescription5
- 239000004575stoneSubstances0.000claimsdescription5
- 239000002023woodSubstances0.000claimsdescription5
- 239000000919ceramicSubstances0.000claimsdescription3
- 230000008859changeEffects0.000claimsdescription3
- 230000002708enhancing effectEffects0.000claimsdescription3
- 230000037433frameshiftEffects0.000claimsdescription3
- 230000004927fusionEffects0.000claimsdescription3
- 239000011159matrix materialSubstances0.000claimsdescription3
- 230000004044responseEffects0.000claimsdescription3
- 230000011218segmentationEffects0.000claimsdescription3
- 230000003595spectral effectEffects0.000claimsdescription3
- 230000000007visual effectEffects0.000abstractdescription4
- 230000007812deficiencyEffects0.000abstractdescription3
- 244000025254Cannabis sativaSpecies0.000description2
- 230000009286beneficial effectEffects0.000description2
- 238000010586diagramMethods0.000description2
- 239000000203mixtureSubstances0.000description2
- 230000001133accelerationEffects0.000description1
- 238000004458analytical methodMethods0.000description1
- 239000003086colorantSubstances0.000description1
- 230000007547defectEffects0.000description1
- 239000012634fragmentSubstances0.000description1
- 230000003287optical effectEffects0.000description1
- 230000008569processEffects0.000description1
Images
Classifications
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N29/00—Investigating or analysing materials by the use of ultrasonic, sonic or infrasonic waves; Visualisation of the interior of objects by transmitting ultrasonic or sonic waves through the object
- G01N29/04—Analysing solids
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N29/00—Investigating or analysing materials by the use of ultrasonic, sonic or infrasonic waves; Visualisation of the interior of objects by transmitting ultrasonic or sonic waves through the object
- G01N29/44—Processing the detected response signal, e.g. electronic circuits specially adapted therefor
- G01N29/4481—Neural networks
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
- G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Acoustics & Sound (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Investigating Or Analyzing Materials By The Use Of Ultrasonic Waves (AREA)
Abstract
Translated fromChinese
Description
Technical Field
The invention relates to a mobile robot pavement material identification method based on sound characteristics, and belongs to the technical field of mobile robot road condition identification and autonomous navigation.
Background
Currently, intelligent products are beginning to appear in every corner of people's life, and can replace people to complete various works, and one of the most widespread applications is intelligent mobile robots. So far, the intelligent mobile robot technology has become mature and perfect, and the functions and application range are continuously expanded. Mobile robots have been successfully used in many fields, such as education, scientific research, medical care, agriculture, industry, logistics, etc.
The autonomous navigation is one of basic functions required by the intelligent mobile robot, and the current road material is accurately known in the outdoor autonomous mobile driving process, which is very important for realizing the autonomous navigation function. The related personnel at home and abroad carry out a great deal of research on the road surface material identification direction, and the real-time, comprehensive, accurate, rapid and reliable road surface material identification technology is the key and difficult point of the current research. At present, the research on the quality classification of pavement materials is mainly divided into three aspects based on laser radar, cameras and other types of sensors (acceleration sensors, optical sensors and the like). Among them, the method based on the laser radar and the camera is to recognize the road surface environment by simulating human vision. However, when the mobile robot encounters different material pavements with similar colors and textures or special conditions such as low-visibility haze weather, correct judgment is difficult to make in a visual mode. Most of the other sensor-based methods focus on the identification of road parameters, including the identification of adhesion coefficients, the identification of road unevenness, the identification of soft and hard roads, and the like, and an effective road material identification method is lacked.
In order to accurately and effectively identify the road surface material, the invention explores a novel road surface material identification method of the mobile robot based on the sound characteristics. The classification and recognition of the road surface material are realized by analyzing and judging the sound information of the road surface material in consideration of analysis from the perspective of hearing. The method can effectively make up for the deficiency of the visual mode.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention aims to provide a mobile robot pavement material identification method based on sound characteristics so as to improve the autonomous navigation capability of the mobile robot. The method comprises the steps of classifying and dividing common road surfaces, collecting sound data of the road surface material, establishing an effective road surface material sound data set, and preprocessing original road surface material sound data, wherein the preprocessing comprises operations of filtering and noise reduction, framing and windowing, end point detection and the like, so that effective fragments of the road surface material sound are obtained. And extracting frequency domain characteristics of sound data of different road surface materials, building a neural network, and performing characteristic learning by taking the sound characteristics of the road surface materials as input signals of the network, thereby realizing classification and identification of the different road surface materials. The invention enriches the technology and theory of pavement material identification, becomes a very beneficial attempt, and has important theoretical significance and application value.
In order to achieve the purpose of the invention and solve the problems existing in the prior art, the invention adopts the technical scheme that: a mobile robot pavement material identification method based on sound characteristics comprises the following steps:
step 2, preprocessing sound data of the road surface material: through segmentation, filtering noise reduction, framing windowing and endpoint detection preprocessing operations, the quality of the sound data of the pavement material is improved, and noise influence is removed, and the method specifically comprises the following substeps:
(a) and (3) dividing: the recording of the road surface material sound is carried out according to a 50-time continuous knocking method, the original data needs to be further divided, the sound of knocking the road surface material each time is used as a single road surface material sound data, i-th class road surface material sound data Si＝{sijJ is more than or equal to 1 and less than or equal to 50, wherein sijJ is the knocking times corresponding to the road surface material sound data;
(b) filtering and denoising: filtering the collected road surface material sound data through a high-pass filter, describing through a formula (1),
where α is the filter coefficient, sij(t) is the original data at the current moment,
for the road surface material sound data at the time t after the filtering and noise reduction processing, the frequency spectrum can be balanced through a filter, the signal-to-noise ratio (SNR) is improved, the proportion of low-frequency information can be suppressed, and the proportion of high-frequency useful components is relatively increased;
(c) framing and windowing: on the whole, the overall road surface material sound characteristic of the knocking road surface has time-varying characteristic and is not stable road surface material sound data, but the recorded road surface material sound data is considered to be stable in a short time, in order to meet the Fourier transformation condition, the road surface material sound data needs to be divided into short-time stable road surface material sound data, the frame division and interception time period ranges from 10 ms to 30ms, in order to ensure that the characteristic parameter continuously and stably changes, some frames are uniformly reserved between two frames without overlapping parts, the change of each overlapping part is called frame shift, the value range is 1/4-1/2 of the frame length, and the road surface material sound data f after frame division and framing is between 1/4-1/2ijWindowing to reduce sub-frame artifactsInfluence, windowed road texture Sound data cij＝fijXxw, wherein the window function w adopts a Hamming window;
(d) and (3) end point detection: determining the initial position and the end position of the effective road material sound data by setting different volume thresholds, and firstly calculating the windowed road material sound data cijVolume maximum value vmax inijAnd minimum value vminijAnd subtracting to obtain the transformation range value vdif of the road surface material sound dataijRespectively taking vmin from the lowest pavement material sound data as a starting pointij+vdifij×0.1、vminij+vdifij×0.01、vminij+vdifijTaking three different threshold values of x 0.05 as the end point position judgment of the road surface material sound data, starting to record the starting point when the threshold value is exceeded, judging whether the sound data at the next moment is larger than the threshold value or not until the end point lower than the threshold value is detected and recorded, and taking the road surface material sound data between the final starting point position and the final position lower than the threshold value as the effective road surface material sound data e for knocking the road surfaceij；
Step 3, Mel cepstrum feature extraction of the road surface material sound: the Mel cepstrum feature is a most common feature extraction method in sound data, and has the advantages of strong anti-interference capability and capability of reflecting feature differences among different sound data, so that the sound data of the road surface material can be effectively distinguished, and the method specifically comprises the following substeps:
(a) road surface material sound data e obtained by detecting end pointsijCarrying out one-dimensional Fourier transform to obtain corresponding frequency domain signals
Wherein N represents the number of sampling points in each frame of pavement material sound data, k is the sampling point, and the power spectrum gij＝|pij(k)|2；
(b) The power spectrum g of each frame of road surface material sound dataijPutting the obtained mixture into a Mel triangular filter bank for filtering, and calculating logarithmic energy qijAnd is described by the following formula,
wherein h isijRepresenting the frequency response of the mel filters, M being the number of mel filters;
(c) to q isijDiscrete cosine transform is carried out to obtain the final Mel cepstrum characteristic coefficient, which is described by the following formula,
wherein melijRepresenting a Mel cepstrum characteristic coefficient, v representing the order of the Mel cepstrum coefficient, and N representing the number of sampling points in each frame of pavement material sound data;
(d) for the obtained mel-frequency cepstrum characteristic coefficient melijCarrying out artificial marking and constructing a training set;
step 4, constructing a deep convolution neural network for training: constructing a deep convolutional neural network, and based on Mel cepstrum characteristic coefficients mel of sound data of different pavement materialsijClassifying to realize the identification of the pavement material, and mainly comprising the following substeps:
(a) enhancing the characteristics, adding some auxiliary characteristics, such as chrominance frequency characteristics, Mel frequency spectrum characteristics, spectral contrast characteristics and hue centroid characteristics in audio, forming 193-dimensional characteristic vectors through characteristic splicing and fusion, finally forming 196-dimensional characteristic vectors through zero padding and alignment, further converting the characteristics into a two-dimensional matrix form (14 × 14), and performing dimension expansion on an input end in the network to obtain a 3D tensor (14 × 14 × 1);
(b) the encoding end comprises 1 convolutional layer Conv1, parameters areconvolutional kernel size 1 multiplied by 1,step length 1, same filling and output channel number 32, Batch Normalization (BN) processing is carried out, and a Leaky _ ReLU activation function is adopted for activation; 3 lightweight depth separable convolutional layers DConv2, DConv3, DConv4 connected in series; the parameters of DConv2 are convolution kernel size 3 × 3,step size 1, same filling and output channel number 32, batch normalization processing is performed, and a Leaky _ ReLU activation function is used for activation; parameters of the pooling layer Maxpool1 are 2 × 2 maximum pooling, step size 2, no padding; the parameters of DConv3 are convolution kernel size 3 × 3,step size 1, same filling and output channel number 64, batch normalization processing is carried out, and a Leaky _ ReLU activation function is adopted for activation; the parameters of DConv4 are convolution kernel size 3 × 3,step size 1, same filling and output channel number 64, batch normalization processing is carried out, and a Leaky _ ReLU activation function is adopted for activation; parameters of the pooling layer Maxpool2 are 3 × 3 maximal pooling, step length 2 andfilling value 1, and then the parameters are flattened into 1024-dimensional feature vectors;
(c) the decoding end first performs Dropout operation to prevent overfitting, the ratio is set to 0.5, and then 3 full-connection layers Dense1, Dense2 and Dense3 are included; wherein the number of output neurons of the Dense1 is 512, and a Leaky _ ReLU activation function is adopted for activation; the number of output neurons of Dense2 is 128, a Leaky _ ReLU activation function is adopted for activation, and the number of output neurons of Dense3 is 10;
and 5, identifying the pavement material based on the trained neural network model.
The invention has the beneficial effects that: a mobile robot pavement material identification method based on sound characteristics comprises the following steps: (1) the method comprises the steps of (1) collecting and establishing a road surface material sound data set, (2) preprocessing road surface material sound data, (3) extracting Mel cepstrum characteristics of road surface material sounds, (4) constructing a deep convolution neural network for training, and (5) identifying the road surface material based on a trained neural network model. Compared with the prior art, the invention has the following advantages: firstly, the method realizes the classification and identification of the pavement material by using the sound characteristics from the perspective of hearing, and can effectively make up for the deficiency of visual mode; secondly, the method realizes effective extraction of the road surface material sound characteristics based on the Mel cepstrum characteristics, builds a deep convolution neural network model, and can effectively perform multi-material road surface identification based on the material sound characteristics.
Drawings
FIG. 1 is a flow chart of the method steps of the present invention.
FIG. 2 is a diagram of road surface texture images and corresponding texture audio data.
In the figure: (a) the sound data map is a sound data map of a grass road surface, (b) the sound data map is a sound data map of a grass road surface, (c) the sound data map is a sound data map of a tile road surface, (d) the sound data map is a sound data map of a tile road surface, (e) the sound data map is an asphalt road surface, (f) the sound data map is an asphalt road surface, (g) the sound data map is a sound data map of a natural stone road surface, (h) the sound data map is a sound data map of a natural stone road surface, (i) the sound data map is a sound data map of a cement road surface, (j) the sound data map is a sound data map of a cement road surface, (k) the sound data map of a rubber road surface, (l) the sound data map of a rubber road surface, (m) the sound data map of a road tile, (n) the sound data map of a road tile, (o) the sound data map of a wood road surface, (p) the sound data map of a wood road surface, (q) the sound data map of a cobblestone road surface, (r) the cobblestone pavement, (s road surface,(s) the sound data map of a gravel road surface, and (t) the sound data map of a gravel road surface.
FIG. 3 is a diagram of road surface material classification training accuracy based on a neural network.
Detailed Description
The invention will be further explained with reference to the drawings.
As shown in fig. 1, a method for identifying a road surface material of a mobile robot based on sound characteristics includes the following steps:
Step 2, preprocessing sound data of the road surface material: through segmentation, filtering noise reduction, framing windowing and endpoint detection preprocessing operations, the quality of the sound data of the pavement material is improved, and noise influence is removed, and the method specifically comprises the following substeps:
(a) and (3) dividing: the recording of the road surface material sound is carried out according to a 50-time continuous knocking method, the original data needs to be further divided, the sound of knocking the road surface material each time is used as a single road surface material sound data, i-th class road surface material sound data Si＝{sijJ is more than or equal to 1 and less than or equal to 50, wherein sijJ is the knocking times corresponding to the road surface material sound data;
(b) filtering and denoising: filtering the collected road surface material sound data through a high-pass filter, describing through a formula (1),
where α is the filter coefficient, sij(t) is the original data at the current moment,
for the road surface material sound data at the time t after the filtering and noise reduction processing, the frequency spectrum can be balanced through a filter, the signal-to-noise ratio (SNR) is improved, the proportion of low-frequency information can be suppressed, and the proportion of high-frequency useful components is relatively increased;
(c) framing and windowing: on the whole, the overall road surface material sound characteristic of the knocking road surface has time-varying characteristic and is not stable road surface material sound data, but the recorded road surface material sound data is considered to be stable in a short time, in order to meet the Fourier transformation condition, the road surface material sound data needs to be divided into short-time stable road surface material sound data, the frame division and interception time period ranges from 10 ms to 30ms, in order to ensure that the characteristic parameter continuously and stably changes, some frames are uniformly reserved between two frames without overlapping parts, the change of each overlapping part is called frame shift, the value range is 1/4-1/2 of the frame length, and the road surface material sound data f after frame division and framing is between 1/4-1/2ijWindowing to reduce the influence of framing, windowed road surface material sound data cij＝fijXxw, wherein the window function w adopts a Hamming window;
(d) and (3) end point detection: determining the initial position and the end position of the effective road material sound data by setting different volume thresholds, and firstly calculating the windowed road material sound data cijVolume maximum value vmax inijAnd minimum value vminijAnd subtracting to obtain the transformation range value vdif of the road surface material sound dataijRespectively taking vmin from the lowest pavement material sound data as a starting pointij+vdifij×0.1、vminij+vdifij×0.01、vminij+vdifijTaking three different threshold values of x 0.05 as the end point position judgment of the road surface material sound data, starting to record the starting point when the threshold value is exceeded, judging whether the sound data at the next moment is larger than the threshold value or not until the end point lower than the threshold value is detected and recorded, and taking the road surface material sound data between the final starting point position and the final position lower than the threshold value as the effective road surface material sound data e for knocking the road surfaceij；
Step 3, Mel cepstrum feature extraction of the road surface material sound: the Mel cepstrum feature is a most common feature extraction method in sound data, and has the advantages of strong anti-interference capability and capability of reflecting feature differences among different sound data, so that the sound data of the road surface material can be effectively distinguished, and the method specifically comprises the following substeps:
(a) road surface material sound data e obtained by detecting end pointsijCarrying out one-dimensional Fourier transform to obtain corresponding frequency domain signals
Wherein N represents the number of sampling points in each frame of pavement material sound data, k is the sampling point, and the power spectrum gij＝|pij(k)|2；
(b) The power spectrum g of each frame of road surface material sound dataijPutting the obtained mixture into a Mel triangular filter bank for filtering, and calculating logarithmic energy qijAnd is described by the following formula,
wherein h isijRepresenting the frequency response of the mel filters, M being the number of mel filters;
(c) to q isijDiscrete cosine transform is carried out to obtain the final Mel cepstrum characteristic coefficient, which is described by the following formula,
wherein melijRepresenting a Mel cepstrum characteristic coefficient, v representing the order of the Mel cepstrum coefficient, and N representing the number of sampling points in each frame of pavement material sound data;
(d) for the obtained mel-frequency cepstrum characteristic coefficient melijCarrying out artificial marking and constructing a training set;
step 4, constructing a deep convolution neural network for training: constructing a deep convolutional neural network, and based on Mel cepstrum characteristic coefficients mel of sound data of different pavement materialsijClassifying to realize the identification of the pavement material, and mainly comprising the following substeps:
(a) enhancing the characteristics, adding some auxiliary characteristics, such as chrominance frequency characteristics, Mel frequency spectrum characteristics, spectral contrast characteristics and hue centroid characteristics in audio, forming 193-dimensional characteristic vectors through characteristic splicing and fusion, finally forming 196-dimensional characteristic vectors through zero padding and alignment, further converting the characteristics into a two-dimensional matrix form (14 × 14), and performing dimension expansion on an input end in the network to obtain a 3D tensor (14 × 14 × 1);
(b) the encoding end comprises 1 convolutional layer Conv1, parameters areconvolutional kernel size 1 multiplied by 1,step length 1, same filling and output channel number 32, Batch Normalization (BN) processing is carried out, and a Leaky _ ReLU activation function is adopted for activation; 3 lightweight depth separable convolutional layers DConv2, DConv3, DConv4 connected in series; the parameters of DConv2 are convolution kernel size 3 × 3,step size 1, same filling and output channel number 32, batch normalization processing is performed, and a Leaky _ ReLU activation function is used for activation; parameters of the pooling layer Maxpool1 are 2 × 2 maximum pooling, step size 2, no padding; the parameters of DConv3 are convolution kernel size 3 × 3,step size 1, same filling and output channel number 64, batch normalization processing is carried out, and a Leaky _ ReLU activation function is adopted for activation; the parameters of DConv4 are convolution kernel size 3 × 3,step size 1, same filling and output channel number 64, batch normalization processing is carried out, and a Leaky _ ReLU activation function is adopted for activation; parameters of the pooling layer Maxpool2 are 3 × 3 maximal pooling, step length 2 andfilling value 1, and then the parameters are flattened into 1024-dimensional feature vectors;
(c) the decoding end first performs Dropout operation to prevent overfitting, the ratio is set to 0.5, and then 3 full-connection layers Dense1, Dense2 and Dense3 are included; wherein the number of output neurons of the Dense1 is 512, and a Leaky _ ReLU activation function is adopted for activation; the number of output neurons of Dense2 is 128, a Leaky _ ReLU activation function is adopted for activation, and the number of output neurons of Dense3 is 10;
and 5, identifying the road surface material based on the trained neural network model, wherein the result is shown in figure 3.
Claims (1)
1. A mobile robot pavement material identification method based on sound characteristics is characterized by comprising the following steps:
step 1, collecting and establishing a road surface material sound data set: the pavement materials are divided into 10 categories, namely grassland pavement, ceramic tile pavement, asphalt pavement, natural stone pavement, cement pavement, rubber pavement, floor tile pavement, wood board pavement, cobblestone pavement and gravel pavement; collecting road surface material sound data by adopting a sound collector according to the rule that every 50 knocks of each type of road surface form a group, and establishing a road surface material sound data set S ═ SiI is more than or equal to 1 and less than or equal to 10, wherein i is the road surface material type, and S isiThe data is the sound data of the i-th type pavement material;
step 2, preprocessing sound data of the road surface material: through segmentation, filtering noise reduction, framing windowing and endpoint detection preprocessing operations, the quality of the sound data of the pavement material is improved, and noise influence is removed, and the method specifically comprises the following substeps:
(a) and (3) dividing: the recording of the road surface material sound is carried out according to a 50-time continuous knocking method, the original data needs to be further divided, the sound of knocking the road surface material each time is used as a single road surface material sound data, i-th class road surface material sound data Si＝{sijJ is more than or equal to 1 and less than or equal to 50, wherein sijJ is the knocking times corresponding to the road surface material sound data;
(b) filtering and denoising: filtering the collected road surface material sound data through a high-pass filter, describing through a formula (1),
where α is the filter coefficient, sij(t) is the original data at the current moment,
for the road surface material sound data at the time t after the filtering and noise reduction processing, the frequency spectrum can be balanced through a filter, the signal-to-noise ratio (SNR) is improved, the proportion of low-frequency information can be suppressed, and the proportion of high-frequency useful components is relatively increased;
(c) framing and windowing: on the whole, the overall road surface material sound characteristic of the knocking road surface has time-varying characteristic and is not stable road surface material sound data, but the recorded road surface material sound data is considered to be stable in a short time, in order to meet the Fourier transformation condition, the road surface material sound data needs to be divided into short-time stable road surface material sound data, the frame division and interception time period ranges from 10 ms to 30ms, in order to ensure that the characteristic parameter continuously and stably changes, some frames are uniformly reserved between two frames without overlapping parts, the change of each overlapping part is called frame shift, the value range is 1/4-1/2 of the frame length, and the road surface material sound data f after frame division and framing is between 1/4-1/2ijPerforming windowing to reduce pointsInfluence of frame, sound data c of windowed road surface materialij＝fijXxw, wherein the window function w adopts a Hamming window;
(d) and (3) end point detection: determining the initial position and the end position of the effective road material sound data by setting different volume thresholds, and firstly calculating the windowed road material sound data cijVolume maximum value vmax inijAnd minimum value vminijAnd subtracting to obtain the transformation range value vdif of the road surface material sound dataijRespectively taking vmin from the lowest pavement material sound data as a starting pointij+vdifij×0.1、vminij+vdifij×0.01、vminij+vdifijTaking three different threshold values of x 0.05 as the end point position judgment of the road surface material sound data, starting to record the starting point when the threshold value is exceeded, judging whether the sound data at the next moment is larger than the threshold value or not until the end point lower than the threshold value is detected and recorded, and taking the road surface material sound data between the final starting point position and the final position lower than the threshold value as the effective road surface material sound data e for knocking the road surfaceij；
Step 3, Mel cepstrum feature extraction of the road surface material sound: the Mel cepstrum feature is a most common feature extraction method in sound data, and has the advantages of strong anti-interference capability and capability of reflecting feature differences among different sound data, so that the sound data of the road surface material can be effectively distinguished, and the method specifically comprises the following substeps:
(a) road surface material sound data e obtained by detecting end pointsijCarrying out one-dimensional Fourier transform to obtain corresponding frequency domain signals
Wherein N represents the number of sampling points in each frame of pavement material sound data, k is the sampling point, and the power spectrum gij＝|pij(k)|2；
(b) The power spectrum g of each frame of road surface material sound dataijPutting into Mel triangular filter bank for filtering, and calculating pairsNumber energy qijAnd is described by the following formula,
wherein h isijRepresenting the frequency response of the mel filters, M being the number of mel filters;
(c) to q isijDiscrete cosine transform is carried out to obtain the final Mel cepstrum characteristic coefficient, which is described by the following formula,
wherein melijRepresenting a Mel cepstrum characteristic coefficient, v representing the order of the Mel cepstrum coefficient, and N representing the number of sampling points in each frame of pavement material sound data;
(d) for the obtained mel-frequency cepstrum characteristic coefficient melijCarrying out artificial marking and constructing a training set;
step 4, constructing a deep convolution neural network for training: constructing a deep convolutional neural network, and based on Mel cepstrum characteristic coefficients mel of sound data of different pavement materialsijClassifying to realize the identification of the pavement material, and mainly comprising the following substeps:
(a) enhancing the characteristics, adding some auxiliary characteristics, such as chrominance frequency characteristics, Mel frequency spectrum characteristics, spectral contrast characteristics and hue centroid characteristics in audio, forming 193-dimensional characteristic vectors through characteristic splicing and fusion, finally forming 196-dimensional characteristic vectors through zero padding and alignment, further converting the characteristics into a two-dimensional matrix form (14 × 14), and performing dimension expansion on an input end in the network to obtain a 3D tensor (14 × 14 × 1);
(b) the encoding end comprises 1 convolutional layer Conv1, parameters are convolutional kernel size 1 multiplied by 1, step length 1, same filling and output channel number 32, Batch Normalization (BN) processing is carried out, and a Leaky _ ReLU activation function is adopted for activation; 3 lightweight depth separable convolutional layers DConv2, DConv3, DConv4 connected in series; the parameters of DConv2 are convolution kernel size 3 × 3, step size 1, same filling and output channel number 32, batch normalization processing is performed, and a Leaky _ ReLU activation function is used for activation; parameters of the pooling layer Maxpool1 are 2 × 2 maximum pooling, step size 2, no padding; the parameters of DConv3 are convolution kernel size 3 × 3, step size 1, same filling and output channel number 64, batch normalization processing is carried out, and a Leaky _ ReLU activation function is adopted for activation; the parameters of DConv4 are convolution kernel size 3 × 3, step size 1, same filling and output channel number 64, batch normalization processing is carried out, and a Leaky _ ReLU activation function is adopted for activation; parameters of the pooling layer Maxpool2 are 3 × 3 maximal pooling, step length 2 and filling value 1, and then the parameters are flattened into 1024-dimensional feature vectors;
(c) the decoding end first performs Dropout operation to prevent overfitting, the ratio is set to 0.5, and then 3 full-connection layers Dense1, Dense2 and Dense3 are included; wherein the number of output neurons of the Dense1 is 512, and a Leaky _ ReLU activation function is adopted for activation; the number of output neurons of Dense2 is 128, a Leaky _ ReLU activation function is adopted for activation, and the number of output neurons of Dense3 is 10;
and 5, identifying the pavement material based on the trained neural network model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011597385.6ACN113514544A (en) | 2020-12-29 | 2020-12-29 | Mobile robot pavement material identification method based on sound characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011597385.6ACN113514544A (en) | 2020-12-29 | 2020-12-29 | Mobile robot pavement material identification method based on sound characteristics |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113514544Atrue CN113514544A (en) | 2021-10-19 |
Family
ID=78060872
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011597385.6APendingCN113514544A (en) | 2020-12-29 | 2020-12-29 | Mobile robot pavement material identification method based on sound characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113514544A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114234061A (en)* | 2021-12-20 | 2022-03-25 | 北京工业大学 | A neural network-based intelligent discrimination method for water leakage sound of water supply pipelines operating under pressure |
| CN114544772A (en)* | 2022-04-26 | 2022-05-27 | 华南农业大学 | Duck egg crack detection device and method based on convolutional neural network and sound spectrum |
| CN115422965A (en)* | 2022-07-22 | 2022-12-02 | 上汽大众汽车有限公司 | Vehicle-interior noise-based vehicle running road surface identification method and system |
| CN116698977A (en)* | 2023-08-07 | 2023-09-05 | 山东科技大学 | Coal gangue recognition sensor and recognition method based on sound signals |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009046359A2 (en)* | 2007-10-03 | 2009-04-09 | University Of Southern California | Detection and classification of running vehicles based on acoustic signatures |
| CN106821337A (en)* | 2017-04-13 | 2017-06-13 | 南京理工大学 | A kind of sound of snoring source title method for having a supervision |
| CN107545902A (en)* | 2017-07-14 | 2018-01-05 | 清华大学 | A kind of article Material Identification method and device based on sound characteristic |
| US20180151183A1 (en)* | 2016-11-29 | 2018-05-31 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and device for searching according to speech based on artificial intelligence |
| CN108550375A (en)* | 2018-03-14 | 2018-09-18 | 鲁东大学 | A kind of emotion identification method, device and computer equipment based on voice signal |
| CN110909637A (en)* | 2019-11-08 | 2020-03-24 | 清华大学 | Outdoor mobile robot terrain recognition method based on visual-touch fusion |
| CN111399505A (en)* | 2020-03-13 | 2020-07-10 | 浙江工业大学 | Mobile robot obstacle avoidance method based on neural network |
- 2020
- 2020-12-29CNCN202011597385.6Apatent/CN113514544A/enactivePending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009046359A2 (en)* | 2007-10-03 | 2009-04-09 | University Of Southern California | Detection and classification of running vehicles based on acoustic signatures |
| US20180151183A1 (en)* | 2016-11-29 | 2018-05-31 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and device for searching according to speech based on artificial intelligence |
| CN106821337A (en)* | 2017-04-13 | 2017-06-13 | 南京理工大学 | A kind of sound of snoring source title method for having a supervision |
| CN107545902A (en)* | 2017-07-14 | 2018-01-05 | 清华大学 | A kind of article Material Identification method and device based on sound characteristic |
| CN108550375A (en)* | 2018-03-14 | 2018-09-18 | 鲁东大学 | A kind of emotion identification method, device and computer equipment based on voice signal |
| CN110909637A (en)* | 2019-11-08 | 2020-03-24 | 清华大学 | Outdoor mobile robot terrain recognition method based on visual-touch fusion |
| CN111399505A (en)* | 2020-03-13 | 2020-07-10 | 浙江工业大学 | Mobile robot obstacle avoidance method based on neural network |
Non-Patent Citations (1)
| Title |
|---|
| 姜囡: "《语音信号识别技术与实践》", 31 January 2020, 东北大学出版社* |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114234061A (en)* | 2021-12-20 | 2022-03-25 | 北京工业大学 | A neural network-based intelligent discrimination method for water leakage sound of water supply pipelines operating under pressure |
| CN114544772A (en)* | 2022-04-26 | 2022-05-27 | 华南农业大学 | Duck egg crack detection device and method based on convolutional neural network and sound spectrum |
| CN114544772B (en)* | 2022-04-26 | 2022-08-05 | 华南农业大学 | Device and method for detecting duck egg cracks based on convolutional neural network and voice frequency spectrum |
| CN115422965A (en)* | 2022-07-22 | 2022-12-02 | 上汽大众汽车有限公司 | Vehicle-interior noise-based vehicle running road surface identification method and system |
| CN116698977A (en)* | 2023-08-07 | 2023-09-05 | 山东科技大学 | Coal gangue recognition sensor and recognition method based on sound signals |
| CN116698977B (en)* | 2023-08-07 | 2023-10-31 | 山东科技大学 | Coal gangue recognition sensor and recognition method based on sound signals |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113514544A (en) | Mobile robot pavement material identification method based on sound characteristics | |
| CN113111902B (en) | Pavement material identification method based on voice and image multi-mode collaborative learning | |
| CN110399840B (en) | A fast lawn semantic segmentation and boundary detection method | |
| CN103218831B (en) | A kind of video frequency motion target classifying identification method based on profile constraint | |
| CN114998852A (en) | Intelligent detection method for road pavement diseases based on deep learning | |
| CN102682303B (en) | Crowd exceptional event detection method based on LBP (Local Binary Pattern) weighted social force model | |
| CN111598142B (en) | An Outdoor Terrain Classification Method for Wheeled Mobile Robots | |
| CN110119687A (en) | Detection method based on the road surface slight crack defect that image procossing and convolutional neural networks combine | |
| CN109035188A (en) | A kind of intelligent image fusion method based on target signature driving | |
| CN110400293B (en) | A no-reference image quality assessment method based on deep forest classification | |
| CN108681737B (en) | A method for image feature extraction under complex illumination | |
| CN101551863A (en) | Method for extracting roads from remote sensing image based on non-sub-sampled contourlet transform | |
| CN103218619A (en) | Image aesthetics evaluating method | |
| CN110111290B (en) | Infrared and visible light image fusion method based on NSCT and structure tensor | |
| CN104636755A (en) | Face beauty evaluation method based on deep learning | |
| CN103295221A (en) | Water surface target motion detecting method simulating compound eye visual mechanism and polarization imaging | |
| CN118781491B (en) | Homeland utilization classification method based on image segmentation | |
| CN112419258B (en) | Robust environment sound identification method based on time-frequency segmentation and convolutional neural network | |
| CN111724074A (en) | A pavement lesion detection and early warning method and system based on deep learning | |
| CN109086803A (en) | A kind of haze visibility detection system and method based on deep learning and the personalized factor | |
| CN109214331A (en) | A kind of traffic haze visibility detecting method based on image spectrum | |
| CN114966836A (en) | Earthquake P wave pickup method based on SAIP-Unet network | |
| CN115184998A (en) | Rayleigh wave frequency dispersion curve automatic extraction method based on improved U-net neural network | |
| CN119672555A (en) | A remote sensing image object segmentation method based on hierarchical feature extraction and decoding | |
| CN111881922B (en) | Insulator image recognition method and system based on salient features |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20211019 | |
| RJ01 | Rejection of invention patent application after publication |