Disclosure of Invention
The invention provides a method, a device, equipment and a storage medium for extracting a water body of a flood disaster, which are used for solving the defects of low extraction precision and extraction accuracy of the water body of the flood disaster in the prior art and achieving the purpose of extracting water body information based on SAR images of the flood disaster with high precision and high accuracy.
The invention provides a flood disaster water body extraction method, which comprises the following steps:
acquiring a water body SAR image to be extracted, wherein the water body SAR image to be extracted comprises an image formed by a dual polarized wave band and a spectral characteristic wave band of a flood disaster SAR image;
Inputting the SAR image to be extracted of the water body into a preset flood extraction convolutional neural network to obtain a flood disaster water body information distribution map;
The preset flood extraction convolutional neural network is used for carrying out residual convolution and attention mechanism correction on the SAR image to be extracted of the water body, and carrying out cavity convolution on the corrected SAR image to be extracted of the water body to obtain the flood disaster water body information distribution map.
According to the method for extracting the flood disaster water body provided by the invention, the preset flood extraction convolutional neural network comprises a preset residual error neural network and a preset cavity convolutional network, the SAR image to be extracted of the water body is input into the preset flood extraction convolutional neural network to obtain a flood disaster water body information distribution diagram, and the method comprises the following steps:
Inputting the SAR image to be extracted of the water body into a preset residual neural network for residual convolution and attention mechanism correction to obtain a first water body information distribution diagram after the accurate correction of the edge of the water body of the flood disaster;
and inputting the first water body information distribution map into a preset cavity convolution network to perform change processing of a preset expansion scale, and obtaining a complete flood disaster water body information distribution map by extracting the edge of the flood disaster water body.
According to the method for extracting the water body of the flood disaster, provided by the invention, the preset residual neural network comprises P residual sub-networks, and when each residual sub-network is respectively connected with an attention mechanism correction sub-network, the SAR image to be extracted of the water body is input into the preset residual neural network to carry out residual convolution and attention mechanism correction, so as to obtain a first water body information distribution diagram after the edge fine correction of the water body of the flood disaster, and the method comprises the following steps:
Inputting the SAR image to be extracted of the water body into a q residual error sub-network for convolution, batch normalization and linear arrangement treatment to obtain a q residual error convolution image;
Inputting the q residual convolution image into a q attention mechanism correction sub-network for channel correction and space correction processing to obtain a q correction image;
Inputting the q correction image into a q+1th residual error sub-network for convolution, batch normalization and linear finishing treatment to obtain a q+1th residual error convolution image;
Inputting the (q+1) th residual convolution image into a (q+1) th attention mechanism correction sub-network for channel correction and space correction processing to obtain a (q+1) th correction image;
and adding 1 to the value of q, and repeatedly executing the steps of inputting the q+1th residual convolution image into the q+1th attention mechanism correction sub-network to perform channel correction and space correction processing until a P-th correction image is obtained, wherein the P-th correction image is a first water body information distribution diagram after the edge of the flood disaster water body is precisely corrected, and q is [1],. P ], and q+1 is [ 2],. P ].
According to the method for extracting the flood disaster water body provided by the invention, when the preset cavity convolution network comprises Q different expansion rates, the first water body information distribution map is input into the preset cavity convolution network to carry out the change processing of the preset expansion scale, so that the complete flood disaster water body information distribution map is extracted from the edge of the flood disaster water body, and the method comprises the following steps:
Inputting the first water body information distribution diagram into a preset cavity convolution network to perform different expansion scale processing to obtain Q tiny water body characteristic information diagrams with different expansion scales;
carrying out fusion processing on the fine water feature information graph to obtain a fusion image;
and performing decoding processing corresponding to the residual convolution and the attention mechanism correction on the fusion image to obtain a complete flood disaster water body information distribution diagram by extracting the flood disaster water body edge.
According to the method for extracting the flood disaster water body, provided by the invention, the training process of the preset flood extraction convolutional neural network comprises the following steps:
The method comprises the steps of obtaining a training sample image set, wherein the training sample image set comprises training sample images, and the training sample images are SAR images obtained by carrying out sample labeling, regular grid cutting and data enhancement processing on the SAR images to be extracted from the water body;
Training a preset initial flood extraction convolutional neural network according to the training sample image set to obtain the preset flood extraction convolutional neural network.
According to the method for extracting the flood disaster water body provided by the invention, the preset initial flood extraction convolutional neural network is trained according to the training sample image set to obtain the preset flood extraction convolutional neural network, and the method comprises the following steps:
performing iterative training on a preset initial flood extraction convolutional neural network according to the training sample image set, and obtaining an intermediate flood extraction convolutional neural network obtained after the iterative training of the round;
judging whether the accumulated training round number corresponding to the round of iterative training reaches a preset round number threshold value or not;
if the accumulated training round number reaches the preset round number threshold value, taking the intermediate flood extraction convolutional neural network obtained after the round of iterative training as the preset flood extraction convolutional neural network;
and if the accumulated training round number does not reach the preset round number threshold, training the intermediate flood extraction convolutional neural network to obtain the preset flood extraction convolutional neural network.
According to the flood disaster water body extraction method provided by the invention, the determining process of the preset round number threshold value comprises the following steps:
dividing the training sample image set into a training sample and a verification sample according to a preset proportion;
Training a preset initial flood extraction convolutional neural network according to the training sample, and obtaining an intermediate flood extraction convolutional neural network obtained after training of a preset number of rounds;
Verifying the intermediate flood extraction convolutional neural network according to the verification sample to obtain the value of the evaluation index of the intermediate flood extraction convolutional neural network;
judging whether the value of the evaluation index reaches a preset standard value or not;
If the value of the evaluation index reaches the preset standard value, taking the accumulated training round number corresponding to the intermediate flood extraction convolutional neural network as the preset round number threshold;
and if the value of the evaluation index does not reach the preset standard value, training the intermediate flood extraction convolutional neural network to obtain the preset round number threshold value.
The invention also provides a flood disaster water body extraction device, which comprises:
the acquisition module is used for acquiring a water body SAR image to be extracted, wherein the water body SAR image to be extracted comprises an image formed by a dual polarized wave band and a spectral characteristic wave band of a flood disaster SAR image;
The determining module is used for inputting the SAR image to be extracted of the water body into a preset flood extraction convolutional neural network to obtain a flood disaster water body information distribution map;
The preset flood extraction convolutional neural network is used for carrying out residual convolution and attention mechanism correction on the SAR image to be extracted of the water body, and carrying out cavity convolution on the corrected SAR image to be extracted of the water body to obtain the flood disaster water body information distribution map.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the steps of the flood disaster water body extraction method according to any one of the above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the flood disaster water extraction method as described in any one of the above.
According to the method, the device, the equipment and the storage medium for extracting the flood disaster water body, the image formed by the dual polarized wave band and the spectral characteristic wave band of the SAR image of the flood disaster is firstly used as the SAR image to be extracted of the water body, and then the SAR image to be extracted of the water body is input into a preset flood extraction convolutional neural network for water body extraction, so that a flood disaster water body information distribution map is obtained. Because the preset flood extraction convolutional neural network is used for carrying out residual convolution and attention mechanism correction on the SAR image to be extracted from the water body and carrying out cavity convolution on the corrected SAR image to be extracted from the water body to obtain a flood disaster water body information distribution map, the aims of accurately correcting and reliably extracting the water body edge with obvious characteristics of the SAR image to be extracted from the water body and further capturing the characteristic information of the water body edge with more details and unobvious characteristics can be fulfilled, so that the water body information in the obtained flood disaster water body information distribution map is richer and complete, and the extraction precision and the extraction accuracy of the flood disaster water body are effectively improved.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Flood disasters are one of the most serious natural disasters worldwide, which cause a great deal of casualties and economic losses to the world. According to the Chinese water and drought disaster publication issued by the water conservancy department, the flood disasters cause direct economic loss of 1682.88 hundred million yuan in China in 1998-2019, and the death of 1288 people can be seen, so that the flood disasters bring great loss to the economic development of China and the life safety of people. Therefore, timely and accurate acquisition of the space-time distribution range of flood inundation has extremely important significance for relevant departments to formulate disaster-fighting schemes and post-disaster loss evaluation.
At present, flood monitoring mainly comprises a hydrologic site monitoring method and a remote sensing monitoring method, wherein the hydrologic site monitoring method is high in accuracy, but has the defects of few site numbers, incapability of monitoring a spatial distribution range and the like, and the remote sensing technology gradually becomes a main method for flood monitoring due to the advantages of short revisiting time, capability of monitoring the spatial distribution range and the like. However, since flood water is often accompanied by bad weather, the conventional optical sensor is affected by cloud rain, and it is difficult to provide an optical image of high quality without clouds. In contrast, SAR has an all-day and all-weather monitoring capability, is not easily affected by severe weather such as cloud and rain, and plays an increasingly important role in flood monitoring and extraction.
The flood monitoring and extracting method based on SAR data mainly comprises a threshold method, an object-oriented method, an active contour method and a data fusion method. However, due to the shortcomings of speckle noise, uneven gray level distribution and the like of the SAR image, under the conditions of large data volume and complex flood inundation scene, the traditional SAR data flood monitoring method cannot well meet the requirements in large-area flood monitoring. Under the background, combining SAR data with intelligent water extraction algorithms is becoming a research hotspot.
Furthermore, the deep learning algorithm is mainly applied to the tasks of semantic segmentation, target detection, image classification and the like in the remote sensing field, and achieves good effects, and the deep learning algorithm has strong applicability in the multiband remote sensing image information extraction process because the deep learning method avoids a complex feature selection process and can directly acquire features from an original image. In 2015, long et al propose a semantic segmentation method based on a full convolution network FCN, solve the problem of image segmentation at a semantic level and classify images at a pixel level, wherein FCN is a first convolution neural network for semantic segmentation, but the network has the defects of simplistic up-sampling operation, loss of detail information and the like. Then Ronneberger and the like improve the FCN network, and a U-Net network with a coding and decoding structure is provided, and the network can integrate the characteristics of low resolution and high resolution, so that the image segmentation accuracy is greatly improved. Thereafter, scholars have proposed many classical semantic segmentation models, such as Deeplab v, unet++, HRNet, etc., in order to improve the accuracy and performance of semantic segmentation.
In recent years, the deep learning model is gradually applied to remote sensing image water body extraction, such as Chen Qian, and the like, a convolutional neural network is adopted to extract water bodies, and the effectiveness of the deep learning method for extracting the water bodies is proved by comparing the water body extraction with the traditional NDWI and other extraction methods, chen and the like design a new convolutional neural network model for extracting urban surface water bodies in high-resolution remote sensing images, higher water body extraction precision is obtained, wang and the like propose a multi-scale lake water body extraction network model, and small lake water bodies can be well extracted.
However, the research of extracting water body by the existing deep learning model mainly focuses on optical images, but the research on SAR images is relatively less, and the defects that water body and shadow are difficult to distinguish, small water body and river are difficult to extract completely, and the extraction precision is not high exist. The key of SAR image monitoring flood is also the identification and extraction of water information, which is the same as the optical remote sensing image.
Based on the above problems, the invention provides a method for extracting a flood disaster water body, wherein an execution subject of the method for extracting the flood disaster water body can be a flood disaster water body extraction device, and the flood disaster water body extraction device can be realized to be part or all of terminal equipment in a mode of software, hardware or combination of software and hardware. Alternatively, the terminal device may be a personal computer (Persodal Computer, PC), a portable device, a notebook computer, a smart phone, a tablet computer, a portable wearable device, or other electronic devices, such as a tablet computer, a mobile phone, or the like. The invention is not limited to the specific form of the terminal device.
It should be noted that, the execution body of the method embodiment described below may be part or all of the terminal device described above. The following method embodiments are described taking an execution body as a terminal device as an example.
Fig. 1 is a flow chart of the flood disaster water body extraction method provided by the invention, as shown in fig. 1, the flood disaster water body extraction method comprises the following steps:
Step S110, acquiring a water body SAR image to be extracted, wherein the water body SAR image to be extracted comprises an image formed by a dual polarized wave band and a spectral characteristic wave band of a flood disaster SAR image.
Specifically, when the terminal equipment acquires an SAR image to be extracted from a water body, firstly acquiring the SAR image aiming at a flood disaster, performing preprocessing operations such as orbit correction, thermal noise removal, radiation calibration, speckle filtering, terrain correction, decibelization, clipping, embedding and the like on the SAR image by utilizing SNAP software to obtain a backscattering distribution map of the SAR image, further acquiring a dual-polarized wave band of the backscattering distribution map, wherein the dual-polarized wave band can comprise a VH wave band and a VV wave band, then obtaining spectral characteristics SDWI wave bands derived from polarization of the VH wave band and the VV wave band by utilizing Python programming, and finally taking a three-wave-band image consisting of the VH wave band, the VV wave band and the SDWI wave band as the SAR image to be extracted from the water body.
The electric field vector of the energy pulse emitted by the radar may be polarized in the vertical or horizontal plane. And, regardless of the wavelength, the radar signal may transmit a horizontal (H) or vertical (V) electric field vector, receive a return signal that is horizontal (H) or vertical (V), or both. Therefore, the radar remote sensing system for the SAR radar generally adopts four polarization modes, HH, VV, HV and VH, where HH and VV are co-polarized, and HV and VH are non-directional (or cross) polarized.
And step S120, inputting the SAR image to be extracted of the water body into a preset flood extraction convolutional neural network to obtain a flood disaster water body information distribution map.
The preset flood extraction convolutional neural network is used for carrying out residual convolution and attention mechanism correction on the SAR image to be extracted of the water body, and carrying out cavity convolution on the corrected SAR image to be extracted of the water body to obtain the flood disaster water body information distribution map.
Specifically, when the terminal device obtains the SAR image to be extracted from the water body, according to a preset feature extraction algorithm and a feature expansion algorithm, firstly extracting water body information with obvious features in the SAR image to be extracted from the water body, such as a broad lake surface, a small pond and the like, and then further extracting more water body information with unobvious features, such as a small river with more details and the like and a tiny water ditch with difficult recognition and the like on the basis of ensuring the extraction accuracy and the extraction precision, so as to obtain a complete flood disaster water body information distribution diagram of water body extraction. When the SAR image to be extracted of the water body is determined aiming at the Sentinel-1SAR image of the 7 th month and the 26 th day of 2020 in the occurrence period of the flood disaster, the flood disaster water body information distribution diagram shown in the figure 2 can be obtained.
It should be noted that, in order to improve the extraction accuracy and the extraction precision, the feature extraction algorithm used in the invention includes an algorithm of combining a depth residual error network with an attention mechanism, because the depth residual error network can reduce the complexity of a network model and improve the processing speed of the network model, and the attention mechanism can not only extract more semantic features, but also enable the terminal equipment to have the functions of ignoring irrelevant information and focusing important information like a human brain. Therefore, the feature extraction algorithm can accurately and accurately extract the water body information with obvious water body features in the SAR image to be extracted from the water body, such as the water body information with obvious features, which is easy to submerge in a wide lake surface, a small water body, a pond and a flood disaster. Alternatively, the depth residual network may be the depth residual network (Deep residual network, resNet) 18 and the attention mechanism may be the simultaneous spatial compression and channel Excitation (Concurrent SPATIAL AND Squeeze AND CHANNEL specification, scse) attention mechanism.
According to the flood disaster water body extraction method provided by the invention, firstly, an image formed by a dual polarized wave band and a spectral characteristic wave band of a flood disaster SAR image is used as a water body SAR image to be extracted, and then the water body information distribution map of the flood disaster water body is obtained through the process of inputting the water body SAR image to be extracted into a preset flood extraction convolutional neural network for water body extraction. Because the preset flood extraction convolutional neural network is used for carrying out residual convolution and attention mechanism correction on the SAR image to be extracted from the water body and carrying out cavity convolution on the corrected SAR image to be extracted from the water body to obtain a flood disaster water body information distribution map, the aims of accurately correcting and reliably extracting the water body edge with obvious characteristics of the SAR image to be extracted from the water body and further capturing the characteristic information of the water body edge with more details and unobvious characteristics can be fulfilled, so that the water body information in the obtained flood disaster water body information distribution map is richer and complete, and the extraction precision and the extraction accuracy of the flood disaster water body are effectively improved.
Based on the defect of low extraction precision and extraction accuracy of flood disaster water bodies in the prior art, the invention provides a flood extraction convolutional neural network (Flood Extraction Network, FENet) based on SAR data for extracting water body information in a flood disaster, wherein a FENet network consists of a coding correction network and a hole convolutional network, the coding correction network adopts a Resnet network, a residual sub-network in the Resnet network is connected with a attention mechanism correction sub-network, so that characteristic images obtained after coding and correction are input into the hole convolutional network with different expansion rates, deep semantic features are extracted by different scales, and the images subjected to hole convolution are further processed to the decoding corresponding to the coding correction process to the original size. In addition, when the attention mechanism adopted by the FENet network corrects the sub-network to be Scse attention mechanism, the accuracy of model prediction can be further improved.
Therefore, when the preset flood extraction convolutional neural network includes the preset residual neural network and the preset hole convolutional network, step S120 may be implemented by:
firstly, inputting the SAR image to be extracted of the water body into a preset residual neural network for residual convolution and attention mechanism correction to obtain a first water body information distribution diagram after the accurate correction of the water body edge of the flood disaster, and then inputting the first water body information distribution diagram into a preset cavity convolution network for the change treatment of a preset expansion scale to obtain a complete flood disaster water body information distribution diagram extracted from the water body edge of the flood disaster.
Specifically, when the terminal device obtains the SAR image to be extracted from the water body, the SAR image to be extracted from the water body can be input into a preset residual neural network to carry out residual convolution and attention correction, namely, the preset residual neural network carries out residual convolution on the SAR image to be extracted from the water body, and then carries out attention correction on the SAR image after the residual convolution, so that a first water body information distribution diagram after the water body edge fine correction of the flood disaster is obtained, wherein the first water body information distribution diagram comprises water body information with obvious characteristics and water body information after the water body edge is accurately corrected.
Further, when the terminal equipment determines that the preset residual neural network outputs the first water body information distribution diagram, the first water body information distribution diagram can be further input into the preset cavity convolutional neural network to be subjected to expansion processing of a plurality of different expansion rates, so that characteristic information of water body edges with more details and unobvious characteristics in the SAR image to be extracted from the water body can be captured, and the water body information is richer and comprehensive.
According to the flood disaster water body extraction method provided by the invention, the water body information with obvious water body characteristics is extracted through the preset residual neural network with the residual convolution and attention mechanism correction functions, and more small water body information with unobvious water body characteristics and easy to ignore is extracted through the preset cavity convolution neural network with different expansion processing functions, so that the accuracy and the reliability of extraction of the flood disaster water body information are improved.
Optionally, when each residual sub-network is connected to one attention mechanism correction sub-network, the specific process of inputting the SAR image to be extracted of the water body into the preset residual sub-network to perform residual convolution and attention mechanism correction to obtain the first water body information distribution diagram after the flood disaster water body edge fine correction includes:
Inputting the SAR image to be extracted of the water body into a q residual error sub-network for convolution, batch normalization and linear arrangement treatment to obtain a q residual error convolution image;
Inputting the q residual convolution image into a q attention mechanism correction sub-network for channel correction and space correction processing to obtain a q correction image;
Inputting the q correction image into a q+1th residual error sub-network for convolution, batch normalization and linear finishing treatment to obtain a q+1th residual error convolution image;
Inputting the (q+1) th residual convolution image into a (q+1) th attention mechanism correction sub-network for channel correction and space correction processing to obtain a (q+1) th correction image;
and adding 1 to the value of q, and repeatedly executing the steps of inputting the q+1th residual convolution image into the q+1th attention mechanism correction sub-network to perform channel correction and space correction processing until a P-th correction image is obtained, wherein the P-th correction image is a first water body information distribution diagram after the edge of the flood disaster water body is precisely corrected, and q is [1],. P ], and q+1 is [ 2],. P ].
Specifically, when the preset residual neural network is a Resnet network comprising 4 residual sub-networks and each attention mechanism correction sub-network represents Scse attention mechanisms, the terminal equipment can input the water body to be extracted SAR image into the 1 st residual sub-network to carry out convolution, batch normalization and linear arrangement processing to obtain the 1 st residual convolution image, then input the 1 st residual convolution image into the 1 st attention mechanism correction sub-network to carry out channel correction and space correction processing to obtain the 1 st correction image, then input the 1 st correction image into the 2 nd residual sub-network to carry out convolution, batch normalization and linear arrangement processing to obtain the 2 nd residual convolution image, input the 2 nd residual convolution image into the 2 nd attention mechanism correction sub-network to carry out channel correction and space correction processing to obtain the 2 nd correction image, then input the 2 nd correction image into the 3 rd residual sub-network to carry out convolution, then carry out convolution processing to obtain the 3 rd convolution image, namely, carry out the 4 th convolution image is input into the 3 rd convolution sub-network to carry out channel correction and the 4 th convolution correction processing to obtain the 4 th convolution image, namely, the 4 th convolution image is subjected to the 4 th convolution correction image is subjected to the channel correction and the 4 th correction processing to obtain the 4 th convolution image, the first water body information distribution diagram is also a characteristic image obtained after P steps (such as 4 steps) of coding correction.
In the actual processing process, each residual sub-network may perform convolution, batch normalization and linear finishing processes as shown in fig. 3, or may directly add to the image x to obtain a corrected image after performing convolution twice, batch normalization twice and linear finishing processes as shown in fig. 3 (a), or may add to the image x after convolution once after performing convolution twice, batch normalization twice and linear finishing processes as shown in fig. 3 (b), where x is a corrected image of a water body to be extracted SAR image or any one of 1 st corrected image to P-1 st corrected image, y is any one of 1 st corrected image to P-th corrected image, and Relu represents the linear finishing process.
In addition, each attention mechanism correction sub-network can execute channel correction and space correction processing of each residual convolution image as shown in fig. 4, wherein the size of each residual convolution image is (C, H, W) respectively, C represents the color channel of each residual convolution image, H represents the height of each residual convolution image, and W represents the width of each residual convolution image, the channel correction processing is to perform global average pooling processing on the residual convolution image, and then perform channel correction by using relu functions and sigmoid functions respectively after two convolutions, finally perform channel correction by using a channel multiplication mode to obtain a first image after channel correction, and perform space information correction by using a spatial multiplication mode to obtain a second image after space correction, and further combine the first image after channel correction and the second image after space correction by using a channel addition mode to obtain a corrected image after channel correction and space correction and the convolution image with the same size as the corresponding residual image. According to the invention, after each attention mechanism correction sub-network is added to each residual error sub-network, the dimension of each residual error convolution image is not changed, so that the purpose of accurately correcting the edge of the water body in each residual error convolution image is realized.
It should be noted that, although the common depth residual error network includes Resnet a 18, resnet a 34, resnet a 50, resnet a 101 and Resnet a 152, in order to reduce complexity of the network model and increase processing speed of the network model, the present invention adopts a Resnet18 network and selects 4 residual error sub-networks in the Resnet18 network to achieve the purpose of extracting water information with obvious water characteristics, which greatly improves use efficiency of the Resnet network, and can further improve accuracy of model prediction when each residual error sub-network is connected with a Scse attention mechanism.
According to the flood disaster water body extraction method provided by the invention, the purpose of rapidly and accurately extracting water body information with obvious characteristics is realized by means of the P residual sub-networks and the mode that each residual sub-network is respectively connected with one attention mechanism correction sub-network, so that the purpose of greatly improving the prediction precision of a network model on the basis of accurately correcting the edge of the water body can be realized.
Optionally, when the preset cavity convolution network includes Q different expansion rates, inputting the first water body information distribution map into the preset cavity convolution network to perform a change process of a preset expansion scale, so as to obtain a specific process of extracting a complete flood disaster water body information distribution map from a flood disaster water body edge, which may include:
Inputting the first water body information distribution map into a preset cavity convolution network to perform different expansion scale processing to obtain Q tiny water body characteristic information maps with different expansion scales, performing fusion processing on the tiny water body characteristic information maps to obtain a fusion image, and performing decoding processing corresponding to the residual convolution and attention mechanism correction on the fusion image to obtain a complete flood disaster water body information distribution map extracted from the edge of the flood disaster water body.
Specifically, when the preset hole convolution network includes 4 different expansion rates, the terminal device inputs the first water information distribution map after the accurate correction of the edge of the flood disaster water body into the preset hole convolution network to perform hole convolution processing with different expansion rates, so as to obtain 4 feature images with different expansion rates, (a) further performs fusion processing (such as channel addition processing) on the 4 feature images with different expansion rates, so as to obtain a fusion image, and finally performs P-step decoding operation corresponding to the P-step coding correction on the fusion image, so that the flood disaster water body information distribution map with the same size as the original water body to be extracted SAR image can be obtained, wherein the flood disaster water body information distribution map includes water body information with more obvious features and water body information with less obvious features, as shown in fig. 5, a hole convolution schematic diagram with different expansion rates is shown in fig. 5, (a) is a hole convolution schematic diagram with expansion rate of 1, (b) is a hole convolution schematic diagram with expansion rate of 2, and (c) is a hole convolution schematic diagram with expansion rate of 4, and the core convolution is a convolution schematic diagram with expansion rate of 1 can be regarded as a standard expansion rate. Because the cavity convolution with different expansion rates has different receptive fields, the different receptive fields are very important for distinguishing mountain shadows with similar spectral characteristics from water bodies, but if the expansion rate is set to be too large, the characteristic information of the tiny water bodies is lost too much. Therefore, the invention combines the cavity convolutions with the expansion rates of 1, 2, 4 and 8, extracts semantic features with different scales, and achieves the aim of extracting the edges of the water body more completely in a mode of capturing the feature information of more tiny water body rivers when the expansion rate is properly reduced.
According to the method for extracting the flood disaster water body, provided by the invention, by means of carrying out cavity convolution processing on the first water body information distribution diagram with different expansion scales and carrying out fusion processing and decoding processing on the characteristic images corresponding to the expansion scales, the purpose that more tiny water body information with unobvious water body characteristics can be captured on the premise that the water body information with obvious characteristics in SAR images including a flood disaster area can be accurately and accurately extracted is realized, and the extraction precision and the extraction efficiency of the flood disaster water body are greatly improved.
Optionally, in order to improve flexibility of training the initial flood extraction convolutional neural network and improve reliability of the preset flood extraction convolutional neural network, a training process of the preset flood extraction convolutional neural network may include:
the method comprises the steps of obtaining a training sample image set, wherein the training sample image set comprises training sample images, the training sample images are SAR images obtained by carrying out sample labeling, regular grid cutting and data enhancement processing on the SAR images to be extracted of the water body, and training a preset initial flood extraction convolutional neural network according to the training sample image set to obtain the preset flood extraction convolutional neural network.
Specifically, the terminal device trains the preset initial flood extraction convolutional neural network, which can also be considered as a process of training the preset initial FENet network, wherein the initial FENet network consists of an initial Resnet network and an initial hole convolutional network.
In addition, before training the initial FENet network, a training sample image set can be acquired from the SAR image to be extracted from the water body, and the acquisition process of the training image set comprises the following steps:
Firstly, two typical areas are cut out from a SAR image to be extracted from a water body and are respectively used as a training image and a test image, no intersection exists between the two typical areas, and the conditions met by each typical area comprise water bodies of different types, such as calm open water bodies, water bodies submerging ground features, fine river water bodies and the like, the more the water body types are, the better the larger the size of each typical area is.
And then respectively carrying out refined sample labeling processing on the training image and the test image to obtain a training label image corresponding to the training image and a test label image corresponding to the test image, wherein the refined sample labeling processing process comprises the steps of combining the existing high-resolution optical remote sensing image data as an aid, and respectively carrying out boundary labeling on suspected water bodies in the training image and the test image based on experience of researchers so as to obtain the training label image and the test label image.
And performing regular grid cutting and data enhancement on the test image and the test label image to obtain a test sample image set comprising N test sample images. The data enhancement comprises horizontal overturning, vertical overturning and diagonal mirroring operations on the image and the corresponding label image, and the data enhancement is used for obtaining sufficient samples and avoiding the phenomenon of fitting in the network training process.
Taking a test image as an example to carry out regular grid cutting explanation, the process comprises cutting each sample from left to right and from top to bottom in the test image according to the preset sample size, wherein the adjacent samples in the cut samples have overlapping areas, and the size of the overlapping areas is preset (for example, 20). For samples that are adjacent to the border of a row or column, samples of a preset sample size are now cut from top to bottom (or from bottom to top) for the last row, and samples of a preset sample size are also cut from left to right (or from right to left) for the last column, with the overlap area between adjacent samples being less than a preset size (e.g., less than 20) for such samples.
And finally, training the initial FENet network by using a training sample image set until the trained network reaches the preset network precision, and obtaining the preset flood extraction convolutional neural network.
According to the flood disaster water body extraction method provided by the invention, the initial flood extraction convolutional neural network is trained by the training sample image set obtained from the SAR image to be extracted from the water body, the training sample image set of the initial flood extraction convolutional neural network is obtained from the SAR image to be extracted from the water body, and the SAR image to be extracted from the water body comprises the dual polarized wave band and the spectral feature wave band, so that the network precision and the network reliability can be improved by training the initial flood extraction convolutional neural network by using the training sample image set, and a foundation is laid for the subsequent rapid and accurate extraction of the water body information.
Optionally, training the preset initial flood extraction convolutional neural network according to the training sample image set to obtain a specific process of the preset flood extraction convolutional neural network may include:
Performing iterative training on a preset initial flood extraction convolutional neural network according to the training sample image set, obtaining an intermediate flood extraction convolutional neural network obtained after the iterative training of the present round, judging whether the accumulated training round number corresponding to the iterative training of the present round reaches a preset round number threshold value, taking the intermediate flood extraction convolutional neural network obtained after the iterative training of the present round as the preset flood extraction convolutional neural network if the accumulated training round number reaches the preset round number threshold value, and training the intermediate flood extraction convolutional neural network if the accumulated training round number does not reach the preset round number threshold value to obtain the preset flood extraction convolutional neural network.
Specifically, the terminal device trains the initial flood extraction convolutional neural network by using M training sample images, wherein each training sample image trains FENet the network once and then is called as performing one training, and the M training sample images train FENet the network once and then are called as performing one round of training. After the terminal device trains the preset number of rounds by using M training sample images, the terminal device can judge whether the accumulated training number of rounds after the training reaches a preset number of rounds threshold, stop training when the accumulated training number of rounds reaches the preset number of rounds threshold, test the corresponding network when the training is stopped by using a testing sample image set comprising N testing sample images, determine that the values of the 5 indexes of the Precision, recall ratio (Recall), F1 index (F1), water intersection ratio (IoU) and homogeneous intersection ratio (mIoU) all reach the highest, at this time, determine that the corresponding network converges when the training is stopped, and obtain the preset flood extraction convolutional neural network, otherwise, when the accumulated training number of rounds does not reach the preset number of rounds threshold, continue to execute the preset number of rounds training of the network obtained after the training until the accumulated training number of rounds reaches the preset number of rounds threshold, and obtain the preset flood extraction convolutional neural network. The calculation formulas of the 5 indexes are respectively as follows:
Wherein n represents the number of the network predicted water body categories, TP represents the number of real water body pixels of the network predicted water body, FP represents the number of real non-water body pixels of the network predicted water body, FN represents the number of real water body pixels of the network predicted non-water body, precision represents Precision, recall represents Recall rate, F1 represents F1 index, mIoU represents homogeneous mixing ratio, ioU represents water body mixing ratio, F1 index comprehensively considers Precision rate and Recall rate, higher value represents better model extraction effect, homogeneous mixing ratio mIoU comprehensively considers model extraction water body mixing ratio and non-water body mixing ratio, higher value represents better model extraction effect, and F1 and mIoU are comprehensive evaluation indexes for measuring network models.
It should be noted that, since each training sample image training FENet network is referred to as performing a training after once, and each of M training sample images training FENet network is referred to as performing a round of training after once, when the process of performing a training for each training sample image is known, the process of performing a training for each of M training sample images training FENet network is also known.
The process of training each training sample image comprises the steps of performing P-step coding correction in an initial flood extraction convolutional neural network, performing first-step residual convolution and attention correction processing on the training sample image to obtain a first correction coding image when the value of P is 4 and the initial flood extraction convolutional neural network comprises 4 initial residual sub-networks and each initial residual sub-network is respectively connected with an initial Scse attention mechanism correction sub-network, performing second-step residual convolution and attention correction processing on the first correction coding image to obtain a second correction coding image, performing third-step residual convolution and attention correction processing on the second correction coding image to obtain a third correction coding image, and performing fourth-step residual convolution and attention correction processing on the third correction coding image to obtain a fourth correction coding image, wherein the fourth coding image can be called a characteristic image obtained after coding and correction.
And then carrying out cavity convolution processing on the characteristic image, namely, carrying out cavity convolution processing on the characteristic image to obtain Q characteristic images with different expansion scales, carrying out fusion processing (such as channel addition) on the characteristic images with different expansion scales, carrying out P-step decoding corresponding to the P-step coding correction on the fused image, and obtaining a decoded image which has the same size as the original training sample image and is trained for one time after the initial flood extraction convolutional neural network.
According to the flood disaster water body extraction method provided by the invention, the purpose of rapidly and efficiently obtaining the preset flood disaster water body extraction convolutional neural network is realized by training whether the accumulated number of rounds of the preset initial flood disaster extraction convolutional neural network reaches the preset round number threshold value or not through the training sample image set, so that the network precision is ensured, the training complexity is greatly reduced, and the network training speed is accelerated.
Optionally, the determining process of the preset round number threshold may include:
The training sample image set is divided into training samples and verification samples according to a preset proportion, a preset initial flood extraction convolutional neural network is trained according to the training samples, an intermediate flood extraction convolutional neural network obtained after the preset number of rounds of training is obtained, the intermediate flood extraction convolutional neural network is verified according to the verification samples, the value of an evaluation index of the intermediate flood extraction convolutional neural network is obtained, whether the value of the evaluation index reaches a preset standard value is judged, the accumulated training round number corresponding to the intermediate flood extraction convolutional neural network is used as the preset round number threshold value if the value of the evaluation index reaches the preset standard value, and the intermediate flood extraction convolutional neural network is trained if the value of the evaluation index does not reach the preset standard value, so that the preset round number threshold value is obtained.
The preset standard value can be used for representing the loss value, the precision and other parameters of the network, and the like, so that the network can meet the convergence requirement.
Specifically, the terminal device judges whether the network after the training is converged or not, and compares the accumulated training round number after the training with a preset round number threshold. Therefore, the specific value of the preset round number threshold is important.
For the determining process of the preset round number threshold, the M training sample images may be divided into training samples and verification samples according to a preset proportion, where the preset proportion may be 6:4 or 7:3, or other proportions, so long as the number of training samples is ensured to be greater than the number of verification samples.
After training a preset initial flood extraction convolutional neural network preset round number by using a training sample, the terminal equipment can acquire an intermediate flood extraction convolutional neural network obtained after the preset round number training, evaluate the intermediate flood extraction convolutional neural network by using a verification sample to obtain the value of an evaluation index of the intermediate flood extraction convolutional neural network, wherein the value of the evaluation index can comprise the value of a loss function, the network precision and the like, when the value of the evaluation index of the intermediate flood extraction convolutional neural network is determined to reach a preset standard value, the accumulated training round number corresponding to the intermediate flood extraction convolutional neural network is used as a preset round number threshold value, and when the value of the evaluation index of the intermediate flood extraction convolutional neural network is determined not to reach the preset standard value, the intermediate flood extraction convolutional neural network can be trained again by using the training sample until the value of the evaluation index after the verification sample is used by the trained network reaches the preset standard value.
According to the flood disaster water body extraction method provided by the invention, the purpose of accurately and reliably determining the threshold value of the preset number of rounds is realized by firstly using the training samples in the training sample image set to train a certain number of rounds of training the network and then using the verification samples in the training sample image set to verify whether the network meets the accuracy requirement, so that a powerful basis is provided for quickly and accurately obtaining the preset flood extraction convolutional neural network in the follow-up process.
In the actual treatment process, in order to verify the effectiveness of the method of the present invention, comparing it with the traditional SAR image flood monitoring method, two typical areas as shown in FIG. 6 can be selected, wherein the first water body of the area accounts for 33.4% and the second water body of the area accounts for 5.8%. Visual interpretation is carried out under the assistance of a high-resolution optical remote sensing image to obtain real water information, the real water information marked by manpower is compared with the water information obtained by an algorithm, and various evaluation indexes are adopted to quantitatively evaluate the extraction precision of the water information, so that the method provided by the invention is verified, and the method is particularly from the recall ratio, the precision ratio, the false alarm ratio and the false alarm ratioThe evaluation index is used for checking the effectiveness and reliability of the method provided by the invention, and the calculation formulas of the four indexes are as follows:
Fa=1-P (8)
wherein, P (TW) represents a real water body pixel which is interpreted by manual visual interpretation, P (AW) represents a water body pixel which is extracted by a method, R represents the recall ratio of the extraction method, the recall ratio represents the degree of the extracted water body range approaching to the real water body, the higher the value is, the better the value is, P represents the precision of the extraction method, the higher the value is, the better the value is, Fa represents the false alarm rate of the extraction method; The method comprises the method, the Otsu method (namely Otsu global threshold method) and an object-oriented method, and the extraction precision pairs of the three methods are shown in a table 1.
TABLE 1
Based on the illustration of fig. 6 and table1, the water body ratios of the selected comparison areas are different, and the advantages and disadvantages of the method of the invention and the traditional method are checked from the angles of different water body ratios. The result shows that the Otsu global threshold method has higher recall ratio and precision ratio under the condition of high water occupation ratio and better water extraction effect, has the worst water extraction precision under the condition of low water occupation ratio, has certain advantages in the aspect of fine water extraction compared with the Otsu global threshold method, but has lower extraction precision than the preset flood extraction convolutional neural network in the method, and has the worst extraction precision for the water information extraction condition of large-scale flood disasters. Among the three methods, the preset flood extraction convolutional neural network in the method has the best effect and highest precision. For large-scale macroscopic water body information extraction, the Otsu global threshold method is difficult to find a proper threshold value to extract the water body information, and a great amount of priori knowledge is required for the segmentation rule and the classification rule of the object-oriented method. The accuracy comparison analysis proves that the preset flood extraction convolutional neural network in the method has higher accuracy and better effect than the traditional flood monitoring method.
The flood disaster water body extraction device provided by the invention is described below, and the flood disaster water body extraction device described below and the flood disaster water body extraction method described above can be correspondingly referred to each other.
Fig. 7 illustrates a flood disaster water body extraction device, as shown in fig. 7, wherein the flood disaster water body extraction device 700 comprises an acquisition module 710, a determination module 720, and a determination module, wherein the acquisition module 710 is used for acquiring a water body to-be-extracted SAR image, the water body to-be-extracted SAR image comprises an image formed by a dual polarized wave band and a spectral characteristic wave band of the flood disaster SAR image, the determination module 720 is used for inputting the water body to-be-extracted SAR image into a preset flood extraction convolutional neural network to obtain a flood disaster water body information distribution map, and the preset flood extraction convolutional neural network is used for carrying out residual convolution and attention mechanism correction on the water body to-be-extracted SAR image, and carrying out cavity convolution on the corrected water body to-be-extracted SAR image to obtain the flood disaster water body information distribution map.
Optionally, the determining module 720 may be specifically configured to input the SAR image to be extracted of the water body into a preset residual neural network to perform residual convolution and attention mechanism correction to obtain a first water body information distribution diagram after the accurate correction of the edge of the water body of the flood disaster, and input the first water body information distribution diagram into a preset cavity convolution network to perform a change process of a preset expansion scale to obtain a complete flood disaster water body information distribution diagram extracted from the edge of the water body of the flood disaster.
Optionally, the determining module 720 may be further configured to input the SAR image to be extracted from the water body into a q residual sub-network for convolution, batch normalization and linear arrangement to obtain a q residual convolution image, input the q residual convolution image into a q attention mechanism correction sub-network for channel correction and space correction to obtain a q correction image, input the q correction image into a q+1 residual sub-network for convolution, batch normalization and linear arrangement to obtain a q+1 residual convolution image, input the q+1 residual convolution image into a q+1 attention mechanism correction sub-network for channel correction and space correction to obtain a q+1 correction image, add 1 to a q value, and repeatedly execute the steps of inputting the q+1 residual convolution image into the q+1 attention mechanism correction sub-network for channel correction and space correction until a P correction image is obtained, and the P correction image is an accurate flooding edge correction [ after the water body is subjected to the water body edge correction, p+1, q is subjected to the water body disaster [ e.1, q+1 ].
Optionally, the determining module 720 may be further specifically configured to input the first water body information distribution map to a preset cavity convolution network to perform different expansion scale processing to obtain Q tiny water body feature information maps with different expansion scales, perform fusion processing on the tiny water body feature information maps to obtain a fused image, and perform decoding processing corresponding to the residual convolution and the attention mechanism correction on the fused image to obtain a complete flood disaster water body information distribution map extracted from the edge of the flood disaster water body.
Optionally, the determining module 720 may be further configured to obtain a training sample image set, where the training sample image set includes a training sample image, where the training sample image is an SAR image obtained by performing sample labeling, regular grid clipping, and data enhancement processing on an SAR image to be extracted from the water body, and training a preset initial flood extraction convolutional neural network according to the training sample image set to obtain the preset flood extraction convolutional neural network.
Optionally, the determining module 720 may be further specifically configured to perform iterative training on a preset initial flood extraction convolutional neural network according to the training sample image set, obtain an intermediate flood extraction convolutional neural network obtained after the iterative training of the present round, determine whether the number of accumulated training rounds corresponding to the iterative training of the present round reaches a preset round threshold, use the intermediate flood extraction convolutional neural network obtained after the iterative training of the present round as the preset flood extraction convolutional neural network if the number of accumulated training rounds reaches the preset round threshold, and train the intermediate flood extraction convolutional neural network to obtain the preset flood extraction convolutional neural network if the number of accumulated training rounds does not reach the preset round threshold.
Optionally, the determining module 720 may be further specifically configured to divide the training sample image set into a training sample and a verification sample according to a preset proportion, train a preset initial flood extraction convolutional neural network according to the training sample, obtain an intermediate flood extraction convolutional neural network obtained after a preset number of rounds of training, verify the intermediate flood extraction convolutional neural network according to the verification sample, obtain a value of an evaluation index of the intermediate flood extraction convolutional neural network, determine whether the value of the evaluation index reaches a preset standard value, use an accumulated training round number corresponding to the intermediate flood extraction convolutional neural network as the preset round number threshold value if the value of the evaluation index reaches the preset standard value, and train the intermediate flood extraction convolutional neural network if the value of the evaluation index does not reach the preset standard value, so as to obtain the preset round number threshold value.
Fig. 8 illustrates a physical schematic diagram of an electronic device, as shown in fig. 8, the electronic device 800 may include a processor 810, a communication interface (Communications Interface) 820, a memory 830, and a communication bus 840, where the processor 810, the communication interface 820, and the memory 830 perform communication with each other through the communication bus 840. The processor 810 can call logic instructions in the memory 830 to execute a method for extracting a flood disaster water body, wherein the method comprises the steps of obtaining a water body to-be-extracted SAR image, wherein the water body to-be-extracted SAR image comprises an image formed by a dual polarized wave band and a spectral characteristic wave band of the flood disaster SAR image, inputting the water body to-be-extracted SAR image into a preset flood extraction convolutional neural network to obtain a flood disaster water body information distribution map, and the preset flood extraction convolutional neural network is used for carrying out residual convolution and attention mechanism correction on the water body to-be-extracted SAR image and carrying out cavity convolution on the corrected water body to-be-extracted SAR image to obtain the flood disaster water body information distribution map.
Further, the logic instructions in the memory 830 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. The storage medium includes a U disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, an optical disk, or other various media capable of storing program codes.
On the other hand, the invention also provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, wherein the computer program comprises program instructions, when the program instructions are executed by a computer, the computer can execute the flood disaster water body extraction method provided by the methods, the method comprises the steps of obtaining a water body to-be-extracted SAR image, wherein the water body to-be-extracted SAR image comprises an image formed by a dual polarized wave band and a spectral characteristic wave band of the flood disaster SAR image, inputting the water body to-be-extracted SAR image into a preset flood extraction convolutional neural network to obtain a flood disaster water body information distribution map, and the preset flood extraction convolutional neural network is used for carrying out residual convolution and attention mechanism correction on the water body to-be-extracted SAR image and carrying out cavity convolution on the corrected water body to-be-extracted SAR image to obtain the flood disaster water body information distribution map.
In still another aspect, the present invention further provides a non-transitory computer readable storage medium, on which a computer program is stored, the computer program being implemented when executed by a processor to perform the above-provided method for extracting a flood disaster water body, where the method includes obtaining a water body to-be-extracted SAR image, where the water body to-be-extracted SAR image includes an image composed of a dual polarized band and a spectral feature band of the flood disaster SAR image, inputting the water body to-be-extracted SAR image into a preset flood extraction convolutional neural network to obtain a flood disaster water body information distribution map, where the preset flood extraction convolutional neural network is configured to perform residual convolution and attention mechanism correction on the water body to-be-extracted SAR image, and performing hole convolution on the corrected water body to-be-extracted SAR image to obtain the flood disaster water body information distribution map.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
It should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention, and not for limiting the same, and although the present invention has been described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the technical solution described in the above-mentioned embodiments may be modified or some technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solution of the embodiments of the present invention.