Disclosure of Invention
The embodiment of the application provides a land area measurement method, an unmanned aerial vehicle and a medium based on an image segmentation network, which can effectively improve the efficiency of measuring the land area.
In a first aspect, an embodiment of the present application provides a land area measurement method based on an image segmentation network, which is applied to an unmanned aerial vehicle, and the method includes:
determining a target land area, the target land area comprising a plurality of monitoring nodes;
determining a target inspection path according to a plurality of monitoring nodes, and shooting a land image at each monitoring node according to the target inspection path, wherein the monitoring nodes are position nodes corresponding to image center points of the land image;
Acquiring target distance information, wherein the target distance information is the field distance between a preset reference object and the monitoring node;
Acquiring the number of target pixels, wherein the number of target pixels is the number of pixels of the reference object from the image center point of the land image to the reference point corresponding to the land image;
determining a target GSD according to the target distance information and the target pixel number;
Inputting the land image into a pre-trained image segmentation network model for image segmentation processing to obtain a target segmentation image;
And obtaining the land area of the target land area according to the target segmentation image and the target GSD.
In some embodiments, the image segmentation network model is trained according to the following steps:
determining the position information of the land image, and acquiring a map remote sensing image corresponding to the position information, wherein the image size of the land image is the same as that of the map remote sensing image;
acquiring a first annotation image corresponding to the land image and acquiring a second annotation image corresponding to the map remote sensing image;
inputting the land image, the map remote sensing image, the first annotation image and the second annotation image into a preset cross-modal alignment network model for image alignment processing to obtain a target training image;
And acquiring an initial image segmentation network model, and training the initial image segmentation network model according to the target training image to obtain the image segmentation network model.
In some embodiments, the cross-modal alignment network model includes a first encoder, a second encoder, a third encoder, a fourth encoder, and a first decoder, and the inputting the land image, the map remote sensing image, the first labeling image, and the second labeling image to a preset cross-modal alignment network model performs image alignment processing to obtain a target training image includes:
inputting the first marked image and the second marked image to the first encoder for encoding processing to obtain a first feature vector;
Inputting the map remote sensing image to the second encoder for encoding processing to obtain a second feature vector;
inputting the land image to the third encoder for encoding processing to obtain a third feature vector;
Inputting the first feature vector, the second feature vector and the third feature vector into the fourth encoder respectively for encoding processing to obtain a new first feature vector, a new second feature vector and a new third feature vector;
Performing feature alignment processing on the new first feature vector and the new second feature vector to obtain first intermediate data;
Performing feature alignment processing on the new second feature vector and the new third feature vector to obtain second intermediate data;
Respectively inputting the first intermediate data and the second intermediate data to the first decoder for decoding processing to obtain an aligned target labeling image and a target land image;
and carrying out image merging processing on the target annotation image and the target soil image to obtain the target training image.
In some embodiments, the image segmentation network model includes a fifth encoder and a second decoder, the fifth encoder includes a feature extraction network and a first convolution layer, the second decoder includes a second convolution layer, a third convolution layer, a first upsampling layer, and a second upsampling layer, the inputting the land image into a pre-trained image segmentation network model for image segmentation processing, to obtain a target segmented image, including:
inputting the land image into the feature extraction network to perform feature extraction to obtain a fourth feature vector;
Inputting the fourth feature vector into the first convolution layer to perform convolution operation to obtain a fifth feature vector, and inputting the fifth feature vector into the first upsampling layer to perform upsampling to obtain a sixth feature vector;
Inputting the fourth feature vector into the second convolution layer to perform convolution operation to obtain a seventh feature vector;
Performing feature stitching on the sixth feature vector and the seventh feature vector to obtain an eighth feature vector;
and sequentially inputting the eighth feature vector to the third convolution layer and the second upsampling layer to obtain the target segmentation image.
In some embodiments, the target segmented image includes a plurality of map patches of a land type, the deriving a land area of the target land area from the target segmented image and the target GSD includes:
determining the number of pixels of each map patch;
And determining the product of the pixel number of each map patch and the target GSD as the land area of the land type corresponding to the map patch.
In some embodiments, the image segmentation network-based land area measurement method further comprises:
calculating longitude and latitude information of each graph plaque according to a preset longitude and latitude conversion rule;
calculating the graph size information of each graph plaque according to a preset edge detection algorithm;
and storing the longitude and latitude information and the graphic size information into a preset database.
In some embodiments, the image segmentation network model is trained from a cross entropy loss function.
In a second aspect, an embodiment of the present application provides a unmanned aerial vehicle, including:
The system comprises a target land area determining module, a monitoring module and a control module, wherein the target land area determining module is used for determining a target land area, and the target land area comprises a plurality of monitoring nodes;
The land image acquisition module is used for determining a target inspection path according to a plurality of monitoring nodes and shooting land images at each monitoring node according to the target inspection path, wherein the monitoring nodes are position nodes corresponding to image center points of the land images;
The target distance information acquisition module is used for acquiring target distance information, wherein the target distance information is the field distance between a preset reference object and the monitoring node;
The target pixel number acquisition module is used for acquiring the target pixel number, wherein the target pixel number is the number of pixels of the reference object at the position, corresponding to the land image, of the reference point, which is far from the image center point of the land image;
the target GSD acquisition module is used for determining a target ground sampling distance value GSD according to the target distance information and the target pixel number;
The target segmentation image acquisition module is used for inputting the land image into a pre-trained image segmentation network model to perform image segmentation processing to obtain a target segmentation image;
and the land area acquisition module is used for acquiring the land area of the target land area according to the target segmentation image and the target GSD.
In a third aspect, an embodiment of the present application provides a drone, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the image segmentation network-based land area measurement method according to the first aspect when the computer program is executed.
In a third aspect, embodiments of the present application also provide a computer-readable storage medium storing computer-executable instructions for performing the land area measurement method based on the image segmentation network according to the first aspect.
The embodiment of the application provides a land area measurement method, an unmanned aerial vehicle and a medium based on an image segmentation network, wherein the method comprises the steps of determining a target land area, determining a target routing path according to the plurality of monitoring nodes, shooting a land image at each monitoring node according to the target routing path, wherein the monitoring nodes are position nodes corresponding to image center points of the land image, acquiring target distance information, wherein the target distance information is a preset land distance between a reference object and the monitoring nodes, acquiring target pixel number, wherein the target pixel number is the pixel number of the reference object, corresponding to the land image, of the reference object from the image center point of the land image, determining a target ground sampling distance value D according to the target distance information and the target pixel number, inputting the land image into a pre-trained image segmentation network model for image segmentation processing, and obtaining a target segmentation image, and obtaining the land area of the target land area according to the target segmentation image and the target GSD. According to the scheme provided by the embodiment of the application, the land area of the target land area can be measured by combining the unmanned aerial vehicle and the image segmentation network model, so that the land area measuring efficiency is effectively improved.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
It will be appreciated that although functional block diagrams are depicted in the device diagrams, logical sequences are shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the block diagrams in the device. The terms first, second and the like in the description, in the claims and in the above-described figures, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order.
The application provides a land area measurement method, an unmanned aerial vehicle and a medium based on an image segmentation network, wherein the method comprises the steps of determining a target land area, determining a target routing path according to the plurality of monitoring nodes, shooting a land image at each monitoring node according to the target routing path, acquiring target distance information, wherein the target distance information is a position node corresponding to an image center point of the land image, acquiring target distance information, the target distance information is a field distance between a preset reference object and the monitoring nodes, acquiring target pixel number, the target pixel number is a pixel number of the reference object, corresponding to the ground image, of the reference object, from the image center point of the land image, determining a target ground sampling distance value GSD according to the target distance information and the target pixel number, inputting the ground image into a pre-trained image segmentation network model for image segmentation processing, and acquiring a target segmentation image according to the target segmentation image and the target GSD, and acquiring the land area of the target land area. According to the scheme provided by the embodiment of the application, the land area of the target land area can be measured by combining the unmanned aerial vehicle and the image segmentation network model, so that the land area measuring efficiency is effectively improved.
Embodiments of the present application will be further described below with reference to the accompanying drawings.
As shown in fig. 1, fig. 1 is a flowchart illustrating steps of a land area measurement method based on an image segmentation network according to an embodiment of the present application, and the embodiment of the present application provides a land area measurement method based on an image segmentation network, which is applied to an unmanned aerial vehicle, and includes, but is not limited to, the following steps:
Step S110, determining a target land area, wherein the target land area comprises a plurality of monitoring nodes;
Step S120, determining a target inspection path according to a plurality of monitoring nodes, and shooting land images at each monitoring node according to the target inspection path, wherein the monitoring nodes are position nodes corresponding to image center points of the land images;
step S130, obtaining target distance information, wherein the target distance information is the field distance between a preset reference object and a monitoring node;
Step S140, obtaining the number of target pixels, wherein the number of target pixels is the number of pixels of a reference point corresponding to a reference object in a land image, which is far from the image center point of the land image;
Step S150, determining a target GSD according to the target distance information and the target pixel number;
Step S160, inputting the land image into a pre-trained image segmentation network model for image segmentation processing to obtain a target segmentation image;
step S170, obtaining the land area of the target land area according to the target segmentation image and the target GSD.
It should be noted that, unmanned aerial vehicle of this embodiment can adopt unmanned aerial vehicle of arbitrary model, can ensure that duration is enough accomplished patrol and examine can to in order to obtain the land image, can carry on remote sensing equipment in unmanned aerial vehicle, this embodiment does not improve specific hardware structure, can realize the function can.
It should be noted that, the embodiment of the present application does not limit a specific method for determining the target routing inspection path, and may be obtained by performing path planning on a plurality of monitoring nodes in the target land area through a genetic algorithm.
The specific image type of the land image is not limited in this embodiment, and the land image in this embodiment may be an orthographic image or an oblique image, and may be selected by those skilled in the art according to actual needs.
It should be noted that, the embodiments of the present application are not limited to the specific number of the reference objects, and those skilled in the art may determine the reference objects according to actual requirements.
It should be noted that, the specific implementation manner of determining the target GSD according to the target distance information and the target pixel number in the embodiment of the present application may be obtained according to the following formula:
GSD＝γ4×σ14+γ3×σ33+γ2×σ22+γ1×σ1+γ0;
Where i is the number of reference objects, i e (1, 2,., N), σi is a coefficient factor, σi＝hi/mi,hi is target distance information, mi is the number of target pixels, and γ is a preset coefficient, it can be understood that acquiring the target GSD can provide an effective data basis for acquiring the land area of the target land area.
It can be understood that the target land area can be set according to actual requirements, after the unmanned aerial vehicle determines a target inspection path and receives an inspection instruction, the target land area can be measured according to the target inspection path and each monitoring node with a preset height in the target land area, the land image is input into an image segmentation network model trained in advance of the unmanned aerial vehicle to be subjected to image segmentation processing, a target segmentation image is obtained, and the land area of the target land area is obtained according to the target GSD and the target segmentation image.
In addition, referring to fig. 2, in an embodiment, the image segmentation network model in the embodiment shown in fig. 1 is trained according to the following steps:
Step S210, determining the position information of a land image, and acquiring a map remote sensing image corresponding to the position information, wherein the image size of the land image is the same as that of the map remote sensing image;
Step S220, a first annotation image corresponding to the land image is obtained, and a second annotation image corresponding to the map remote sensing image is obtained;
Step S230, inputting a land image, a map remote sensing image, a first annotation image and a second annotation image into a preset cross-modal alignment network model for image alignment processing to obtain a target training image;
Step S240, an initial image segmentation network model is obtained, and training is carried out on the initial image segmentation network model according to the target training image, so that the image segmentation network model is obtained.
It should be noted that, the embodiment of the application is not limited to a specific way of determining the position information of the land image and acquiring the map remote sensing image corresponding to the position information, and may be that the unmanned aerial vehicle acquires the position information of the current unmanned aerial vehicle by calling the API interface of the map navigation software while shooting the land image, and acquires the map remote sensing image corresponding to the position information and consistent with the size of the land image.
It should be noted that, the embodiment of the present application is not limited to the land type labeling information contained in the first labeling image and the second labeling image, and the land type labeling information may include labeling information of a home, a cultivated land, a forest land, a fish pond, or the like.
It can be understood that the image segmentation network model obtained by training the initial image segmentation network model according to the land image, the map remote sensing image, the first annotation image and the second annotation image can effectively improve the accuracy of image segmentation, so that the data availability of the land area is improved.
In addition, referring to fig. 3, in an embodiment, the cross-modal alignment network model includes a first encoder, a second encoder, a third encoder, a fourth encoder, and a first decoder, and step S230 in the embodiment shown in fig. 2 further includes, but is not limited to, the following steps:
step S310, inputting the first marked image and the second marked image into a first encoder for encoding processing to obtain a first feature vector;
Step S320, inputting the map remote sensing image to a second encoder for encoding processing to obtain a second feature vector;
Step S330, inputting the land image to a third encoder for encoding processing to obtain a third feature vector;
Step S340, the first feature vector, the second feature vector and the third feature vector are respectively input to a fourth encoder for encoding processing, so as to obtain a new first feature vector, a new second feature vector and a new third feature vector;
Step S350, performing feature alignment processing on the new first feature vector and the new second feature vector to obtain first intermediate data;
step S360, performing feature alignment processing on the new second feature vector and the new third feature vector to obtain second intermediate data;
Step S370, the first intermediate data and the second intermediate data are respectively input to a first decoder for decoding processing, and an aligned target labeling image and a target land image are obtained;
And step S380, carrying out image merging processing on the target annotation image and the target soil image to obtain a target training image.
It should be noted that, referring to fig. 7, a specific structure of a cross-modal alignment network model in an embodiment of the present application may be described below as a specific example, where the specific steps for acquiring a target training image in this embodiment are as follows: the cross-modal alignment network model shown in FIG. 7 includes a first encoder, a second encoder, a third encoder, The fourth encoder and the first decoder input the first labeling image and the second labeling image to the first encoder for encoding processing to obtain a first feature vector Ai, input the map remote sensing image to the second encoder for encoding processing to obtain a second feature vector Bi, input the land image to the third encoder for encoding processing to obtain a third feature vector Ci, i E (1, 2, the..once, S) which is the total number of Ai、Bi、Ci, and encode the first feature vector Ai, the second characteristic vector Bi and the third characteristic vector Ci are respectively input into a fourth encoder for encoding processing to obtain a new first characteristic vector ai, The method comprises the steps of obtaining a new second feature vector bi and a new third feature vector ci, carrying out feature alignment processing on the new first feature vector and the new second feature vector to obtain first intermediate data, carrying out feature alignment processing on the new second feature vector and the new third feature vector to obtain second intermediate data, respectively inputting the first intermediate data and the second intermediate data into a first decoder to carry out decoding processing to obtain an aligned target labeling image and a target land image, carrying out image merging processing on the target labeling image and the target land image to obtain a target training image, and carrying out feature alignment processing between the new first feature vector and the new second feature vector and between the new second feature vector and the new third feature vector by combining a preset alignment loss function to improve the accuracy of the first intermediate data and the second intermediate data, wherein the expression of the alignment loss function is as follows:
Lmatch＝Lalign+Lreson;
Wherein, the expression of Lalign is:
Lalign＝α×Lalign1+β×Lalign2;
wherein, the expression of Lalign1,Lalign2 is respectively:
Lalign1＝max(1+Dis(ai,bi)-Dis(ai,bj),0);
Lalign2＝max(1+Dis(ai,bi)-Dis(ai,bj),0);
Wherein, the expression of Lreson is:
Lreson＝α×Lreson1+β×Lreson2;
Wherein, the expression of Lreson1,Lreson2 is respectively:
Wherein, theTo annotate the image for the aligned object,For the aligned target earth image, alpha and beta are coefficient factors, so that two alignment weights can be balanced, and a specific numerical value can be determined according to actual conditions, and the value of the embodiment of the application is 0.5.
It will be appreciated that acquiring the target training image can provide an effective data basis for training the initial image segmentation network model.
In addition, in an embodiment, the image segmentation network model includes a fifth encoder and a second decoder, the fifth encoder includes a feature extraction network and a first convolution layer, the second decoder includes a second convolution layer, a third convolution layer, a first upsampling layer and a second upsampling layer, referring to fig. 4, step S160 in the embodiment shown in fig. 1 further includes, but is not limited to, the following steps:
Step S410, inputting the land image into a feature extraction network to perform feature extraction to obtain a fourth feature vector;
Step S420, inputting the fourth feature vector into the first convolution layer for convolution operation to obtain a fifth feature vector, and inputting the fifth feature vector into the first upsampling layer for upsampling to obtain a sixth feature vector;
step S430, inputting the fourth feature vector into the second convolution layer for convolution operation to obtain a seventh feature vector;
Step S440, performing feature stitching processing on the sixth feature vector and the seventh feature vector to obtain an eighth feature vector;
and S450, sequentially inputting the eighth feature vector to the third convolution layer and the second upsampling layer to obtain the target segmentation image.
It should be noted that, referring to fig. 8, the specific structure of the image segmentation network model in the embodiment of the present application may refer to fig. 8, where the image segmentation network model includes a fifth encoder and a second decoder, the fifth encoder includes a feature extraction network and a first convolution layer, the second decoder includes a second convolution layer, a third convolution layer, a first upsampling layer and a second upsampling layer, and the embodiment of the present application is not limited to the specific structure of each network layer in the image segmentation network model, for example, the feature extraction network may include a convolution layer with a convolution kernel of 1×1, a convolution layer with a three-layer convolution kernel of 3×3 and a hole rate of 6, and a pooling layer. For example, the convolution kernels of the first convolution layer and the second convolution layer are 1×1, the convolution kernel of the third convolution layer is 3×3, and the up-sampling times of the first up-sampling layer and the second up-sampling layer are 4.
In order to improve the accuracy of obtaining the target segmented image by the image segmentation network model, the embodiment of the application trains the image segmentation network model through a cross entropy loss function, and the cross entropy loss function has the following formula:
Wherein C is an identification of a land type, p epsilon (p0,…,pC-1) is a probability distribution, and each element pi represents a probability that each graph patch in the target segmented image belongs to an i-th land type. y e (y0,…,yC-1) is onehot that the target labeling image belongs to, when the graph patch in the target segmented image belongs to the ith class, yi =1, otherwise yi =0.
It will be appreciated that acquiring the target segmented image can provide an effective data basis for acquiring the land area of the target land area.
In addition, referring to fig. 5, in an embodiment, the target segmentation image includes a plurality of land-type map patches, and step S170 in the embodiment shown in fig. 1 further includes, but is not limited to, the following steps:
Step S510, determining the pixel quantity of each map patch;
In step S520, the product between the number of pixels of each map patch and the target GSD is determined as the land area of the land type corresponding to the map patch.
It can be understood that by determining the number of pixels of each of the map tiles of different land types in the target segmented image, determining the product between the number of pixels of each map tile and the target GSD as the land area of the land type corresponding to the map tile, the intelligent calculation of the land area can be realized, and compared with the technical scheme of manual actual measurement, the efficiency of measuring the land area is improved.
In addition, in an embodiment, referring to fig. 6, the land area measurement method based on the image segmentation network according to the embodiment of the present application further includes, but is not limited to, the following steps:
Step S610, calculating longitude and latitude information of each graph plaque according to a preset longitude and latitude conversion rule;
step S620, calculating the graph size information of each graph plaque according to a preset edge detection algorithm;
Step S630, storing the longitude and latitude information and the graphic size information into a preset database.
It should be noted that, the embodiment of the present application is not limited to the file storage format when storing the latitude and longitude information and the graphic size information into the preset database, and may be a vector file format, a raster file format, or a compressed raster file format corresponding to the geographic information system GIS, which can be adjusted by those skilled in the art according to actual needs.
It can be understood that longitude and latitude information of each graph patch is calculated according to a preset longitude and latitude conversion rule, graph size information of each graph patch is calculated according to a preset edge detection algorithm, and the longitude and latitude information and the graph size information are stored in a preset database, so that a land planning related worker can conveniently read the information, and an effective data base is provided for subsequent land planning work.
In addition, referring to fig. 9, fig. 9 is a schematic block diagram of a drone according to another embodiment of the present application, in an embodiment, a drone 900 is provided, where the drone 900 includes:
a target land area determination module 910, the target land area determination module 910 configured to determine a target land area, the target land area including a plurality of monitoring nodes;
the land image acquisition module 920 is configured to determine a target inspection path according to a plurality of monitoring nodes, and shoot a land image at each monitoring node according to the target inspection path, where the monitoring node is a position node corresponding to an image center point of the land image;
The target distance information acquisition module 930, the target distance information acquisition module 930 is configured to acquire target distance information, where the target distance information is a field distance between a preset reference object and a monitoring node;
The target pixel number acquisition module 940 is configured to acquire a target pixel number, where the target pixel number is a number of pixels of the reference object at a reference point corresponding to the land image from an image center point of the land image;
The target GSD acquisition module 950, where the target GSD acquisition module 950 is configured to determine a target ground sampling distance value GSD according to the target distance information and the target pixel number;
the target segmentation image acquisition module 960 is used for inputting the land image into a pre-trained image segmentation network model for image segmentation processing to obtain a target segmentation image;
the land area acquisition module 970, the land area acquisition module 970 is configured to obtain the land area of the target land area according to the target segmentation image and the target GSD.
The specific implementation manner of the unmanned aerial vehicle for obtaining the land area of the target land area is basically the same as the specific embodiment of the land area measurement method based on the image segmentation network, and is not repeated here.
Referring additionally to fig. 10, fig. 10 is a block diagram of a drone according to another embodiment of the present application, and one embodiment of the present application further provides a drone 1000, where the drone 1000 includes a memory 1010, a processor 1020, and a computer program stored on the memory 1010 and executable on the processor 1020.
The processor 1020 and the memory 1010 may be connected by a bus or other means.
The non-transitory software programs and instructions required to implement the image segmentation network-based land area measurement method of the above-described embodiments are stored in the memory 1010, and when executed by the processor 1020, the image segmentation network-based land area measurement method applied to the unmanned aerial vehicle 1000 in the above-described embodiments is performed, for example, the method steps S110 to S170 in fig. 1, the method steps S210 to S240 in fig. 2, the method steps S310 to S380 in fig. 3, the method steps S410 to S450 in fig. 4, the method steps S510 to S520 in fig. 5, and the method steps S610 to S630 in fig. 6 described above are performed.
The above described apparatus embodiments are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, i.e. may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
Furthermore, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions that are executed by a processor or controller, for example, by one of the processors in the embodiment of the unmanned aerial vehicle 1000, and that may cause the processor to perform the land area measurement method based on the image segmentation network applied to the unmanned aerial vehicle in the embodiment described above, for example, to perform the method steps S110 to S170 in fig. 1, the method steps S210 to S240 in fig. 2, the method steps S310 to S380 in fig. 3, the method steps S410 to S450 in fig. 4, the method steps S510 to S520 in fig. 5, and the method steps S610 to S630 in fig. 6 described above. Those of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.