Disclosure of Invention
The embodiment of the application provides an unmanned aerial vehicle identification method and system for low-altitude security, which are used for solving the problem of poor precision in the prior art.
In a first aspect, an embodiment of the present application provides an unmanned aerial vehicle identification method for low-altitude security, including:
generating an optical compensation parameter set optimized for unmanned aerial vehicle imaging in real time based on a joint optimization model of the environment light intensity change rate and the background motion vector field in the low-altitude security scene;
Acquiring an original optical sequence in a target capture window through the optical compensation parameter set, and constructing a time domain deconvolution kernel by combining the unmanned aerial vehicle motion characteristic to generate an enhanced optical image with the motion blur resistance characteristic;
Extracting anti-sample defense features from the enhanced optical image, and performing nonlinear weighted fusion on the anti-sample defense features by constructing a disturbance intensity evaluation function to form an anti-disturbance feature mask;
Performing airspace superposition operation on the enhanced optical image and the disturbance-resisting feature mask, and performing unmanned aerial vehicle target confidence estimation on the superimposed image by adopting a multi-scale residual error network based on an attention mechanism;
and generating an unmanned aerial vehicle identification result with environmental interference resistance and attack resistance robustness according to the target confidence estimation result.
Optionally, generating, in real time, an optical compensation parameter set optimized for unmanned aerial vehicle imaging based on a joint optimization model of an environmental light intensity change rate and a background motion vector field in a low-altitude security scene includes:
Acquiring environment light intensity data in a low-altitude security scene in real time through a multichannel light intensity sensor, analyzing and extracting frequency domain features of the light intensity change rate based on a time sequence, and constructing a light intensity change rate function model;
Performing multi-scale pyramid decomposition on the light intensity change rate function model, and calculating an initial motion vector of sparse feature points on each layer of pyramid to form a background motion vector field;
Coupling the light intensity change rate function model with a background motion vector field, establishing a joint optimization equation taking light intensity smooth constraint and motion vector field energy minimization as objective functions, and iteratively solving the joint optimization equation by adopting an alternate direction multiplier method until the joint optimization equation converges to a stable state;
And dynamically adjusting the exposure time, the gain coefficient and the filtering threshold of the optical compensation module according to the output parameters of the joint optimization equation to generate an optical compensation parameter set optimized for unmanned aerial vehicle imaging.
Optionally, the original optical sequence is acquired in the target capturing window through the optical compensation parameter set, and a time domain deconvolution kernel is constructed by combining the unmanned aerial vehicle motion characteristic, so as to generate an enhanced optical image with the motion blur resistance characteristic, which comprises the following steps:
Based on the optical compensation parameter set, carrying out channel-division strengthening treatment on N frames of optical signals continuously collected in a target capture window, wherein after each frame of optical signals are separated according to RGB three channels, point multiplication operation is carried out on each frame of optical signals and gain coefficient matrixes of corresponding channels respectively, and an original optical sequence is generated;
Performing time domain feature modeling on the original optical sequence, extracting brightness gradient features and chromaticity related features of each pixel point in the time dimension through a multi-scale expansion convolution kernel, and outputting feature tensors with space-time relevance;
Constructing a time domain deconvolution kernel based on the characteristic tensor and the unmanned aerial vehicle motion characteristic, constructing a motion consistency constraint equation by minimizing an optical flow residual error between adjacent frames, introducing unmanned aerial vehicle motion track smoothness constraint, and solving by adopting a constraint optimization algorithm to obtain a pixel-level time domain deconvolution kernel matrix;
And performing space-time domain joint deconvolution operation on the time domain deconvolution kernel and the original optical sequence, wherein each spatial position is respectively subjected to back projection calculation along a time axis, and the phase response characteristic of the deconvolution kernel is adjusted through an iterative updating strategy to generate an enhanced optical image with the motion blur resistance characteristic.
Optionally, coupling the light intensity change rate function model with a background motion vector field, establishing a joint optimization equation taking light intensity smooth constraint and motion vector field energy minimization as objective functions, and iteratively solving the joint optimization equation by adopting an alternate direction multiplier method until the joint optimization equation converges to a stable state, including:
Designing coupling term parameters, and fusing light intensity smooth constraint of a light intensity change rate function model with a background motion vector field through weighting factors to form a joint optimization equation;
decomposing the joint optimization equation into a first sub-problem of constraining the intensity gradient between adjacent pixels by introducing an intensity smoothing regularization term of the intensity change rate function model and a second sub-problem of constraining the spatial continuity of motion vectors by introducing an energy term of a back-shadow motion vector field;
Respectively constructing an augmented Lagrangian function for the first sub-problem and the second sub-problem, and introducing a Lagrangian multiplication sub-term and a quadratic punishment term of linear constraint into an objective function;
after each variable is updated, synchronously updating the augmented Lagrangian multiplier, and dynamically adjusting the weight coefficient of the secondary penalty term through a self-adaptive step length adjustment strategy;
And iteratively solving the joint optimization equation by adopting an alternate direction multiplier method based on the secondary penalty term, calculating relative error norms in two adjacent iterations, and judging that the joint optimization equation is converged to a stable state when the relative error norms are smaller than a preset threshold value.
Optionally, constructing a time domain deconvolution kernel based on the feature tensor and the unmanned aerial vehicle motion characteristic, constructing a motion consistency constraint equation by minimizing an optical flow residual error between adjacent frames, introducing the unmanned aerial vehicle motion track smoothness constraint, and solving by adopting a constraint optimization algorithm to obtain a pixel-level time domain deconvolution kernel matrix, including:
Based on the multi-scale space-time correlation characteristics of the characteristic tensor, extracting space-time local contrast characteristics, generating an initial parameter set of a dynamic convolution kernel, carrying out motion track prior correction on the initial parameter set by combining high-speed translation, hover shake and rotor wing periodic motion modes in the motion characteristics of the unmanned aerial vehicle, and mapping the initial parameter set to a weight space of an deconvolution kernel to form a time domain deconvolution kernel;
Performing space-time joint analysis on the time domain deconvolution kernel and a minimized adjacent inter-frame optical flow residual function, introducing unmanned aerial vehicle motion track smoothness constraint, and constructing a motion consistency constraint equation taking a pixel displacement vector as a variable, wherein the unmanned aerial vehicle motion track smoothness constraint comprises unmanned aerial vehicle acceleration upper limit and heading continuity conditions;
And (3) carrying out space domain and time domain optimization through iteration of a constraint optimization algorithm, and adjusting weight distribution of an optical flow residual error function according to smoothness constraint of the unmanned aerial vehicle motion track in each iteration until the mean square error of the optical flow residual error converges to a preset precision range, and outputting a pixel-level time domain deconvolution kernel matrix meeting the motion consistency constraint.
Optionally, performing time domain feature modeling on the original optical sequence, extracting luminance gradient features and chromaticity related features of each pixel point in a time dimension through a multi-scale expansion convolution kernel, and outputting feature tensors with space-time relevance, including:
Decomposing the time domain characteristics of an original optical sequence into a brightness gradient component of a pixel-by-pixel brightness change matrix obtained through adjacent inter-frame difference operation and a chromaticity correlation component for describing cross-frame chromaticity correlation by adopting a covariance matrix of a chromaticity channel;
performing a depth separable convolution operation on the luminance gradient component and the chrominance associated component through each branch of the multi-scale dilation convolution kernel, respectively;
and performing cross-scale correlation modeling on the multi-branch output characteristics based on the depth separable convolution operation, aligning convolution results with different expansion rates according to time dimension, and outputting characteristic tensors with space-time correlation.
Optionally, extracting the anti-sample defense feature from the enhanced optical image, and performing nonlinear weighted fusion on the anti-sample defense feature by constructing a disturbance intensity evaluation function to form an anti-disturbance feature mask, including:
Performing multi-scale spatial filtering processing on the enhanced optical image, and generating a denoised standardized optical image by combining an adaptive noise suppression algorithm with local contrast enhancement operation;
Feature extraction is carried out based on the denoised standardized optical image, high-frequency texture features and low-frequency structural features of a multi-rotor structure and metal reflection characteristics of the unmanned aerial vehicle are screened through a cross-channel attention mechanism, and an anti-sample defense feature matrix is constructed, wherein the attention mechanism focuses on key parts of the rotor and the fuselage of the unmanned aerial vehicle;
Dynamically calculating weight coefficients of defense characteristic channels of each countermeasure sample by using a disturbance intensity evaluation function, and adjusting weight distribution by adopting a layer-by-layer reverse gradient accumulation strategy to realize nonlinear weighted fusion of characteristic spaces;
And outputting an anti-disturbance feature mask by iteratively optimizing the semantic consistency of the mask boundary and the original image of the anti-sample defense feature matrix after nonlinear weighted fusion.
Optionally, performing spatial domain superposition operation on the enhanced optical image and the disturbance-countermeasure feature mask, and performing unmanned aerial vehicle target confidence estimation on the superimposed image by adopting a multi-scale residual error network based on an attention mechanism, including:
Normalizing the disturbance resisting feature mask to enable the numerical range of the disturbance resisting feature mask to be matched with the pixel distribution of the enhanced optical image, and generating a superimposed image through airspace superposition operation;
Constructing a multi-scale processing module, performing multi-scale residual error network operation based on an attention mechanism on the superimposed image, and generating a multi-scale fusion feature tensor;
inputting the multiscale fusion feature tensor into a confidence estimation unit, executing local feature statistic calculation, and generating an unmanned aerial vehicle target confidence estimation value.
Optionally, generating the unmanned aerial vehicle recognition result with the environmental interference resistance and the attack resistance robustness according to the unmanned aerial vehicle target confidence estimation result includes:
based on the target confidence estimation result of the unmanned aerial vehicle, calculating an environment interference intensity index in real time by adopting a sliding window mechanism, and starting an adaptive threshold adjustment algorithm when detecting that continuous frame confidence fluctuation exceeds a preset threshold;
performing interference pattern recognition on the confidence coefficient abnormal region through a self-adaptive threshold adjustment algorithm to generate an environment interference feature mask;
Starting an antagonistic sample detection module for a low-confidence target, and analyzing feature space disturbance sensitivity by adopting a gradient back propagation method to generate a robustness assessment coefficient;
And combining the unmanned aerial vehicle target confidence estimation result, the environment interference feature mask and the robustness assessment coefficient to generate an unmanned aerial vehicle identification result with environment interference resistance and attack resistance robustness.
In a second aspect, an embodiment of the present application provides an unmanned aerial vehicle identification system for low-altitude security, including:
The generation module is used for generating an optical compensation parameter set optimized for unmanned aerial vehicle imaging in real time based on a joint optimization model of the environment light intensity change rate and the background motion vector field in a low-altitude security scene, and is also used for acquiring an original optical sequence in a target capture window through the optical compensation parameter set, constructing a time domain deconvolution kernel by combining the unmanned aerial vehicle motion characteristic, and generating an enhanced optical image with the motion blur resistance characteristic;
the processing module is used for extracting anti-sample defense features from the enhanced optical image, and performing nonlinear weighted fusion on the anti-sample defense features by constructing a disturbance intensity evaluation function to form an anti-disturbance feature mask;
the estimation module is used for performing airspace superposition operation on the enhanced optical image and the disturbance resisting feature mask, and performing target confidence estimation on the superimposed image by adopting a multi-scale residual error network based on an attention mechanism;
The generation module is also used for generating an unmanned aerial vehicle identification result with environmental interference resistance and attack resistance robustness according to the target confidence estimation result.
According to the method, an optical compensation parameter set optimized for unmanned aerial vehicle imaging is generated in real time based on a joint optimization model of an environment light intensity change rate and a background motion vector field in a low-altitude security scene, an original optical sequence is acquired in a target capture window through the optical compensation parameter set, a time domain deconvolution kernel is built by combining unmanned aerial vehicle motion characteristics to generate an enhanced optical image with anti-motion blur characteristics, anti-sample defense features are extracted from the enhanced optical image, nonlinear weighted fusion is conducted on the anti-sample defense features through a disturbance intensity evaluation function, an anti-disturbance feature mask is formed, the enhanced optical image and the anti-disturbance feature mask are subjected to airspace superposition operation, a multi-scale residual error network based on an attention mechanism is adopted to conduct target confidence estimation on the superimposed image, and an unmanned aerial vehicle recognition result with environmental disturbance resistance and anti-attack robustness is generated according to a target confidence estimation result.
The technical scheme of the application has the following beneficial effects:
The application realizes the self-adaptive dynamic adjustment of the optical compensation parameter by fusing the light intensity change rate and the background motion vector field, and remarkably improves the imaging stability and the target capturing quality in a complex environment. And eliminating image blurring caused by target high-speed movement through a time domain deconvolution kernel, recovering detail information, and improving the definition and the intelligibility of a target area. And the anti-attack area is accurately detected and identified, the robustness of image processing and target identification is enhanced, and the false detection rate caused by the anti-attack is reduced. And through a multi-scale feature extraction and attention mechanism, the target detection precision and stability are improved, and a high-confidence recognition result in a complex scene is ensured. And combining confidence driving screening and multi-frame fusion strategies to generate high-reliability unmanned aerial vehicle identification results, and remarkably improving the environmental adaptability and anti-interference capability of the low-altitude security system.
The method comprises the steps of acquiring environment light intensity data in a low-altitude security scene in real time through a multichannel light intensity sensor, extracting frequency domain features of light intensity change rate based on time sequence analysis, constructing a light intensity change rate function model, carrying out multi-scale pyramid decomposition on the light intensity change rate function model, calculating initial motion vectors of sparse feature points on each layer of pyramid to form a background motion vector field, coupling the light intensity change rate function model with the background motion vector field, establishing a joint optimization equation with light intensity smooth constraint and motion vector field energy minimized as objective functions, adopting an alternate direction multiplier method to iteratively solve the joint optimization equation until the joint optimization equation converges to a stable state, and dynamically adjusting exposure time, gain coefficient and filtering threshold of an optical compensation module according to output parameters of the joint optimization equation to generate an optical compensation parameter set optimized for unmanned aerial vehicle imaging.
These and other aspects of the application will be more readily apparent from the following description of the embodiments.
Detailed Description
In order to enable those skilled in the art to better understand the present application, the following description will make clear and complete descriptions of the technical solutions according to the embodiments of the present application with reference to the accompanying drawings.
In some of the flows described in the specification and claims of the present application and in the foregoing figures, a plurality of operations occurring in a particular order are included, but it should be understood that the operations may be performed out of order or performed in parallel, with the order of operations such as 101, 102, etc., being merely used to distinguish between the various operations, the order of the operations themselves not representing any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types.
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.
Fig. 1 is a flowchart of an unmanned aerial vehicle identification method for low-altitude security, according to an embodiment of the present application, as shown in fig. 1, the method includes:
101. generating an optical compensation parameter set optimized for unmanned aerial vehicle imaging in real time based on a joint optimization model of the environment light intensity change rate and the background motion vector field in the low-altitude security scene;
In this step, the ambient light intensity change rate refers to the rate of time-dependent change of the illumination intensity in the low-altitude security scene, and is usually calculated by time sequence data collected by a light intensity sensor.
The background motion vector field is a vector field describing the motion direction and speed of a background object in a low-altitude security scene, is generated by an optical flow estimation or feature matching method and is used for representing dynamic interference (such as tree shaking and cloud drift) of the scene background.
The optical compensation parameter set comprises parameters such as exposure time, gain coefficient and the like, and is used for adjusting the optical system to cope with the change of the environmental light intensity and the background motion interference, and is used for inhibiting the environmental light interference and compensating imaging blurring caused by the high-speed motion of the unmanned aerial vehicle.
And acquiring light intensity data in a scene in real time through a light intensity sensor, and calculating a time difference sequence to obtain the change rate of the light intensity of the environment. And carrying out optical flow estimation on the video frame sequence, extracting a background motion vector field, and removing abnormal vectors through motion consistency clustering. And constructing a joint optimization model, and coupling the light intensity change rate with a background motion vector field, wherein an objective function comprises a light intensity smooth constraint and a motion vector field energy minimization term. And adopting an alternate direction multiplier method to iteratively solve the joint optimization model, and outputting an optical compensation parameter set generated in real time.
In a low-altitude security scene, an unmanned aerial vehicle monitoring system collects environment light intensity data through a light intensity sensor to calculate the change rate of the environment light intensity data, and meanwhile, optical flow estimation is carried out on a video frame sequence to generate a background motion vector field. Inputting the two parameters into a joint optimization model, and generating an optical compensation parameter set optimized for unmanned aerial vehicle imaging in real time, wherein the optical compensation parameter set is used for adjusting the exposure time and gain coefficient of a camera so as to cope with illumination change and background motion interference.
102. Acquiring an original optical sequence in a target capture window through the optical compensation parameter set, and constructing a time domain deconvolution kernel by combining the unmanned aerial vehicle motion characteristic to generate an enhanced optical image with the motion blur resistance characteristic;
the original optical sequence refers to a sequence of consecutive frame images acquired within a target capture window.
The temporal deconvolution kernel is a temporal filter for removing motion blur in an image sequence, recovering a sharp image by deconvolution operation.
And adjusting camera parameters by using the optical compensation parameter set, and collecting an original optical sequence in a target capture window. And constructing a time domain deconvolution kernel by combining the unmanned aerial vehicle motion characteristics (including high-speed translation, hover dithering and rotor wing periodic motion modes), estimating a motion blur track based on an optical flow field, introducing unmanned aerial vehicle motion track smoothness constraints (such as an acceleration upper limit and a heading continuity condition), and optimizing kernel function parameters through regularization constraints. Performing a temporal deconvolution operation on the original optical sequence to generate an enhanced optical image having anti-motion blur characteristics.
The unmanned aerial vehicle flies obliquely at a speed of 15m/s, and the target capture window tracks the movement of the unmanned aerial vehicle and intercepts 5 frames of blurred images. Optical flow analysis shows that the target forms a zig-zag trajectory in the time domain (maximum displacement 8 pixels/frame). And modeling a PSF (point spread function) by the deconvolution core according to the Z-shaped track by combining the motion characteristics of the unmanned aerial vehicle, and introducing the smoothness constraint optimization kernel function parameters of the motion track of the unmanned aerial vehicle. After regularized deconvolution, the target rotor wing texture definition is significantly improved (edge gradient increases from 0.3 to 0.7), and the background tree smear length is reduced by 70%.
103. Extracting anti-sample defense features based on a multi-rotor structure and metal reflection characteristics of the unmanned aerial vehicle from the enhanced optical image, and performing nonlinear weighted fusion on the anti-sample defense features by constructing a disturbance intensity evaluation function to form an anti-disturbance feature mask;
The challenge sample defense features refer to features extracted from the enhanced optical image that characterize the challenge, such as abnormal gradient distribution or local texture mutations, for detecting potential challenge regions.
The disturbance intensity evaluation function is used for quantifying the disturbance intensity of the anti-sample defense features, generating an anti-disturbance feature mask through nonlinear weighted fusion, and identifying high-risk areas.
Extracting anti-sample defense features from an enhanced optical image, including characterizing an abnormal region of the image based on periodic texture features of a multi-rotor structure of the unmanned aerial vehicle, highlight region features caused by metal reflection characteristics, gradient amplitude, local contrast and texture complexity, constructing a disturbance intensity evaluation function, calculating weight coefficients of the features based on physical characteristics (such as rotor symmetry and metal reflection intensity) and feature sensitivity of the unmanned aerial vehicle, quantifying potential risks of anti-attack, generating an anti-disturbance feature mask through nonlinear weighted fusion, identifying potential anti-attack regions in the image, and providing risk prompts for subsequent processing.
For example, after generating the enhanced optical image, the system extracts the high-light region features caused by the periodic texture features and the metal reflection features of the multi-rotor structure of the unmanned aerial vehicle, combines the gradient amplitude and the local contrast features, calculates the weight coefficient through the disturbance intensity evaluation function, generates a disturbance resisting feature mask, and identifies a possible attack resisting region in the image. For example, when an area in the image is detected with periodic anomalies in rotor texture, abrupt changes in metallic reflection intensity, and abnormally high gradient magnitudes, the system determines that the area may be vulnerable to attack and marks the area as a high risk area in the mask.
104. Performing airspace superposition operation on the enhanced optical image and the disturbance-resisting feature mask, and performing unmanned aerial vehicle target confidence estimation on the superimposed image by adopting a multi-scale residual error network based on an attention mechanism;
The spatial superposition operation refers to pixel-level fusion of the enhanced optical image and the disturbance-resistant feature mask, so as to generate a composite image containing attack-resistant information, and more comprehensive input data is provided for subsequent recognition.
The multi-scale residual error network is a deep learning model based on an attention mechanism and is used for extracting multi-scale characteristics of an image and estimating target confidence coefficient, so that reliability of an identification result is improved.
And performing airspace superposition operation on the enhanced optical image and the disturbance resisting feature mask to generate a composite image, fusing target details and attack resisting information, extracting features of the composite image by adopting a multi-scale residual error network, focusing a key region by using an attention mechanism, enhancing the significance of target features, outputting an unmanned aerial vehicle target confidence estimation result through a full connection layer, evaluating the reliability of the identification result, and providing a basis for subsequent screening.
For example, the system performs airspace superposition on the enhanced optical image and the disturbance resisting feature mask, inputs a multi-scale residual error network for feature extraction, outputs a target confidence coefficient estimation result, evaluates the reliability of unmanned aerial vehicle identification, and reduces the confidence coefficient score of a certain area in the composite image when the area is marked as a high-risk disturbance resisting area so as to avoid false identification.
105. And generating an unmanned aerial vehicle identification result with environmental interference resistance and attack resistance robustness according to the target confidence estimation result.
The target confidence estimation result refers to the reliability score of the recognition result output by the multi-scale residual error network and is used for screening the high-confidence recognition result. The identification result comprises unmanned aerial vehicle model, position and threat level information.
The anti-environmental interference and anti-attack robustness refers to the capability of the recognition system to still keep high precision under complex environment and anti-attack, and the reliability of recognition results is ensured.
And screening high-confidence recognition results according to the target confidence estimation results, removing low-confidence false recognition, improving recognition accuracy, enhancing the robustness of the system to the environment and attack resistance by fusing multi-frame recognition results, eliminating the influence of single-frame false recognition, outputting a final unmanned aerial vehicle recognition result, and ensuring the accuracy and reliability of the unmanned aerial vehicle recognition result in a complex environment.
The system screens high-confidence recognition results according to the target confidence estimation results, robustness is enhanced through multi-frame fusion, and an accurate unmanned aerial vehicle recognition result is finally output.
The method comprises the steps of 101-105, by means of joint optimization of the environment light intensity change rate and the background motion vector field, generating an optical compensation parameter set optimized for unmanned aerial vehicle imaging in real time, effectively aiming at dynamic illumination and background motion interference, eliminating motion blur through a time domain deconvolution kernel, generating a clear enhanced optical image, extracting anti-sample defense features and generating an anti-disturbance feature mask, identifying potential anti-attack areas, carrying out target confidence estimation by adopting a multi-scale residual error network, screening high-confidence identification results, generating unmanned aerial vehicle identification results with environmental interference resistance and anti-attack robustness, and remarkably improving identification accuracy and reliability in low-altitude security scenes.
In order to solve the limitation of a single parameter compensation method in a dynamic complex scene, aiming at the problem of imaging quality reduction caused by dynamic illumination intense change and complex background motion interference in a low-altitude security scene, a joint optimization model is constructed by fusing environment light intensity change rate data acquired in real time by a light intensity sensor with background motion vector field data extracted by a video frame sequence, a multi-source data coupling constraint design objective function is utilized, and a dynamic compensation parameter is solved in real time by adopting an alternating optimization algorithm, so that dynamic self-adaptive adjustment of optical system parameters is realized, and in some embodiments, an optical compensation parameter set optimized for unmanned aerial vehicle imaging is generated in real time based on the joint optimization model of the environment light intensity change rate and the background motion vector field in the low-altitude security scene, and the method comprises the following steps:
201. Acquiring environment light intensity data in a low-altitude security scene in real time through a multichannel light intensity sensor, analyzing and extracting frequency domain features of the light intensity change rate based on a time sequence, and constructing a light intensity change rate function model;
In step 201, the multi-channel light intensity sensor refers to a sensor integrated with a multi-spectrum photosensitive unit, and can synchronously collect multi-band light intensity data such as visible light, infrared light, and the like. The light intensity change rate is the light intensity fluctuation rate calculated based on the time sequence data and reflects the dynamic change characteristic of the environment light intensity. The light intensity change rate function model refers to a mathematical model constructed by frequency domain characteristics (such as dominant frequency and amplitude) and is used for quantifying the regularity of illumination change.
In the embodiment of the application, the environment light intensity data is collected by the multichannel light intensity sensor at the frequency of 100Hz, and the time sequence data is processed in a segmentation way by adopting a sliding window (the window length is 3 seconds and the overlapping rate is 50%). And performing Fast Fourier Transform (FFT) on the light intensity data in each window, extracting main frequency components (0.1 Hz-10 Hz) and amplitudes thereof, and constructing a light intensity change rate function model.
202. Performing multi-scale pyramid decomposition on the light intensity change rate function model, and calculating an initial motion vector of sparse feature points on each layer of pyramid to form a background motion vector field;
In step 202, multi-scale pyramid decomposition refers to decomposing the light intensity change rate function model into components of different spatial scales (e.g., high frequency details, low frequency trends). The sparse feature points are key points extracted through a corner detection algorithm and are used for representing stable structural features in a scene. The background motion vector field is a set of background object motion vectors calculated by an optical flow method and reflects the motion trend of a non-target object.
In the embodiment of the application, three layers of Gaussian pyramid decomposition (scale factor is 2) are carried out on the light intensity change rate function model, and a Harris corner detection algorithm (threshold is set to 0.01) is adopted on each layer of pyramid to extract sparse feature points. Then, calculating motion vectors of the feature points based on a Lucas-Kanade optical flow method, fitting a global motion model (such as affine transformation) through a RANSAC algorithm, removing abnormal vectors (such as flyers and fallen leaves) with residual errors exceeding 2 pixels, and generating a smooth background motion vector field.
203. Coupling the light intensity change rate function model with a background motion vector field, establishing a joint optimization equation taking light intensity smooth constraint and motion vector field energy minimization as objective functions, and iteratively solving the joint optimization equation by adopting an alternate direction multiplier method until the joint optimization equation converges to a stable state;
In step 203, the light intensity smoothing constraint refers to limiting the abrupt change of the light intensity rate, so as to avoid overexposure or underexposure of the image caused by severe fluctuation of illumination. Motion vector field energy minimization suppresses abnormal transitions in background motion by constraining the spatial continuity of motion vectors. The joint optimization equation refers to a multi-objective optimization model coupling light intensity and motion constraints for generating stability compensation parameters.
In the embodiment of the application, a joint optimization equation is constructed, and light intensity smooth constraint (gradient of light intensity change rate is constrained through first order difference) is combined with motion vector field energy minimization (motion continuity is ensured through total variation regularization). And decomposing the optimization problem into a light intensity compensation sub-problem and a motion compensation sub-problem by adopting an alternate direction multiplication sub-method (ADMM), and solving alternately and iteratively. In each iteration, the light sub-problem updates parameters through a gradient descent method, and the motion sub-problem optimizes the vector field through a conjugate gradient method until the residual norm is smaller than a preset threshold (such as 0.01).
204. And dynamically adjusting the exposure time, the gain coefficient and the filtering threshold of the optical compensation module according to the output parameters of the joint optimization equation to generate an optical compensation parameter set optimized for unmanned aerial vehicle imaging.
In step 204, the exposure time refers to a parameter for adjusting the exposure time of the camera, so as to balance the brightness of the image and the motion blur. The gain coefficient is a control image signal magnification for suppressing noise in a dark light environment. The filtering threshold is a parameter (e.g., median filter window size) that dynamically adjusts the image processing algorithm for noise and artifact suppression. The optical compensation parameter set is used for inhibiting ambient light interference and compensating imaging blurring caused by high-speed movement of the unmanned aerial vehicle.
In the embodiment of the application, the optical compensation module is dynamically adjusted according to the output parameters of the joint optimization equation, and the exposure time is adjusted to be proportionally prolonged (for example, from 1/1000s to 1/500 s) if the light intensity change rate is more than 100 lux/s. The gain factor is adjusted to dynamically set the gain based on the ambient light intensity average (e.g., the gain increases to 12dB at light intensity <200 lux). The filtering threshold is adjusted to adjust the median filtering window size based on the average speed of the motion vector field (e.g., a 5 x 5 window is used at a speed >5 pixels/frame). The generated set of optical compensation parameters may be adapted to the scene change in real time.
The following is a specific example:
The illumination intensity of the low-altitude security scene in the dusk time is reduced from 1000 lux to 200 lux in 5 minutes, and the background cloud layer moves at the speed of 2 m/s. Step 201, the multichannel light intensity sensor detects that the illumination intensity is reduced from 1000 lux to 200 lux, and the change rate is 160 lux/min. The system performs time sequence analysis on the light intensity data, extracts a main frequency component (0.5 Hz) through Fourier transformation, constructs a light intensity change rate function model, and quantifies illumination dynamic fluctuation. Step 202, performing three-layer pyramid decomposition on the light intensity change rate function model, and extracting sparse feature points (Harris corner detection, and setting a threshold to be 0.01) respectively. Calculating an initial motion vector by using a Lucas-Kanade optical flow method, removing abnormal values such as birds or fallen leaves by using a RANSAC algorithm, generating a smooth background motion vector field, and representing the moving trend of a cloud layer at the speed of 2 m/s. Step 203 constructs a joint optimization equation, and adopts an ADMM algorithm to carry out iterative solution by combining light intensity smoothness constraint (restraining illumination abrupt noise) and motion vector field energy minimization (guaranteeing motion continuity). Each iteration takes <10ms until the residual norm is less than the preset threshold (0.01), and the compensation parameter is output, namely the exposure time is 1/300s, and the gain is 10dB. Step 204 dynamically adjusts the optical system according to the output parameters, adjusts the exposure time from 1/1000s to 1/300s, increases the gain factor to 10dB, and simultaneously applies 3×3 median filtering to suppress noise. The standard deviation of the brightness of the compensated image is reduced from 35 to 12, the background blurring is reduced by 50%, and the definition of the target area is obviously improved.
In summary, the optical compensation parameter set optimized for unmanned aerial vehicle imaging is generated in real time through the joint optimization of the multichannel light intensity sensor and the background motion vector field, so that the problem of imaging quality reduction caused by dynamic illumination change and complex background motion interference in a low-altitude security scene is effectively solved, imaging definition and recognition stability of an unmanned aerial vehicle target are remarkably improved, and a high-quality input data basis is provided for subsequent anti-motion blur processing and anti-attack defense.
In order to solve the problem of image blurring caused by high-speed movement of a target in a low-altitude security scene, imaging parameters (such as exposure time and gain) in a target capture window are dynamically adjusted through an optical compensation parameter set to obtain an original optical sequence, a time domain deconvolution kernel is constructed based on target movement track modeling and unmanned aerial vehicle movement characteristics, motion blurring is eliminated and details are restored through deconvolution operation, the defect that the traditional fixed parameter imaging cannot adaptively compensate the motion blurring in the dynamic scene is overcome, a clear image is reversely solved through a time domain degradation model, and high-definition and anti-blurring optical input data are provided for subsequent target detection and identification.
In some embodiments, the method includes obtaining an original optical sequence in a target capture window through the optical compensation parameter set, constructing a time domain deconvolution kernel in combination with unmanned aerial vehicle motion characteristics, and generating an enhanced optical image with anti-motion blur characteristics, including:
301. Based on the optical compensation parameter set, carrying out channel-division strengthening treatment on N frames of optical signals continuously collected in a target capture window, wherein after each frame of optical signals are separated according to RGB three channels, point multiplication operation is carried out on each frame of optical signals and gain coefficient matrixes of corresponding channels respectively, and an original optical sequence is generated;
In step 301, the gain coefficient matrix is a parameter matrix designed for each of the three channels of RGB, and is used for dynamically adjusting the brightness enhancement ratio of each channel. The multi-channel enhancement process is to decompose each frame of optical signal into three independent channels of red, green and blue, and respectively perform gain adjustment to optimize color balance. The original optical sequence is a continuous frame image set generated after the channel strengthening treatment, and the time domain relevance and the space detail are reserved.
In the embodiment of the application, based on gain coefficients (such as R channel gain 1.2, G channel 1.0 and B channel 0.8) in the optical compensation parameter set, continuous 5-frame optical signals in the target capture window are subjected to channel separation processing. The specific process is to decompose each frame of image into RGB three independent channel gray scale images, to execute point multiplication operation (such as R channel pixel value x 1.2) on each channel pixel value and corresponding gain coefficient matrix, to re-combine the enhanced three channel images to generate original optical sequence. For example, in low light scenes, red light attenuation is compensated for by increasing the R-channel gain, avoiding the overall bluing of the image.
302. Performing time domain feature modeling on the original optical sequence, extracting brightness gradient features and chromaticity related features of each pixel point in the time dimension through a multi-scale expansion convolution kernel, and outputting feature tensors with space-time relevance;
In step 302, temporal feature modeling is to analyze the brightness and chromaticity change rule of pixels in continuous frames, and capture motion trajectories and illumination fluctuation features. The multi-scale expansion convolution kernel is a convolution kernel with different time spans (e.g., 3 frames, 5 frames, 7 frames) for extracting short-time abrupt changes and long-time trend features. The spatio-temporal correlation feature tensor refers to a tensor structure containing pixel luminance gradient, chromaticity correlation and time-sequential motion information.
In the embodiment of the application, time domain feature modeling is performed on an original optical sequence, a convolution kernel with 3 expansion rates (expansion rate 1 corresponds to 3 frame spans and expansion rate 2 corresponds to 5 frame spans) is adopted, the brightness gradient (adjacent frame difference value) and chromaticity relevance (RGB channel covariance) of each pixel are calculated in a sliding mode along a time axis, feature graphs with different scales are spliced into three-dimensional tensors, and the feature tensors with space-time relevance are output.
303. Constructing a time domain deconvolution kernel based on the characteristic tensor and the unmanned aerial vehicle motion characteristic, constructing a motion consistency constraint equation by minimizing an optical flow residual error between adjacent frames, introducing unmanned aerial vehicle motion track smoothness constraint, and solving by adopting a constraint optimization algorithm to obtain a pixel-level time domain deconvolution kernel matrix;
The time domain deconvolution kernel in step 303 refers to a filter that dynamically adjusts parameters based on spatio-temporal characteristics for removing motion blur. The motion consistency constraint equation ensures the continuity of the motion vectors between adjacent frames through optical flow residual minimization. The pixel-level time domain deconvolution kernel matrix is a deconvolution kernel which is independently designed for each pixel and is suitable for local motion difference.
The unmanned aerial vehicle motion characteristics comprise priori knowledge of unmanned aerial vehicle high-speed translation, hover shake and rotor periodic motion, and the priori knowledge is used for guiding optical flow residual calculation and deconvolution kernel parameter optimization.
In the embodiment of the application, dense optical flow estimation is carried out on adjacent frames, an optical flow field residual error (deviation between predicted motion and actual motion) is calculated, motion track smoothness correction is carried out on the optical flow residual error by combining unmanned aerial vehicle motion characteristics (such as high-speed translation, hover shake and rotor wing periodic motion), the optical flow residual error is minimized to serve as constraint conditions, an objective function is designed by combining space-time characteristic tensor and unmanned aerial vehicle motion characteristics, wherein the unmanned aerial vehicle motion characteristics are used for constraining weight distribution of the optical flow residual error, a constraint optimization algorithm (such as a projection gradient method) is adopted for iteratively solving a deconvolution kernel matrix, deconvolution kernel parameters are adjusted according to unmanned aerial vehicle motion track smoothness constraint in each iteration, and each pixel is independently optimized to ensure that the deconvolution kernel adapts to local motion difference.
304. And performing space-time domain joint deconvolution operation on the time domain deconvolution kernel and the original optical sequence, wherein each spatial position is respectively subjected to back projection calculation along a time axis, and the phase response characteristic of the deconvolution kernel is adjusted through an iterative updating strategy to generate an enhanced optical image with the motion blur resistance characteristic.
In step 304, the spatial-temporal joint deconvolution operation is to perform deconvolution in both the spatial and temporal dimensions simultaneously, restoring the target details. The back projection is calculated as a back tracing pixel motion trail along a time axis and correcting the fuzzy pixel value. The phase response characteristic adjustment means that phase parameters of the deconvolution kernel are dynamically optimized to match the target motion frequency.
In the embodiment of the application, the space-time domain joint deconvolution operation is executed by the time domain deconvolution kernel and the original optical sequence to generate the enhanced optical image with motion blur resistance. Firstly, back projection calculation is carried out on each space position along a time axis, and characteristic distribution of motion blur is extracted by analyzing time domain information in an optical sequence. And secondly, adjusting the phase response characteristic of the deconvolution kernel by adopting an iterative updating strategy, and optimizing the parameters of the deconvolution kernel by minimizing a reconstruction error function to ensure that the deconvolution kernel can effectively compensate motion blur. In each iteration, updating the phase response of the deconvolution kernel, and carrying out joint optimization by combining spatial domain information, so as to gradually improve the image reconstruction accuracy. And finally, generating an enhanced optical image through space-time domain joint deconvolution operation, obviously reducing the influence of motion blur, improving the definition and detail restoring capability of the image, and providing high-quality data support for subsequent analysis and processing.
In order to realize the self-adaptive dynamic adjustment of optical compensation parameters, aiming at the problem of imaging parameter mismatch caused by coupling between illumination mutation and background motion interference in a dynamic scene, a joint optimization equation is constructed by fusing an intensity change rate function model (quantized illumination fluctuation rule) and a background motion vector field (representing the motion trend of a non-target object), illumination mutation noise is restrained by introducing light intensity smooth constraint, motion continuity is ensured by combining motion vector field energy minimization, an Alternating Direction Multiplier Method (ADMM) is adopted to decompose and optimize variables and alternately iterate and solve, parameter dynamic balance is realized by residual convergence judgment, and the defect that the traditional single-dimensional compensation method has parameter conflict or slow convergence in a complex scene is overcome.
In some embodiments, coupling the light intensity change rate function model with a background motion vector field, establishing a joint optimization equation using light intensity smoothness constraint and motion vector field energy minimization as objective functions, and iteratively solving the joint optimization equation by adopting an alternate direction multiplier method until the joint optimization equation converges to a stable state, wherein the method comprises the following steps:
401. designing coupling term parameters, and fusing light intensity smooth constraint of a light intensity change rate function model with a background motion vector field through weighting factors to form a joint optimization equation;
In step 401, the coupling term parameters are used to correlate the light intensity change rate with the weighting factor of the background motion vector, balancing the influence of the two types of constraints on the optimization result. The light intensity smoothness constraint limits the gradient difference of the light intensity change rate between adjacent pixels, and avoids noise amplification caused by illumination abrupt change. The weighting factor fusion integrates the two types of constraints into a unified objective function through a preset weighting coefficient (such as 0.6 pair of optical intensity constraints and 0.4 pair of motion constraints).
In the embodiment of the application, the coupling item parameters are designed according to the physical relevance of the light intensity sensor data and the background motion vector field. And normalizing the light intensity change rate data and the motion vector field data to the same dimension (such as a 0-1 interval), dynamically adjusting a weight factor based on scene characteristics (such as when dynamic illumination is intense, the light intensity constraint weight is increased to 0.7), fusing a light intensity smoothing term and a motion energy term through the weight factor, forming a joint optimization equation, and ensuring the synergy of the two types of constraints.
402. Decomposing the joint optimization equation into a first sub-problem of constraining the intensity gradient between adjacent pixels by introducing an intensity smoothing regularization term of the intensity change rate function model and a second sub-problem of constraining the spatial continuity of motion vectors by introducing an energy term of a back-shadow motion vector field;
in step 402, a first sub-problem is an optimization problem centered on a light intensity smoothing regularization term for constraining light intensity gradient consistency between adjacent pixels. The second sub-problem is an optimization problem with motion vector field energy terms as the core, for ensuring spatial continuity of motion vectors.
In the embodiment of the application, the joint optimization equation is decomposed into two independent solving sub-problems, the motion vector field parameters are fixed, the light intensity parameters are optimized by adopting a gradient descent method, and the light intensity distribution is updated by calculating the gray gradient difference (such as 3 multiplied by 3 neighborhood) of adjacent pixels. And fixing the light intensity parameter, optimizing the motion vector field by using a conjugate gradient method, updating the motion field by restraining the direction consistency (such as cosine similarity > 0.9) of adjacent vectors, alternately iterating the two sub-problems, and transmitting an optimization result through an intermediate variable to realize global convergence.
403. Respectively constructing an augmented Lagrangian function for the first sub-problem and the second sub-problem, and introducing a Lagrangian multiplication sub-term and a quadratic punishment term of linear constraint into an objective function;
In step 403, the lagrangian function is to introduce lagrangian multipliers and quadratic penalty terms into the objective function, converting the constraint optimization problem into an unconstrained form. The linear constraint term is used for forcing the physical association of the light intensity parameter and the motion parameter, for example, the light intensity change needs to be matched with the background motion trend.
In the embodiment of the application, an extended Lagrangian function is constructed for each sub-problem, lagrangian sub-items are added in the light intensity smoothing items, the relevance of the light intensity parameters and the motion field is forced, a secondary punishment item is introduced in the motion energy item, and the relaxation degree of constraint conditions is balanced. The Lagrangian multiplier initial value is set to 0 and the quadratic penalty weight coefficient is initialized to 1.0.
404. After each variable is updated, synchronously updating the augmented Lagrangian multiplier, and dynamically adjusting the weight coefficient of the secondary penalty term through a self-adaptive step length adjustment strategy;
in step 404, the adaptive step size adjustment strategy dynamically adjusts the multiplier update step size according to the iteration residual, accelerating convergence and avoiding concussion. The secondary penalty term weight coefficient controls the parameter of the constraint condition strictness, and the larger the weight is, the stronger the constraint is.
In the embodiment of the application, parameters are synchronously updated after each iteration, the Lagrangian multiplier is updated proportionally according to the residual error of the current solution and the constraint condition (for example, the residual error is increased by 10% and the step length is increased by 20%), and if the residual error change rate of two adjacent iterations exceeds a threshold value (for example, 5%), the weight coefficient of the secondary penalty term is adaptively increased (for example, the weight coefficient is adjusted from 1.0 to 1.2). Setting the upper weight limit (e.g., 2.0) prevents excessive penalties from causing model stiffness.
405. And iteratively solving the joint optimization equation by adopting an alternate direction multiplier method based on the secondary penalty term, calculating relative error norms in two adjacent iterations, and judging that the joint optimization equation is converged to a stable state when the relative error norms are smaller than a preset threshold value.
In step 405, the alternate direction multiplier method is a decomposition-co-ordination optimization framework that achieves global optimization by solving the sub-problems alternately. The relative error norm is a difference measure of the results of two adjacent iterations and is used to determine convergence.
In embodiments of the present application, primary optimization objectives (e.g., minimizing bandwidth allocation costs) and constraint conditions (e.g., energy consumption does not exceed a threshold) are determined. And adding a quadratic penalty term into the objective function for balancing the accuracy and stability of the solution. The initial values of the main variable, the auxiliary variable and the Lagrangian multiplier are set, and a zero vector or a reasonable value based on priori knowledge is usually taken. A penalty coefficient (for controlling the intensity of the penalty term) and a convergence threshold (for determining whether the algorithm converges) are set. And fixing the auxiliary variable and the Lagrangian multiplier, and solving the sub-problem of the main variable. And fixing the main variable and Lagrangian multiplier, and solving the sub-problem of the auxiliary variable. And updating the Lagrangian multiplier according to the residual error of the main variable and the auxiliary variable, and adjusting the intensity of the penalty term. And calculating the difference value of the main variable of the two adjacent iterative solutions, and dividing the difference value by the norm of the main variable of the current iterative solution to obtain the relative error norm. If the relative error norm is less than a predetermined threshold (e.g., one part per million), the decision algorithm converges to a steady state.
The unmanned aerial vehicle flies in the sand and dust weather, the illumination intensity changes by 300 lux per minute due to sand and dust concentration fluctuation, and the background sand and dust particles move in an irregular track. The fluctuation dominant frequency is detected to be 0.6Hz (the sand concentration periodically changes) by the light intensity change rate, the motion vector field shows the average speed of sand particles to be 3m/s, the weight factors are dynamically distributed (the light intensity constraint is 0.65 and the motion constraint is 0.35), and a joint optimization equation is constructed. After the sub-problems are resolved, the first sub-problem optimizes the light intensity parameters through a gradient descent method to reduce the gradient difference of adjacent pixels by 40%, and the second sub-problem optimizes the motion field through a conjugate gradient method to improve the vector direction consistency to 0.85. An augmented Lagrangian function is constructed, the initial penalty weight is reset to 1.0, and the multiplier is initialized to 0. And dynamically adjusting the penalty weight to 1.3 according to the residual error change rate, and increasing the multiplier step length by 15%. After 12 rounds of ADMM iteration, the relative error norm drops to 0.008, and convergence is determined. The final output light intensity parameter (exposure time 1/400 s) and the motion compensation parameter (filtering threshold 5×5) reduce the image noise caused by sand interference by 60%, and improve the definition of the target contour by 55%.
In summary, the method solves the problems of convergence speed and stability of coupling optimization of light intensity and motion parameters in a dynamic scene through coupling item parameter design, sub-problem decomposition, augmented Lagrange function construction and self-adaptive ADMM optimization.
In order to eliminate image blurring caused by target high-speed motion, a time domain deconvolution kernel is constructed based on space-time correlation information (such as brightness gradient and chromaticity correlation) extracted by a feature tensor aiming at the problem of insufficient adaptability of a traditional deconvolution kernel caused by complex and changeable target motion trail in a dynamic scene, a motion consistency constraint equation is designed by minimizing optical flow residual errors between adjacent frames to ensure that the deconvolution kernel is matched with the actual target motion trail, and a constraint optimization algorithm (such as a projection gradient method) is adopted to iteratively solve a pixel-level deconvolution kernel matrix to solve the problem of performance bottleneck of a global deconvolution kernel in a local motion difference scene.
In some embodiments, a time domain deconvolution kernel is constructed based on the feature tensor and the unmanned aerial vehicle motion characteristics, a motion consistency constraint equation is constructed by minimizing an optical flow residual error between adjacent frames, and a unmanned aerial vehicle motion track smoothness constraint is introduced, and a constraint optimization algorithm is adopted to solve and obtain a pixel-level time domain deconvolution kernel matrix, which comprises:
501. Based on the multi-scale space-time correlation characteristics of the characteristic tensor, extracting space-time local contrast characteristics, generating an initial parameter set of a dynamic convolution kernel, carrying out motion track prior correction on the initial parameter set by combining high-speed translation, hover shake and rotor wing periodic motion modes in the motion characteristics of the unmanned aerial vehicle, and mapping the initial parameter set to a weight space of an deconvolution kernel to form a time domain deconvolution kernel;
in step 501, the spatio-temporal local contrast features characterize changes in detail caused by motion of the object by analyzing the difference in luminance (e.g., rate of change of pixel values of adjacent frames) of local areas of the image in both the temporal and spatial dimensions.
And the dynamic convolution kernel initial parameter set is an initial filter parameter generated based on the space-time characteristics and used for describing the fuzzy track under different motion modes.
And (3) mapping a weight space, namely mapping the initial parameters to a weight matrix of a deconvolution core through nonlinear transformation, and adapting to different movement speeds and directions.
The unmanned aerial vehicle motion characteristics comprise unmanned aerial vehicle high-speed translation, hovering shake and rotor wing periodic motion modes, and are used for carrying out motion track priori correction on an initial parameter set, so that deconvolution kernels are matched with the actual motion modes of the unmanned aerial vehicle.
In the embodiment of the application, firstly, the space-time local contrast characteristic is extracted from the characteristic tensor, the space-time contrast (the average value of the brightness difference between the current frame and the front and back frames) is calculated for the 3×3 neighborhood of each pixel, and the high-contrast area is screened (the threshold value is set to be 0.2).
And secondly, generating an initial parameter set of a dynamic convolution kernel based on the contrast characteristic, and carrying out motion track priori correction on the initial parameter set by combining high-speed translation, hover shake and rotor wing periodic motion modes in the unmanned aerial vehicle motion characteristic. For example:
a high-speed translation area, which corresponds to the short-time high-frequency kernel and is suitable for blurring caused by rapid movement;
hover shaking area, corresponding to middle frequency core, adapting to the blurring caused by tiny shaking;
and the periodic motion area of the rotor wing corresponds to the periodic kernel and is suitable for periodic blurring caused by rotation of the rotor wing.
And finally, mapping the corrected initial parameters to a weight space of the deconvolution kernel through a fully connected neural network to generate a time domain deconvolution kernel.
502. And carrying out space-time joint analysis on the time domain deconvolution kernel and the minimized optical flow residual error function between adjacent frames, introducing unmanned aerial vehicle motion track smoothness constraint, and constructing a motion consistency constraint equation taking a pixel displacement vector as a variable, wherein the unmanned aerial vehicle motion track smoothness constraint comprises unmanned aerial vehicle acceleration upper limit and heading continuity conditions.
In step 502, the optical flow residual function is a function that measures the difference between the neighboring inter-prediction motion vectors and the actual pixel displacement, and is used to evaluate the accuracy of motion estimation. The motion consistency constraint equation is an optimization equation constructed by taking the minimum of the optical flow residual as the target, and the deconvolution kernel is forced to be matched with the actual motion trail of the target. The unmanned plane motion track smoothness constraint comprises an unmanned plane acceleration upper limit and a course continuity condition, and is used for constraining an optical flow residual function and ensuring that a deconvolution kernel is matched with an actual unmanned plane motion mode.
In the embodiment of the application, first, a dense optical flow algorithm (such as Farneback algorithm) is adopted to calculate the pixel displacement vector of the adjacent frame. A residual (e.g., euclidean distance) of the predicted displacement (derived from the deconvolution kernel parameters) from the actual optical flow vector is calculated for each pixel.
And secondly, taking the sum of squares of the residual errors of the optical flows as an objective function, and constructing a constraint equation by combining the space-time local contrast characteristic and the unmanned plane motion track smoothness constraint (such as an acceleration upper limit and a heading continuity condition). For example:
The acceleration upper limit constraint is that the change rate of the optical flow residual error is limited, and abnormal residual error caused by high-speed movement of the unmanned aerial vehicle is avoided;
And the course continuity constraint ensures smooth transition of the optical flow residual error when the course of the unmanned plane changes, and avoids residual error jump caused by course mutation.
And finally, iteratively solving a motion consistency constraint equation through an optimization algorithm (such as a gradient descent method) until the optical flow residual error converges to a preset precision range.
503. And (3) carrying out space domain and time domain optimization through iteration of a constraint optimization algorithm, and adjusting weight distribution of an optical flow residual error function according to smoothness constraint of the unmanned aerial vehicle motion track in each iteration until the mean square error of the optical flow residual error converges to a preset precision range, and outputting a pixel-level time domain deconvolution kernel matrix meeting the motion consistency constraint.
In step 503, the spatial domain optimization is performed by adjusting the weight distribution of the deconvolution kernel in the image plane, enhancing the restoration capability of the local motion characteristics, and dynamically adjusting the weight attenuation coefficient of the high motion area in combination with the motion track smoothness constraint (such as the upper acceleration limit) of the unmanned plane. And optimizing a time domain, namely optimizing the response characteristic of the deconvolution kernel on a time axis, matching a periodic motion mode of a target, and adjusting the time span of the deconvolution kernel based on periodic motion characteristics such as the rotating speed of the rotor wing of the unmanned aerial vehicle.
In the embodiment of the application, firstly, initial deconvolution kernel parameters and optical flow residual function weight distribution are set, and a weight attenuation coefficient is initialized according to the motion track smoothness constraint of the unmanned plane (for example, the high-speed translation region weight attenuation coefficient is set to be 0.8).
And secondly, updating the space weight of the deconvolution kernel by adopting a projection gradient method, optimizing the weight distribution of a high-contrast area (such as the rotor edge of the unmanned aerial vehicle) preferentially, and dynamically adjusting the area weight of the residual function according to the upper limit constraint of the acceleration of the unmanned aerial vehicle. For example:
Reducing weight attenuation coefficient and inhibiting residual interference caused by abnormal movement in an acceleration overrun region;
and adding a weight penalty term in the course mutation area to avoid residual jump caused by discontinuous course.
Then, the time span of the deconvolution kernel is adjusted based on the target motion frequency (such as the rotor rotation speed of 20 Hz), and the time window length is set according to the periodic motion characteristic of the rotor (such as 5 frames corresponding to the half period of rotor rotation), so that low-frequency background interference is restrained.
And finally, stopping optimizing when the mean square error of the optical flow residual error is continuously iterated for 3 times and the change rate is less than 1%, and outputting the pixel-level time domain deconvolution kernel matrix.
The unmanned aerial vehicle flies at a speed of 20m/s in stormy weather, and the raindrops cause high-frequency noise of the background and periodic blurring is caused by rotor movement. The method comprises the steps of extracting space-time local contrast characteristics from characteristic tensors, detecting a rotor wing area contrast average value of 0.5 (threshold value of 0.2), generating an initial short-time high-frequency kernel (time span of 3 frames), generating a long-time low-frequency kernel (time span of 7 frames), calculating an optical flow residual error, displaying rotor wing area residual error as 3.2 pixels (threshold value of 2.5 pixels), constructing a motion consistency constraint equation, endowing rotor wing area residual error weight of 1.5 times, improving rotor wing area kernel weight by 30% through space domain optimization, and matching rotor wing 20Hz motion frequency through time domain optimization. After 15 iterations, the mean square error of the optical flow residual error is reduced from the initial 5.6 to 0.8, and the convergence condition is satisfied. Rotor texture clarity is increased from 0.4 (SSIM) to 0.75, with 70% reduction in raindrop noise interference. The target tracking and positioning error is reduced from 6 pixels to 1.5 pixels, so that the real-time detection requirement is met.
In conclusion, the problem of insufficient deconvolution kernel self-adaptation capability in complex dynamic scenes is solved by means of space-time local contrast characteristic extraction, motion consistency constraint equation construction and space-time joint optimization.
In order to accurately capture the rapid motion of a target, aiming at the problems that the motion speed of the target in a dynamic scene is changeable, and the space-time correlation characteristic is difficult to effectively capture by a traditional single time domain convolution kernel, the luminance gradient characteristic (representing the brightness change caused by motion) and the chromaticity correlation characteristic (reflecting the color consistency under illumination fluctuation) of pixel points are extracted in a layered manner in a time dimension by designing a multi-scale expansion convolution kernel (such as a short 3-frame span and a long 7-frame span), the local and global characteristics of different time spans are fused, a three-dimensional characteristic tensor with space-time correlation is constructed, the defect that the characteristic representation capability of the traditional method is insufficient in a complex motion mode is overcome, in some embodiments, the time domain characteristic modeling is carried out on the original optical sequence, the luminance gradient characteristic and the chromaticity correlation characteristic of each pixel point in the time dimension are extracted by the multi-scale expansion convolution kernel, and the characteristic tensor with space-time correlation is output, and the method comprises:
601. decomposing the time domain characteristics of an original optical sequence into a brightness gradient component of a pixel-by-pixel brightness change matrix obtained through adjacent inter-frame difference operation and a chromaticity correlation component for describing cross-frame chromaticity correlation by adopting a covariance matrix of a chromaticity channel;
In step 601, the pixel-by-pixel brightness change matrix is a matrix generated by calculating brightness differences of corresponding pixels of adjacent frames, and represents brightness changes caused by target motion. The chrominance correlation component describes the correlation of colors between different frames (e.g. whether the red channel variation is synchronized with the blue channel) based on the covariance matrix of the RGB channels. The time domain feature decomposition divides the original optical sequence into two independent dimensions of brightness and chromaticity, and the motion and illumination effects are analyzed respectively.
In the embodiment of the application, time domain feature decomposition is performed on an original optical sequence, for continuous 5-frame images, adjacent frame brightness difference values (such as difference values of a t frame and a t+1 frame) are calculated pixel by pixel, a 4-layer brightness change matrix is generated, RGB channel data of each frame is extracted, a cross-frame covariance matrix, such as the covariance values of a red channel and a blue channel in the continuous frames, chromaticity consistency is quantized, and brightness gradient components (4-layer matrix) and chromaticity associated components (3×3 covariance matrix) are output for subsequent multi-scale convolution processing.
602. Performing a depth separable convolution operation on the luminance gradient component and the chrominance associated component through each branch of the multi-scale dilation convolution kernel, respectively;
In step 602, the multi-scale dilation convolution kernel has convolution kernels of different time spans (e.g., dilation rate 1 for 3 frame spans and dilation rate 2 for 5 frame spans) for capturing short-term abrupt changes and long-term trends. The depth separable convolution decomposes the standard convolution into a depth convolution (channel-by-channel processing) and a point-by-point convolution (channel fusion), reduces the computational effort and improves the feature discrimination.
In the embodiment of the application, the depth separable convolution is carried out on the luminance component and the chrominance component, 3 convolution branches (expansion rates 1,2 and 3) with different expansion rates are arranged, the convolution branches respectively correspond to short-time, medium-time and long-time feature extraction, the layer-by-layer convolution is carried out on the luminance gradient component (single channel) to extract the motion features with different time spans, the independent convolution of channels is carried out on the chrominance related component (3 channels) respectively, the color correlation is reserved, and the output of each branch is fused through the channel of the 1 multiplied by 1 convolution, for example, the short-time luminance features and the long-time chrominance features are related across the channels.
603. Performing cross-scale correlation modeling on multi-branch output characteristics based on the depth separable convolution operation, aligning convolution results with different expansion rates according to time dimension, and outputting characteristic tensors with space-time correlation;
in step 603, cross-scale correlation modeling aligns and splices feature maps of different time spans in time axis, constructing a spatio-temporal correlation tensor. The feature maps with different scales have the same time length through interpolation or clipping in time dimension alignment, so that the subsequent fusion is facilitated.
In the embodiment of the application, cross-scale associated modeling is carried out on multi-branch output, linear interpolation is carried out on a long-term feature map with expansion rate of 3, the time dimension of the long-term feature map is consistent with that of a short-term feature map (such as interpolation from 5 frames to 7 frames), feature maps with different expansion rates are spliced according to channel dimensions to form a three-dimensional tensor (time multiplied by space multiplied by channel), space-time features are further fused through 3D convolution, and feature tensors comprising global motion trend and local detail are output.
The unmanned aerial vehicle flies at 15m/s in a sand storm environment, the illumination intensity of the background sand dust fluctuates by 500 lux per minute, and irregular motion blur exists between the target and the background. The dust shielding results in the maximum brightness difference of 80 (0-255 range) of adjacent frames, and a high dynamic brightness change matrix is generated. The yellow hue of the dust causes the red and green channel covariance values to rise to 0.8 (about 0.3 for a normal scene). Short-time dilation convolution (3 frame span) captures the rapid vibrations of the unmanned rotor (frequency 25 Hz). The long-term dilation convolution (7 frame span) extracts the slow diffusion trend (speed 0.5 m/s) of the cloud of sand. The long-term feature map is interpolated from 7 frames to 9 frames and spliced with the short-term feature map. The correlation of rotor wing high-frequency vibration and sand dust low-frequency motion is enhanced by 3D convolution, and a characteristic tensor is output. The space-time characteristic distinction degree of the rotor wing area is improved by 50%, and the false detection rate caused by sand and dust interference is reduced by 65%. The prediction error of the target motion trail is reduced from 5 pixels to 1.2 pixels, and the real-time tracking requirement under the complex environment is met.
In order to enhance the defending capability of the low-altitude security system against the attack of the anti-sample, aiming at the problem of the image recognition precision reduction caused by the attack of the anti-sample in the low-altitude security scene, the disturbance intensity evaluation function is constructed to quantify the potential risk of the attack by extracting the defending characteristic of the anti-sample (such as local gradient direction consistency and frequency domain energy concentration) in the enhanced optical image, the multidimensional characteristic is integrated by adopting a nonlinear weighted fusion strategy (such as a random forest model or a attention mechanism), generating an anti-disturbance feature mask, identifying high risk areas, solving the defect of insufficient detection capability of a traditional method on novel anti-attack, in some embodiments, extracting anti-sample defense features in the enhanced optical image, performing nonlinear weighted fusion on the anti-sample defense features by constructing a disturbance intensity evaluation function, forming the anti-disturbance feature mask, and comprising:
701. Performing multi-scale spatial filtering processing on the enhanced optical image, and generating a denoised standardized optical image by combining an adaptive noise suppression algorithm with local contrast enhancement operation;
In step 701, the multi-scale spatial filtering process performs a layering process on the image using filters of different sizes (e.g., 3×3, 5×5, 7×7), while retaining details and suppressing noise. The adaptive noise suppression algorithm dynamically adjusts the filtering strength according to the local noise level, and avoids detail loss caused by excessive smoothing. Local contrast enhancement operations enhance detail visibility of local areas of an image through histogram equalization or contrast stretching.
In the embodiment of the application, the enhanced optical image is subjected to multi-scale spatial filtering, the images are subjected to smoothing processing by using 3×3, 5×5 and 7×7 Gaussian filters respectively, detailed information of different scales is reserved, the filtering intensity is dynamically adjusted based on local noise variance (such as standard deviation of pixel values in a 3×3 neighborhood), strong filtering is adopted when the noise level is high, weak filtering is adopted when the noise level is low, local histogram equalization is carried out on the filtered images, the contrast of a target area (such as an unmanned plane rotor wing) is improved, and the denoised standardized optical image is generated.
702. Feature extraction is carried out based on the denoised standardized optical image, and a cross-channel attention mechanism is used for screening high-frequency texture features and low-frequency structural features of a multi-rotor structure and metal reflection characteristics of the unmanned aerial vehicle, so as to construct an anti-sample defense feature matrix;
In step 702, cross-channel attention mechanisms dynamically screen important features in RGB channels by attention weights, enhancing the discrimination of high frequency textures from low frequency structures, where the attention mechanisms focus on the drone rotor and fuselage critical parts. The high-frequency texture features represent detailed information such as edges, corner points and the like in the image and are used for detecting abnormal textures introduced by the attack resistance. The low frequency structural features describe the overall profile and regional distribution of the image for evaluating the impact of the challenge on the global structure.
In the embodiment of the application, feature extraction is performed based on a denoised standardized optical image, the image is decomposed into three independent channels of RGB, gradient amplitude and direction of each channel are calculated respectively, weight coefficients (such as red channel weight 0.6, green channel weight 0.3 and blue channel weight 0.1) of each channel are calculated through a cross-channel attention mechanism, high-frequency textures and low-frequency structural features are dynamically fused, and the screened features are spliced according to channel dimensions to generate an anti-sample defense feature matrix for subsequent disturbance intensity evaluation.
703. Dynamically calculating weight coefficients of defense characteristic channels of each countermeasure sample by using a disturbance intensity evaluation function, and adjusting weight distribution by adopting a layer-by-layer reverse gradient accumulation strategy to realize nonlinear weighted fusion of characteristic spaces;
in step 703, the disturbance intensity evaluation function calculates the potential risk level against the attack based on the feature matrix, and assigns weights to the different feature channels. The layer-by-layer reverse gradient accumulation strategy optimizes weight distribution through reverse propagation, and the detection precision of a high disturbance area is enhanced.
In the embodiment of the application, feature weighted fusion is performed by using a disturbance intensity evaluation function, an initial weight (such as high-frequency texture weight 0.7 and low-frequency structure weight 0.3) is allocated to each feature channel, gradient values of the feature channels are calculated through back propagation, weight distribution (such as high-frequency texture weight is raised to 0.8) is dynamically adjusted, and the weighted feature channels are fused according to a nonlinear function (such as Sigmoid) to generate a disturbance intensity distribution map.
704. And outputting an anti-disturbance feature mask by iteratively optimizing the semantic consistency of the mask boundary and the original image of the anti-sample defense feature matrix after nonlinear weighted fusion.
In step 704, the mask boundaries are iteratively optimized by iteratively adjusting the mask edges over a number of iterations to conform to semantic information (e.g., target contours) of the original image. Semantic consistency-ensuring that the mask area is precisely aligned with the target area in the image (e.g., drone fuselage, rotor).
In the embodiment of the application, multi-dimensional information such as gradient amplitude, pixel intensity distribution, texture characteristics and the like is extracted from the countermeasure sample. The features are weighted by a nonlinear function (e.g., sigmoid) to generate a fused feature matrix. A high-dimensional matrix containing the fused features is generated as input to a subsequent mask optimization. Based on the fused feature matrix, an initial mask is generated covering the primary area of disturbance rejection. And calculating the semantic similarity between the boundary area of the mask and the original image, and evaluating the semantic consistency of the mask. If the semantic similarity is lower than a threshold (such as 0.9), the mask boundary is adjusted, the mask coverage is enlarged or reduced, and the boundary area is ensured to be aligned with the semantic information of the original image. And repeating the semantic consistency evaluation and the boundary adjustment until the semantic similarity between the mask boundary and the original image reaches a preset threshold value. The mask's defenses (e.g., challenge sample recognition accuracy) are tested against semantic consistency (e.g., similarity of boundary regions to the original image) on the validation set. Outputting the optimized disturbance resisting characteristic mask.
In summary, the problem of image recognition accuracy degradation caused by sample attack resistance is solved by the multi-scale filtering, cross-channel attention feature extraction, disturbance intensity evaluation and mask optimization.
Aiming at the problems of image semantic information damage and target detection accuracy reduction caused by disturbance resistance in a low-altitude security scene, the method comprises the steps of performing airspace superposition operation on an enhanced optical image and a disturbance resistance feature mask, fusing original image details and disturbance area identification information, extracting multiscale features (such as local textures and global structures) from the superimposed image by adopting a multiscale residual error network based on an attention mechanism, dynamically focusing a key area (such as an unmanned plane rotor wing) through attention weights, and outputting a target confidence degree estimation result, thereby solving the problems of insufficient feature extraction and inaccurate target positioning in the disturbance resistance scene in the traditional method. In some embodiments, performing spatial domain superposition operation on the enhanced optical image and the disturbance rejection feature mask, and performing target confidence estimation on the superimposed image by using a multi-scale residual error network based on an attention mechanism, including:
801. Normalizing the disturbance resisting feature mask to enable the numerical range of the disturbance resisting feature mask to be matched with the pixel distribution of the enhanced optical image, and generating a superimposed image through airspace superposition operation;
In step 801, the normalization process scales the pixel values of the anti-perturbation feature mask to the same range of values (e.g., 0-255) as the enhanced optical image, facilitating subsequent overlay operations. And carrying out spatial domain superposition operation to carry out pixel weighted fusion on the normalized mask and the enhanced image, and generating a superimposed image containing the disturbance area identification.
In the embodiment of the application, the anti-disturbance characteristic mask is subjected to normalization processing, and the mask pixel value is linearly mapped from 0-1 range to 0-255 and is matched with the pixel distribution of the enhanced optical image. And (3) fusing the normalized mask and the enhanced image according to pixel weighting (such as mask weight 0.3 and image weight 0.7), and generating a superimposed image. The perturbed regions are marked with translucent red, and the undisturbed regions preserve the original image details.
802. Constructing a multi-scale processing module, performing multi-scale residual error network operation based on an attention mechanism on the superimposed image, and generating a multi-scale fusion feature tensor;
In step 802, the multi-scale processing module includes convolution layers (e.g., 3×3,5×5, 7×7) of different receptive fields for extracting local details and global structural features. The attention mechanism enhances the pertinence of feature extraction by dynamically weighting the focus key region (e.g., the unmanned rotor). The multi-scale fusion feature tensor is used for splicing feature graphs with different scales into a three-dimensional tensor, and local and global information is reserved.
In the embodiment of the application, a multi-scale processing module is constructed and residual network operation is executed, local texture and global contour features are extracted by using 3×3, 5×5 and 7×7 convolution kernels respectively, weight coefficients (such as rotor wing region weight 0.8 and background region weight 0.2) of the scale features are calculated through an attention mechanism, and key region features are dynamically fused. And splicing the feature graphs with different scales according to the channel dimension to generate a multi-scale fusion feature tensor for subsequent confidence estimation.
803. Inputting the multiscale fusion feature tensor into a confidence estimation unit, executing local feature statistic calculation, and generating a target confidence estimation value.
In step 803, the multi-scale processing module includes convolution layers (e.g., 3×3, 5×5, 7×7) of different receptive fields for extracting local details and global structural features. The attention mechanism enhances the pertinence of feature extraction by dynamically weighting the focus key region (e.g., the unmanned rotor). The multi-scale fusion feature tensor is used for splicing feature graphs with different scales into a three-dimensional tensor, and local and global information is reserved.
In the embodiment of the application, a multi-scale processing module is constructed and residual network operation is executed, local texture and global contour features are extracted by using 3×3, 5×5 and 7×7 convolution kernels respectively, weight coefficients (such as rotor wing region weight 0.8 and background region weight 0.2) of each scale feature are calculated through an attention mechanism, key region features are dynamically fused, feature graphs with different scales are spliced according to channel dimensions, and multi-scale fusion feature tensors are generated for subsequent confidence estimation.
The unmanned aerial vehicle flies in a strong wind environment, and high-frequency noise (psnr=28db) is injected into the image against attack, so that target detection fails. The pixel values of the disturbance-resistant feature mask are mapped from 0-1 to 0-255. The mask and the enhanced image are superimposed at a weight of 0.3:0.7, and the disturbance areas are marked with translucent red. Rotor texture and fuselage contour features are extracted using 3 x 3, 5 x 5, 7 x 7 convolution kernels. The rotor zone weight is raised to 0.8 and the background zone weight is lowered to 0.2. Feature mean and variance are calculated for the 3 x 3 neighborhood of rotor region, with mean significantly higher than background region. The confidence of the rotor area is increased to 0.9, and the confidence of the background noise area is reduced to 0.2. The detection accuracy of the rotor wing area is improved to 95%, and the false detection rate caused by attack resistance is reduced by 80%. The target detection precision is recovered from 60% to 90%, and the low-altitude security requirement is met.
In summary, the problem of target detection accuracy reduction under the disturbance resisting scene is solved by the scheme through normalized superposition, the multi-scale residual error network and confidence estimation.
In order to improve the target recognition accuracy and stability of the low-altitude security system in a complex dynamic scene, aiming at the problem of reduced recognition accuracy of the unmanned aerial vehicle caused by environmental interference (such as illumination mutation and background motion) and counterattack (such as high-frequency noise and image tampering) in the low-altitude security scene, based on a target confidence estimation result (such as local region confidence score), a low-confidence false detection region is removed through a dynamic threshold screening and multi-frame fusion strategy, the recognition weight of the high-confidence target region is enhanced, the recognition result is optimized by combining space-time context information, the defect that the conventional method is high in false detection rate and poor in robustness in the complex scene is overcome, and in some embodiments, the unmanned aerial vehicle recognition result with the environmental interference resistance and the counterattack robustness is generated according to the target confidence estimation result, the method comprises the following steps:
901. Based on the target confidence estimation result, calculating an environment interference intensity index in real time by adopting a sliding window mechanism, and starting an adaptive threshold adjustment algorithm when detecting that continuous frame confidence fluctuation exceeds a preset threshold;
In step 901, the trend of the target confidence change is analyzed in real time through a sliding window (e.g., a window length of 5 frames, an overlapping rate of 50%). The environmental interference intensity index quantifies the intensity of environmental interference based on the magnitude of the confidence fluctuation (e.g., adjacent frame confidence difference). The self-adaptive threshold adjustment algorithm dynamically adjusts the confidence threshold according to the interference intensity, and screens the high-reliability target area. In the embodiment of the application, the environmental interference intensity index is calculated based on the target confidence estimation result, sliding window analysis is performed on the confidence coefficient data of the continuous 5 frames, the confidence coefficient mean value and variance in the window are calculated, if the confidence coefficient difference value of the adjacent frames exceeds a preset threshold value (such as 0.2), the environmental interference is judged to exist, and the interference intensity index is set to be the absolute value of the difference value. When the interference intensity index is more than 0.3, starting an adaptive threshold adjustment algorithm, increasing the confidence coefficient threshold from 0.6 to 0.7, and screening a high-confidence region.
902. Performing interference pattern recognition on the confidence coefficient abnormal region through a self-adaptive threshold adjustment algorithm to generate an environment interference feature mask;
In step 902, an image region with a confidence anomaly region confidence fluctuation amplitude exceeding a preset threshold value characterizes a potential interference region. Interference pattern recognition recognizes the type of interference (e.g., abrupt illumination, background motion) through feature analysis (e.g., texture, motion trajectory). The ambient interference feature mask is used to mark the binarization mask of the interference area to assist in subsequent recognition optimization.
In the embodiment of the application, the interference pattern recognition is carried out on the confidence abnormal region, the texture characteristics (such as gradient direction histogram) and the motion characteristics (such as optical flow vector) of the abnormal region are extracted, the interference type is recognized through a random forest model (comprising 100 decision trees), for example, the gradient direction of the illumination mutation region is concentrated, and the optical flow vector of the background motion region is consistent. An ambient interference feature mask is generated based on the recognition result, and the interference region is marked (e.g., the illumination mutation region is marked with yellow, and the background motion region is marked with green).
903. Starting an anti-sample detection module for the low-confidence target, and analyzing the disturbance sensitivity of the characteristic space by adopting a gradient back propagation method to generate a robustness assessment coefficient.
In step 903, the challenge sample detection module detects potential challenge regions by analyzing feature space disturbance sensitivity. The gradient back-propagation method quantifies the impact on the challenge by back-propagating the sensitivity of the computational features to the disturbance. The robustness assessment coefficient characterizes an index of the target area's resistance to attack resistance, and the higher the coefficient is, the stronger the robustness is.
In the embodiment of the application, starting up the low confidence target to detect the disturbance gradient value of the counter-propagation calculation feature pair through counter-propagation, quantifying the disturbance sensitivity (for example, gradient amplitude >0.5 represents high sensitivity), generating a robustness evaluation coefficient based on the gradient value, for example, the region coefficient of the gradient amplitude <0.2 is set to be 1.0, and the region coefficient of the gradient amplitude >0.5 is set to be 0.3. And outputting a robustness evaluation coefficient of each target area for subsequent recognition optimization.
904. And combining the target confidence estimation result, the environment interference feature mask and the robustness assessment coefficient to generate an unmanned aerial vehicle identification result with environment interference resistance and attack resistance robustness.
In step 904, the joint optimization integrates the confidence estimation result, the interference feature mask and the robustness factor to generate a robust recognition result. The stability of the identification result in a complex scene is improved through dynamic screening and fusion strategies.
In the embodiment of the application, a robust recognition result is generated by combining multi-dimensional information, a high-reliability target area (for example, confidence coefficient >0.7 and robust coefficient > 0.8) is screened according to a confidence coefficient threshold value and a robust coefficient, the screening result is fused with an interference feature mask, for example, a low-confidence target of an illumination mutation area is ignored, and a high-confidence target of a background motion area is reserved. And generating an unmanned aerial vehicle identification result with environmental interference resistance and attack resistance robustness for subsequent tracking and decision.
The unmanned aerial vehicle flies in a heavy rain environment, raindrops cause image high-frequency noise (psnr=28db), and the illumination intensity fluctuates by 300 lux per minute due to the shielding of cloud layers. The continuous 5-frame confidence mean fluctuation amplitude is 0.25 (threshold value 0.2), and the interference intensity index is set to 0.25. The confidence threshold is raised from 0.6 to 0.7, and the high-confidence region is screened. The gradient direction of the illumination abrupt change area is concentrated, and the optical flow vector of the background raindrop area is dispersed. And identifying illumination mutation and raindrop interference, and generating an environment interference feature mask. The gradient amplitude of the raindrop region is 0.6, and the robustness coefficient is set to 0.3. And generating a coefficient, namely setting the gradient amplitude of the unmanned plane body region to be 0.1 and setting the robustness coefficient to be 1.0. Target regions with confidence >0.7 and robustness factor >0.8 are preserved. And ignoring the low confidence target of the raindrop region, and generating a robust recognition result. The unmanned aerial vehicle recognition accuracy is improved from 65% to 92%, and the false detection rate caused by raindrop interference is reduced by 85%. The false detection rate of the illumination mutation area is reduced by 70%, and the real-time detection requirement under the complex environment is met.
In summary, the problem of reduced unmanned aerial vehicle recognition accuracy caused by environmental interference and challenge in a low-altitude security scene is solved by the sliding window interference detection, the self-adaptive threshold adjustment, the challenge sample detection and the multi-dimensional joint optimization.
Fig. 2 is a schematic structural diagram of an unmanned aerial vehicle recognition device for low-altitude security protection according to an embodiment of the present application, where, as shown in fig. 2, the device includes:
The generation module 21 is used for generating an optical compensation parameter set optimized for unmanned aerial vehicle imaging in real time based on a joint optimization model of the environment light intensity change rate and the background motion vector field in a low-altitude security scene, wherein the generation module 21 is also used for acquiring an original optical sequence in a target capture window through the optical compensation parameter set, constructing a time domain deconvolution kernel by combining the unmanned aerial vehicle motion characteristic, and generating an enhanced optical image with an anti-motion blur characteristic;
the processing module 22 extracts anti-sample defense features from the enhanced optical image, and performs nonlinear weighted fusion on the anti-sample defense features by constructing a disturbance intensity evaluation function to form an anti-disturbance feature mask;
The estimation module 23 performs spatial domain superposition operation on the enhanced optical image and the disturbance resisting feature mask, and performs target confidence estimation on the superimposed image by adopting a multi-scale residual error network based on an attention mechanism;
the generating module 21 is further configured to generate an unmanned aerial vehicle recognition result with environmental interference resistance and attack resistance robustness according to the target confidence estimation result.
The unmanned aerial vehicle identification device for low-altitude security in fig. 2 may execute the unmanned aerial vehicle identification method for low-altitude security in the embodiment shown in fig. 1, and its implementation principle and technical effects are not repeated. The specific manner in which the modules and units perform the operations in the unmanned aerial vehicle identification device for low-altitude security in the above embodiment is described in detail in the embodiments related to the method, and will not be described in detail herein.
It should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present application, and not for limiting the same, and although the present application has been described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the technical solution described in the above-mentioned embodiments may be modified or some technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solution of the embodiments of the present application.