Disclosure of Invention
The application provides a panoramic stitching method and device for airport monitoring videos and electronic equipment, which are convenient for panoramic stitching of airport monitoring videos.
The panoramic stitching method of the airport monitoring videos comprises the steps of obtaining a first monitoring video and a second monitoring video aiming at a target airport, wherein the target airport corresponds to a plurality of airport monitoring videos, the first monitoring video and the second monitoring video are any two monitoring videos in the airport monitoring videos, determining an interference pattern according to the first monitoring video and the second monitoring video, conducting static background extraction operation and dynamic target removal operation on the interference pattern to obtain a first area to be stitched and a second area to be stitched, determining a first light wave field corresponding to a first camera for shooting the first monitoring video and a second light wave field corresponding to a second camera for shooting the second monitoring video, and determining the panoramic monitoring pattern based on the first area to be stitched and the second area to be stitched and combining the first light wave field and the second light wave field.
By adopting the technical scheme, the feature information in the first monitoring video and the second monitoring video is converted into the high-frequency feature map of phase interference through the introduction of the interference pattern map, so that the static background and the dynamic target in each monitoring video can be accurately identified. The dynamic target removing operation can effectively filter the interference of the dynamic targets such as airplanes, ground service vehicles and the like on the splicing process, ensure the definition and accuracy of the area to be spliced, and eliminate the ghost or blurring problem. By establishing an independent light wave field model for each camera, the problem of distortion caused by shooting angles, field of view ranges and perspective deformation of the cameras is solved. The light wave field contains rich phase information, so that natural alignment and fusion between multi-view videos can be realized, dislocation and deformation at joints are eliminated, and smooth transition of splicing results is ensured. The static background extraction operation based on the interference pattern graph utilizes the high-frequency phase consistency characteristic of the static background to stably extract the static background information in a complex illumination condition or a scene in which a dynamic target frequently moves. The method avoids the problem that the traditional background extraction method based on the inter-frame difference is invalid due to illumination change or dynamic interference, and improves the robustness of a splicing algorithm. By combining the first light wave field and the second light wave field, high-frequency details shot in the multi-video are accurately fused into the panoramic image, so that the spliced image is clearer in space and detail level. Meanwhile, the light wave field technology can dynamically compensate color difference and brightness change among coverage areas of different cameras, and ensures color consistency of panoramic images. The combination of the interference pattern and the light wave field reduces the calculation amount of multi-step feature matching and complex registration operation in the traditional splicing technology, so that panoramic splicing can realize near real-time updating in a high-frame-rate video stream environment. The method can adapt to background feature extraction and splicing under different illumination conditions by utilizing the interference pattern and the frequency domain characteristics of the light wave field, and the spliced panoramic image can keep stable quality and consistency no matter under the condition of day, night or weather change. Therefore, panoramic stitching of airport monitoring videos is facilitated.
Optionally, determining an interference pattern diagram according to the first monitoring video and the second monitoring video specifically comprises performing fourier transform on the first monitoring video and the second monitoring video frame by frame to obtain a first high-frequency texture and a second high-frequency texture, and calculating phase changes between the first high-frequency texture and the second high-frequency texture to generate the interference pattern diagram.
By adopting the technical scheme, the Fourier transform is used for converting the monitoring video frame in the time domain into the frequency domain, so that the high-frequency texture features in the image are highlighted, and the features mainly correspond to the detail information in the video. By acquiring the high-frequency textures of the first and second monitoring videos, key features can be extracted more accurately, aliasing of background textures and dynamic targets is avoided, and accuracy of feature comparison is improved. The photographed pictures of different cameras have certain parallax, and the dynamic targets are represented as discontinuous phase changes in video frames. By calculating the phase change between high frequency textures, dynamic objects and static backgrounds can be sharply detected and distinguished. The static background has high phase consistency in a plurality of frames, and the dynamic target causes phase fluctuation due to the continuous change of the position. With this feature, the disturbance of the dynamic object can be removed more accurately. The interference pattern generates an image combining high-frequency phase characteristics of two videos through phase change calculation. The image directly reflects the key texture area and phase consistency information in the video frame, and is the basic data of panoramic stitching. Compared with the traditional edge detection or feature point matching, the interference pattern diagram integrates the frequency domain characteristics and the phase information, and is more suitable for panoramic stitching of dynamic complex scenes. Due to the view angle difference of the multiple cameras, certain distortion and dislocation exist in the video frame, and high-frequency textures in two sections of videos can be aligned accurately through Fourier transformation and phase correction.
The method comprises the steps of inputting an interference pattern diagram into a light wave propagation model, extracting a static background to obtain a first result, removing a dynamic target in the interference pattern diagram by a high-pass filter to obtain a second result, and determining the first region to be spliced and the second region to be spliced by the light wave propagation model according to the first result and the second result.
By adopting the technical scheme, the interference pattern diagram is processed by adopting the light wave propagation model, so that the light wave propagation characteristics can be utilized to highlight the key texture information in the static background, and meanwhile, the interference of the dynamic target is restrained. In a scene, the light wave propagation characteristic of the static background has higher stability, and static targets such as runways, buildings and the like can be clearly separated. The influence of dynamic interference on light wave propagation is weakened, so that the static background extraction is more accurate. The high-pass filter processes the dynamic targets in the interference pattern diagram, so that the interference of the discontinuous dynamic targets with higher frequency on panoramic stitching can be eliminated. The dynamic object appears as a discontinuous distribution in the high frequency features, while the high pass filter is able to accurately identify and remove these high frequency disturbances. After the dynamic target is eliminated, the splicing area only keeps the background characteristic, so that the problem of double image or dislocation of the dynamic target in splicing is avoided. And (3) performing double operations of static background extraction and dynamic target removal, and fusing the two results through a light wave propagation model to accurately position the first region to be spliced and the second region to be spliced. The static background extraction provides high-quality background texture information, and the high-pass filter effectively eliminates dynamic interference, so that the high-quality background texture information and the high-pass filter are combined to ensure that the texture consistency in a splicing area is strong and the dynamic interference is less. The splicing area determining process reduces the mixing of low-frequency background and high-frequency interference, and ensures the quality of a splicing result. The static background extraction ensures the robustness of the splicing result, and the stability of the spliced image can be maintained even if the dynamic target changes frequently. The dynamic target removal reduces visual interference in real-time monitoring, so that the panoramic view is clearer and more reliable. The splicing process in the multi-camera monitoring scene is simplified, and the processing efficiency and the automation degree are improved. The method is suitable for real-time panoramic stitching tasks and meets the high timeliness requirement of airport monitoring.
Optionally, the determining a first light wave field corresponding to a first camera for shooting the first monitoring video and a second light wave field corresponding to a second camera for shooting the second monitoring video specifically includes constructing an initial light wave field corresponding to the first camera and an initial light wave field corresponding to the second camera according to the interference pattern diagram, correcting the initial light wave field corresponding to the first camera by using nonuniform refraction, and generating perspective distortion of the initial light wave field corresponding to the second camera and the first light wave field and the second light wave field.
By adopting the technical scheme, an accurate light wave propagation model is built for the shooting area of each camera by constructing the initial light wave field corresponding to the first camera and the initial light wave field corresponding to the second camera. This helps to describe the viewing angle and optical characteristics of each camera in detail. The light wave field modeling enables the video frames shot by each camera to accurately reflect the light propagation process based on the physical rule, and the splicing error caused by optical distortion is avoided. By means of non-uniform refraction correction, the optical wave field of the camera is accurately corrected, distortion caused by the optical lens is corrected, and each element in the image is ensured to be aligned with the real world more accurately. The non-uniform refraction correction effectively eliminates local distortion of different cameras caused by lens distortion, and enhances geometric consistency of spliced images. After correction, the geometric shape of the image is closer to the actual scene, and the dislocation phenomenon caused by distortion in the traditional splicing method is avoided. By correcting perspective distortion, image distortion caused by different photographing angles is eliminated. After perspective distortion correction, the light wave field of each camera is more in line with the actual view angle relation, so that a plurality of monitoring videos can be spliced together naturally. By optimizing the light wave field, the splicing error caused by shooting angle difference, perspective distortion and nonuniform refraction among different cameras in the traditional splicing is reduced, and the panoramic image is ensured to be more natural and finer. The method can process video streams shot by a plurality of cameras, and can ensure the optical and geometric correction of each video even under the conditions of different visual angles and different optical distortions, thereby realizing more accurate splicing, improving the splicing quality under the scene of a plurality of cameras, adapting to the configuration of a plurality of visual angles such as wide angle, long focus and the like, and enhancing the adaptability and the stability of splicing.
The panoramic monitoring image is determined by combining the first light wave field and the second light wave field, and specifically comprises the steps of obtaining an overlapping area between the first area to be spliced and the second area to be spliced, smoothing the overlapping area by a Gaussian beam model to obtain a first splicing result, determining key alignment features according to the overlapping area, aligning the first area to be spliced and the second area to be spliced by adopting interference phase change according to the key alignment features to obtain a second splicing result, and generating the panoramic monitoring image according to the first splicing result and the second splicing result.
By adopting the technical scheme, the overlapping area is subjected to smoothing treatment through the Gaussian beam model, so that the significance of a joint or a transition area between two areas to be spliced can be reduced, and the splicing result is smoother and more natural. The application of the Gaussian beam model is beneficial to smooth transition in the overlapped area, abrupt joints or distortion in the splicing process are avoided, and the panoramic image shows a seamless splicing effect. The splicing error caused by illumination, texture or visual angle change is reduced, and the overall visual continuity is enhanced. By analyzing the overlapping area and determining the key alignment features, the alignment process of the stitching is more accurate, and the most important geometric structures in the images are ensured to be aligned accurately after the stitching. The identification of the key alignment features helps to precisely align the two regions to be spliced and avoid texture dislocation or geometric distortion caused by inaccurate alignment. The method can extract important features in complex scenes and dynamic environments, and ensures consistency of key information during image stitching. The interference phase change alignment can accurately calibrate the relative position of the areas to be spliced based on the interference pattern diagram, so that the geometrical consistency and the optical consistency of the spliced images are enhanced. The interference phase change alignment provides a physical level alignment way that can handle viewing angle differences, optical distortions and perspective changes in complex scenes. And the first splicing result and the second splicing result after smooth transition are combined to generate a panoramic monitoring image, so that the naturalness and consistency of the final spliced image are ensured. The application of key alignment features and interference phase changes can provide accurate alignment results in a dynamic environment and prevent the dynamic targets from affecting the splicing effect. The Gaussian beam smoothing reduces the influence of fast moving objects in a dynamic scene on a splicing result, and enhances the visual consistency of a panoramic image.
The method comprises the steps of determining a target dynamic region according to a second light wave field, predicting a motion track of a dynamic target by analyzing energy change of the target dynamic region in the second light wave field, determining a shielding region in the second region to be spliced, carrying out interpolation reconstruction on the shielding region according to the motion track of the dynamic target, filling texture information lost due to shielding of the dynamic target, supplementing missing texture details in a second splicing result through low-frequency light wave components in the texture information, obtaining a third splicing result, and generating the panoramic monitoring map according to the first splicing result and the third splicing result.
By adopting the technical scheme, the moving target can be identified and tracked in advance by analyzing the target dynamic region and predicting the motion trail of the dynamic target, and the shielding or distortion influence of the moving target in the splicing process is avoided. The prediction and tracking of the dynamic target can ensure the consistency of the target in image splicing, and the splicing error or fracture caused by the occurrence of the dynamic target is avoided. The dynamic target motion path is recognized in advance, so that the splicing process can be adjusted in real time, and spatial dislocation caused by target motion in the splicing process is avoided. The interpolation reconstruction technology of the shielding region can effectively fill in the texture loss caused by shielding of the dynamic target, and ensures the integrity of the spliced image in all regions. Texture information is filled through interpolation reconstruction technology, and the splicing result can keep visual integrity, so that blank or distortion areas are avoided during splicing. And for the texture fuzzy region generated in the splicing process, the lost detail information can be recovered by using the low-frequency light wave component to complement, so that the fineness and the accuracy of the image are improved. The low-frequency light wave component complement technology can accurately recover details caused by blurring or loss in the splicing process, ensure that texture details of images are more real, and improve visual quality. By complementing the missing details, the blurred part in the splicing process is avoided, and the final panoramic image is clearer and more exquisite. The method enables the splicing result to better embody the continuity of the real scene, and avoids abrupt or discontinuous parts caused by shielding or dynamic targets in the image. The final panoramic image is ensured to have clear and fine details in each area, and the whole effect is more natural. The dynamic target prediction and the interpolation reconstruction of the shielding area can effectively cope with challenges in complex dynamic environments such as airports, and the adaptability to factors such as dynamic targets and shielding in the splicing process is improved, so that the spliced image can cope with changes in actual scenes, and the dynamic environments are prevented from producing excessive influence on splicing precision.
Optionally, the method further comprises the steps of obtaining a target video frame in a third monitoring video, wherein the third monitoring video is any one monitoring video except the first monitoring video and the second monitoring video in a plurality of airport monitoring videos, dynamically adjusting a panoramic framework of a third light wave field corresponding to the third monitoring video according to the target video frame, and optimizing consistency of the panoramic framework by adopting an autocorrelation calibration mechanism of an interference phase so as to eliminate drift problem generated by splicing.
By adopting the technical scheme, the panoramic framework of the third light wave field can be optimized according to the change of the actual video content by dynamically adjusting the panoramic framework according to the target video frame, so that the geometric precision in the splicing process is improved. The panoramic framework is dynamically adjusted to be beneficial to adapting to the change of the environment in the monitoring video, so that the visual angle and the alignment relation of each camera in the splicing process are accurately adjusted. The adjustment can effectively process splicing deviation caused by different camera positions and visual angle differences, and the geometric consistency and accuracy of spliced images are enhanced. And an auto-correlation calibration mechanism of an interference phase is adopted to optimize the panoramic framework, so that the drift problem caused by view angle deviation or image dislocation does not occur in the splicing process of the panoramic image. The mechanism can ensure the consistency of images when processing the splicing of a plurality of visual angles, avoid the dislocation, distortion or unnatural transition in the spliced images and improve the overall quality. By introducing an autocorrelation calibration mechanism, the stability and robustness of the stitching process is enhanced, especially in the face of dynamic scenes or coordination problems between multiple cameras. The adaptability of the splicing system in a dynamic environment is enhanced, so that the splicing among the plurality of camera view angles can be kept consistent under any environmental condition. For the dynamic environment of an airport, the optimization mechanism can stably process small changes in a large-scale monitoring scene, and high-quality output of a final panoramic image is ensured. Via accurate adjustment of the panoramic skeleton and the calibration mechanism, visual flaws caused by a stitching algorithm are reduced, and the quality of a final panoramic monitoring image is optimized. The consistency and consistency of the panoramic monitoring image are enhanced, the nature of the transition part of the image is ensured, and the content of the image is clear.
The panoramic stitching device of the airport monitoring videos comprises an acquisition module and a processing module, wherein the acquisition module is used for acquiring a first monitoring video and a second monitoring video of a target airport, the target airport corresponds to a plurality of airport monitoring videos, the first monitoring video and the second monitoring video are any two monitoring videos in the airport monitoring videos, the processing module is used for determining an interference pattern according to the first monitoring video and the second monitoring video, the processing module is also used for carrying out static background extraction operation and dynamic target removal operation on the interference pattern to obtain a first area to be stitched and a second area to be stitched, the processing module is also used for determining a first light wave field corresponding to a first camera used for shooting the first monitoring video and a second light wave field corresponding to a second camera used for shooting the second monitoring video, and the processing module is also used for determining a panoramic area to be stitched and the second area to be stitched based on the first wave field and the second wave field.
In a third aspect of the application there is provided an electronic device comprising a processor, a memory for storing instructions, a user interface and a network interface, both for communicating to other devices, the processor being for executing instructions stored in the memory to cause the electronic device to perform a method as described above.
In a fourth aspect of the application there is provided a computer readable storage medium storing instructions which, when executed, perform a method as described above.
In summary, one or more of the technical solutions provided in the present application at least have the following technical effects or advantages:
By introducing the interference pattern, the characteristic information in the first monitoring video and the second monitoring video is converted into a high-frequency characteristic pattern of phase interference, so that the static background and the dynamic target in each monitoring video can be accurately identified. The dynamic target removing operation can effectively filter the interference of the dynamic targets such as airplanes, ground service vehicles and the like on the splicing process, ensure the definition and accuracy of the area to be spliced, and eliminate the ghost or blurring problem. By establishing an independent light wave field model for each camera, the problem of distortion caused by shooting angles, field of view ranges and perspective deformation of the cameras is solved. The light wave field contains rich phase information, so that natural alignment and fusion between multi-view videos can be realized, dislocation and deformation at joints are eliminated, and smooth transition of splicing results is ensured. The static background extraction operation based on the interference pattern graph utilizes the high-frequency phase consistency characteristic of the static background to stably extract the static background information in a complex illumination condition or a scene in which a dynamic target frequently moves. The method avoids the problem that the traditional background extraction method based on the inter-frame difference is invalid due to illumination change or dynamic interference, and improves the robustness of a splicing algorithm. By combining the first light wave field and the second light wave field, high-frequency details shot in the multi-video are accurately fused into the panoramic image, so that the spliced image is clearer in space and detail level. Meanwhile, the light wave field technology can dynamically compensate color difference and brightness change among coverage areas of different cameras, and ensures color consistency of panoramic images. The combination of the interference pattern and the light wave field reduces the calculation amount of multi-step feature matching and complex registration operation in the traditional splicing technology, so that panoramic splicing can realize near real-time updating in a high-frame-rate video stream environment. The method can adapt to background feature extraction and splicing under different illumination conditions by utilizing the interference pattern and the frequency domain characteristics of the light wave field, and the spliced panoramic image can keep stable quality and consistency no matter under the condition of day, night or weather change. Therefore, panoramic stitching of airport monitoring videos is facilitated.
Detailed Description
In order that those skilled in the art will better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments.
In describing embodiments of the present application, words such as "for example" or "for example" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "such as" or "for example" in embodiments of the application should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "or" for example "is intended to present related concepts in a concrete fashion.
In the description of embodiments of the application, the term "plurality" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating an indicated technical feature. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
Airports are important public transportation hubs, and relate to complex scenes such as airplane taxiing, parking apron scheduling, ground work, passenger activities and the like. In order to ensure safe and efficient operation, modern airports are widely deployed with multi-camera monitoring systems to cover critical areas such as runways, taxiways, tarmac, and the like. However, due to the wide airport scene space and frequent dynamic targets, videos captured by multiple cameras face challenges when panoramic stitching.
The existing panoramic stitching technology relies on traditional methods such as feature point matching or image registration. However, in airport monitoring scenarios, dynamic objects such as aircraft taxiing, ground service vehicles and personnel movement create significant interference with the splice results, resulting in ghosting and splice errors. In addition, the visual angle difference among the multiple cameras can also cause distortion of a splicing area, so that the seam is obvious, and the image fusion is unnatural. Therefore, the traditional feature point matching and image registration method is poor in effect when processing panoramic stitching of airport monitoring videos.
In order to solve the above technical problems, the present application provides a panoramic stitching method of airport monitoring video, and referring to fig. 1, fig. 1 is a flow chart of a panoramic stitching method of airport monitoring video according to an embodiment of the present application. The panoramic stitching method is applied to a server and comprises the following steps of S110 to S150:
S110, acquiring a first monitoring video and a second monitoring video aiming at a target airport, wherein the target airport corresponds to a plurality of airport monitoring videos, and the first monitoring video and the second monitoring video are any two monitoring videos in the plurality of airport monitoring videos.
Specifically, the server needs to select a surveillance video for the target airport from the surveillance system. A target airport refers to an airport of interest, possibly a particular airport. Wherein the server is a separate computer system or a server cluster for managing the monitoring system of the airport. The target airport has a plurality of surveillance video sources. For example, airports may deploy multiple cameras to cover different areas, such as runways, taxiways, gates, parking lots, and the like. Each camera generates a surveillance video. Of these multiple surveillance videos, the server selects two particular videos, referred to as a first surveillance video and a second surveillance video. The two videos come from different cameras, possibly monitoring videos covering different areas of the field. The two videos selected from the plurality of monitoring videos need not be fixed, but any two monitoring videos. That is, the first video and the second video may be any two monitoring videos, and may be two cameras from different positions or different perspectives.
For example, assume that there is one airport in which a plurality of monitoring cameras are deployed to cover different areas. The cameras are respectively arranged at the positions of the runway, the parking apron, the boarding gate and the like. The first surveillance video is a surveillance video captured by one camera from an airport runway. This video may cover the taxiing, takeoff and landing of aircraft on the runway. The second monitoring video is a monitoring video shot by another camera from the parking apron, and shows scenes such as airplane parking and ground crew operation on the parking apron.
S120, determining an interference pattern diagram according to the first monitoring video and the second monitoring video.
Specifically, the interference pattern is an image based on the mutual influence between different frames in the videos, and the generated interference pattern can reveal the relation between different videos, especially the change of a dynamic target by analyzing the image details in two videos. The generation of the interference pattern map is related to the frequency domain processing of the image. The method can analyze the slight change between two video frames through Fourier transform and other technologies to find the phase difference between the two video frames. These differences are manifested as interference effects produced by dynamic objects in the image. The figure can reveal detail differences between different videos, especially in dynamic environments such as moving objects and scene changes.
In one possible implementation, the method for determining the interference pattern graph according to the first monitoring video and the second monitoring video specifically comprises the steps of carrying out Fourier transform on the first monitoring video and the second monitoring video frame by frame to obtain a first high-frequency texture and a second high-frequency texture, calculating phase change between the first high-frequency texture and the second high-frequency texture, and generating the interference pattern graph.
In particular, the fourier transform may transform a signal from the time domain to the frequency domain. In the frequency domain, different parts of the image will be represented as different frequency components. For video, the fourier transform may help extract high and low frequency information in the image. The high frequency part represents details in the image, such as edges, textures, etc., and the low frequency part represents smooth areas of the image, such as the background. Performing a fourier transform on a frame-by-frame basis means that the server performs a fourier transform on each frame of image in the video, rather than an overall process. This enables a detailed analysis of each moment in the video, capturing the changes in the dynamic object. After fourier transformation, the frequency domain of the video frame may contain high frequency textures. These high frequency textures contain detailed parts in the image, especially variations associated with dynamic objects. The first high-frequency texture and the second high-frequency texture represent high-frequency components of the first monitor video and the second monitor video, respectively. Since the surveillance video is from different perspectives, the high frequency content in the two videos may be different, particularly when there is a dynamic object, the change of these dynamic elements may be reflected on the high frequency texture. The phase change refers to a phase difference between frequency components of two images in the frequency domain. By calculating the phase change between the high frequency textures of the two videos, fine changes in the image, in particular the motion of the dynamic object, can be revealed. When a plurality of monitoring videos are spliced, dynamic targets can cause changes among video frames, and calculation of the phase changes can help to accurately align dynamic elements in different videos. The interferogram is the result of calculating the high frequency texture phase change of the two video segments. It exhibits differences between the two video segments, in particular variations due to dynamic objects. The interference pattern diagram can reveal information such as relative displacement between two video sources, movement track of a target and the like, which is very important for accurately aligning different visual angles and dynamic targets in the splicing process.
For example, assume that there are two surveillance videos, one from the runway camera of the airport and the other from the apron camera. These two videos capture dynamic scenes of different areas, such as aircraft taxiing on a runway and ground crew operations on an apron, respectively. The first surveillance video contains dynamic scenes of taxiing of the aircraft, and the movement of the aircraft on the runway causes detail changes in the video, particularly in surrounding areas of the aircraft. The second monitoring video is shot by the scene of the ground staff operating the airplane on the tarmac, and dynamic changes are also caused, such as the movement of the staff or the use of equipment. The server first performs fourier transform on each frame of image in the video of the runway camera and the video of the apron camera, and converts the images into the frequency domain. In this way we can get a high frequency texture, i.e. a detailed part in the image. By comparing the high frequency textures of the runway video and the tarmac video, the server calculates the phase change of the detail parts in the two video segments. For example, when an aircraft is taxiing on a runway, it may change the high frequency texture of the video, creating a phase difference. Similarly, when the ground crew is operating on the tarmac, changes in the video may also occur. By analyzing the phase changes of the two videos, the server generates an interference pattern map. This figure reveals the dynamic difference between the two videos, especially in the high frequency region of the image. These differences are related to the motion trajectories of the dynamic objects such as aircraft taxiing, ground crew operations, etc.
S130, performing static background extraction operation and dynamic target removal operation on the interference pattern diagram to obtain a first region to be spliced and a second region to be spliced.
Specifically, an interference pattern diagram has been previously generated, which shows the difference between the dynamic object and the background caused by the phase change between two monitoring video frames. The figure reveals the distinction between dynamic objects and static background. Static background extraction refers to identifying the unchanged portion of the video from the interference pattern diagram and distinguishing it from the dynamic object. The background portion is an object or region of video that is stationary, while the dynamic object is a portion of video that has significant movement or changes. This operation may utilize image processing techniques such as background modeling, frame difference methods, etc. to extract stationary background regions in the video. For example, markings on runways, buildings, etc. are static backgrounds, and personnel, vehicles on airports are dynamic targets. Dynamic object removal refers to removing all changes caused by dynamic objects in the interferogram so that only background information remains. By removing these dynamic targets, they can be avoided from interfering with the splice results during the splicing process. Dynamic objects may cause varying degrees of distortion, ghosting, or stitching errors in the two video sources. Thus, by removing these dynamic targets, the server can get a more accurate splice area. The dynamic target removal operation may be accomplished using a high pass filter or a background subtraction method. The high pass filter helps to remove the low frequency portion while leaving the high frequency portion. The region to be spliced refers to a part of images extracted from the interference pattern diagram, and the part of images are static background regions after the dynamic targets are removed, and are suitable for splicing operation. The first region to be spliced and the second region to be spliced are respectively extracted regions from two monitoring videos, and the regions are combined into a seamless panorama in the subsequent splicing step.
In a possible implementation manner, the method comprises the steps of performing static background extraction operation and dynamic target removal operation on an interference pattern diagram to obtain a first region to be spliced and a second region to be spliced, and specifically comprises the steps of inputting the interference pattern diagram into a light wave propagation model, and obtaining a first result through static background extraction; and determining a first area to be spliced and a second area to be spliced by adopting a light wave propagation model according to the first result and the second result.
In particular, a light wave propagation model is a mathematical model that describes how light waves or image information propagates. In this context, the light wave propagation model is used to extract the static background portion in the interference pattern map. It helps identify which regions are static backgrounds and distinguish between dynamic targets. During the static background extraction process, the server identifies the time-invariant portions of the interference pattern diagram through the light wave propagation model. These background portions include areas of airport sites, buildings, reticles, etc. that do not change over time or object movement. The first result of the static background extraction is the background portion extracted from the interferogram. It helps to separate stable, mosaicable regions from complex monitored images. The second result is an image after the dynamic object is removed, and the background information of the dynamic object such as a person, a vehicle and the like is removed. And further determining the image area suitable for splicing by the server through the light wave propagation model according to the first result and the second result. The areas are a first area to be spliced and a second area to be spliced, which are respectively from two monitoring videos, and the interference of a dynamic target is removed, so that the splicing is ready.
S140, determining a first light wave field corresponding to a first camera for shooting a first monitoring video and a second light wave field corresponding to a second camera for shooting a second monitoring video.
In particular, a light wave field is a mathematical model describing the propagation of light waves in an image or video. In an embodiment of the application, the light wave field is used to represent the relationship between camera view angle and scene. Each camera's scene taken through its view angle will produce different light wave fields that will affect the appearance of the video image, such as refraction of light, see-through effects, and distortion. For a first surveillance video, the server will determine the optical wave field of that camera based on which camera the video was taken by. The determination of the light wave field is based on the position, angle, focal length, shooting environment and other factors of the camera. These factors determine the perspective effect, the field of view and the possible distortion of the image taken by the camera. Similarly, the camera of the second monitoring video also has a corresponding light wave field. The second camera may be located in a different position and the angle may be different, so that the light wave field of the camera will be different from the light wave field of the first camera. The server will determine its light wave field based on the position, angle, etc. of the second camera.
In one possible implementation manner, determining a first light wave field corresponding to a first camera for shooting a first monitoring video and a second light wave field corresponding to a second camera for shooting a second monitoring video specifically comprises constructing an initial light wave field corresponding to the first camera and an initial light wave field corresponding to the second camera according to an interference pattern diagram, correcting the initial light wave field corresponding to the first camera by using nonuniform refraction, generating perspective distortion of the initial light wave field corresponding to the second camera, and generating the first light wave field and the second light wave field.
Specifically, the server determines how to construct the initial light wave fields of the two cameras by analyzing the interference pattern diagram. These initial light wavefields are preliminary modeling of each camera view angle, representing the light wave propagation characteristics in the scene captured by the camera. In an actual scene, different cameras can generate different light wave propagation effects on capturing the same scene. Especially, under the conditions of different visual angles, different shooting distances, different focal lengths and the like of cameras, the shot images can generate certain distortion, in particular perspective distortion, namely, the difference between the sizes of distant objects and the sizes of near objects is obvious. The non-uniform refraction correction is to adjust the non-uniform refraction effects, so as to ensure that the corresponding relation between the light wave field generated by the view angle of the camera and the actual scene is more accurate. Through the correction, parallax difference between cameras can be reduced, so that the splicing effect is more natural. After the correction is completed, a first light wave field and a second light wave field are generated, wherein the two light wave fields respectively represent accurate descriptions of the scene by the two cameras, and factors such as viewing angles, refraction, perspective and the like are considered. The two light wave fields can be used for guiding the subsequent panoramic stitching process, so that the two images can be better aligned, and distortion or unnatural seams generated during stitching are reduced.
And S150, determining a panoramic monitoring map based on the first area to be spliced and the second area to be spliced and combining the first light wave field and the second light wave field.
Specifically, in the airport monitoring video stitching process, the first to-be-stitched area and the second to-be-stitched area refer to image areas obtained through the aforementioned operations of background extraction, dynamic target removal and the like. These areas are the portions of the video that actually need to be stitched, possibly due to overlapping or close-up of the picture content taken by the different cameras. The first light wave field and the second light wave field refer to light wave propagation characteristics of a scene photographed by each camera. These light wave fields describe the visual characteristics of the scene, including camera shooting angle, distance, illumination, etc. These light wave fields play an important role in the alignment and fusion of images in the stitching process, ensuring that the transitions between the stitched areas are natural and free of obvious seams. Combining the first region to be stitched, the second region to be stitched and their respective light wave fields means that the server will utilize the overlapping part of the two regions and the properties of the two light wave fields for image stitching. Specifically, the stitching algorithm may smooth, align, and fuse the images based on the overlapping regions of the two regions, ensuring that the finally generated panoramic image is more natural and consistent in vision. Finally, based on the above information, the server generates a panoramic monitoring map of the entire monitoring area, which is a wide-range image of the content photographed by a plurality of cameras. The panoramic monitoring view provides a complete airport monitoring view, which can cover a plurality of areas, and is convenient for management personnel to monitor all corners of an airport from a uniform view angle.
In a possible implementation manner, a panoramic monitoring map is determined based on a first area to be spliced and a second area to be spliced and by combining a first light wave field and a second light wave field, and the panoramic monitoring map specifically comprises the steps of obtaining an overlapping area between the first area to be spliced and the second area to be spliced, smoothing the overlapping area by a Gaussian beam model to obtain a first splicing result, determining key alignment features according to the overlapping area, aligning the first area to be spliced and the second area to be spliced by adopting interference phase change according to the key alignment features to obtain a second splicing result, and generating the panoramic monitoring map according to the first splicing result and the second splicing result.
Specifically, in the video mosaic for monitoring, there may be a portion where two video areas to be stitched overlap, which is a portion where image contents photographed by two cameras intersect. This overlap region is the region where the stitching algorithm needs to pay attention to, since it must be ensured that the overlap region is visually seamless when stitched. The gaussian beam model is a mathematical model for simulating the propagation of an optical wave, with smooth and soft transition characteristics. When the images are spliced, the overlapping area is processed by adopting a Gaussian beam model, so that smooth transition can be realized, and obvious seams or abrupt transition possibly occurring during splicing can be reduced. Through the step, a first splicing result, namely the spliced image after preliminary treatment, is obtained, and the spliced image has a natural transition effect. In the overlapping areas, there are some features that may contribute to the alignment of the images, such as textures, corner points or edges, etc. These features are referred to as critical alignment features. By analyzing the image content of the overlapping areas, the algorithm can determine which features are most important for stitching and use these features for subsequent fine alignment. The interferometric phase variation technique can be used to analyze the phase difference between two images and align the key features of the two images by adjusting the position, rotation, etc. of the images. Here, the algorithm uses the critical alignment features of the overlapping regions to perform phase correction and position adjustment, thereby ensuring accurate stitching of the two regions. And finally, aligning through interference phase change to obtain a second splicing result, wherein the second splicing result is a spliced image after fine alignment. And finally, the server combines the first splicing result with the second splicing result to generate a final panoramic monitoring chart. This panorama covers a larger airport area and ensures a natural, seamless transition between images.
For example, assume that there is an airport monitoring system that includes two cameras, one on the east and west sides of the runway. The east-side camera 1 photographs the east half of the runway and the west-side camera 2 photographs the west half of the runway. In the pictures shot by the two cameras, a part of the pictures on the east side and the west side are overlapped, such as the joint of two sections of runways. Assume that the east and west pictures have an overlapping portion in the middle of the racetrack, which is the key area for stitching. In the splicing process, the overlapping area is smoothed first using a gaussian beam model. For example, overlapping areas in the middle of the runway may be seamed due to different illumination, different angles of capture, and the like. Through the Gaussian beam model, the overlapped area can be smoothly transited, the abrupt sense during splicing is reduced, and a first splicing result is formed. In the overlapping area, there may be some marking features, such as edge lines of the runway, ground markings, etc., which may serve as key alignment features. By analyzing these features, the algorithm determines the correspondence of these points in the two images. The server fine aligns the two regions by an interferometric phase change algorithm based on the critical alignment features of the overlapping regions. For example, runway edges or ground markings may be precisely aligned to ensure that images in the two video streams do not have significant misalignment when stitched. And after the interference phase is adjusted, a second splicing result is obtained, and the image is more accurate and seamless. And finally, combining the first splicing result and the second splicing result to generate a large panoramic monitoring image, wherein the image comprises images of the whole airport runway, so that seamless connection is ensured, and monitoring personnel can conveniently and comprehensively check the situation of an airport.
In one possible implementation, a target dynamic region is determined according to a second light wave field, a motion track of a dynamic target is predicted by analyzing energy change of the target dynamic region in the second light wave field, a shielding region in a second region to be spliced is determined, interpolation reconstruction is carried out on the shielding region according to the motion track of the dynamic target, texture information lost due to shielding of the dynamic target is filled, a third splicing result is obtained by complementing missing texture details through low-frequency light wave components in a texture fuzzy region in a second splicing result, and a panoramic monitoring diagram is generated according to the first splicing result and the third splicing result.
Specifically, by observing the energy change of the dynamic region of the target, the moving direction and speed of the target can be estimated. Dynamic objects can produce energy fluctuations or variations in the video, and therefore analyzing these variations can help predict the motion profile of the object. This helps to know where dynamic targets will appear, so that repair of occlusion areas is done in subsequent processing. Occlusion regions refer to regions where the image is occluded or covered due to the presence of dynamic objects, resulting in loss of image detail for those regions. For example, a traveling vehicle may obstruct a portion of the ground or other object, causing a portion of the image to be invisible. By analyzing the image, the server can determine these occluded areas and take them as the key areas to be repaired. Interpolation reconstruction is a commonly used image restoration technique. In this step, the server presumes the contents of the occluded area based on the previously predicted motion trajectories of the dynamic objects. For example, if a vehicle obstructs a background area, the server may infer the previously obscured background image of the vehicle by interpolation techniques and fill in the image information to the obstructed area, recovering the lost texture. The areas of texture blurring may be due to a number of reasons, such as differences in camera angle, focus problems, or uneven illumination. For these areas, the server repairs the blurred areas using a complement method of low frequency light wave components by analyzing existing texture information, e.g., color, illumination characteristics, etc. The low-frequency light wave component mainly reflects the large-scale structure and shape in the image, and the restoration of the texture information can enable the image details to be more complete, and finally a third splicing result, namely the spliced image after repair and optimization, is obtained. In this step, the server combines the first splicing result and the third splicing result to generate a final panoramic monitoring chart. The panoramic view integrates images from different cameras, ensures the integrity of the whole monitoring area, and the images are still clear and seamless under the influence of a dynamic target.
In a possible implementation manner, referring to fig. 2, fig. 2 is another flow chart of a panoramic stitching method of an airport monitoring video according to an embodiment of the present application. The method comprises the steps of S210 to S230, wherein the step of S210 is to acquire a target video frame in a third monitoring video, the third monitoring video is any one monitoring video except the first monitoring video and the second monitoring video in a plurality of airport monitoring videos, the step of S220 is to dynamically adjust a panoramic skeleton of a third light wave field corresponding to the third monitoring video according to the target video frame, and the step of S230 is to conduct consistency optimization on the panoramic skeleton by adopting an autocorrelation calibration mechanism of an interference phase so as to eliminate drift problem generated by splicing.
Specifically, the third surveillance video refers to selecting other videos than the first and second surveillance videos among the plurality of airport surveillance videos. This video may come from the perspective of other cameras in the airport. The goal of this step is to extract key target video frames from this third video, i.e. important image frames related to stitching in the monitoring. Assuming that there are multiple cameras at the airport, the first surveillance video captured by camera 1 and the second surveillance video captured by camera 2 are responsible for covering both sides of the runway, while the third surveillance video captured by camera 3 may be responsible for monitoring the tarmac or taxiway. By extracting the target frame in the third video, the server can use the information of this frame to perform panorama stitching. The third light wave field refers to a light wave propagation model corresponding to the third surveillance video, and this light wave field represents the view angle and imaging characteristics of the camera 3. The panoramic skeleton is a virtual frame or structure representing the basic layout of the overall surveillance video splice. During stitching, the server needs to dynamically adjust the light wave field according to the image content and viewing angle of the third video in order to ensure that the images are properly aligned when composing the panorama. If the angle of view of the camera 3 is different from that of the camera 1 or the camera 2, or the position of the camera 3 is changed, the server needs to dynamically adjust the light wave field according to the image information of the third monitoring video so as to maintain the panoramic space consistency.
Wherein the interference phase relates to the phase difference between different images or light waves. In image stitching, the interference phase is used to evaluate the degree of alignment between images. An autocorrelation calibration mechanism is an algorithm that optimizes the consistency of the splice by comparing the phase changes between different video frames, reducing visual errors, such as drift, e.g., image misalignment or distortion, at the splice. The mechanism helps the server dynamically adjust the splicing result by comparing the phase changes of different visual angles and visual distances, ensures accurate alignment of the contents of different video sources, and avoids unnatural gaps or offsets generated during image splicing. It is assumed that during stitching the images between camera 1 and camera 2 are already aligned very accurately, but the images of camera 3 are not aligned accurately due to the difference in viewing angle or movement. The server detects the inconsistency by using an autocorrelation calibration mechanism of the interference phase and optimizes the panoramic framework, thereby eliminating the problem of splicing drift caused by inconsistent viewing angles. Drift problems refer to the situation that when images are stitched from multiple camera views, misalignment or misplacement of the images may occur. This problem is caused by errors caused by different perspectives of the cameras, different focal lengths or movements. By applying an autocorrelation calibration mechanism of the interference phase, the server can accurately correct the errors, eliminate drift in image stitching, and enable the final stitching result to be more natural and seamless. If the camera in the third monitoring video is slightly deviated due to vibration or position, the shot picture is slightly deviated from the pictures of the first and second monitoring videos, and the server automatically corrects the errors through an interference phase autocorrelation calibration mechanism, so that the images of the third video can be correctly spliced with the images of the first two videos.
The application further provides a panoramic stitching device of the airport monitoring video, and referring to fig. 3, fig. 3 is a schematic block diagram of the panoramic stitching device of the airport monitoring video. The panoramic stitching device is a server, the server comprises an acquisition module 31 and a processing module 32, the acquisition module 31 acquires a first monitoring video and a second monitoring video aiming at a target airport, the target airport corresponds to a plurality of airport monitoring videos, the first monitoring video and the second monitoring video are any two monitoring videos in the plurality of airport monitoring videos, the processing module 32 determines an interference pattern according to the first monitoring video and the second monitoring video, the processing module 32 performs static background extraction operation and dynamic target removal operation on the interference pattern to obtain a first area to be stitched and a second area to be stitched, the processing module 32 determines a first light wave field corresponding to a first camera for shooting the first monitoring video and a second light wave field corresponding to a second camera for shooting the second monitoring video, and the processing module 32 determines the panoramic monitoring pattern based on the first area to be stitched and the second area to be stitched and combining the first light wave field and the second light wave field.
In one possible implementation, the processing module 32 determines an interference pattern diagram according to the first surveillance video and the second surveillance video, and specifically includes that the processing module 32 performs fourier transform on the first surveillance video and the second surveillance video frame by frame to obtain a first high-frequency texture and a second high-frequency texture, and the processing module 32 calculates a phase change between the first high-frequency texture and the second high-frequency texture to generate the interference pattern diagram.
In a possible implementation manner, the processing module 32 performs a static background extraction operation and a dynamic target removal operation on the interference pattern diagram to obtain a first region to be spliced and a second region to be spliced, and specifically includes that the processing module 32 inputs the interference pattern diagram into a light wave propagation model to obtain a first result through static background extraction, the processing module 32 removes a dynamic target in the interference pattern diagram by using a high-pass filter to obtain a second result, and the processing module 32 determines the first region to be spliced and the second region to be spliced by using the light wave propagation model according to the first result and the second result.
In one possible implementation, the processing module 32 determines a first light wave field corresponding to a first camera for capturing a first monitoring video and a second light wave field corresponding to a second camera for capturing a second monitoring video, and specifically includes the processing module 32 constructing an initial light wave field corresponding to the first camera and an initial light wave field corresponding to the second camera according to an interference pattern, and the processing module 32 corrects the initial light wave field corresponding to the first camera by using non-uniform refraction, and generates a first light wave field and a second light wave field by using perspective distortion of the initial light wave field corresponding to the second camera.
In a possible implementation manner, the processing module 32 determines a panoramic monitoring map based on a first region to be spliced and a second region to be spliced and combines a first light wave field and a second light wave field, and specifically includes the steps that the acquisition module 31 acquires an overlapping region between the first region to be spliced and the second region to be spliced, the processing module 32 smoothes the overlapping region by using a Gaussian beam model to obtain a first splicing result, the processing module 32 determines key alignment features according to the overlapping region, the processing module 32 aligns the first region to be spliced and the second region to be spliced by using interference phase changes according to the key alignment features to obtain a second splicing result, and the processing module 32 generates the panoramic monitoring map according to the first splicing result and the second splicing result.
In one possible implementation manner, the processing module 32 determines a target dynamic region according to the second light wave field, the processing module 32 predicts the motion track of the dynamic target by analyzing the energy change of the target dynamic region in the second light wave field, the processing module 32 determines an occlusion region in the second region to be spliced, the processing module 32 performs interpolation reconstruction on the occlusion region according to the motion track of the dynamic target to fill texture information lost due to occlusion of the dynamic target, the processing module 32 complements the missing texture details in the second splicing result by adopting low-frequency light wave components to the texture information to obtain a third splicing result, and the processing module 32 generates a panoramic monitoring graph according to the first splicing result and the third splicing result.
In a possible implementation manner, the obtaining module 31 obtains a target video frame in a third monitoring video, where the third monitoring video is any one monitoring video except the first monitoring video and the second monitoring video in the plurality of airport monitoring videos, the processing module 32 dynamically adjusts a panoramic skeleton of a third light wave field corresponding to the third monitoring video according to the target video frame, and the processing module 32 performs consistency optimization on the panoramic skeleton by adopting an auto-correlation calibration mechanism of an interference phase so as to eliminate drift problem generated by splicing.
It should be noted that, when the apparatus provided in the foregoing embodiment implements the functions thereof, only the division of the foregoing functional modules is used as an example, in practical application, the foregoing functional allocation may be implemented by different functional modules, that is, the internal structure of the device is divided into different functional modules, so as to implement all or part of the functions described above. In addition, the embodiments of the apparatus and the method provided in the foregoing embodiments belong to the same concept, and specific implementation processes of the embodiments of the method are detailed in the method embodiments, which are not repeated herein.
The application further provides an electronic device, and referring to fig. 4, fig. 4 is a schematic structural diagram of the electronic device according to the embodiment of the application. The electronic device may comprise at least one processor 41, at least one network interface 44, a user interface 43, a memory 45, at least one communication bus 42.
Wherein a communication bus 42 is used to enable connected communication between these components.
The user interface 43 may include a Display screen (Display) and a Camera (Camera), and the optional user interface 43 may further include a standard wired interface and a standard wireless interface.
The network interface 44 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.
Wherein processor 41 may comprise one or more processing cores. The processor 41 connects various parts within the overall server using various interfaces and lines, performs various functions of the server and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 45, and invoking data stored in the memory 45. Alternatively, the processor 41 may be implemented in at least one hardware form of digital signal Processing (DIGITAL SIGNAL Processing, DSP), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 41 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like, the GPU is used for rendering and drawing contents required to be displayed by the display screen, and the modem is used for processing wireless communication. It will be appreciated that the modem may not be integrated into the processor 41 and may be implemented by a single chip.
The Memory 45 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 45 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 45 may be used to store instructions, programs, code, a set of codes, or a set of instructions. The memory 45 may include a stored program area that may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-described respective method embodiments, etc., and a stored data area that may store data, etc., involved in the above-described respective method embodiments. The memory 45 may also optionally be at least one memory device located remotely from the aforementioned processor 41. As shown in fig. 4, an operating system, a network communication module, a user interface module, and an application program of a panorama stitching method of an airport monitoring video may be included in the memory 45 as a computer storage medium.
In the electronic device shown in fig. 4, the user interface 43 is mainly used to provide an input interface for a user to obtain data input by the user, and the processor 41 may be used to invoke an application program in the memory 45 for storing a panoramic stitching method of airport monitoring video, which when executed by one or more processors, causes the electronic device to perform the method as in one or more of the embodiments described above.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all of the preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
The application also provides a computer readable storage medium storing instructions. When executed by one or more processors, cause an electronic device to perform the method as described in one or more of the embodiments above.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, such as a division of units, merely a division of logic functions, and there may be additional divisions in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some service interface, device or unit indirect coupling or communication connection, electrical or otherwise.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in whole or in part in the form of a software product stored in a memory, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method of the various embodiments of the present application. The memory includes various media capable of storing program codes, such as a USB flash disk, a mobile hard disk, a magnetic disk or an optical disk.
The foregoing is merely exemplary embodiments of the present disclosure and is not intended to limit the scope of the present disclosure. That is, equivalent changes and modifications are contemplated by the teachings of this disclosure, which fall within the scope of the present disclosure. Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a scope and spirit of the disclosure being indicated by the claims.