Method for removing jitter of look-around system of police motorcycleTechnical Field
The invention relates to the technical field of motorcycles, in particular to a method for removing jitter of a look-around system of a police motorcycle.
Background
The panoramic system of the police motorcycle is characterized in that multi-channel videos are synchronously input, and when the panoramic video is formed, a camera is driven to generate micro wave jitter due to high-frequency vibration of an engine of the motorcycle, so that the problems of edge dislocation, pixel drift and the like when the multi-channel videos are spliced in a panoramic mode are caused;
in the prior art, mechanical anti-shake or optical anti-shake is generally adopted; mechanical anti-shake is realized by a mechanical structure which is ingenious in design and complex, mechanical vibration can be relieved to a certain extent, however, the mechanical structure is complex, easy to wear and high in processing and maintenance cost, the mechanical structure needs to be modified, and the mounted panoramic camera is fixed on the motorcycle through a connecting rod, and the motorcycle engine transmits vibration to the top camera module, so that on one hand, more buffer space is not available, and on the other hand, the motorcycle has high vibration frequency and small vibration amplitude, so that the mechanical structure cannot effectively reduce the vibration effect under the linear transmission vibration with high frequency and low amplitude; in addition, the panoramic camera also needs to realize panoramic stitching of multi-view video, and when the vibration frequency of each camera has deviation, the scheme can not solve the problems of dislocation and pixel offset at the picture stitching position;
the optical anti-shake can remove the shake of the picture through algorithms such as picture image comparison or pixel optical flow tracking of a plurality of lenses, is realized by the algorithm, has low cost, but in the aspect of splicing the panoramic picture, because a single lens is adopted in each direction, the multi-lens comparison cannot be applied, and the single lens and the multi-lens shooting mode can be adopted for comparison; moreover, optical anti-shake is generally used for solving large-amplitude and low-frequency vibration, and is difficult to apply in high-frequency and low-amplitude scenes such as motorcycles;
to this end, we propose a method for debouncing a look-around system for police motorcycles.
Disclosure of Invention
In order to solve the problems existing in the scheme, the invention provides a method for removing jitter of a look-around system of a police motorcycle, which is used for solving the problems of edge dislocation, pixel drift and the like when the look-around video is spliced in the look-around process of multi-channel videos, wherein the multi-channel videos of the look-around system of the police motorcycle are synchronously input, and when the look-around video is formed, a camera is driven to generate micro wave jitter due to high-frequency vibration of an engine of the motorcycle.
The purpose of the invention can be realized by the following technical scheme:
a method for removing jitter of a look-around system of a police motorcycle comprises the following steps:
the method comprises the following steps: six lenses are adopted to form a look-around system, all the lenses are connected through rigid connecting pieces, and a transparent cover is sleeved outside the look-around system;
step two: collecting vibration parameters of six sensors to calculate vibration parameters of the multiple paths of lenses, and splicing the multiple paths of lenses in a static state;
step three: under the condition of vibration, adding an additional value brought by the vibration to a fine adjustment parameter matrix w of the six-axis sensor, and then repeating the splicing process of the multiple paths of lenses in a static state to realize splicing;
step four: under the vibration condition, the reverse calculation and correction of the vibration parameters of the whole rigid connecting piece are realized through the optical flow tracking of multiple paths of pictures, and the vibration parameters are mutually complemented with the six-axis sensor to realize splicing.
Further, the splicing process of the multi-path lens in the static state is as follows:
the first step is as follows: performing static splicing correction, and creating a splicing parameter matrix for splicing pictures between adjacent lenses; the splicing parameter matrix comprises a wide parameter matrix a, a high parameter matrix b and a fine tuning parameter matrix w;
the second step is that: generating an independent pixel of each lens according to the splicing parameter matrix;
the third step: and splicing the generated independent pixels of each lens to realize splicing of the multiple paths of lenses in a static state.
Further, generating an independent pixel of each lens according to the splicing parameter matrix, specifically:
setting each lens pixel to behave as: width, height; (x1, y1) represents the pixel of the first lens, and the splicing parameter matrixes of the adjacent first lens and the adjacent second lens are respectively represented as a [1,2], b [1,2] and w [1,2 ];
the picture relationship after edge splicing is as follows: in the new frame, (x1, y1) in the scene-frame is represented by (x1 ', y 1'); (x1 ', y 1') represents an individual pixel of edge-stitched rear lens one; wherein:
x1`=x1*a[1,2]+y1*b[1,2]+w[1,2][x1,y1];
y1`=x1*a[1,2]+y1*b[1,2]+w[1,2][x1,y1];
and by analogy, obtaining independent pixels after splicing all the lens edges.
Further, the generated independent pixels of each lens are spliced, specifically:
X=[x1`+(w+x2`)+(w*2+x3`)+(w*3+x4`)+(w*4+x5`)+(w*5+x6`)];
y ═ Y1 ' + Y2 ' + Y3 ' + Y4 ' + Y5 ' + Y6; where (X, Y) represents the pixels of the spliced picture in the multi-shot still state.
Further, after vibration occurs, estimating the floating vibration of each lens through vibration parameters of six sensors, wherein the vibration parameters of the six sensors are six and are respectively a rotation angle of x \ y \ z and an acceleration of x \ y \ z; the distances from the six lenses to the six-axis sensor are equal and are denoted by d;
and obtaining the rotation offset angle of each lens picture according to the rotation angle and the acceleration of x \ y \ z, thereby realizing the deflection calculation of the fine tuning parameter matrix w and obtaining the additional value brought by the vibration.
Further, when each lens vibrates up and down, a deviation value exists between the pixels of the previous frame and the next frame, and the deviation value is recorded as the position of x \ y and the rotation angle deviation;
the deviation value is equal to an action distance value after six-axis rotation passes through the radius d, the deviation values of the six lenses are integrated together, and the unified calculation of the deflection angles of the six lenses is completed, so that the vibration parameters of the six-axis sensor are solved, and the reverse calculation of the vibration parameters of the whole rigid connecting piece is realized.
Furthermore, the vibration parameters of the six sensors are compared with the vibration parameters collected by the six sensors, the vibration parameters of the whole rigid connecting piece are corrected, and the corrected vibration parameters are used as the vibration parameters collected by the six sensors to realize splicing.
Furthermore, the six-axis sensor is connected with each path of lens through a rigid connecting piece, and the vibration parameters of the six-axis sensor and each path of lens are synchronous and consistent through the rigid connecting piece.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the invention, the synchronization and the consistency of the vibration parameters of the six-axis sensor and each camera are realized through the rigid connecting piece, then the vibration parameters of the multi-path lens are calculated through the vibration parameters of the six axes, the splicing of the multi-path lens in a static state is realized, under the vibration condition, only the additional value brought by the vibration needs to be added to the fine-tuning parameter matrix of the six-axis sensor, and the splicing efficiency and the accuracy are improved;
2. according to the invention, through optical flow tracking of multiple paths of pictures, the reverse calculation and correction of vibration parameters of the whole rigid connecting piece are realized, and the vibration parameters are mutually complemented with six axes, so that the association of pixel-level vibration tracking and mechanical vibration detection is realized, and the splicing efficiency and accuracy are improved; the problems of edge dislocation, pixel drift and the like when the multi-path video is subjected to all-round view splicing are solved; the definition and the accuracy of the video pictures are improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of the present invention.
Fig. 2 is a schematic structural diagram of the panoramic system of the present invention.
Fig. 3 is a distance diagram of six lens distance six-axis sensors according to the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the following embodiments, and it should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1 to 3, a ring vision system debouncing method for police motorcycles includes the following steps:
the method comprises the following steps: six lenses are adopted to form a look-around system, all the lenses are connected through rigid connecting pieces, and a transparent cover is sleeved outside the look-around system and used for protecting the look-around system;
the six-axis sensor is connected with each path of lens through a rigid connecting piece, and the synchronization and the consistency of the vibration parameters of the six-axis sensor and each path of lens are realized through the rigid connecting piece,
step two: collecting vibration parameters of six sensors to calculate vibration parameters of the multiple paths of lenses, and splicing the multiple paths of lenses in a static state;
the splicing process of the multiple paths of lenses in the static state is as follows:
the first step is as follows: performing static splicing correction, and creating a splicing parameter matrix for splicing pictures between adjacent lenses; the splicing parameter matrix comprises a wide parameter matrix a, a high parameter matrix b and a fine tuning parameter matrix w;
the second step is that: setting each lens pixel to behave as: width, height;
wherein (x1, y1) represents the pixel of the first lens, and the splicing parameter matrixes of the adjacent first lens and the adjacent second lens are respectively represented as a [1,2], b [1,2] and w [1,2 ];
the picture relationship after edge splicing is as follows: in the new frame, (x1, y1) in the scene-frame is represented by (x1 ', y 1');
x1`=x1*a[1,2]+y1*b[1,2]+w[1,2][x1,y1];
y1 ═ x1 a [1,2] + y1 b [1,2] + w [1,2] [ x1, y1 ]; by analogy, obtaining independent pixels formed by splicing all lens edges, wherein the independent pixels comprise (x1 ', y 1'), (x2 ', y 2'), (x2 ', y 3'), (x4 ', y 4'), (x5 ', y 5') and (x6 ', y 6');
the third step: splicing the generated independent pixels of each lens to realize splicing of multiple paths of lenses in a static state; the method specifically comprises the following steps:
X=[x1`+(w+x2`)+(w*2+x3`)+(w*3+x4`)+(w*4+x5`)+(w*5+x6`)];
y ═ Y1 ' + Y2 ' + Y3 ' + Y4 ' + Y5 ' + Y6; wherein (X, Y) represents the pixels of the spliced picture in the multi-path lens static state;
step three: under the condition of vibration, adding an additional value brought by the vibration to a fine adjustment parameter matrix w of the six-axis sensor, and then repeating the splicing process of the multiple paths of lenses in a static state to realize splicing; the method specifically comprises the following steps:
after vibration occurs, firstly, estimating the downward floating vibration of each lens through vibration parameters of six sensors, wherein the vibration parameters of the six sensors are six and are respectively a rotation angle of x \ y \ z and an acceleration of x \ y \ z; the distances from the six lenses to the six-axis sensor are equal and are denoted by d;
obtaining a rotation offset angle of each lens picture according to the rotation angle and the acceleration of x \ y \ z, thereby realizing the deflection calculation of the fine tuning parameter matrix w and obtaining the additional value brought by vibration;
adding an additional value brought by vibration into the fine-tuning parameter matrix w, and then repeating the splicing process of the multiple paths of lenses in a static state to realize splicing;
according to the invention, the synchronization and the consistency of the vibration parameters of the six-axis sensor and each camera are realized through the rigid connecting piece, then the vibration parameters of the multi-path lens are calculated through the vibration parameters of the six axes, the splicing of the multi-path lens in a static state is realized, under the vibration condition, only the additional value brought by the vibration needs to be added to the fine-tuning parameter matrix of the six-axis sensor, and the splicing efficiency and the accuracy are improved;
step four: optical flow tracking of multiple paths of pictures is carried out, reverse calculation and correction of vibration parameters of the whole rigid connecting piece are realized, the optical flow tracking and the correction are mutually complemented with a six-axis sensor, association of pixel-level vibration tracking and mechanical vibration detection is realized, and splicing efficiency and accuracy are improved; the method specifically comprises the following steps:
when each lens vibrates up and down, a deviation value exists between the pixels of the previous frame and the next frame, and the deviation value is recorded as the position of x \ y and the rotation angle deviation;
the deviation value is equal to an action distance value after six-axis rotation passes through the radius d, the deviation values of the six lenses are integrated together, and the unified calculation of the deflection angles of the six lenses is completed, so that the vibration parameters of the six-axis sensor are solved, and the reverse calculation of the vibration parameters of the whole rigid connecting piece is realized;
the solved vibration parameters of the six-axis sensor are compared with the vibration parameters collected by the six-axis sensor, so that the vibration parameters of the whole rigid connecting piece are corrected;
executing the third step according to the corrected vibration parameters to realize splicing;
according to the invention, through optical flow tracking of multiple paths of pictures, the reverse calculation and correction of vibration parameters of the whole rigid connecting piece are realized, and the vibration parameters are mutually complemented with six axes, so that the association of pixel-level vibration tracking and mechanical vibration detection is realized, and the splicing efficiency and accuracy are improved; the problems of edge dislocation, pixel drift and the like when the multi-channel video is subjected to all-round-looking splicing due to the fact that when the multi-channel video of the police motorcycle all-round-looking system is synchronously input and the camera is driven to generate micro-wave jitter due to high-frequency vibration of a motorcycle engine when the all-round-looking video is formed are solved; the definition and the accuracy of the video pictures are improved.
The working principle of the invention is as follows:
a method for removing shake of a look-around system of a police motorcycle comprises the steps that during working, six lenses are adopted to form the look-around system, all the lenses are connected through a rigid connecting piece, a transparent cover is sleeved outside the look-around system for protection, and through the rigid connecting piece, synchronization and consistency of vibration parameters of a six-axis sensor and all the cameras are achieved; firstly, performing static splicing correction, and creating a splicing parameter matrix for splicing pictures between adjacent lenses; obtaining independent pixels after splicing of all lens edges according to the splicing parameter matrix, and splicing the generated independent pixels of each lens to realize splicing of multiple paths of lenses in a static state; under the condition of vibration, the added value brought by the vibration is only added to a fine adjustment parameter matrix of the six-axis sensor, so that the splicing efficiency and accuracy are improved; meanwhile, reverse calculation and correction of vibration parameters of the whole rigid connecting piece are realized through optical flow tracking of multiple paths of pictures, the vibration parameters are mutually complemented with a six-axis sensor, correlation of pixel-level vibration tracking and mechanical vibration detection is realized, splicing efficiency and accuracy are improved, and the problems of edge dislocation, pixel drift and the like when multiple paths of videos are subjected to all-round splicing due to the fact that a camera is driven to generate micro wave jitter by high-frequency vibration of a motorcycle engine when a all-round video is formed and multi-path videos of an all-round system for police motorcycles are synchronously input are solved; the definition and the accuracy of the video pictures are improved.
In the description herein, references to the description of "one embodiment," "an example," "a specific example" or the like are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The preferred embodiments of the invention disclosed above are intended to be illustrative only. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best utilize the invention. The invention is limited only by the claims and their full scope and equivalents.