Disclosure of Invention
In order to overcome the problems in the related art, the invention discloses a battery defect detection method and system based on a multi-mode sensor.
According to a first aspect of the disclosed embodiments of the present invention, there is provided a battery defect detection method based on a multi-modal sensor, the method being applied to a battery defect detection system based on a multi-modal sensor, the system including an image acquisition module, a tactile sensor module, a processor, and a motion control platform, the method including:
The image acquisition module is used for acquiring images of the battery to be detected on the motion control platform to acquire images to be detected;
analyzing the image to be detected by the processor according to a preset image processing algorithm, determining a target area suspected to have defects in the image, and extracting visual features corresponding to the target area;
Scanning the target area through the touch sensor, and extracting touch characteristics through the processor according to a scanning result of the target area;
Obtaining, by the processor, a visual preliminary defect probability corresponding to the visual feature using a random forest algorithm, and obtaining a haptic preliminary defect probability corresponding to the haptic feature;
and acquiring target defect probability under target probability distribution according to the visual preliminary defect probability and the tactile preliminary defect probability by the processor and the D-S theory so as to determine whether the target region has target defects or not according to the target defect probability.
Optionally, the obtaining, by the processor, the target defect probability under the target probability distribution according to the visual preliminary defect probability and the tactile preliminary defect probability and the D-S theory, so as to determine whether the target region has the target defect according to the target defect probability, includes:
Converting the visual preliminary defect probability and the tactile preliminary defect probability into visual defect basic probability distribution and tactile defect basic probability distribution of a D-S theory, and calculating conflict coefficients of an image acquisition module and a tactile sensor;
If the conflict coefficient is larger than or equal to a preset coefficient threshold, correcting the visual defect basic probability distribution and the tactile defect basic probability distribution through the weight coefficient of the image acquisition module and the weight coefficient of the tactile sensor, and acquiring target defect probability under target probability distribution according to the corrected visual defect basic probability distribution and the corrected tactile defect basic probability distribution;
If the conflict coefficient is smaller than a preset coefficient threshold value, acquiring target defect probability under target probability distribution according to the visual defect basic probability distribution and the tactile defect basic probability distribution;
If the target defect probability is greater than or equal to a preset defect threshold corresponding to the target defect, determining that the target region has the target defect;
And if the target defect probability is smaller than a preset defect threshold corresponding to the target defect, determining that the target region does not have the target defect.
Optionally, the obtaining the target defect probability under the target probability distribution according to the corrected visual defect basic probability distribution and the corrected haptic defect basic probability distribution, or the obtaining the target defect probability under the target probability distribution according to the visual defect basic probability distribution and the haptic defect basic probability distribution includes:
Determining target defect probability according to the corrected visual defect basic probability distribution and the corrected tactile defect basic probability distribution or according to the visual defect basic probability distribution and the tactile defect basic probability distribution through a preset target defect probability formula,
The target defect probability formula is as follows:
, a target defect probability in the target probability allocation,The probability of defects in the visual defect basic probability distribution is given,K is a collision coefficient, wherein K is the probability of defects in the basic probability of the tactile defects.
Optionally, the correcting the visual defect basic probability distribution and the tactile defect basic probability distribution through the weight coefficient of the image acquisition module and the weight coefficient of the tactile sensor includes:
Correcting the visual defect basic probability distribution through the weight coefficient of the image acquisition module to obtain corrected visual defect basic probability distribution, wherein,The probability of defects in the visual defect basic probability distribution is given,The weight coefficient of the image acquisition module;
Correcting the haptic defect basic probability distribution through a haptic sensor weight coefficient to obtain corrected haptic defect basic probability distribution, wherein,For the probability of defects in the haptic defect base probability,Is a haptic sensor weight coefficient.
Optionally, the motion control platform comprises a mechanical arm and a placement table, the placement table is used for placing a battery to be detected, the mechanical arm is used for moving the battery to be detected, the image acquisition module is used for acquiring images of the battery to be detected on the motion control platform, and the image acquisition module comprises:
acquiring a battery to be detected on the placing table through an image acquisition module, and acquiring an image to be detected of one detection surface of the battery to be detected;
And moving the battery to be detected in the process of acquiring the images by the mechanical arm so as to ensure that the images to be detected of all detection surfaces of the battery to be detected are acquired.
Optionally, a planar coordinate system is disposed on the placement table, and the scanning, by the tactile sensor, the target area includes:
determining a coordinate position of the target area in the plane coordinate system;
Determining a contact path of the tactile sensor when scanning the target area according to the coordinate position;
And scanning the target area according to the contact path through the touch sensor.
According to a second aspect of the disclosed embodiments of the present invention, there is provided a multi-modal sensor-based battery defect detection system for implementing the multi-modal sensor-based battery defect detection method according to the first aspect of the disclosed embodiments of the present invention, the system including an image acquisition module, a tactile sensor module, a processor, and a motion control platform;
The image acquisition module and the touch sensor module are arranged above the motion control platform and are respectively and electrically connected with the processor, and the motion control platform is used for placing the battery to be detected.
Optionally, the motion control platform includes a mechanical arm and a placement table, the placement table is used for placing the battery to be detected, and the mechanical arm is used for moving the battery to be detected.
Optionally, the mechanical arm is configured to move the battery to be detected in the process of acquiring an image, so as to ensure that the image of each detection surface of the battery to be detected is acquired, and after the processor analyzes the image of the battery to be detected and determines a target area through an image processing algorithm, move the battery to be detected to align a detection surface corresponding to the target area to the touch sensor.
Optionally, a plane coordinate system is arranged on the placing table.
In summary, through the technical scheme in the disclosed embodiments of the present invention, the following beneficial effects can be brought:
(1) Firstly classifying visual features and tactile features by using a random forest, outputting preliminary defect probability, fusing confidence coefficients of a plurality of sensors by using a D-S theory, and improving the reliability of final judgment by adopting a multi-mode fusion algorithm;
(2) The high-resolution industrial camera and the device with low equivalent lattice of the array type touch sensor are adopted as the image acquisition module and the touch sensor, so that the cost in the battery defect detection process is effectively controlled relative to the 3D camera with high price;
(3) The suspected defects of the target area are further verified through the touch sensor, the area which is detected by the touch sensor in a focused mode is concentrated in the target area, the problem that the working efficiency of the touch sensor is low can be solved, and the defect detection efficiency of the battery is effectively improved.
Additional features and advantages of the present disclosure will be set forth in the detailed description which follows.
Detailed Description
The following describes in detail the embodiments of the present disclosure with reference to the drawings. It should be understood that the detailed description and specific examples, while indicating and illustrating the disclosure, are not intended to limit the disclosure.
The battery defect detection method in the embodiment of the invention collects the battery shell data with multiple dimensions by combining visual detection and tactile detection, fuses the data with multiple dimensions (2D visual data+3D tactile data) to obtain a high-precision detection result, and identifies the defects on the surface of the battery shell. The battery defect detection method is applied to a battery defect detection system based on a multi-mode sensor, and the system comprises an image acquisition module, a touch sensor module, a processor and a motion control platform, wherein the image acquisition module and the touch sensor module are respectively used for acquiring data of visual dimension and touch dimension, the processor is used for analyzing the data to obtain a defect detection result after fusion processing is carried out on the data, the motion control platform can be used for placing a battery shell to be detected, and each battery shell is moved to enable each battery shell to face the image acquisition module or the touch sensor to be detected.
Fig. 1 is a flow chart illustrating a method for detecting defects of a battery based on a multi-mode sensor according to an exemplary embodiment, and the method includes:
In step 101, the image acquisition module acquires an image to be detected of the battery to be detected on the motion control platform.
Illustratively, the motion control platform includes a robotic arm for positioning the battery to be inspected and a positioning table for moving the battery to be inspected. It can be understood that the housing of each battery to be detected comprises a plurality of detection surfaces, and when the image acquisition module acquires the image to be detected, the image to be detected of each detection surface needs to be completely acquired.
Specifically, fig. 2 is a schematic flow chart of an image acquisition method according to fig. 1, and as shown in fig. 2, the step 101 includes:
In step 1011, the image acquisition module acquires the image to be detected of the detection surface of the battery to be detected by acquiring the battery to be detected on the placement table.
In step 1012, the battery to be detected is moved by the mechanical arm during the image capturing process, so as to ensure that the images to be detected on all the detection surfaces of the battery to be detected are captured.
For example, the battery to be detected of the placement table is moved by the mechanical arm, so that each detection surface is aligned to the image acquisition module, and the image acquisition of the detection surface is completed.
In step 102, the processor analyzes the image to be detected according to a preset image processing algorithm, determines a target area suspected to have a defect in the image, and extracts visual features corresponding to the target area.
For example, when the processor analyzes the image to be detected, each detection surface is analyzed separately. The preset image processing algorithm comprises a multi-light source imaging algorithm, an edge enhancement algorithm, a contour extraction algorithm and other existing image processing modes. The specific step of analyzing the image to be detected according to a preset image processing algorithm is to fuse the images to be detected of the detection surfaces acquired by the image acquisition modules with different angles aiming at each detection surface so as to highlight defect characteristics. And (3) calculating the image gradient of the image to be detected of the detection surface by using a Sobel or Scharr operator, strengthening linear characteristics (for example, scratches are usually represented as long-strip-shaped high-gradient areas), determining the outline of a welding line area by using an outline extraction technology, and highlighting a concave area by adopting a local contrast enhancement mode and the like. In addition, the oxidation area, the size deviation area, the weld pore area, the crack area, the welding slag area, the cold joint area and the like on the surface of the battery shell can also be determined by the existing image processing algorithm, and the detailed description is omitted in the disclosed embodiment of the invention. It is understood that the weld region, the recess region, the weld void region, the crack region, and the like are target regions suspected of having defects. The target area is called as suspected defect, and the 2D visual data acquired by the image acquisition module is easily affected by reflection of light on the outer surface of the metal and other environmental light, so that a certain deviation exists in the detection result. In order to ensure the accuracy of the detection results of the defects of each detection surface of the battery case, the defect detection results obtained through the 2D visual data are called target areas suspected to have defects, and the suspected defects are further verified through the following steps 103-105.
The visual features corresponding to the target region include texture features (LBP, haralick), shape descriptors (Hu moments), and deep learning features (CNN middle layer output).
In step 103, the target area is scanned by the tactile sensor, and tactile features are extracted by the processor according to the scanning result of the target area.
Illustratively, the "suspected defects" of the target area are further verified by the tactile sensor, touching the target area weld zone contact force/pressure distribution, cold joint, structural deformation, scratches, pits, bump deformation, etc. The tactile features corresponding to the target area comprise pressure distribution mean/variance, force sense time series features (zero crossing rate) and surface roughness.
The placement stage is provided with a planar coordinate system, specifically, fig. 3 is a flowchart illustrating a method for scanning a target area by using a touch sensor according to fig. 1, and as shown in fig. 3, the step 103 includes:
In step 1031, a coordinate position of the target area in the planar coordinate system is determined.
In step 1032, a contact path is determined for the tactile sensor as it scans the target area based on the coordinate location.
In step 1033, the target area is scanned by the tactile sensor according to the contact path.
For example, the planar coordinate system uses the vertex of the lower left corner of the placing table as the center of a circle, uses the lower side edge of the placing table as the straight line where the x axis is located, uses the left side edge of the placing table as the straight line where the y axis is located, and each point in the placing table can find the corresponding coordinate position in the planar coordinate system. After the coordinate position corresponding to the target area is determined, a reasonable touch sensor scanning contact path can be set according to the position of the target area and the size of the target area, so that the touch sensor scans according to the specified contact path.
In addition, after the target area is determined, the detection surface corresponding to the target area can be aligned to the touch sensor by moving the battery to be detected through the mechanical arm, so that the touch sensor can conveniently execute the next scanning action.
In step 104, a visual preliminary defect probability corresponding to the visual feature is obtained by the processor using a random forest algorithm, and a haptic preliminary defect probability corresponding to the haptic feature is obtained.
Illustratively, a random forest classifier is established, visual features and tactile features are respectively used as inputs of the random forest classifier, and visual preliminary defect probability and tactile preliminary defect probability are obtained according to the output of the random forest classifier.
In step 105, the processor obtains a target defect probability under a target probability distribution according to the visual preliminary defect probability and the tactile preliminary defect probability and the D-S theory, so as to determine whether a target defect exists in the target area according to the target defect probability.
By way of example, the probability distribution of the target defects is obtained by converting the probability of the visual preliminary defects and the probability of the tactile preliminary defects into the output confidence under the D-S theory, so as to determine whether the target defects exist in the target area. Among them, the D-S theory is called Dempster-Shafer theory. Is a method of dealing with uncertainty problems, the uncertainty information being described by "interval estimation" rather than "point estimation". In the embodiment of the invention, the visual features and the tactile features are classified by utilizing a random forest, the preliminary defect probability is output, the confidence coefficients of a plurality of sensors are fused by utilizing a D-S theory, and the reliability of final judgment is improved by adopting a multi-mode fusion algorithm.
Specifically, fig. 4 is a flow chart of a target defect determining method according to the method shown in fig. 1, and as shown in fig. 4, the step 105 includes:
in step 1051, the visual preliminary defect probability and the haptic preliminary defect probability are converted into a visual defect basic probability distribution and a haptic defect basic probability distribution of the D-S theory, and collision coefficients of the image acquisition module and the haptic sensor are calculated.
Illustratively, the calculation formula of the collision coefficient K is:, The probability of defects in the visual defect base probability is assigned,Is the probability of a defect in the basic probability of the haptic defect.
In step 1052, if the conflict coefficient is greater than or equal to the preset coefficient threshold, the visual defect basic probability distribution and the tactile defect basic probability distribution are corrected by the image acquisition module weight coefficient and the tactile sensor weight coefficient, and the target defect probability under the target probability distribution is obtained according to the corrected visual defect basic probability distribution and the corrected tactile defect basic probability distribution.
For example, if the conflict coefficient is greater than or equal to the preset coefficient threshold, it is indicated that there is a high contradiction between the 2D visual data collected by the image collection module and the 3D tactile data collected by the tactile sensor, and at this time, the visual defect basic probability distribution and the tactile defect basic probability distribution need to be corrected.
Specifically, the visual defect basic probability distribution and the tactile defect basic probability distribution are corrected through the weight coefficient of the image acquisition module and the weight coefficient of the tactile sensor, and the method comprises the following steps:
Correcting the visual defect basic probability distribution through the weight coefficient of the image acquisition module to obtain corrected visual defect basic probability distribution, wherein,The probability of defects in the visual defect base probability is assigned,And the weight coefficient is an image acquisition module.
Correcting the haptic defect basic probability distribution through the haptic sensor weight coefficient to obtain corrected haptic defect basic probability distribution, wherein,To be a probability of a defect in the basic probability of a haptic defect,Is a haptic sensor weight coefficient. The weight coefficient of the image acquisition module and the weight coefficient of the touch sensor are obtained through experiments.
In step 1053, if the conflict factor is smaller than the preset factor threshold, the target defect probability under the target probability distribution is obtained according to the visual defect basic probability distribution and the haptic defect basic probability distribution.
For example, if the conflict coefficient is smaller than the preset coefficient threshold, it is indicated that the 2D visual data collected by the image collection module and the 3D tactile data collected by the tactile sensor are not in high contradiction, and the visual defect basic probability distribution and the tactile defect basic probability distribution do not need to be corrected, so that the target defect probability can be determined directly through the visual defect basic probability distribution and the tactile defect basic probability distribution.
Specifically, the method for obtaining the target defect probability under the target probability distribution according to the corrected visual defect basic probability distribution and the corrected tactile defect basic probability distribution, or obtaining the target defect probability under the target probability distribution according to the visual defect basic probability distribution and the tactile defect basic probability distribution comprises the following steps:
Determining target defect probability according to the corrected visual defect basic probability distribution and the corrected tactile defect basic probability distribution or the visual defect basic probability distribution and the tactile defect basic probability distribution through a preset target defect probability formula,
The target defect probability formula is as follows:
, for a target defect probability in the target probability allocation,The probability of defects in the visual defect base probability is assigned,K is a collision coefficient, which is the probability of defects in the basic probability of the haptic defects.
In step 1054, if the target defect probability is greater than or equal to the preset defect threshold corresponding to the target defect, it is determined that the target region has the target defect.
In step 1055, if the target defect probability is smaller than the preset defect threshold corresponding to the target defect, it is determined that the target region has no target defect.
By way of example, it can be appreciated that the types of target defects include weld seams, pits, oxidation, slag, cold joint, etc., the annotation library contains a preset defect threshold corresponding to each target type, and the target defect type can be preliminarily determined through analysis of the 2D visual data+3D tactile data, if the target defect probability is greater than or equal to the preset defect threshold corresponding to the target defect type, the target defect is determined to exist in the target area, otherwise, the target defect is determined not to exist in the target area. Aiming at a preset defect threshold value in a standard library, a high-quality labeling library (comprising main defect types) can be established in an initial stage, active learning is introduced in a middle stage to reduce incremental cost, and high-efficiency sustainable model optimization is realized by long-term combination of data and transfer learning.
Fig. 5 is a schematic structural diagram of a battery defect detection system based on a multi-mode sensor according to an exemplary embodiment, as shown in fig. 5, the system 500 includes an image acquisition module 510, a touch sensor module 520, a processor (not shown in the figure), and a motion control platform (including a mechanical arm 531 and a placement platform 532), where the image acquisition module 510 and the touch sensor module 520 are disposed above the motion control platform (an image acquisition module 510 may be additionally disposed on a side of the motion control platform to acquire images of other detection surfaces of a battery to be detected), and are electrically connected to the processor, respectively, where the motion control platform is used for placing the battery to be detected.
Specifically, the image acquisition module 510 may be a high resolution industrial camera, containing circularly polarized light sources to suppress reflections and structured light, and may emit light sources of different angles from different colors to suppress reflections. The tactile sensor 520 may be an array type tactile sensor. The high-resolution industrial camera and the array type touch sensor are low in price, and cost in the battery defect detection process can be effectively controlled.
The image acquisition module 510 is used for acquiring images of a battery to be detected on the motion control platform to obtain images to be detected, the touch sensor 520 is used for scanning the target area, the processor is used for analyzing the images to be detected according to a preset image processing algorithm to determine a target area suspected to be defective in the images and extract visual features corresponding to the target area, the touch features are extracted according to the scanning result of the target area, the random forest algorithm is used for acquiring visual preliminary defect probability corresponding to the visual features and tactile preliminary defect probability corresponding to the touch features, the target defect probability distributed according to the visual preliminary defect probability and the tactile preliminary defect probability is acquired according to the D-S theory, and whether the target area has target defects or not is determined according to the target defect probability.
The motion control platform comprises a mechanical arm 531 and a placing table 532, wherein the placing table 532 is used for placing the battery to be detected, the mechanical arm 531 is used for moving the battery to be detected, the battery to be detected is moved in the process of collecting images so as to ensure that the images of all detection surfaces of the battery to be detected are collected, and after the processor analyzes the images of the battery to be detected through an image processing algorithm and determines a target area, the processor moves the battery to be detected to align the detection surfaces corresponding to the target area to the touch sensor.
The placement table is provided with a planar coordinate system. The method comprises the steps of determining a coordinate position of a target area in the plane coordinate system, determining a contact path of the touch sensor when the touch sensor scans the target area according to the coordinate position, and scanning the target area according to the contact path through the touch sensor.
In summary, the invention discloses a battery defect detection method and system based on a multi-mode sensor, the method comprises the steps of acquiring images of a battery to be detected, acquiring the images to be detected, analyzing the images to be detected according to a preset image processing algorithm, determining a target area suspected to be defective and visual characteristics in the images, scanning the target area, extracting tactile characteristics, acquiring visual preliminary defect probability corresponding to the visual characteristics and tactile preliminary defect probability corresponding to the tactile characteristics by utilizing a random forest algorithm, acquiring target defect probability under target probability distribution according to the visual preliminary defect probability and the tactile preliminary defect probability, and determining whether target defects exist in the target area according to the target defect probability. The method can combine the tactile sensor with the image acquisition, the 2D visual data and the 3D tactile data are fused in a multi-mode, the reliability of final judgment is improved, and the cost of the detection equipment is effectively controlled.
The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details of the embodiments described above, and various simple modifications may be made to the technical solutions of the present disclosure within the scope of the technical concept of the present disclosure, and all the simple modifications belong to the protection scope of the present disclosure.
In addition, the specific features described in the foregoing embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, the present disclosure does not further describe various possible combinations.
Moreover, any combination between the various embodiments of the present disclosure is possible as long as it does not depart from the spirit of the present disclosure, which should also be construed as the disclosure of the present disclosure.