Disclosure of Invention
Aiming at the defects in the prior art, the tea bud screening method based on machine vision solves the problem of low tea bud screening precision in the prior art.
In order to achieve the aim of the invention, the technical scheme adopted by the invention is that the tea bud screening method based on machine vision comprises the following steps:
s1, screening suspicious bud area pixel points from a tea image to obtain a bud area significant filtering image;
S2, calculating a fluff coefficient value of the pixel point for the bud region significant filtering image;
S3, screening the nap pixel points according to the nap coefficient value, carrying out partition treatment on the nap pixel points, and then intercepting a nap region from the bud region significant filtering image;
s4, carrying out feature fusion on the pixel value and the nap coefficient value of the nap area by adopting a feature fusion unit to obtain fusion features;
S5, calculating bud morphology values of the villus regions, classifying the fusion characteristics by using a classification model, and obtaining tea bud regions based on characteristic enhancement of the bud morphology values.
Further, S1 comprises the following sub-steps:
s11, calculating the average value of pixel values of all pixel points on the tea image;
S12, marking pixel points with pixel values larger than or equal to the average value of the pixel values as suspected bud area pixel points on the tea image, and discarding other pixel points to obtain a bud area significant image;
s13, discarding isolated pixel points on the bud region significant image to obtain a bud region significant filter image.
Further, S2 comprises the following sub-steps:
s21, converting the bud region significant filtering image into an HSI color space, and extracting an I component;
s22, calculating the brightness contrast of each pixel point on the bud region significant filtering image according to the component I;
s23, performing remarkable filtering image gray scale processing on the bud region to obtain a gray scale value of each pixel point;
S24, calculating texture complexity of each pixel point on the bud region significant filtering image according to the gray value of each pixel point;
s25, adding the brightness contrast and texture complexity of the same pixel point to obtain a fluff coefficient value.
Further, the formula for calculating the brightness contrast in S22 is:
Wherein mui is the brightness contrast of the ith pixel point, Ii is the I component of the ith pixel point on the bud region significant filter image, Ic is the mean value of the I component in the neighborhood range of the ith pixel point on the bud region significant filter image, and I is the number of the pixel point on the bud region significant filter image.
Further, the formula for calculating the texture complexity in S24 is:
Wherein εi is the texture complexity of the ith pixel, Gi is the gray value of the ith pixel, Gi,j is the gray value of the jth pixel in the neighborhood of the ith pixel, Gc is the average gray value in the neighborhood, i is the number of the pixel on the image of the bud region significant filter, j is the number of the pixel in the neighborhood, and N is the size of the neighborhood.
Further, S3 comprises the following sub-steps:
S31, taking pixel points with the nap coefficient value larger than a nap coefficient threshold value as nap pixel points on the bud region significant filtering image;
s32, taking an undivided fluff pixel point as an initial growth point;
s33, judging the initial growth pointIf the non-partitioned villus pixel points exist in the neighborhood range, classifying the non-partitioned villus pixel points and the initial growing points into a candidate area, and jumping to the step S34, if not, classifying the initial growing points into a candidate area, and jumping to the step S32;
s34, taking a fluff pixel point which does not become an overgrowth point in the edge of the candidate area as a new growth point;
S35, judging new growth pointsWhether non-partitioned villus pixel points exist in the neighborhood range, if so, classifying the non-partitioned villus pixel points as the candidate region, and jumping to step S34, if not, directly jumping to step S34 until all edge fluff pixel points in the candidate areaNon-partitioned fluff pixel points do not exist in the neighborhood range;
S36, jumping to the step S32 until all fluff pixel points on the bud region significant filtering image have corresponding candidate regions;
S37, discarding candidate areas with the pixel points less than the threshold value;
S38, setting a minimum circumscribed rectangle in each remaining candidate area;
S39, cutting down the villus region from the bud region significant filtering image according to the position of the minimum circumscribed rectangle.
Further, the expression of the feature fusion unit in S4 is:
Wherein X is a fusion feature,Is thatA is the pixel value of the pile area, B is the pile coefficient value of the pile area,Is multiplied by element.
Further, the formula for calculating the bud morphology value in S5 is:
Where γ is the bud morphology value, length is the Length of the fluff region, and Width is the Width of the fluff region.
Further, the classification model in S5 comprises a CNN network, a characteristic enhancement unit and a full connection layer;
the input end of the CNN is used for inputting fusion characteristics, and the output end of the CNN is connected with the first input end of the characteristic enhancement unit;
the second input end of the characteristic enhancement unit is used for inputting bud morphology values, and the output end of the characteristic enhancement unit is connected with the input end of the full-connection layer;
The output end of the full connection layer is used as the output end of the classification model.
Further, the expression of the feature enhancement unit is: wherein Z is the output of the characteristic enhancing unit, Y is the output of the CNN network, and gamma is the bud morphology value.
In summary, the invention has the following beneficial effects:
1. According to the invention, the suspected bud area pixel points are screened from the tea image, and the filter treatment is carried out, so that the tea bud area is effectively enhanced, and the interference of other areas is reduced.
2. According to the invention, the fluff coefficient value of the pixel point is calculated for the bud region significant filtering image, so that the fluff characteristic condition of each pixel point is reflected, and the bud region characteristics are further represented.
3. According to the invention, the fluff pixel points are screened according to the fluff coefficient value, so that the zoning treatment of the fluff pixel points is realized, the area belonging to one tea bud is stripped from the bud area obvious filtering image, the interference of other areas is further reduced, and the bud area characteristics are highlighted.
4. According to the invention, the pixel value of the villus region and the villus coefficient value are subjected to feature fusion, so that the characteristic of tea buds is enhanced, the identification capability of the tea bud shape is improved, the screening accuracy is improved, and the robustness of the method under different illumination conditions and complex backgrounds is improved.
5. According to the invention, bud morphology values are calculated for the villus region, classification processing is carried out on the fusion characteristics by using a classification model, characteristic enhancement is carried out on the basis of the bud morphology values, interference of non-tea bud images is reduced, and screening precision is improved.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and all the inventions which make use of the inventive concept are protected by the spirit and scope of the present invention as defined and defined in the appended claims to those skilled in the art.
As shown in fig. 1, a tea bud screening method based on machine vision comprises the following steps:
s1, screening suspicious bud area pixel points from a tea image to obtain a bud area significant filtering image;
S2, calculating a fluff coefficient value of the pixel point for the bud region significant filtering image;
S3, screening the nap pixel points according to the nap coefficient value, carrying out partition treatment on the nap pixel points, and then intercepting a nap region from the bud region significant filtering image;
s4, carrying out feature fusion on the pixel value and the nap coefficient value of the nap area by adopting a feature fusion unit to obtain fusion features;
S5, calculating bud morphology values of the villus regions, classifying the fusion characteristics by using a classification model, and obtaining tea bud regions based on characteristic enhancement of the bud morphology values.
In the invention, the tea image is an RGB image.
In this embodiment, S1 includes the following sub-steps:
s11, calculating the average value of pixel values of all pixel points on the tea image;
S12, marking pixel points with pixel values larger than or equal to the average value of the pixel values as suspected bud area pixel points on the tea image, and discarding other pixel points to obtain a bud area significant image;
s13, discarding isolated pixel points on the bud region significant image to obtain a bud region significant filter image.
According to the method, the areas with obvious brightness and color characteristics in the tea image can be distinguished preliminarily by calculating the average value of the pixel values, the method can be used for filtering out the background and irrelevant areas quickly, focusing on the image areas possibly containing tea buds, compared with a traditional fixed threshold value method, the average value screening can be more suitable for the image characteristics under different illumination conditions, the points with the pixel values larger than or equal to the average value are marked as suspicious bud areas, the texture and color characteristics of the tea buds can be captured effectively, and a large amount of background noise is filtered preliminarily. The invention can further improve the structural integrity of the image by discarding the isolated pixel points in the bud region significant image, is beneficial to eliminating random noise points and interference in the image, and enhances the accuracy of subsequent image processing.
Isolated pixel pointsThere are no pixels in the neighborhood.
In this embodiment, S2 includes the following substeps:
s21, converting the bud region significant filtering image into an HSI color space, and extracting an I component;
s22, calculating the brightness contrast of each pixel point on the bud region significant filtering image according to the component I;
s23, performing remarkable filtering image gray scale processing on the bud region to obtain a gray scale value of each pixel point;
S24, calculating texture complexity of each pixel point on the bud region significant filtering image according to the gray value of each pixel point;
s25, adding the brightness contrast and texture complexity of the same pixel point to obtain a fluff coefficient value.
According to the invention, the bud region significant filtering image is converted into the HSI color space, the I component is extracted, the I component (Intensity) represents the brightness information of the color, the brightness change of the image can be accurately reflected, and the image is more sensitive to the fine brightness difference of the fluff on the tea bud surface.
Because the fluff is distributed on the tea buds in a staggered way, the brightness contrast of each pixel point on the obvious filtering image of the bud region can show the fluff characteristic. According to the invention, the distribution condition of the fluff is reflected by calculating the texture complexity through the gray value, and the fluff coefficient value is obtained by combining the brightness contrast and the texture complexity, so that the fluff recognition precision is improved.
In the present embodiment, the formula for calculating the brightness contrast in S22 is:
Wherein mui is the brightness contrast of the ith pixel point, Ii is the I component of the ith pixel point on the bud region significant filter image, Ic is the mean value of the I component in the neighborhood range of the ith pixel point on the bud region significant filter image, and I is the number of the pixel point on the bud region significant filter image.
The invention calculates the difference between the I component of each pixel point and the average value of the I components of the neighborhood pixel points, reflects the brightness difference condition, and sets the proportionality coefficient according to the brightness differenceThe larger the luminance difference is, the more remarkable the luminance contrast is.
Because the brightness value of the pixel point on the fluff is higher than that of the pixel point at other parts of the bud region, when the I component is smaller than or equal to the I component mean value of the neighborhood, the brightness contrast of the pixel point is 0.
In this embodiment, the formula for calculating the texture complexity in S24 is:
Wherein εi is the texture complexity of the ith pixel, Gi is the gray value of the ith pixel, Gi,j is the gray value of the jth pixel in the neighborhood of the ith pixel, Gc is the average gray value in the neighborhood, i is the number of the pixel on the image of the bud region significant filter, j is the number of the pixel in the neighborhood, and N is the size of the neighborhood.
In this embodiment, the size of the neighborhood range in steps S22 and S24 is:。
Because the villi are distributed in the bud area in a staggered way, the invention calculates the discrete condition of gray values in the neighborhood range of the pixel point, and the texture is more complex as the texture complexity is larger. Meanwhile, since the gray value of the pixel points on the fluff is higher than that of the pixel points at other parts of the bud region, when the gray value is smaller than the gray value average value in the neighborhood range, the texture complexity is set to 0.
In this embodiment, S3 includes the following substeps:
S31, taking pixel points with the nap coefficient value larger than a nap coefficient threshold value as nap pixel points on the bud region significant filtering image;
s32, taking an undivided fluff pixel point as an initial growth point;
s33, judging the initial growth pointIf the non-partitioned villus pixel points exist in the neighborhood range, classifying the non-partitioned villus pixel points and the initial growing points into a candidate area, and jumping to the step S34, if not, classifying the initial growing points into a candidate area, and jumping to the step S32;
s34, taking a fluff pixel point which does not become an overgrowth point in the edge of the candidate area as a new growth point;
S35, judging new growth pointsWhether non-partitioned villus pixel points exist in the neighborhood range, if so, classifying the non-partitioned villus pixel points as the candidate region, and jumping to step S34, if not, directly jumping to step S34 until all edge fluff pixel points in the candidate areaNon-partitioned fluff pixel points do not exist in the neighborhood range;
In step S35, all edge pile pixel points in the candidate regionWhen the non-partitioned fluff pixel points do not exist in the neighborhood range, the fact that the fluff pixel points do not exist at the periphery of the candidate region is indicated, so that the candidate region cannot be continuously expanded, and the candidate region is distributed completely;
S36, jumping to the step S32 until all fluff pixel points on the bud region significant filtering image have corresponding candidate regions;
in step S36, step S32 is skipped to find the next candidate region;
S37, discarding candidate areas with the pixel points less than the threshold value;
S38, setting a minimum circumscribed rectangle in each remaining candidate area;
S39, cutting down the villus region from the bud region significant filtering image according to the position of the minimum circumscribed rectangle.
In the present invention, since the fluff is discretely distributed, the neighborhood range is set toAnd the fluff pixel points are conveniently searched.
The shape and position of the fluff region in step S39 are the same as the minimum bounding rectangle.
According to the invention, an undivided fluff pixel point is taken as an initial growth point, and the fluff pixel points belonging to a neighborhood range are gradually brought into a zone in a neighborhood searching mode, so that the clustering of the fluff pixel points adjacent in position is realized.
The fluff coefficient threshold is a threshold set for the fluff coefficient value, and the number threshold and the fluff coefficient threshold are specifically set according to experiments or experience.
In this embodiment, the minimum bounding rectangle is the smallest rectangle that can completely cover the candidate region.
The invention realizes the segmentation of the villus region belonging to one bud through the step S3, so that S4 and S5 classify the single villus region and identify whether the villus region belongs to tea buds or not.
As shown in fig. 2, the expression of the feature fusion unit in S4 is:
Wherein X is a fusion feature,Is thatA is the pixel value of the pile area, B is the pile coefficient value of the pile area,Is multiplied by element.
The pile area itself is a two-dimensional area, and since part of the pixel points are discarded in step S12, there may be a partial missing of the pixel values of the pile area, the missing positions are filled with 0, the invention adopts two convolution layers to respectively process the pixel value of the villus region and the villus coefficient value of the villus region, does not change the image size, multiplies the pixel value characteristic and the villus coefficient value characteristic which belong to the same pixel point position, realizes characteristic fusion, and enhances the identification capability of the sprout region.
In this embodiment, the formula for calculating the bud morphology value in S5 is:
Where γ is the bud morphology value, length is the Length of the fluff region, and Width is the Width of the fluff region.
Because of the shape of the tea bud strip, the invention introduces bud morphology values and shows the state of length-width ratio.
As shown in fig. 2, the classification model in S5 includes a CNN network, a feature enhancement unit, and a full connection layer;
the input end of the CNN is used for inputting fusion characteristics, and the output end of the CNN is connected with the first input end of the characteristic enhancement unit;
the second input end of the characteristic enhancement unit is used for inputting bud morphology values, and the output end of the characteristic enhancement unit is connected with the input end of the full-connection layer;
The output end of the full connection layer is used as the output end of the classification model.
In the present embodiment, the expression of the feature enhancing unit is:
wherein Z is the output of the characteristic enhancing unit, Y is the output of the CNN network, and gamma is the bud morphology value.
According to the invention, after the CNN network is adopted to extract the characteristics, the bud morphology value is adopted to enhance the characteristics, so that buds with larger length-width ratio have higher characteristic values, and the screening precision is improved.
According to the invention, the suspected bud area pixel points are screened from the tea image, and the filter treatment is carried out, so that the tea bud area is effectively enhanced, and the interference of other areas is reduced.
According to the invention, the fluff coefficient value of the pixel point is calculated for the bud region significant filtering image, so that the fluff characteristic condition of each pixel point is reflected, and the bud region characteristics are further represented.
According to the invention, the fluff pixel points are screened according to the fluff coefficient value, so that the zoning treatment of the fluff pixel points is realized, the area belonging to one tea bud is stripped from the bud area obvious filtering image, the interference of other areas is further reduced, and the bud area characteristics are highlighted.
According to the invention, the pixel value of the villus region and the villus coefficient value are subjected to feature fusion, so that the characteristic of tea buds is enhanced, the identification capability of the tea bud shape is improved, the screening accuracy is improved, and the robustness of the method under different illumination conditions and complex backgrounds is improved.
According to the invention, bud morphology values are calculated for the villus region, classification processing is carried out on the fusion characteristics by using a classification model, characteristic enhancement is carried out on the basis of the bud morphology values, interference of non-tea bud images is reduced, and screening precision is improved.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.