Device and method for obtaining multipoint chromaticity coordinate values of surface of shot objectTechnical FieldThe invention relates to the fields of color vision, colorimetry, color communication, image processing, computer control, deep learning, application mathematics and the like, in particular to a device and a method for acquiring multipoint chromaticity coordinate values of the surface of a shot object.
BackgroundThe accurate, quick and low-cost acquisition of the multipoint chromaticity coordinate values of the surface of the shot object is an important basis for color communication, the chromaticity coordinate values of the surface of the shot object are mainly acquired by using a colorimeter or a spectrophotometer at present, the chromaticity coordinate values of a single point can be only acquired each time, and the measurement is easily influenced by the three-dimensional shape of the surface of the shot object, so that the measurement result is inaccurate. Although the hyperspectral camera can be used for obtaining the multipoint spectrum distribution of the surface of the measured object so as to calculate the chromaticity coordinate values of each point, the application of the hyperspectral camera is limited by the high price of the hyperspectral camera. The common color camera can only obtain three-channel color images of the measured object, but cannot accurately obtain the multipoint chromaticity coordinate values of the surface of the measured object.
Disclosure of Invention
The invention provides a device and a method for shooting a group image of a shot object and a color block group under irradiation of a multispectral distribution light source by using an image sensor and further calculating multipoint chromaticity coordinate values of the surface of the shot object by using the group image, which solve the problems of difficult measurement, inaccurate measurement, high measurement cost and the like of the multipoint chromaticity coordinate values of the surface of the shot object.
The present invention relates to various aspects of an apparatus and method. Embodiments of the invention may include one or any combination of one or more of the different aspects described herein.
In a first aspect, there is provided an apparatus for acquiring multi-point chromaticity coordinate values of a surface of a subject, comprising a light source group including a plurality of light sources of different spectral distributions, a color block group including a plurality of color blocks having different reflection spectral distributions, a control unit controlling the light source group to irradiate the subject and the color block group with the light sources of different spectral distributions in different periods of time, and controlling the imaging unit to synchronously acquire composite images of the plurality of subjects and the color block group under the irradiation of the light sources of different spectral distributions in corresponding periods of time, the image division alignment unit dividing an image including a single subject and an image including a single color block from the composite images, and aligning the images of the same subject and the same color block obtained under different light source irradiation conditions, respectively, the chromaticity coordinate value calculation unit calculating the multi-point chromaticity coordinate values on the surface of the subject using a regression model.
In a second aspect, there is provided a method of acquiring multi-point chromaticity coordinate values of a surface of a subject, acquiring a group image of a subject and a group of color patches including a plurality of color patches having different reflection spectrum distributions under irradiation of a light source group having different spectrum distributions, dividing an image including a single subject and an image including a single color patch from the group image, respectively aligning images obtained under different light source irradiation conditions of the same subject and the same color patch, and calculating the multi-point chromaticity coordinate values on the surface of the subject by using a regression model.
The half-width range near the peak wavelength of the light source spectrum distribution of at least part of the light source groups is intersected with the half-width range near the sensitive peak wavelength of the S-type, M-type and L-type three human cone cells.
The color lump group comprises a plurality of color lumps which respectively have different reflection spectrum distribution in the half-height wide range near the sensitive peak wavelength of three kinds of S-shaped, M-shaped and L-shaped human cone cells.
The calculating step of the chromaticity coordinate value calculating unit comprises the following steps:
Firstly, establishing a regression model, using color channel component values of color block images acquired under the irradiation of light sources with different spectral distributions as model input data, using chromaticity coordinate values of corresponding color blocks as model output marking data, using color channel component values of shot object images acquired under the irradiation of light sources with different spectral distributions as model input data, using chromaticity coordinate values of corresponding shot object corresponding positions acquired in other modes as model output marking data, training the regression model, and determining regression model parameters;
and then, using a trained regression model, inputting color component values of the shot object image acquired under the irradiation of the light sources with different spectral distributions, and outputting chromaticity coordinate values corresponding to the shot object.
DrawingsThe invention may be better understood by the following preferred but non-limiting embodiments, which are described in detail in conjunction with the accompanying drawings, in which:
FIG. 1 is a schematic view of a photographic image of a subject and a color patch set under illumination from a multi-spectral light source set;
FIG. 2 is a schematic diagram showing a group image of a subject and a color patch group segmented and aligned by an image segmentation and alignment unit;
FIG. 3 is a schematic diagram of a chromaticity coordinate calculation unit;
FIG. 4 shows the spectral distribution curves of a multispectral light source set and the spectral sensitivity curves of three human cone cells of S type, M type and L type;
FIG. 5 shows the reflectance spectrum distribution curves of color lump sets and the spectrum sensitivity curves of three human cone cells of S type, M type and L type;
Fig. 6 is a schematic diagram of a neural network regression model.
Detailed DescriptionIn order to make the technical problems and technical solutions solved by the present invention more clear, the technical solutions in the embodiments of the present invention are further clearly and completely described below with reference to the drawings and the embodiments of the present invention. The described embodiments are some, but not all, embodiments of the invention. The description herein is intended to be illustrative of the invention and is not to be construed as limiting the invention. For the sake of clarity, some features which are not particularly important for the understanding of the invention or which are obvious to a person skilled in the relevant art may not be shown.
This embodiment is an embodiment of the invention in which a plurality of teeth are photographed.
The light source groups in the embodiment are all realized by using light emitting diodes, and nine LEDs which emit light with different colors are respectively red, cold white, medium white, purple, warm white, green, yellow, white and blue LEDs. In the embodiment, a plurality of tooth objects are photographed to obtain the coordinate values of the chromaticity of the tooth surface at a plurality of points. The color block group in this embodiment includes 24 color blocks. The image collected by the image sensor in this embodiment is a three-channel color image.
Referring to fig. 1, a light source group 101 includes a plurality of light sources with different spectral distributions, a control unit 102 controls the light source group 101 to irradiate a subject 105 and a color block group 104 with light rays 107 emitted by the light sources with different spectral distributions in different time periods, the color block group 104 includes a plurality of color blocks 106 with different spectral absorption rates of wavelengths, and the control unit 102 controls an imaging unit 103 to synchronously acquire a plurality of combined images of the subject and the color block group under irradiation of the light sources with different spectral distributions in corresponding time periods.
The control unit can be realized by a general microcontroller, the imaging unit can be realized by a general color CMOS image sensor and an optical lens, and the control unit can be realized by a general relay for controlling the on and off of the LED light source.
Referring to fig. 2, a group image 201 captured under irradiation of light sources of different spectral distributions acquired by an imaging unit is processed by an image division alignment unit 202, first an image 203 of a color patch group is divided from an image 205 of a subject, then an image 206 containing a single subject and an image 204 containing a single color patch are further divided, and images obtained under irradiation of different light sources of the same subject and the same color patch are aligned, respectively.
The image instance segmentation method based on the depth neural network, which is known at present, for example, mask RCNN, can be used to segment the color block group image and the subject image from the group image and further segment the image containing a single subject and the image containing a single color block.
Firstly, 50 front full-mouth teeth and color block group composite images to be tested are shot, the outline of a single tooth and the serial numbers of the teeth are marked in the composite images by using a known image marking tool, and the outline of a single color block and the serial numbers of the color blocks are marked at the same time. On the basis of a pre-trained Mask RCNN network known in the industry, the acquired single tooth image, single color block image and corresponding labeling data are used, and the training image example segmentation depth neural network Mask RCNN is continued by using a known method to obtain a trained neural network. And performing image instance segmentation on the combined image of the teeth and the color patches by using the trained neural network to obtain a single tooth image and a single color patch image.
Images of the same instance under different illumination conditions after segmentation can be aligned using their geometric centers as reference points under different illumination conditions.
Referring to fig. 3, a chromaticity coordinate value calculation unit 301 first uses color channel component values of a color patch image 303 obtained under irradiation of light sources of different spectral distributions as model input data, chromaticity coordinate values 302 of the corresponding color patch as model output annotation data, and color channel component values of a subject image 304 obtained under irradiation of light sources of different spectral distributions as model input data, trains the regression model 306 by using chromaticity coordinate values 305 of the corresponding subject corresponding positions obtained by other means as model output annotation data, determines the regression model 306 parameters, and then inputs color components of the subject image 304 obtained under irradiation of light sources of different spectral distributions in the regression model 306, calculates by the regression model 306, and outputs chromaticity coordinate values 307 corresponding to the subject.
The chromaticity coordinate values of the subject as the regression model training data may be acquired by other apparatuses that measure chromaticity coordinates, for example, using a hyperspectral camera. In this case, a colorimetric dental plate may be used as a photographic subject instead of a real tooth, so as to facilitate acquisition of chromaticity coordinate values of the tooth ratio chromatogram.
Referring to fig. 4, at least a portion of the light source spectrum distribution 404 of the light source set has a half-width range 405 around the peak wavelength that intersects with half-width ranges 406 around the sensitivity peak wavelengths of three human Cone cells, S-type (S-Cone), M-type (M-Cone), L-type (L-Cone). A spectral sensitivity curve 401 for S-type human cones, a spectral sensitivity curve 402 for M-type human cones, and a spectral sensitivity curve 403 for L-type human cones.
The spectral distribution curves corresponding to the red, cool white, medium white, purple, warm white, green, yellow, white and blue LED light sources are shown in the figure by thin solid lines. The thick dotted lines in the figure show the spectral sensitivity curves of three human cone cells of S type, M type and L type. In the graph, the abscissa corresponding to the curve is wavelength, the unit is nanometer, the ordinate is the value after normalization, the light source is corresponding to the intensity, and the S-type, M-type and L-type cone cells are corresponding to the sensitivity.
Referring to fig. 5, the color patch group includes a plurality of color patches having different reflection spectrum distributions in a half-width range 505 around the sensitivity peak wavelength of three kinds of human cone cells of S-type, M-type, and L-type, respectively. A spectrum sensitivity curve 501 of S-type human cone cells, a spectrum sensitivity curve 502 of M-type human cone cells, a spectrum sensitivity curve 503 of L-type human cone cells, and a reflection spectrum distribution curve 504 of color blocks.
The reflection spectrum distribution curves corresponding to 24 color patches are shown in fig. 5, and are shown by thin solid lines. The thick dotted lines in the figure show the spectral sensitivity curves of three human cone cells of S type, M type and L type. In the graph, the abscissa corresponding to the curve is wavelength, the unit is nanometer, the ordinate is the value after normalization, the color block corresponds to the intensity, and the S-type, M-type and L-type cone cells correspond to the sensitivity.
Referring to fig. 6, the regression model is implemented using a 9-layer fully connected neural network, wherein an input layer 601 includes 27 nodes, an output layer includes 3 nodes corresponding to 3 color channel component values of a color image acquired under the irradiation of 9 light sources, an intermediate layer 603 includes layers 2 to 8, and includes 81, 243, 729, 243, 81, and 81 nodes, respectively, each layer uses a sigmoid activation function. Full connection 602 is provided between nodes 605 of each layer.
It should be noted that the drawings and description of the present invention are simplified for a clearer understanding of the present invention, and elements well known in the art are omitted for clarity. Moreover, the removed elements themselves do not facilitate a better understanding of the present invention, and thus are not described in detail. The details omitted as well as modifications or alternative embodiments are within the knowledge of a person of ordinary skill in the art. The present invention is not limited to the above embodiments, and those skilled in the art may still modify the technical solutions described in the above embodiments or substitute some technical features thereof, but these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.