The invention is a divisional application of an invention patent application with application number 201210581560.1, filed 12/27/2012, of 'method and apparatus for three-dimensional enhancement of backscatter human inspection images'.
Disclosure of Invention
In order to solve the problem of poor overall effect of image enhancement in the prior art, a three-dimensional enhancement method and equipment of a backscatter human body inspection image are provided.
In one aspect of the present invention, a method for three-dimensionally enhancing a backscatter image in a backscatter body scanning system is provided, comprising the steps of: obtaining back scattering images of a plurality of sides of an object to be detected in a scanning area; obtaining a human body three-dimensional model of the detected object based on at least one back scattering image; back projecting the back scattering image to the obtained human body three-dimensional model to obtain a three-dimensional image of the detected object; determining an image of entrainment carried by the inspected object based on one or more of the backscatter images; generating a three-dimensional image of the entrainment from the image of the entrainment; fusing the three-dimensional image of the detected object and the three-dimensional image of the entrainment object to generate a fused three-dimensional image; and presenting the fused three-dimensional image on a display.
According to an embodiment of the present invention, the backscatter images of the plurality of side surfaces include a front backscatter image and at least one other backscatter image, and the step of obtaining the three-dimensional model of the human body of the detected object based on the at least one backscatter image includes: extracting contour information of the detected object from the front backscatter image; and obtaining a human body three-dimensional model of the detected object based on the contour information.
According to an embodiment of the present invention, the step of obtaining a three-dimensional model of the human body of the detected object based on at least one backscatter image further includes: extracting key data from the contour information; and extracting the corresponding human body three-dimensional model from the database by referring to the key data.
According to an embodiment of the invention, the step of generating a three-dimensional image of the entrainment from the image of the entrainment comprises: imparting a thickness to the entrainment based on a contrast of the entrainment image, forming a three-dimensional image of the entrainment; and correcting the three-dimensional image of the entrainment object by using the three-dimensional scattering cross section of the entrainment object.
According to an embodiment of the present invention, the step of fusing the three-dimensional image of the object to be inspected and the three-dimensional image of the entrainment includes: and fusing a three-dimensional image generated by the image of the carried object into the three-dimensional image of the detected object according to the corresponding position.
According to an embodiment of the invention, the method further comprises: and generating a two-dimensional projection map from the fused three-dimensional image according to the projection angle selected by the user, and displaying the projection image of the detected object under the projection angle.
According to an embodiment of the present invention, the method further comprises the step of obtaining object information of the object to be examined and obtaining a three-dimensional model of a human body of the object to be examined based on both the object information and the at least one backscatter image.
According to an embodiment of the present invention, the object information is profile information measured with an optical measuring device.
According to an embodiment of the present invention, the object information is ID information of the object to be inspected, and the contour information of the object to be inspected is obtained from a database based on the ID information.
In another aspect of the present invention, an apparatus for three-dimensionally enhancing a backscatter image in a backscatter human body scanning system is presented, comprising: a memory for storing backscatter images of a plurality of sides of an object under examination; a processor for processing the backscatter image stored in the memory as follows: obtaining a human body three-dimensional model of the detected object based on at least one scattered image; back projecting the back scattering image to a human body three-dimensional model to obtain a three-dimensional image of the detected object; determining an image of entrainment carried by the inspected object based on one or more of the backscatter images; generating a three-dimensional image of the entrainment from the image of the entrainment; fusing the three-dimensional image of the detected object and the three-dimensional image of the entrainment object to generate a fused three-dimensional image; and a display for presenting the fused three-dimensional image.
According to an embodiment of the invention, the back scatter images of the plurality of side faces comprise a front back scatter image and at least one other back scatter image, the processor is further configured to: extracting contour information of the detected object from the front backscatter image; and obtaining a human body three-dimensional model of the detected object based on the contour information.
According to an embodiment of the invention, the processor is further configured to: extracting key data from the contour information; and extracting the corresponding human body three-dimensional model from the database by referring to the key data.
According to an embodiment of the invention, the processor is further configured to: imparting a thickness to the entrainment based on a contrast of the entrainment image, forming a three-dimensional image of the entrainment; and correcting the three-dimensional image of the entrainment object by using the three-dimensional scattering cross section of the entrainment object.
According to an embodiment of the invention, the processor is further configured to: and fusing a three-dimensional image generated by the image of the carried object into the three-dimensional image of the detected object according to the corresponding position.
According to an embodiment of the present invention, the processor is configured to generate a two-dimensional projection map from the fused three-dimensional image according to a projection angle selected by a user, and the display displays a projection image of the detected object at the projection angle.
According to an embodiment of the present invention, the processor is further configured to obtain object information of the object to be examined and obtain a three-dimensional model of a human body of the object to be examined based on both the object information and the at least one backscatter image.
According to an embodiment of the present invention, the object information is profile information measured with an optical measuring device.
According to an embodiment of the present invention, the object information is ID information of the object to be inspected, and the processor obtains the contour information of the object to be inspected from a database based on the ID information.
In another aspect of the invention, a computer-readable medium is provided, storing a computer program which, when executed by a processor, performs the steps of: obtaining a human body three-dimensional model of the detected object based on the at least one back scattering image; back projecting the back scattering image to the obtained human body three-dimensional model to obtain a three-dimensional image of the detected object; determining an image of entrainment carried by the inspected object based on one or more of the backscatter images; generating a three-dimensional image of the entrainment from the image of the entrainment; and fusing the three-dimensional image of the detected object and the three-dimensional image of the entrainment object to generate a fused three-dimensional image.
According to the scheme, the articles carried by the human body are subjected to three-dimensional transformation after being divided, the carried articles are protruded on the surface of the human body, and the visual contrast effect is very strong. In addition, the gray scale of the original image is reserved, the noise is not enhanced, and the overall effect is better.
Detailed Description
Specific embodiments of the present invention will be described in detail below, and it should be noted that the embodiments described herein are only for illustration and are not intended to limit the present invention. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that: it is not necessary to employ these specific details to practice the present invention. In other instances, well-known structures, materials, or methods have not been described in detail in order to avoid obscuring the present invention.
Throughout the specification, reference to "one embodiment," "an embodiment," "one example," or "an example" means: the particular features, structures, or characteristics described in connection with the embodiment or example are included in at least one embodiment of the invention. Thus, the appearances of the phrases "in one embodiment," "in an embodiment," "one example" or "an example" in various places throughout this specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures, or characteristics may be combined in any suitable combination and/or sub-combination in one or more embodiments or examples. Further, as used herein, the term "and/or" will be understood by those of ordinary skill in the art to include any and all combinations of one or more of the associated listed items.
In view of the problem of the prior art that image enhancement adversely affects the overall effect, some embodiments of the present invention provide a method for three-dimensionally enhancing a backscatter image in a backscatter body scanning system. Backscatter images of a plurality of sides of an object to be inspected within a scanning region are first acquired, which can be obtained by scanning the plurality of sides of a human body using a human body backscatter system. Then, a three-dimensional model of the human body of the inspected object is obtained based on the at least one scattered image. For example, contour information of the human body of the object to be examined is obtained from the backscatter image, and a corresponding three-dimensional model of the human body is obtained based on the contour information. For example, retrieving an appropriate three-dimensional model of the human body from a database. Then, the back scattering image is back projected to the obtained human body three-dimensional model, and a three-dimensional image of the detected object is obtained.
In another aspect, an image of entrainment carried by the inspected object is determined based on one or more of the backscatter images. And generating a three-dimensional image of the entrainment based on the two-dimensional image of the entrainment. For example, a three-dimensional image of the material to be entrained is generated by giving the material to be entrained a thickness based on information such as the gradation or contrast of the image of the material to be entrained.
Next, the three-dimensional image of the object to be inspected and the three-dimensional image of the entrainment are fused to generate a fused three-dimensional image, and the fused three-dimensional image is presented on a display. Therefore, the carried three-dimensional image can be highlighted on the surface of the human body three-dimensional model, and the visual contrast effect is very strong. In addition, the gray scale of the original image is reserved, the noise is not enhanced, and the overall effect is better.
According to the embodiment of the invention, after the three-dimensional image of the carried object is fused on the three-dimensional human body model, an operator can select a two-dimensional projection image along a certain visual angle, for example, the observation visual angle is changed by using a mouse, a keyboard or a shortcut key, so that the three-dimensional display effect is achieved. For example, only a two-dimensional projection view along a certain viewing angle is displayed, the projection direction is changed by sliding a mouse, the projection direction is changed by up, down, left, right, and left keys of a keyboard, and front, left, right, and back images are displayed by shortcut keys.
In addition, in other embodiments of the present invention, a three-dimensional model of the human body can be established by using a priori information of the detected object, for example, known contour information such as height, and information such as fat and thin of the object. In this case, a three-dimensional model of the human body can be established based on key information of the human body to be examined. For example, a three-dimensional model of a human body can be acquired relatively quickly by acquiring key information such as the height and weight of the subject from a database based on the ID of the subject, or by acquiring a contour image of the human body of the subject using an optical camera, extracting key data, and comparing the extracted key data with a model in the database.
According to some embodiments of the invention, after the image subjected to feature matching is back-projected onto the three-dimensional model, image fusion is completed through image deformation, hole filling and other technologies. And giving thickness to the entrainment according to information such as the gray scale or contrast of the entrainment, generating a three-dimensional image from the two-dimensional image, and correcting the three-dimensional image by using a thickness correction table (calibrating and generating the correction table by using some typical materials), thereby obtaining a more proper three-dimensional image of the entrainment.
According to the embodiment of the invention, the three-dimensional graph generated by the carried article is fused on the human body three-dimensional model according to the corresponding position, and because the carried article is outside the surface of the human body, the carried article is convex, so that the image is convex.
FIG. 1 shows a schematic diagram of a human backscatter scanning system according to an embodiment of the invention. As shown in FIG. 1, the human backscatter scanning system includes anX-ray source 10, adetector 30, acontrol apparatus 20, and an image processing anddisplay device 60. Thecontrol device 20 controls theX-ray source 10 to emit X-rays, forms flying spots under the modulation of a flying spot forming device (not shown), scans the surface of the human body of theobject 50, and controls thedetector 30 to detect scattered X-rays of the human body, thereby forming a backscatter image. Thus, after the scanning is performed on a plurality of sides of the human body, back scattering images of the plurality of sides are obtained. The resulting back-scattered images of the sides are then processed in an image processing anddisplay device 60, for example with software.
Fig. 2 shows a flow chart of a method of three-dimensionally enhancing a backscatter image according to an embodiment of the invention. As shown in fig. 2, in step S21, a backscatter image of the object to be examined is obtained, for example, a backscatter image of a human body is obtained by fusing a plurality of side scattered images or a backscatter image of interest is used.
For example, in some embodiments, when backscatter images of different sides are obtained, the images may also be matched and fused first. Taking two images which are scanned on the back and rotate by 45 degrees left and right as an example, the feature matching algorithm is adopted for fusion, and because the scanned image obtained by each examination and the image obtained by the previous examination have strong correlation, the correlation optimization algorithm is utilized to realize rapid fusion.
For example, in the example of fig. 1, the person may stand in three images, namely, a left turn 45 ° on the back side, a right turn 45 ° on the right side, and a right turn on the front side, where the two left and right rotation scanning images on the back side include two sides of the human body. For another example, a circular scanning method may be used to collect the backscatter image around one revolution of the person (person revolution or equipment revolution), and restore the two-dimensional image according to the auxiliary information. Alternatively, a circular arc scan or the like may be used to obtain back-scattered images of multiple sides of the human body of the object to be examined.
Fig. 3 shows an example of the operation of a human backscatter scanning system according to another embodiment of the invention. As shown in fig. 3,apparatus 100 andapparatus 300 are two backscatter apparatuses, each using a flying spot forming device as described in chinese patent application 201010624252.3(CN102565110A), the detectors comprising two backscatter detectors. The probe does not translate left and right, and theobject 500 to be inspected moves while standing on theconveyor 200. When theobject 500 to be inspected is within the scanning range of theapparatus 100, theradiation source 101 of theapparatus 100 emits aradiation beam 102 to irradiate theobject 500 to be inspected, and the backscattered radiation is received by thedetectors 103 and 104.
When theobject 500 to be inspected is conveyed within the scanning range of theapparatus 300, theradiation source 301 of theapparatus 300 emits radiation to irradiate theobject 500 to be inspected, and the backscattered radiation is received by thedetectors 303 and 304. After theobject 500 to be examined passes out of the scanning range of theapparatus 300 on the conveyor belt, the human body scanning is completed. This allows fast acquisition of back-scattered images of multiple sides of the human body.
According to other embodiments of the present invention, the human backscatter scanning system may also use thecamera 400 to quickly obtain an optical image of theobject 500 to be examined, which is used to determine contour information of theobject 500 to be examined.
In step S22, a three-dimensional phantom of the object to be examined is obtained based on the backscattered image, and in step S24, the backscattered image is back-projected onto the three-dimensional phantom to obtain a three-dimensional phantom.
A library of three-dimensional models of the human body is stored in the image processing anddisplay device 60. These three-dimensional models are, for example, a library of reference three-dimensional model supplementary models that are derived from statistics of people of different groups and are formed in an actual backscatter scan using body size information obtained from the scan. For example, a human body three-dimensional model is built according to geometric data (height, full head height, shoulder width, leg length, arm length and the like) of the surface of the human body obtained by scanning an image, and body surface data can be obtained by analyzing the outline of the image, so that the model is modeled as a basic model. This modeling process is completed before the device is shipped from the factory, and the reference model information is stored in the image processing anddisplay device 60.
According to other embodiments of the present invention, a method of local editing of the human body with spline curves proposed by Seo et al (Seo H, Cordier F, Philippon L, et al. Interactive modeling of MPEG-4deformable human body models [ OL ] http:// www.miralab.unige.ch/papers/115.pdf, 2004) may also be employed. And the basic models are deformed and edited according to newly obtained human body surface geometric data to generate a new three-dimensional model, so that the modeling speed is greatly improved.
In some embodiments of the invention, in order to make the modeling speed faster, a typical human body three-dimensional model library is established, the three-dimensional model established each time is numbered, a record document is established, the model number and the surface geometric data are in one-to-one correspondence, a threshold method is adopted to judge whether a new model is needed, the sum of squares of the difference values of new parameters and the existing library model parameters is calculated, and if the sum exceeds a specified threshold value, the new model is stored in the library. Regarding the threshold determination, according to the characteristics of the human body examination equipment, the three-dimensional model is not required to be very accurate to correspond to the real human body, and only the physical and physical characteristics of the human body are required to be approximately reflected, so that the threshold is set to be higher, and the database is not too large.
In addition, a data sharing mode can be adopted, a database established by each device can be copied to other devices for use, a model does not need to be newly built after the devices are used for a period of time, the human body three-dimensional model can be called out from the database only by matching the geometric parameters of the body surface in the normal scanning process, the speed is very high, and the requirement on the human body inspection speed can be met.
In some embodiments of the invention, the three-dimensional model is established based on parameters obtained by scanning image contour analysis, the scanning two-dimensional image is back-projected onto the three-dimensional model according to a contour corresponding method, the projected image is deformed to form some holes, and some positions are blocked and can not be projected, and linear interpolation algorithm is adopted for filling.
In step S23, an image of an entrainment carried by the object to be examined is determined from the backscatter image, and in step S25, a three-dimensional image of the entrainment is generated.
For example, in some embodiments, the entire image contour is extracted, and the human body contour is analyzed and removed according to human body features, so as to realize the segmentation of the entrainment. The method is simple and rapid, the thickness of the divided entrained articles is given according to the contrast, a three-dimensional image is generated from the two-dimensional image, and the thickness is corrected by using a correction table. For example, for low-energy X-ray, the scattering cross section of a high atomic number substance is small, the photoelectric cross section is large, the gray value is low, and the contrast between a thin metal and a human body is large, so that the thickness generated according to the contrast needs to be corrected. For example, the low gray scale is corrected for metals, and the highlight portion is generally corrected for organic substances having a low atomic number. Since the thickness calculation does not need to be accurate, the correction coefficient can be an empirical value.
In step S26, the three-dimensional image of the entrainment and the three-dimensional image of the human body are fused. For example, the divided entrained articles are placed in corresponding positions according to the three-dimensional coordinate system of the human body model. In step S27, the fused three-dimensional image, or the two-dimensional image at a certain viewing angle, is displayed on the display. For example, according to the observation visual angle selected by the user, the three-dimensional image is quickly projected, shortcut keys are arranged to display a plurality of main visual angle images, and corresponding functions can be added according to the actual requirements of the user.
In other embodiments, the corresponding three-dimensional model may also be obtained from object information of the detected object. Fig. 4 shows a flowchart of a method of three-dimensionally enhancing a backscatter image of a human body according to still another embodiment of the present invention.
In step S41, a backscatter image of the object to be inspected and object information are acquired. In step S42, a three-dimensional model of the human body of the test object is obtained based on the object information of the test object. For example, a priori information of the object to be examined, such as known height and other contour information, is obtained. For example, the fat and thin of the object, and the like, to build a three-dimensional model of the human body. In this case, a three-dimensional model of the human body can be established based on key information of the human body to be examined. For example, key information such as the height and the weight of the subject is obtained from the database based on the ID of the subject. For example, at an airport or a port, the name or identification number of the subject may be used to obtain a priori information about the subject. Or an optical camera is used for obtaining a human body outline image of the detected object, key data are extracted, and the key data are compared with the models in the database, so that the human body three-dimensional model is obtained more quickly. Therefore, it is not necessary to create a three-dimensional model of the human body of the object to be examined from the backscatter images.
The remaining steps S43, S44, S45, S46 and S47 in the method shown in fig. 4 correspond to the steps S23, S24, S25, S26 and S27 in fig. 2 described above, respectively, and thus the description of these steps is omitted here.
According to the human body inspection image three-dimensional enhancement method of the embodiment, the image segmentation, matching, three-dimensional modeling, fusion and other methods are used, a plurality of images are fused into a three-dimensional image, the image can be observed from any visual angle, the detected object and the human body have light and shade difference and space position difference, the entrained objects can be seen more easily, and a better visual effect is achieved. In addition, the side entrainment of the human body can also be clearly shown.
According to some embodiments, the articles carried by the human body are subjected to three-dimensional transformation after being divided, the carried articles are protruded on the surface of the human body, the visual contrast effect is very strong, the gray scale of an original image is reserved, noise is not enhanced, the overall effect is better, and meanwhile, the articles carried by the side surface of the human body can be clearly displayed.
The three-dimensional enhancement method for the human body examination image according to the embodiment can meet the requirement of human body examination practicability, namely, the image processing speed is high; the image quality level after being processed by various methods is kept consistent with the original image; the algorithm is required to be concise, and the output result is stable, namely the robustness of the algorithm is required.
The foregoing detailed description has set forth numerous embodiments of a method and apparatus for three-dimensionally enhancing image quality of a backscatter image using schematics, flowcharts, and/or examples. Where such diagrams, flowcharts, and/or examples contain one or more functions and/or operations, it will be understood by those within the art that each function and/or operation within such diagrams, flowcharts, or examples can be implemented, individually and/or collectively, by a wide range of structures, hardware, software, firmware, or virtually any combination thereof. In one embodiment, portions of the subject matter described by embodiments of the invention may be implemented by Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), Digital Signal Processors (DSPs), or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein, in whole or in part, can be equivalently implemented in integrated circuits, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and or firmware would be well within the skill of one of skill in the art in light of this disclosure. In addition, those skilled in the art will appreciate that the mechanisms of the subject matter described herein are capable of being distributed as a program product in a variety of forms, and that an illustrative embodiment of the subject matter described herein applies regardless of the particular type of signal bearing media used to actually carry out the distribution. Examples of signal bearing media include, but are not limited to: recordable type media such as floppy disks, hard disk drives, Compact Disks (CDs), Digital Versatile Disks (DVDs), digital tape, computer memory, etc.; and a transmission type medium such as a digital and/or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.).
While the present invention has been described with reference to several exemplary embodiments, it is understood that the terminology used is intended to be in the nature of words of description and illustration, rather than of limitation. As the present invention may be embodied in several forms without departing from the spirit or essential characteristics thereof, it should also be understood that the above-described embodiments are not limited by any of the details of the foregoing description, but rather should be construed broadly within its spirit and scope as defined in the appended claims, and therefore all changes and modifications that fall within the meets and bounds of the claims, or equivalences of such meets and bounds are therefore intended to be embraced by the appended claims.