S&FRef: 691019
AUSTRALIA
PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT Name and Address of Applicant: Actual Inventor(s): Address for Service: Invention Title: Siemens Audiologische Technik GmbH, of Gebbertstrasse 125, 91058, Erlangen, Germany Gtinter Doemens Frank Forster Torsten Niederdrink Peter Rummel Spruson Ferguson St Martins Tower Level 31 Market Street Sydney NSW 2000 (CCN 3710000177) Panoramic scanner The following statement is a full description of this invention, including the best method of performing it known to me/us:- IAUratIi yj Batch: 5845c 1- Panoramic scanner The invention relates to a method for the threedimensional detection of an object. Furthermore, the invention relates to an apparatus for performing the method and a use of the apparatus and the method.
Methods for the three-dimensional detection and digitization of objects are used for various application purposes, e.g. in the development, production and quality control of industrial products and components. In medical technology, use is made of, for example, optical measurement methods for producing the housings of hearing aids that can be worn in the ear.
For the purpose of individually adapting a housing to the auditory canal of a person wearing a hearing aid, in this case impressions of the patient's outer auditory canal are created by an audiologist by means of a rubber-like plastics composition. In order to be able to employ stereolithographic or similar methods for producing the housings, it is necessary to create three-dimensional computer models from the ear impressions. This procedure has hitherto been effected by the hearing aid manufacturer, where the impressions are measured panoramically three-dimensionally by means of a precision scanner and a 3D computer model of the outer auditory canal is created on the basis of these data. Afterwards, in a laser sintering process, the individually formed housing shell is produced on the basis of the data of the computer model.
The precision scanners used are usually designed as laser scanners in which a laser beam is guided over the surface of the impression in a controlled manner and the backscattered light is observed by a detector (e.g.
a CCD camera) from a direction deviating from the laser beam. The surface coordinates of the impression are 2 then calculated by triangulation. In the case of the known laser scanner VIVID 910 from the company Minolta, a line is generated from the laser beam and is moved over the surface of the object to be detected, e.g. an ear impression. The image of the line is in turn observed by a camera, the surface coordinates of the object to be detected being deduced from the deformation of the line image by triangulation. A rotary stage controller on which the object rotates through 3600 during the scanning serves as an accessory to the known laser scanner.
What is disadvantageous about the known laser scanners is their high procurement costs, which are occasionally also caused by the high-precision mechanism of the rotary stage controllers.
Frank Forster, Manfred Lang, Bernd Radig in "Real-Time Range Imaging for Dynamic Scenes Using Color-Edge Based Structured Light", ICPR '02, Vol. 3, pp. 30645-30648, 2002, disclose a method for the 3D detection of an object by means of structured light. In this case, a projector is used to project a color pattern containing a redundant code with known projection data onto the surface of an object, and the object with the color pattern projected thereon is recorded by a camera from a direction deviating from the projection direction. By decoding the color pattern at each pixel of the camera image, it is possible to determine the associated three-dimensional coordinates of the object surface by means of triangulation. This method permits the reconstruction of a partial region of the surface of the object with a video image.
JP 2001108421 A discloses a 3D scanner for the threedimensional detection of an object. During scanning, the object rotates together with a reference object on which markings are provided. Thus, different views of the object and of the reference object are photo- 3 graphed, the photographs being combined to form a three-dimensional computer model on the basis of the markings on the reference object. What is disadvantageous about the known method is the for some applications inadequate correspondence between the computer model and the real object.
It is an object of the present invention to specify a method and also a panoramic scanner which make it possible to detect three-dimensionally an object, in particular an ear impression, in a comparatively simple and cost-effective manner with the accuracy required for producing a hearing aid housing shell.
This object is achieved by means of a method for the three-dimensional detection of an object having the features in accordance with claim 1. Furthermore, the object is achieved by means of a panoramic scanner in accordance with claim The three-dimensional detection of an object in accordance with the invention requires at least a projector, a camera and means for rotating the projector and the camera relative to the object. The projector projects a two-dimensional pattern, e.g. a color pattern, containing a redundant code with known projection data onto the surface of the object. The color pattern projected on is subsequently recorded by a camera, e.g. a CCD camera, from a direction deviating from the projection direction. By decoding the color pattern at each pixel of the camera image, the associated three-dimensional coordinates of the object surface are determined by means of triangulation.
In order to enable a three-dimensional panoramic view, the object rotates relative to the projector and the camera. For this purpose, the object is preferably situated on a rotary stage controller. The rotary stage controller rotates through a predeterminable angle 4 between two recordings, so that it is possible to record a plurality of object images, e.g. 60, per periphery. During a scan, the object generally rotates once through 3600 about the rotation axis. If only a partial region of an object is to be digitized, then the object may also be rotated through an angle of less than 3600. Furthermore, it is also possible for more than one complete revolution to be performed during the detection of an object in order to increase the accuracy of the 3D model to be generated. By way of example, five completely executed revolutions of the object then constitute a termination criterion for the scan.
In order that a contiguous panoramic view of the object can be generated from these individual images, it is advantageous if the 3D data of the individual images are related to a common coordinate system. For the requisite calibration, in accordance with the invention, markings are provided on the scanner and do not change their position with respect to the object during scanning. With the use of a rotary stage controller, the markings are preferably situated on the rotary stage controller or at the edge of the rotary stage controller. The markings are configured in such a way that a specific number of said markings are visible in each camera image and the angle of rotation of the object relative to the projector and the camera can be gathered from said markings unambiguously and with the required accuracy. In this case, a higher number of markings increases the accuracy of the 3D reconstruction.
In an advantageous manner, the position of the markings that are moved with the object is precisely determined once with respect to a "world coordinate system" and communicated to the evaluation system. It is then possible to determine the relative position of the object with respect to the projector and the camera or 5 the angle of rotation of the rotary stage controller from the position and the coding of the markings recorded in the object image in the coordinate system.
Successively recorded individual images or the 3D data records obtained from the latter can then be combined in a simple manner by means of a corresponding coordinate transformation to form the overall view in the "world coordinate system".
The invention affords the advantage that a synchronization of the individual image recordings with the rotary movement of the object is achieved in a simple and cost-effective manner without this requiring a high-precision and correspondingly expensive mechanism.
A user of the panoramic scanner does not have to perform any calibration or adjustment operations, with the exception of fixing the object to be measured on the rotary stage controller. Consequently, what has been created is a possibility for detecting the 3D panoramic surface of an object, this possibility being simple to control but nevertheless highly precise and cost-effective. The panoramic scanner according to the invention is therefore e.g. especially suitable for use by an audiologist who creates an ear impression of a patient and digitizes it three-dimensionally by means of the scanner, so that the model data obtained can be communicated directly to the manufacturer of a housing shell by data transmission (E-mail or the like). This saves time and costs in the production of a hearing aid housing.
In one embodiment of the invention, a plurality of overlapping object images are recorded in the course of a revolution of the object relative to the camera and the projector. In this case, a plurality of the same markings are then visible in each case in successive object images. With the aid of the common visible markings, the object images are combined in such a way as to produce a so-called image composite. A precise 6 measurement of the markings is not necessary for this purpose, which simplifies the production of the system.
The relative camera coordinates of each recording can be determined by means of a method that is referred to as "cluster compensation" and is known from photogrammetry. A few markings measured in the "world coordinate system" serve for relating the image composite thereto. After this step, the individual object images can then be combined in a simple manner by means of a corresponding coordinate transformation to form the overall view. In order to simplify the calculation, two axes of the "world coordinate system" lie in the plane spanned by the rotary stage controller and the third axis of the "world coordinate system" coincides with the rotation axis of the rotary stage controller.
The markings are preferably configured in such a way that they contain a coding with the extent 1-n, e.g. in the form of a binary code. The markings advantageously contain a few measurement positions (corners, lines, circles or the like). The markings recorded in the object images are automatically detected, decoded and measured in each object image by means of a suitable image processing software. The markings are preferably embodied in such a way that, for each object image, on the basis of the markings contained therein, it is possible to unambiguously assign the spatial position with respect to the camera and the projector.
In one development of the invention, it is provided that the rotation axis about which the object rotates relative to the projector and the camera can be pivoted relative to the projector and the camera. When using a rotary stage controller, the simplest way of achieving this is by tilting the rotary stage controller by a specific angle in at least one direction. This affords advantages in particular in the digitization of ear 7 impressions since the latter may be comparatively fissured. By pivoting the rotation axis, it is possible to prevent shading and thus gaps or inaccuracies in the three-dimensional computer model.
In an advantageous embodiment of the invention, the markings are arranged and configured in such a way that, in addition to the angle of rotation, the angle by which the rotation axis is pivoted with respect to a starting position can also be detected from each object image. In this case, the position of the rotation axis in the preceding object image or an original position may serve as the starting position.
In an alternative development of the invention, at least two cameras arranged offset with respect to one another are present, so that the object can be recorded simultaneously from different viewing angles. The cameras are fitted at a different height with regard to the rotation axis of the object to be detected, so that even undercuts of the object, which would lead to defects in the computer model when using just one camera, can be detected by the further camera. A pivot movement of the rotary stage controller relative to the cameras can thereby be dispensed with. In an advantageous manner, a second projector is also used in addition to a second camera, so that object images are in each case generated by a camera-projector pair.
The self-calibration property of a panoramic scanner according to the invention has the advantage that all the individual 3D object images can be combined in a simple. manner to form a 3D panoramic image. In this case, no stringent requirements are made of the constancy of the rotary movement. A synchronization of the rotary movement with the image recordings is not necessary. It is possible, therefore, to have recourse to a cost-effective mechanism. The accuracy of the 3D 8 detection can easily be increased by increasing the number of images per revolution.
The robustness and accuracy of the measurement rise significantly as a result of a high number of measurement data and in particular as a result of overlapping object images.
The invention is described below on the basis of exemplary embodiments. In the figures: figure 1 shows a diagrammatic sketch of the 3D detection of an object by means of color-coded, structured light, figure 2 shows the basic diagram of a scanner according to the invention, figure 3 shows a scanner according to the invention in a perspective view, figure 4 shows the scanner in accordance with figure 3 with a rotation axis that has been pivoted with respect to figure 3, figure 5 shows the scanner in accordance with figures 3 and 4 with a housing, and figure 6 shows an alternative embodiment of a scanner with two cameras.
Figure 1 illustrates an apparatus 1 which serves for determining the three-dimensional object coordinates of a surface 2 of an object 3 to be detected.
The apparatus 1 has a projector 4, which projects a color pattern 5 onto the surface 2 of the object 3 to be detected. In the case illustrated in figure 1, the color pattern 5 is composed of a series of color 9 stripes lying next to one another. However, it is also conceivable to use a two-dimensional color pattern instead of the one-dimensional color pattern illustrated in figure 1.
In the case of the exemplary embodiment illustrated in figure 1, a projection plane g may be assigned to each point P of the surface 2 of the object 3. Consequently, projection data are coded by the color pattern 5. The color pattern 5 projected onto the surface 2 of the object 3 is converted into an image 7 by a camera 6 in that the point P on the surface 2 is transformed into the point Given a known arrangement of the projector 4 and the camera 6, in particular given a known length of a base path 8, the three-dimensional spatial coordinates of the point P on the surface 2 can be calculated by triangulation. The requisite data reduction and evaluation is performed by an evaluation unit 9.
In order to enable the three-dimensional spatial coordinates of the point P on the surface 2 to be determined from an individual image 7 even when the surface 2 of the object 3 has depth jumps and occlusions, the color pattern 5 is constructed in such a way that the coding of the projection planes g is as robust as possible with respect to errors. Furthermore, errors based on the coloration of the object can be eliminated by means of the coding.
In the case of the exemplary embodiments illustrated in figure 1, the colors of the color pattern 5 are described by the RGB model. The changes in the color values of the color pattern 5 are effected by changes in the color values in the individual color channels R, G and B.
The color pattern is then intended to satisfy the following conditions: 10 Only two color values are used in each color channel. In particular, the minimum value and the maximum value are in each case used in each color channel, so that a total of eight colors are available in the RGB model.
Within a code word, each color channel has at least one color change. This condition enables the individual code words to be decoded.
Color elements lying next to one another differ in at least two color channels. This condition serves in particular for ensuring the error tolerance in particular with respect to depth jumps.
The individual code words of the color pattern have a non-trivial Hamming distance. This condition also serves for increasing the error tolerance when decoding the projection planes g.
The color changes are also combined to form code words with a non-trivial hamming distance.
Mention shall be made below of an example of the color pattern 5 which satisfies the five conditions mentioned above. This color pattern 5 relates to the RGB model with a red color channel R, a green color channel G and a blue color channel B. Since color values in each color channel are only permitted in each case to assume the minimum value and maximum value, a total of eight mixed colors are available, which are respectively assigned the following numbers: Black 0 Blue 1 Green 2 Cyan 3 Red 4 11 Magenta Yellow 6 White 7 A length of four color stripes was chosen for the code words of the color values, with overlapping of adjacent code words in each case with three color stripes.
The color changes were also assigned numerical values.
Since the color value can remain the same, decrease or increase in each of the three color channels, the result is a total of 27 different color changes of the mixed color, which were respectively assigned a number between 0 and 26. The length of the code words assigned to the color changes was chosen to be equal to three color changes, with overlapping of adjacent code words in each case with two color changes.
A search algorithm found the following series of numbers, which describes an exemplary embodiment of the color pattern 5 which satisfies the five conditions mentioned above: 1243070561217414270342127216534171614361605306352717072 416305250747147065035603634743506172524253607 In the exemplary embodiment specified, the first code word comprises the numerals 1243, the second code word comprises the numerals 2430 and the third code word comprises the numerals 4307. The exemplary embodiment shown constitutes a very robust coding.
Figure 2 illustrates the basic diagram of a panoramic scanner according to the invention. The scanner comprises a rotary stage controller .10, which is mounted such that it is rotatable about its axis of symmetry. An ear impression 11 configured according to the individual anatomical characteristics of a person wearing a hearing aid is fixed on the rotary stage 12 controller. The ear impression 11 is intended to be digitized in order to produce an individually formed shell of a hearing aid that can be worn in the ear. The ear impression is detected by means of coded illumination and triangulation. For this purpose, the panoramic scanner comprises a projector 12, which projects a color-coded pattern onto the surface of the ear impression 11. The color pattern projected onto the surface of the ear impression 11 is converted into an image of the ear impression 11 by a CCD camera 13. By virtue of the rotary movement of the rotary stage controller 10, it is possible to record a multiplicity of such imagings from different observation angles. In order that the individual imagings can be assigned the respective observation angle, markings 14 are provided at the outer edge of the rotary stage controller 10. In addition to the ear impression 11, a number of said markings 14 are also detected in each image. The images of the markings 14 are automatically detected, decoded and measured in the object images by means of a computer 15 with a suitable image processing software.
On the basis of the angular information obtained therefrom, a three-dimensional computer model of the ear impression 11 is calculated from the individual imagings. The computer 15 is preferably not part of the actual panoramic scanner, i.e. not arranged with the rotary stage controller 10, the projector 12 and the camera 13 in a common housing. Rather, an external powerful PC with a suitable software may be used as the computer 15. The panoramic scanner then has an interface for connection to the computer Figure 3 shows the panoramic scanner illustrated in the basic diagram in figure 2, in a perspective view. This also reveals the rotary stage controller 10, a projector 12 and also a CCD camera 13 in the respective position in relation to one another. Furthermore, the drive unit for the rotary stage controller 10 can also be discerned in figure 3. Said drive unit comprises a 13 motor 16, which drives the rotary stage controller via a gearwheel 17 and a toothed belt 18. Furthermore, figure 3 illustrates a mechanism that enables not only the rotation movement but also a pivot movement in the case of the rotary stage controller 10. In the exemplary embodiment, the pivot axis 19 runs through the point of intersection between the rotation axis and the surface of the rotary stage controller 10. In the exemplary embodiment, the pivot movement is also effected automatically by means of an electric drive, the motor 16 bringing about both the rotation movement and the pivot movement in the case of the embodiment shown. Specifically, the rotation of the rotary stage controller 10 drives a gearwheel 21A connected thereto, which engages in a toothed piece 21B fixedly anchored in the housing of the scanner and thereby leads to the pivot movement of the drive unit with the motor 16 and the toothed belt 18. The markings 14 provided at the edge of the rotary stage controller 10 can furthermore be seen, which markings make it possible to determine the precise angle of rotation of the rotary stage controller 10 and thus of an object mounted thereon (cf. figure 2) with respect to the projector 12 and the camera 13 from the imagings produced.
At the beginning of the detection of an object, the rotation axis is advantageously situated in the starting position envisaged therefor. This may be effected e.g. by a housing cover (not illustrated) being fixed in a pivotable manner to the housing of the panoramic scanner. Said housing cover must firstly be opened before an object is positioned on the rotary stage controller 10. In the course of said housing cover being opened, the entire rotation unit with the motor 16 and the rotary stage controller 10 is then transferred into its starting position by means of a corresponding mechanism (not illustrated). Consequently, at the beginning of a scan, the rotary stage controller 10 is situated in the starting position 14 illustrated in figure 3 until it finally assumes the end position shown in figure 4 after a plurality of revolutions. The motor 16 is automatically stopped in the end position. On the basis of the markings in the object images, the angle of rotation and the angle by which the rotary stage controller 10 is pivoted from its starting position can be unambiguously gathered from each image. Thus, it is possible to create a 3D model with high accuracy from the individual object images.
As an alternative, the rotary stage controller 10, for execution of the pivot movement, may also be connected to a second motor (not illustrated). The pivot movement may then also be controlled by the computer 15, so that the number of revolutions of the rotary stage controller during which the latter pivots from a starting position into an end position is variable.
In the case of the panoramic scanner in accordance with figure 3, the rotary stage controller, the drive unit of the rotary stage controller, the projector and the camera are accommodated in a common housing illustrated in figure 5. The panoramic scanner thereby constitutes a compact unit that is simple to handle.
The operational control is also very simple since, besides fixing the examination object on the rotary stage controller 10, the user does not have to carry out any further calibration or adjustment operations.
Furthermore, the two housing openings 31 and 32 for the projector and the camera can also be discerned in figure 5. Moreover, the panoramic scanner also comprises a cable 33 for connection to a computer.
Figure 6 shows an alternative embodiment of a panoramic scanner according to the invention. In contrast to the previous exemplary embodiments, the rotary stage controller 60 is not pivotable in the case of this embodiment. In order nevertheless also to be able to 15 detect complicated objects with undercuts, the scanner has two cameras 61 and 62 which are arranged one above the other and thus detect the object from different viewing directions. Furthermore, the projector 63 is not designed as a point radiation source, rather it emits a coded pattern proceeding from a vertically running line. This ensures the projection of the pattern onto all regions of the object that are detected by the cameras. As an alternative, it is also possible to use a plurality of projectors with a point radiation source (not illustrated). By virtue of the use of a plurality of cameras, a pivot movement of the rotary stage controller 60 becomes invalid and the drive unit can be simplified compared with previous exemplary embodiments. Thus, the rotary stage controller 60 is driven directly (without the interposition of a toothed belt) in the exemplary embodiment in accordance with figure 6.
In the case of the panoramic scanner in accordance with figure 6, all the components are enclosed by a common housing, so that this scanner also forms a compact unit that is simple to handle. Furthermore, it is possible to have recourse to cost-effective commercially available components (CCD cameras, projector) and in particular to a simple mechanism.