US6614946B1

Movatterモバイル変換

Info

Publication number: US6614946B1
Application number: US09/679,990
Authority: US
Inventors: Albert D. Edgar; Thomas A. Dundon
Original assignee: Eastman Kodak Co
Current assignee: Kodak Alaris Inc
Priority date: 1999-10-08
Filing date: 2000-10-05
Publication date: 2003-09-02
Also published as: WO2001027688A3; WO2001027688A2; AU1962701A

Abstract

A system and method for correcting defects in an image. A correction area surrounding the defective pixel is dynamically chosen to include a predetermined percentage of non-defective pixels. The percentage is varied as a function of the size of the surrounding region, such that the percentage decreases as the size of the region increases. The shape of the surrounding area may be varied to further enhance the correction procedure. A center of gravity is maintained around the defective pixel by defining subregions surrounding the pixel.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 U.S.C. §119 of U.S. Provisional Application Serial No. 60/158,501 filed Oct. 8, 1999. This application is also related to the Potucek et al U.S. application Ser. No. 09/686,719 filed Oct. 10, 2000, entitled Method and Apparatus for Differential Illumination Image-Capturing and Defect Handling, and the Ford et al U.S. application Ser. No. 09/552,473 entitled Scanner and Method.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to image processing and, particularly, to an improved system and method for correcting defects in images.

2. Description of the Related Art

Digitized images often include imperfections that are not present in the original image. One cause of such defects may be the components of the image capturing system. For example, a scanning surface, or platen, may contain scratches and other optical path obstructions. These optical path obstructions are digitized along with the original image and appear as imperfections in the digitized image.

Another cause of imperfections is defects within the physical medium of the image. For example, a photograph, film negative or other physical medium may be scratched or deformed despite careful handling. In addition, dust, hair, smudges, and the like may be deposited on the surface of the physical medium and will be digitized along with the original image. These defects will also appear in the digitized image as defects.

Conventional software programs are available to identify such defects. Some programs simply identify anything in the image with a certain characteristic, such as high frequency content, as being defective. Other programs allow the user to manually specify the defects. The identification of such defects can be summarized in a defect map. A binary defect map simply marks pixels in an image as defective or non-defective. Other systems facilitate the creation of a continuous defect map, wherein each pixel is marked with a defect value proportional to the severity of any defect at the pixel.

Once a pixel has been marked as defective, conventional imaging programs replace the defective pixel with a replacement amplitude value determined from amplitudes of the surrounding pixels within a predetermined area. In particular, such programs define a fixed area around a defective pixel and then calculate a mean average of the amplitude values of the pixels within that area. This value is then used to fill the defective pixel.

Conventional defect correction programs have several technical disadvantages. For example, conventional defect correction programs generally employ a fixed correction area, wherein a significant number of defective pixels are included, which are then included in the correction averaging calculation. This, in turn, can cause defects in the corrected image.

SUMMARY OF THE INVENTION

These and other drawbacks in the prior art are overcome in large part by a system and method for correcting defects in an image according to the present invention. In accordance with one implementation of the present invention, a method for correcting images is provided. In accordance with this implementation, the correction area surrounding the defective pixel is dynamically chosen to include a predetermined percentage of non-defective pixels. The percentage is varied as a function of the size of the surrounding region, such that the percentage decreases as the size of the region increases.

According to another implementation of the present invention, the shape of the surrounding area may be varied to further enhance the correction procedure. According to this method, subregions surrounding the defective pixel are defined. Once the subregions surrounding the pixel has been defined, the pixel is corrected, for example, by choosing a median average of the pixel amplitudes within the subregions.

Thus, the defect correcting system and method provides an improved method for selecting a replacement value for a defective pixel. This results in a clearer final image having fewer defects than in many conventional imaging systems.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the invention is obtained when the following detailed description is considered in conjunction with the following drawings, in which like reference numerals represent like features, in which:

FIG. 1 is a block diagram of an imaging system employing an image correcting system in accordance with the present invention;

FIG. 2 is an exemplary defect map showing dynamic correction region selection in accordance with the present invention;

FIG. 3 illustrates a flowchart according to one implementation of the present invention;

FIG. 4 illustrates correction subregions according to an implementation of the present invention;

FIG. 5 illustrates a flowchart of subregion selection according to an implementation of the present invention;

FIGS. 6A and 6B illustrate subregion selection according to an implementation of the present invention; and

FIG. 7 illustrates a flowchart of subregion selection according to another implementation of the invention.

DETAILED DESCRIPTION OF THE INVENTION

FIGS. 1-7 illustrate an image defect correcting system and method of operation. As will be discussed in greater detail below, an image defect correcting system according to the present invention dynamically determines a correction area surrounding defective pixels. The correction region is chosen to contain a predetermined percentage or number of non-defective pixels, which varies with the size of the region. In addition, the shape of the correction region may be varied to enhance the image correction process.

Turning now to the drawings and, with particular attention to FIG. 1, anexemplary processing system10 including image processing functionality according to the present invention is shown. As will be described in greater detail below, theprocessing system10 provides enhanced image correction according to the present invention by varying the size and shape of a correction area around a pixel to be corrected. Theprocessing system10 may be embodied as a known computer system, such as an IBM RS 6000 workstation, an Apple G4 computer, or as an IBM PC or PC-compatible computer.

Theprocessing system10 implements animage processing system12, such as a computer program. Theprocessing system10 may further include animage input device14. Theimage input device14 may be embodied as a scanner, digital camera, or any other device suitable for providing an image to be processed, such as a mass storage device. It is noted that theimage processing system10 may be embodied as part of theimage input device14, rather than, as illustrated, a separate unit. Thus, the figures are exemplary only. It is further noted that an exemplaryimage processing system10 is the image processing system described in co-pending U.S. patent application Ser. No. 09/686,719, titled Method and Apparatus for Differential-Illumination Image Capturing and Defect Handling, Attorney Docket No. ASF98-098-PA-US which is hereby incorporated by reference in its entirety as if fully set forth herein.

Theimage processing system12 includes animage correcting system15 including a correctionregion selection unit16 and a replacement value selection unit18. As will be discussed in greater detail below, theimage correcting system15 uses adefect map20 to perform image correction. One method for generating a defect map is described in U.S. Pat. No. 5,266,805, which is hereby incorporated by reference in its entirety as if fully set forth herein. This method transmits infrared light at one surface of a film. A corresponding digitized infrared image is captured on this opposite side. Imperfections are identified because they substantially reduce or totally occlude the infrared light. The digitized infrared image thus provides an accurate map of the location and shape of image defects. Other methods of generating defect maps are known and the invention is not intended to be limited to any particular method. Thus, the above discussion is by way of example only.

In the case of a binary defect map, pixels are classified according to a binary defectiveness metric as defective or non-defective. In the case of a continuous defect map, pixels are characterized by continuous defectiveness metrics. For example, one such defectiveness metric is a level of defectiveness. If values for defectiveness are normalized between 0 for defective and 1 for non-defective, a pixel with a value of 0.5 would be considered half defective. An alternative defectiveness metric is correctability, which is a metric indicating how susceptible a pixel is to a particular method for correction. For example, a pixel might be assigned a correctability of 1, meaning 100% correctable, if it can be corrected using the infrared subtractive technique, even though the defectiveness for that pixel may be only 0.5. For sake of simplicity, the following discussion will use defectiveness terminology, it being understood that other defectiveness metrics may be employed.

Theimage correcting system15 selects a correction area proximate to each defective pixel. In one implementation, the correction area is selected to contain a predetermined percentage of non-defective pixels. In a specific implementation, the percentage varies as a function of the size of the surrounding area, so that the percentage decreases as the size of the region increases. For example, in one implementation, the following formula is used to calculate the percentage of non-defective pixels:

% non-defective pixels (NDP)=100(1/(1+A/25)), (Equation 1)

where:

A=(2r+1)²

r=radius of the region

In this implementation, the term radius is used loosely since, generally, the area in question is square. The sum of non-defective pixels for the case of a binary defect map can be calculated as:

\begin{matrix} ND = \sum_{k = 1}^{A} P_{k} & (Equation 2) \end{matrix}

where:

P_k=1 if the k^thpixel is non-defective

p_k=0 if the k^thpixel is defective

To determine when the area in question is suitable for the next stage of correction, the following condition is tested:

ND>=A*NDP/100 (Equation 2.5)

If the condition in Equation 2.5 is satisfied, then the area is of sufficient size for use in the next stage of correction. If not, then the radius is incremented and the required percentage and actual sum of non-defective pixels (ND) are recalculated. For example, for r=0, the required percentage is nearly 100% and for r=2, the required percentage is 50%.

Use of this method is illustrated more particularly, by way of example, in FIG.2. FIG. 2 illustrates adefect map200. A pixel X to be corrected is denoted201,defective pixels203 are denoted d, andnon-defective pixels205 are denoted n. If r is chosen to be 1, then, usingEquation 1, abox202 surrounding thepixel201 needs 74% (or at least 7 out of 9) non-defective pixels in order to satisfy Equation 2.5. For r=1 and r=2, the criteria are not met. However, if r is chosen to be 3,Equation 1 indicates thatbox204 requires only 33% (or at least 17 out of 49) non-defective pixels, and we see that the condition of Equation 2.5 is satisfied.

In the case of a continuous defect map, a pixel reliability is defined, which is the inverse of a pixel s defect value. The pixel reliability is used as a gauge for a pixel s suitability for use as a substitute for the defective pixel in question. For example, in one implementation, a pixel s reliability is calculated using the following formula:

R=1−K*D (Equation 3)

Where:

R is the reliability

K is a proportionality constant determined empirically

D is the defect value

The range of the pixel s defect value may be set from 0 (completely non-defective) to 1 (completely defective). If K=1, it is seen that R varies from 1 (completely reliable) to 0 (completely unreliable). For K>1, the lowest value of R may be limited to 0.

In determining the appropriate size of the correction area for a continuous defect map,Equation1 above may be used for determining the required percentage of non-defective pixels. The sum of non-defective pixels, ND, now becomes the sum of each pixel s reliability:

ND=whereR_kis the k^thpixel s reliability (Equation 4)

As discussed above, once the correction area has been determined, the amplitudes of the pixels in the correction area are used to determine a corrected amplitude value for the pixel which is to be corrected. A variety of methods for choosing the correction value may be employed. For example, the amplitudes of the pixels in the correction region may be averaged to determine the correction value.

In an implementation employing a binary defect map, only the amplitudes of the non-defective pixels are used to determine the correction amplitude value. For example, if the

correction areas

202 or204 of FIG. 2 define binary defect maps, then the amplitudes of thedefective pixels203 are ignored in determining the correction value. Instead, only the amplitudes of thenon-defective pixels205 are used. According to one implementation, the median amplitude of all non-defective pixels is used as a substitute for thepixel201. Other mathematical techniques may be employed. For example, cubic spline techniques or similar mathematical models may be implemented.

If the defect map is a continuous defect map, then an amplitude value is chosen based on an averaging of the values of pixels in the surrounding region, taking into account or weighting their reliabilities or defectiveness metrics. Thus, the value may be obtained by taking a mean or median average of those pixels having a particular reliability.

A flowchart illustrating operation of one implementation of the present invention is shown in FIG.3. This flowchart represents the steps involved in correcting one pixel in the image, and as such would be repeated for each pixel in the entire image. In one implementation, only the original pixel values (and not the newly corrected pixel values) are used in any of these steps. In astep302, theimage correcting system14 sets the index value r equal to a predetermined index, such as 1. In astep304, theimage correcting system14 calculates the percentage of non-defective pixels allowed for that particular value of r. Alternatively, theimage correcting system14 may access a lookup table (not shown) which contains a list of values of r and corresponding percentages. Then, in astep306, theimage correcting system14 determines the actual percentage of non-defective pixels. For example, theimage correcting system14 may count the non-defective pixels within the region defined by r. The actual percentage of non-defective pixels is compared to the calculated percentage of non-defective pixels in astep308. If the actual percentage is not greater than or equal to the calculated percentage, then in astep310, the index r is incremented and the program loops back tostep304. However, if the actual percentage is greater than or equal to the calculated percentage, then in astep312, a correction value is determined by theimage correcting system14. For example, in the case of the binary defect map, the amplitude values of the non-defective pixels are used to determine a median average. In the case of a continuous defect map, the values of all pixels are used to determine the median. Once the correction value is determined, the image correcting program replaces the defective pixel value with the calculated correction amplitude value, in astep314.

As noted above, the shape of the area surrounding the defective pixel may also be varied. The shape is varied while maintaining a center of gravity on the defective pixel (i.e., the defective pixel is maintained at the center of subregions weighted according to size and number of defective pixels). To maintain the center of gravity, subregions are created around the defective pixel which contain predetermined percentages of non-defective pixels. According to one implementation, the subregions are defined as rectangular quadrants of dimension r×(r+1), where r is the effective radius for the quadrant. Although the subregions are described in terms of four rectangular regions, the subregions may have any of a variety of shapes. The subregions can have effective radii which differ from one another depending upon the proportion of defective pixels within the particular subregion. For example, as shown in FIG. 4, amap400 surrounding the defective pixel399 (X)may be divided into subregions401-404 (Q1-Q4). For subregion401 (Q1), r=3; for subregion402 (Q2), r=2; for subregion403 (Q3), r=3; and for subregion404 (Q4), r=1.

In one implementation, the center of gravity is maintained by determining the number of non-defective pixels for a portion of each subregion associated with a particular increment of r. The sum of non-defective pixels (ND) is multiplied by its value of rand added vectorially to similar products from other subregions. The center of gravity is maintained by constraining the vector sum to be zero. For example, in the case of quadrants, +i is assigned to quadrant401 (Q1), −i can be assigned to quadrant403 (Q3), +j to quadrant402 (Q2), and j to quadrant404 (Q4), where i is the unit vector for the positive x-axis, and j is the unit vector for the positive y-axis. If the vector sum is nonzero, the radius is incremented for the appropriate quadrant to achieve the proper center of gravity.

The following equation is used to determine the vector sum:

VS=r₁ND_Q1î−r₃ND_Q3î+r₂ND_Q2ĵ−r₄ND_Q4ĵ Equation 5

For a binary defect map, ND for each quadrant is calculated using an equation similar to Equation 2 (i.e., the area A is now the area of the quadrant). For a continuous defect map, ND for each quadrant is calculated using an equation similar toEquation 4. In addition, a modified version ofEquation 1, as seen below, is used to constrain the appropriate total percentage of non-defective pixels:

NDP=100*(1/(1+ΣA/25)) Equation 6

As can be seen, the area A inEquation 1 is now the sum of the areas of each subregion. In certain implementations, the sum of the areas can include the central pixel, i.e., for r=0.

For example, in FIG. 4, VS=24i−24i+8j−j=7j, and NDP=43.1% (A=33). It is seen that Equation 2.5 is satisfied. If Equation 2.5 had not been satisfied, the index r would be incremented, and the above-described calculations undertaken again to determine whether the region(s) now satisfy the criteria of Equation 2.5. It is noted that a maximum number of iterations may constrain the image correction system to prevent hysteresis. Moreover, in certain cases, it may not be necessary for the vector sum VS to equal zero (0). Satisfactory image correcting performance may be achieved if VS meets a predetermined threshold value.

Turning now to FIG. 5, a flowchart illustrating operation of this implementation of the invention is shown. In astep502, correctionregion selection unit16 defines an initial index r for each of a predetermined number of subregions. Then, in astep504, using adefect map20, the correctionregion selection unit16 calculates ND for all of the subregions. In astep506, the correctionregion selection unit16 calculates VS and NDP In astep508, the correctionregion selection unit16 determines whether VS=0 (or is below a threshold value). If not, then the value of r for one or more of the subregions is incremented, in astep512, and the system cycles back tostep504. If, instep508, VS=0, then in astep510, it is determined whether Equation 2.5 is satisfied. If not, then one or more values of r are incremented and the system returns to step504. Otherwise, the selection of the subregions is complete. As discussed above, the number of iterations of the process may be constrained to a particular number.

Another method of growing separate subregions is to constrain each subregion to contain above a threshold number of non-defective pixels. Thus, in the case of quadrants, each quadrant contains ¼^thof the total number of non-defective pixels. Initially, each quadrant is the same size. The sum of all non-defective pixels is then determined, and ¼ of that value is used as a threshold for the next iteration of quadrant growth. If a quadrant has already contributed more than this threshold, it is not grown (i.e., r is not increased). Otherwise, r is incremented by 1. When all four quadrants are evaluated in this manner, a new sum of non-defective pixels is calculated and the process repeats. When the total number of non-defective pixels satisfies the requirement ofEquation 1, then the process is finished with each quadrant contributing equally. In the case of a continuous defect map, the sum in question is the sum of reliability values for each pixel.

For example, FIG. 6A and 6B illustrate quadrant growth on the map of FIG.4. Initially, as shown in FIG. 6A, the radius, r, is set to 1. As shown, ND=1, and the quadrant threshold T=ND/4=0.25, and NDP=74%. Quadrants601a-604aare ofdimension 1×2; all quadrants exceptquadrant604ahave two defective pixels. In this case, quadrant601d(Q4) is not grown, but quadrants601a-601c(Q1-Q3) are. The result is seen in FIG.6B. In this case, ND=9, and the threshold T=2.25. As can be seen, of the newly-grownquadrants601b-603b, onlyquadrant602bhas a number of non-defective pixels above the threshold. The process continues until all quadrants are above the associated threshold.

This process is illustrated with reference to FIG.7. In astep702, the radius r_kfor each subregion, k=1 (to m) is set to an initial value, such as 1. In astep704, the system calculates the sum of non-defective pixels (ND), the total required percentage (NDP) , and the threshold required of non-defective pixels per subregion (T). In a quadrant-based system, T=ND/4, the total number of subregions m=4, and the index k (in r_k) varies from m=1 to m=4. Next, in astep706, the sum of non-defective pixels (ND) is compared to the required percentage (NDP), as in Equation 2.5. If this condition is satisfied, a flag, F, is set to TRUE. Otherwise, this flag is set to FALSE. Next, in astep708, the index k is set to 1. Instep710, the sum of non-defective pixels for the quadrant associated with k (ND_k) is calculated and compared to the threshold, T. If ND_kis less than T, then the radius for that quadrant (r_k) is incremented instep712. Otherwise, index k is incremented instep714. The new value of index k is compared against m instep716 and if less than or equal to m, control is returned to step710. Otherwise, step716 checks the flag F. If F is FALSE, control is returned to step704. Otherwise, the process is finished for the current pixel.

Once the subregions have been selected, the correction value may be chosen in a manner similar to that discussed above. However, in another implementation, a substitute value for the amplitude of the pixel399 (FIG. 4) is calculated by first determining a figure of merit for each pixel in a subregion once the size and shape of the subregions are determined. The figure of merit indicates the appropriateness of using that pixel as a substitute. One particular figure of merit is shown below:

FM_k=R_k/r_k Equation 7

where, for the k^thpixel:

FM_kis the figure of merit;

R_kis the reliability;

r_kis the radius.

Within each subregion, the pixel with the highest figure of merit is selected.

The figures of merit for each subregion are then summed:

\begin{matrix} {FM}_{t} = \sum_{k = 1}^{N} {FM}_{k} & (Equation 8) \end{matrix}

where:

FM_kis the figure of merit for the selected pixel from the k^thsubregion;

FM_tis the total figure of merit.

N is the total number of subregions (N=4 for quadrants)

Then, an amplitude value for the defective pixel is chosen according to the following equation:

\begin{matrix} S = \sum_{k = 1}^{N} P_{k} * {FM}_{k} / {FM}_{t} & (Equation 9) \end{matrix}

where:

P_kis the amplitude of the selected pixel from the k^thsubregion;

S is the substitute value.

When using quadrants, the summations in Equations 8 and 9 go to 4. (i.e., Q=4).

The invention described in the above detailed description is not intended to be limited to the specific form set forth herein, but is intended to cover such alternatives, modifications and equivalents as can reasonably be included within the spirit and scope of the appended claims. To aid the Patent Office and any readers of any patent issued on this application in interpreting the claims appended hereto, applicants wish to note that they do not intend any of the appended claims to invoke paragraph 6 of 35 U.S.C. Sec. 112 as it exists on the date of filing hereof unless the words “means for” or “step for” are used in the particular claim.

Claims

What is claimed is:

1. A method for correcting images, comprising:

selecting an area proximate at least one defective pixel, said area selected to include a variable number of pixels having a defectiveness metric above a predetermined threshold; and

replacing said defective pixel with a pixel having a value derived from values of one or more selected pixels within said area.

2. A method according toclaim 1, said replacing said defective pixel including determining an average value of said selected pixels within said area.

3. A method according toclaim 2, including deriving a reliability from selected pixels within said area.

4. A method according toclaim 3, said replacing said defective pixel including determining a median of non-defective pixels within said area.

5. A method according toclaim 4, including weighting the median with a defectiveness metric.

6. A method according toclaim 3, said determining an average value including determining a mean of selected pixels within said area.

7. A method according toclaim 6, including weighting the average with a defectiveness metric.

8. A method according toclaim 1, said selecting an area including selecting one or more subregions chosen to include a variable percentage of non-defective pixels.

9. A method according toclaim 8, said replacing including averaging among selected pixels in said subregions.

10. A method according toclaim 9, said averaging including averaging among said subregions weighted so that a center of gravity is coincident with said defective pixel.

11. A method according toclaim 10, wherein said on or more subregions define quadrants.

12. An image processing system, comprising:

a correction region selection unit operable to dynamically select a correction area around at least one defective pixel, said area including a variable level of non-defective pixels within a defectiveness metric above a predetermined threshold; and

replacing said defective pixel with a pixel having, a value derived from values of one or more selected pixels within said area.

13. An image processing system according toclaim 12, said replacement value selection unit configured to determine an average value of said selected pixels within said area.

14. An image processing system according toclaim 13, said replacement value selection unit configured to derive a reliability from selected pixels within said area.

15. An image processing system as recited inclaim 12, wherein said image processing system comprises a scanner.

16. An image processing system, comprising:

a correction region selection unit operable to dynamically select a correction area around at least one defective pixel, said area including a variable level of non-defective pixels; and

a replacement value selection unit configured to (i) select a replacement value for said defective pixel, wherein said replacement value selection unit configured to determine an average value of the selected pixels within said area, (ii) derive a reliability from selected pixels within said area, and (iii) determine a median of non-defective pixels within said area.

17. An image processing system according toclaim 16, said replacement value selection unit configured to weight the median with a defectiveness metric.

18. An image processing system according toclaim 17, said replacement value selection unit configured to determine a mean of selected pixels within said area.

19. An image processing system according toclaim 16, said replacement value selection unit configured to weight the average with a defectiveness metric.

20. An image processing system, comprising:

a replacement value selection unit configured to select a replacement value for said defective pixel, wherein said replacement value selection unit configured to select one or more subregions chosen to include a variable percentage of non-defective pixels.

21. An image processing system according toclaim 20, said replacement value selection unit configured to average among selected pixels in said subregions.

22. An image processing system according toclaim 21, said replacement value selection unit configured to average among said subregions weighted so that a center of gravity s coincident with said defective pixel.

23. An image processing system according toclaim 22, wherein said on or more subregions define quadrants.

24. An image processing system, comprising:

a replacement value selection unit configured to select a replacement value for said defective pixel, wherein said image processing system comprises a reflective-type scanner.

25. A method for correcting images, comprising:

selecting one or more variably-sized regions proximate a defective pixel;

replacing said defective pixel with a pixel having a value derived from values of one or more of pixels within said one or more variably-sized regions; and

selecting said one or more regions to have a predetermined level of pixels having a defectiveness metric above a predetermined threshold.

26. A method according toclaim 25, said replacing said defective pixel comprising determining an average value of said values of said one or more pixels.

27. A method according toclaim 25, including deriving a reliability from selected pixels within said one or more variably-sized regions.

28. A method according toclaim 27, said replacing said defective pixel including determining a median of non-defective pixels within said one or more variably-sized regions.

29. A method according toclaim 28, including weighting the median with a defectiveness metric.

30. A method according toclaim 25, said determining an average value including determining a mean of selected pixels within said one or more variably-sized regions.

31. A method according toclaim 30, including weighting the average with a defectiveness metric.

32. A method according toclaim 31, said averaging including averaging among said subregions weighted so that a center of gravity is coincident with said defective pixel.