WO2000079781A1 - Image enhancement system - Google Patents

Abstract

A method of determining an interpreted line in a line doubling system including the steps of determining the value of pixels on adjacent lines, determining the Root Mean Square (RMS) value of the pixels on adjacent lines, and utilizing the RMS value to create the interpreted line.

Description

IMAGE ENHANCEMENT SYSTEM

FIELD OF THE INVENTION

The present invention relates to scan line doublers for increasing the number of apparent scan lines of a display device to reduce the visibility of the scan line structure of the picture image. More particularly, the present invention relates to a television, computer monitor or video projector scan line doubter which includes a method that overcomes the limitations of pixel interpolation by scan line averaging. BACKGROUND OF THE INVENTION When all other sources of error and distortion have been removed or minimised by correction or compensation, standard PAL or NTSC colour video images become limited in quality by perceptibility of the line scan structure.

Subjective visibility of the line scan structure is a direct consequence of the limited number of horizontal scan lines in the standard composite picture, and is further a direct consequence of the field by field interlace. A conventional PAL television frame at 25HZ repetition rate is composed of two fields eg F0 and F1. Each field includes 312.5 scan lines, each of which are separated by an unilluminated strip or band. Successive fields are offset so that the scan lines of the next field occupy the unilluminated strips of the present field. This arrangement is followed to minimize perception of 25Hz flicker in the resultant image.

The need to increase the number of scan lines is particularly evident in the application of Field Sequential 3D. In this instance, odd lines of the video image are used to carry the left eye image and even lines the right eye image. Thus, after de-multiplexing, the image intended for each eye is at half the resolution of the original video standard.

One approach to reducing the visibility of the line scan structure of the image calls for estimating, or interpolating, picture elements of additional scan lines from the picture elements already present in the picture image scanned in the conventional format. This prior approach is known in the art as "scan line doubling" or "line doubling", and calls for doubling the number of scan lines from 312.5 to 625 lines per field. Thus 625 lines are presented each 50th of a second. One prior approach to pixel interpolation is carried out by an intra-field or spatial domain process. The pixel for the unilluminated band between two scan lines is derived as the average of the pixel amplitude and hue of the pixel in the scan line directly above and of the pixel in the scan line directly below. The main drawback of this approach is the reduced resolution or softness of the resultant picture in the vertical dimension at edges and some perceptible 25Hz vertical flicker in the instance of sharp vertical transitions within the picture image. OBJECTIVES OF THE INVENTION

A general objective of the present invention is to provide an improved method and apparatus for television scan line doubling and display. The invention overcomes a number of limitations of the line averaging techniques of prior art and may be simply implemented in readily available hardware or software.

A more specific objective of the invention is to include a method whereby the additional pixel amplitude and hue may be determined via mathematical calculation or a lookup table and applied based upon specific characteristics of the overall image.

With the above objectives in mind, the present invention provides a method of determining an interpreted line in a line doubling system including the steps of: determining the amplitude and hue of pixels on adjacent lines; determining the Root Mean Square (RMS) value of the amplitude and hue of the pixels on adjacent lines; utilizing the RMS value to create said interpreted line. The Root mean Square value may be calculated in hardware or software for each set of adjacent pixels. Alternatively, a lookup table could be used to approximate the Root Mean Square value.

These and other objects, aspects, advantages and features of the present invention will be more fully understood and appreciated upon consideration of the following detailed description of a preferred embodiment, presented in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 depicts a segment of a video image.

Figure 2 depicts two tables showing average of two pixels, A and B, and the RMS of A and B. Figure 3 depicts a hardware implementation of the invention.

Figure 4 depicts a simplified hardware implementation of the invention.

Figure 5 depicts the difference between two values A and B.

Figure 6 depicts a lookup table based upon the difference between A and B. Figure 7 depicts a Pseudo RMS value of A and B.

Figure 8 depicts the true RMS value of A and B DETAILED DESCRIPTION OF A PREFERRED EMBODIMENTS

Figure 1 depicts a segment of a video image consisting of Field Zero line zero (F0), Field One line one (F1) and interpreted lines L0 and L1. The individual pixels per line are indicated such that the first pixel on line one of Field 0 is marked PF0,1 the second pixel PF0,2 and the nth pixel PF0,n. Similar terminology is used throughout the figure.

The prior art has described one method of adding additional lines to an interlaced image by inserting an additional pixel having a value equal to the average of the value of the pixel in the line above and the pixel in the line below i.e. if the pixel above has a value A and the pixel below has a value B then the additional pixel will have a value equal to:

New pixel = (A+BV2

The main shortcoming of this technique is that the resultant image tends to look softer or slightly out of focus. This is direct limitation of using a simple averaging technique. For example, if we consider a black and white image as the input source and the above line to be peak white and the below line to be black then the interpolated line will be grey.

In order to overcome this limitation it is considered desirable to insert an interpolated line containing pixels that are more closely associated with the luminance value of the brighter of the two pixels A and B.

In a preferred embodiment this invention discloses a new technique to achieve this by taking the Root Mean Square (RMS) of the values of A and B rather than the average.

This is illustrated in figure 2 which contains two tables. The first table depicts A along the X axis and B along the Y axis and the average of A and B at the intersection.

The second table depicts A along the X axis and B along the Y axis and the RMS value of A and B at the intersection. The RMS value is calculated from:

RMS Value = (A²+ B²)/2)

By comparing the Average and RMS tables it is evident that: 1. When A = B, both mean and RMS process yield the same result;

2. When A > B or B > A then the result is squewed closer to the larger value, which is the desired result.

This RMS processing may be implemented in either hardware or software.

A means of implementing the process in hardware is illustrated in figure 3. A pixel from a line in field FO and a corresponding pixel from the line below in F1 are both passed simultaneously to squarer circuits. The output from each squarer is added and this result subsequently divided by two. This intermediate result has its square root taken and the resulting value becomes the new pixel.

If the image to be line doubled is in colour then the original image may well be in RGB format. If this is the case then each of the individual R, G and B values will require to be processed using the RMS method. Such a hardware implementation will require six squaring circuits and three square root circuits.

Both the squaring and square root functions are comparatively difficult to implement in hardware. It is therefore another objective of this invention to disclose an alternative preferred embodiment that enables an RMS value to be calculated in an efficient and effective manner.

Figure 4 discloses an alternative preferred embodiment that simplifies the hardware implementation of the RMS process.

Figure 4 depicts a Read Only Memory (ROM) that requires an input address and provides data output dependant upon the input address. For illustrative purposes only, consider the pixel values to be quantised to 8 bits i.e. 256 individual levels. The quantised pixels PFO.n and PF1 ,n are used to form an address for the ROM. At each unique address is stored a byte that approximates to the RMS value of PF0,n and PF1 ,n.

It is desirable that the RMS process can be implemented within an ASIC or FPGA. Using the ROM process of Figure 4, and assuming 8 bit RGB video then the number of input-output lines, external to the ASIC or FPGA, required to address the ROM's becomes excessive. Whilst a single ROM could be multiplexed across the R,G and B signals this may cause timing problems. In order to implement the RMS process within an ASIC or FPGA a simplified implementation is disclosed.

An alternative preferred embodiment that simplifies the look up requirement such that the RMS process could be implemented within an ASIC or FPGA may operate as follows:

Given above and below pixels A and B; If A < B then swap A and B such that A is always greater than or equal to

B;

If A = B then the new pixel = A;

Take the difference between A and B;

Use the difference to index into a lookup table; Add the value from the lookup table to B;

Use this result as the value of the new pixel.

In a practical implementation, comprising 8 bit RGB video, the lookup table would be contained in a ROM and the difference information would be used as the address, which would typically be 8 bits, of the data located in the ROM. Thus in this implementation the ROM would contain a maximum of 256 addresses each containing an 8 bit value.

Figures 5 through 8 illustrate this simplified process as follows. In figure 5 the A value is horizontal and the B value vertical. The table contains the difference between A and B where A > B or A = B. Note: In order to simplify the explanation A and B are assumed to take values of between 0 and 100 in steps of 10.

In figure 6 the table depicts the value that would be stored in the lookup table for each difference between A and B.

Figure 7 shows the effect of applying the previously disclosed method of approximating the RMS value of A and B. In this figure the A value is again horizontal and the B value vertical. The union of A and B within the table is the approximate RMS value of A and B, or a so called "Pseudo RMS" value.

If the results of figure 7 are compared with the true RMS values of A and

B shown in figure 8 ( which has been rounded up to zero decimal places ) it will be seen that the percentage error in most cases is small and not significant in this particular application. This is due to the fact the eye is not sensitive to small variations in colour or intensity.

The objective of this invention is to overcome the previously described shortcomings of simply adding an additional pixel that is the mean of A and B.

This is a achieved by squewing the value of the additional pixel towards the larger value pixel. Since, in the preferred embodiments, the value of the additional pixel is determined by deriving the RMS value of two existing pixels from vertically opposed lines and the RMS value determined, either derived accurately or an approximation, from a lookup table, then the values contained within the lookup table can be altered to provide the most aesthetically pleasing images. In a preferred embodiment, different look up tables, or the same lookup table with different weightings, could be used depending upon the overall characteristics of the original image.

For example, should the overall image be particularly dark then it would be preferable to use interpolated pixels that are closer to the average value. Alternatively, if the image contains areas of high contrast then an alternative table, or the same table with different weightings, may be used with values that enhance the contrast differences.

It will be appreciated by those skilled in the art that these techniques may be applied to the whole image or selectively over the image such that different areas of the image may use different look up tables, or apply different weightings. The use of different tables or weightings could be determined by, but not limited to, brightness, contrast, colour, shading, hue, saturation, or marked differences between these values over the image being processed.

Whilst the method and apparatus of the present invention has been summarised and explained by an illustrative application in television line doubling, it will be appreciated by those skilled in the art that many widely varying embodiments and applications are within the teaching and scope of the present invention, and that the examples presented herein are by way of illustration only and should not be construed as limiting the scope of this invention.

Claims

THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS:

1. A method of determining an interpreted line in a line doubling system including the steps of: determining the value of pixels on adjacent lines; determining the Root Mean Square (RMS) value of the pixels on adjacent lines; utilizing the RMS value to create said interpreted line.

2. A method as claimed in claim 1 wherein the step of determining the value of pixels on adjacent lines includes determining the amplitude and hue of said pixels.

3. A method of determining an interpreted line in a line doubling system including the steps of: determining the amplitude and hue of pixels on adjacent lines; determining the Root Mean Square (RMS) value of the amplitude and hue of the pixels on adjacent lines; utilizing the RMS value to create said interpreted line.

4. A method of determining an interpreted line in a line doubling system including the steps of: determining the value of R, G and B components of pixels on adjacent lines; determining the Root Mean Square (RMS) value of each said R, G and B components of the pixels on adjacent lines; utilizing each RMS value to create said interpreted line

5. A method as claimed in any preceding claim wherein said RMS value of an interpreted pixel is determined by:

RMS Value = V(A²+ B²)/2}

Wherein A represents a pixel immediately above said interpreted pixel, and B represents a pixel immediately below said interpreted pixel.

6. A method as claimed in any one of claims 1 to 4 wherein said RMS value is approximated.

7. A method as claimed in claim 6 wherein said approximate RMS value is determined from a lookup table.

8. A method of determining an interpreted line in a line doubling system including the steps of: determining the value of pixels on adjacent lines; accessing a memory device utilising the value of said adjacent pixels as an address for said memory device wherein an approximate Root Mean Square (RMS) value of the pixels on adjacent lines is stored at said address; utilizing the RMS value to create said interpreted line.

9. A method as claimed in claim 8 wherein said memory device is a ROM.

10. A method of determining an interpreted line in a line doubling system including the steps of: determining the value of pixels on adjacent lines; consulting a lookup table to determine the approximate Root Mean Square (RMS) value of said adjacent pixels utilising the RMS value to create said interpreted line.

11. A method of determining an interpreted line in a line doubling system including the steps of: determining the value of a first pixel located above an interpreted pixel; determining the value of a second pixel located below an interpreted pixel; calculating the difference value between said first and second pixels; utilising said difference value to index a lookup table and obtain a table value; O 00/79781 -| Q PCT/AUOO/00673

calculating an added value by adding said table value to the lesser of said first and second pixels; assigning said calculated value to said interpreted pixel.

12. A method as claimed in claim 11 wherein said table value is predefined such that said added value is the approximate Root Mean Square (RMS) value of said first and second pixels.

13. A method as claimed in claim 11 or 12 wherein said table value is stored in a memory device and said difference value is the address to obtain said table value.

14. A method as claimed in any one of claims 11 to 13 wherein said difference value is calculated by: comparing the value of said first and second pixels; if said second pixel value is greater than said first pixel value then said first and second pixel values are interchanged; subtracting said second pixel value from said first pixel value.

15. A method as claimed in any one of claims 11 to 13 wherein said difference value is calculated by: comparing the value of said first and second pixels; if said first pixel value is greater than said second pixel value then said first and second pixel values are interchanged; subtracting said first pixel value from said second pixel value.

16. A method as claimed in any one of claims 7 to 13, including a plurality of lookup tables, and wherein the table to be utilised is selected based on the properties of the overall image.

17. A method as claimed in any one of claims 7 to 13 further including the step of adding or subtracting a constant to said added value, wherein said constant is selected based on the properties of the overall image.

18. A system for determining an interpreted line in a line doubling system including: an analysis means to determine the value of pixels on adjacent lines; and a calculation means to determine the Root Mean Square (RMS) value of pixel values detected by said analysis means on said adjacent lines; wherein said RMS value determined by said calculation means is utilised to create said interpreted line.

19. A system as claimed in claim 18 wherein said analysis means determines the amplitude and hue of the adjacent pixels.

20. A system as claimed in claim 18 or 19 wherein said analysis means determines the value of R, G and B components of the adjacent pixels, and said calculation means determines the RMS value for each said R, G and B component.

21. A system as claimed in any one of claims 18 to 20 wherein said calculation means determines the RMS value by:

RMS Value = V (A²+ B²)/2} wherein A represents a pixel immediately above said interpreted pixel, and B represents a pixel immediately below said interpreted pixel.

22. A system as claimed in any one of claims 18 to 20 wherein said RMS value is approximated.

23. A system as claimed in claim 22 further including a memory means to store approximate RMS values and wherein said adjacent pixel values form an address to access said memory means.

24. A system as claimed in claim 23 wherein said memory means is a ROM.

25. A system as claimed in claim 22 further including a storage means to store approximate RMS values in a lookup table and wherein said adjacent pixel values form an address to access said lookup table.

26. A system for determining an interpreted line in a line doubling system including: an analysis means to determine the value of a first pixel located above an interpreted pixel, and the value of a second pixel located below said interpreted; a calculation means to determine a difference value between said first and second pixels; a storage means to store a lookup table, wherein said difference value is an index to said lookup table, said index returning a table value; an adding means to add said table value to the lesser of said first and second pixel values; wherein the value returned from said adding means is assigned to said interpreted pixel.

27. A system as claimed in claim 26 wherein said lookup table is predefined such that the value returned from said adding means is the approximate RMS value of said first and second pixel values.

28. A system as claimed in any one of claims 25 to 27 wherein said storage means further includes a plurality of lookup tables, said system further including a selection means to select an appropriate table based on the properties of the overall image.

29. A system as claimed in any one of claims 25 to 27 wherein said adding means further adds or subtracts a constant to said table value, said constant being based on the properties of the overall image.

30. A method or system substantially as herein before described with reference to the accompanying drawings.

DATED this 15th day of June, 2000 DYNAMIC DIGITAL DEPTH RESEARCH PTY LTD WATERMARK PATENT & TRADEMARK ATTORNEYS 4TH FLOOR, "DURACK CENTRE" 263 ADELAIDE TERRACE PERTH W.A. 6000 AUSTRALIA