US20160227188A1 - Calibrating range-finding system using parallax from two different viewpoints and vehicle mounting the range-finding system - Google Patents

Abstract

A range-finding system includes two imaging devices to capture multiple images from two different viewpoints and a parallax calculator to calculate parallax based on the multiple images captured by the two imaging devices. The parallax calculator includes an estimation unit to estimate a correction value based on the amount of image deviation in a lateral direction corresponding to pixel position in the images captured by the two imaging devices and a correction unit to correct a pre-correction parallax or an image based on the correction value estimated by the estimation unit.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
• This patent application is a continuation of and claims priority under 35 U.S.C. §§120/121 to U.S. patent application Ser. No. 13/798,321, filed on Mar. 13, 2013, which is based on and claims priority pursuant to 35 U.S.C. §119 to Japanese Patent Application Nos. 2012-063705, filed on Mar. 21, 2012 and 2013-003413, filed on Jan. 11, 2013 in the Japan Patent Office, the entire disclosures of each of which are hereby incorporated by reference herein.
BACKGROUND
• 1. Technical Field
• The present invention relates to a range-finding system including a stereo camera system and a vehicle mounting the range-finding system.
• 2. Related Art
• Collision avoidance systems involving the use of in-vehicle stereo camera systems have become more common. A stereoscopic image of the area in front of the vehicle is generated using the stereo camera systems, and an obstacle is detected and a distance to the obstacle is measured based on the generated stereoscopic image. The driver can then be alerted to take corrective action to avoid a collision or maintain a safe minimum distance between vehicles. Alternatively, the system can engage a control device such as the brakes and the steering.
• FIG. 10 is a diagram for explaining a principle of range-finding in which two cameras are disposed in parallel. Acamera100awith a focal length f, an optical center O₀and an imaging plane S₀is disposed such that a direction of an optical axis extends in a upper direction inFIG. 10, and acamera100bwith the same focal length f is deposed in parallel with and spaced by a distance B with respect to thecamera100aon the right side.
• An image of a subject A which is located at a distance d from an optical center O₀of thecamera100ain the direction of the optical axis forms an image at a point P₀which is the intersection of a straight line A-O₀and an imaging plane S₀. With respect to thecamera100b,the subject A forms an image at a point P₁on an imaging plane S₁. Here, an intersection of a line and the imaging plane S₁, which line passes through an optical center O₁of thecamera100band is parallel to the line A-O₀, is indicated by P′₀, and the distance between the point P′₀and the point P₁is indicated by p. The point P′₀is the same as the point P₀on thecamera100a.The distance p indicates a deviation amount P′₀-P₁of the points on the images captured by these twocameras100aand100band is called parallax. Then, a triangle A₀-O₀-O₁and a triangle O₁-P′₀-P₁are geometrically similar and thus the following formula 1 is given.
• $\begin{array}{cc}d=\frac{\mathrm{Bf}}{p}& \left(1\right)\end{array}$
• where B represents the base line length of the triangle. If the base line length B and the focal length f are known, the distance d can be determined based on the parallax p.
• In order for the above-described principle of range-finding by stereo camera system to hold, the twocameras100aand100bmust be disposed precisely, as shown inFIG. 10. However, when the two cameras are mounted in a vehicle, it is difficult to dispose the cameras in parallel without some error, and therefore parallax correction techniques using signal processing are proposed.
• As one example, after the stereo camera system is assembled, by capturing a test chart set at a known position, relative positions of the two cameras are accurately measured. Then, using the measured data and deforming the captured image, a pseudo-parallel arrangement can be realized. At this time, the image data is modified by affine conversion.
• In another example, in order to correct non-linear deviation, such as aberration of distortion of the optical lenses, which cannot be modified only by affine conversion, a table is used the contents of which are determined based on data of a captured test chart set at a known position.
• In yet another example, a technique to correct the deviation caused by distortion and temperature change in actual use is proposed. During actual use, it is difficult to capture the known test chart, unlike during the manufacturing process. Instead, using only how parallel the dividing line on the road is, the deviation amount is calculated.
• Herein, to maintain consistently accurate range-finding in the above-described range-finding device, precise calibration is needed. Accordingly, various calibration techniques are proposed.
• Known calibration techniques fall roughly into two categories: Those having set times for executing calibration and those having a degree of flexibility in modifying the image.
• More specifically, the timing for executing calibration can be divided into two types: Those done during manufacturing and those done while in use. For example, in the calibration in manufacturing, the optical distortion of the lens and deviations arising in assembling are measured and corrected in manufacturing. By contrast, in the calibration in use, the influenced by temperature change and vibration over time is measured and corrected. The flexibility of the image deformation means, for example, affine conversion, non-linear conversion, and parallel movement having a predetermine offset.
• However, in the calibration in use, a known object such as a test chart cannot be captured, and image conversion other than with a particular offset cannot be executed. Therefore, it is difficult to measure image deviation in the lateral direction accurately. However, when two cameras are set at ideal position, such as without aberration in assembling, corresponding points between a left image captured by a left camera and a right image captured by a right camera always show parallax in the lateral direction. Therefore, if a vertical component of the position difference between the corresponding points is present, it can be easily assumed that there is image deviation in the vertical direction based on the position difference between the corresponding points. By contrast, with the parallax in the lateral direction, a true value thereof depends on the distance to the object, and therefore detecting or estimating the amount of the deviation in the lateral direction is difficult.
• In yet another example, using how parallel is the dividing line and the number of motionless objects, the parallax offset, that is, a parallel moving amount of the entire image in the left image and the right image in the lateral direction is estimated. However, image deviation in the lateral direction of the stereo camera system having two cameras provided in parallel is not limited to the parallel movement. Accordingly, the image deviation in the lateral direction cannot be measured and corrected without using the known test chart in use.
• The types of lateral deviation possible in a stereo camera system are described below. The lateral deviation generated in the stereo camera system includes lateral deviation caused by rotation around a vertical axis and horizontal deviation caused by change in distortion characteristics. When a line of vision of the camera is slightly changed in a given direction, the lateral deviation caused by rotation around a vertical axis is moved to an opposite direction to the predetermined captured image. However, at this time, the image is not moved strictly in parallel.
• For example, as illustrated inFIG. 11A, when acamera100 is tilted to the left, a rectangle positioned in front of the camera is deformed as shown inFIG. 11B. AlthoughFIG. 11B exaggerates the effect, when thecamera100 is tilted to the left, a rectangle positioned in front of the camera is deformed from therectangle110 to atrapezoid120. In this case, the entire image is deviated to the right, the size of the right side image is reduced, and the left side of the image approaches the center position. Therefore, compared to a position B adjacent to the center position of the image, a left side point A is greatly moved to the right side. The right side of the image is expanded and the moved far from the center position, and a right end C is greatly moved to the right side.
• As a result, the image deviation to right side is not uniform over the entire image, as illustrated inFIG. 12. In a comparative example, if the angle of deviation is very small, thetrapezoid120 shown inFIG. 11B is very near to therectangle110, which is regarded as pseudo-parallel movement. However, in order to detect the parallax more accurately, the parallax should be corrected not just regarding simple parallel movement but also as to deviation that varies depending on the pixel position.
• Next, the lateral deviation caused by the change in the distortion characteristics is described. The focus lens system generally has aberration distortion characteristics. In an initial state, using the above-described table, non-linear deviation can be corrected but the distortion characteristics may change over time. In this case, the change in the center area of the image is small, but at the edges of the image the image deviation may be something other than parallel movement, such as expansion or contraction.
• Furthermore, if deviation other than that which is symmetrical around the optical axis occurs, such as, tilt or eccentricity of the lenses, anisotropic distortion is generated, resulting in even more complicated deviation.
SUMMARY
• In one aspect of this disclosure, there is a range-finding system that includes two imaging devices and a parallax calculator. The two imaging devices capture multiple images from two different viewpoints. The parallax calculator calculates parallax based on the multiple images captured by the two imaging devices. The parallax calculator includes an estimation unit and a correction unit. The estimation unit estimates a correction value based on the amount of image deviation in a lateral direction corresponding to pixel position in the images captured by the two imaging devices. The correction unit corrects a pre-correction parallax or an image based on the correction value estimated by the estimation unit.
• In another aspect of this disclosure, there is provided a novel a vehicle including the above-described range-finding system.
BRIEF DESCRIPTION OF THE DRAWINGS
• A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:
• FIG. 1 is a hardware block diagram illustrating a range-finding system including a stereo camera system according to the present disclosure;
• FIG. 2 is a software block diagram illustrating a configuration of a parallax calculator in the range-finding system according to a first embodiment of the present disclosure;
• FIG. 3 is a flow chart illustrating a process executed by a parallax image generation processor shown inFIG. 2;
• FIG. 4 is one example of an image divided into three image regions;
• FIG. 5 is a graph illustrating calculation examples of parallax correction values according to the first embodiment;
• FIG. 6 is a software block diagram illustrating a configuration of a parallax calculator in the range-finding system according to a second embodiment of the present disclosure;
• FIG. 7 is a flow chart illustrating a modification process for a reference image;
• FIG. 8 is a flow chart illustrating a modification process for a comparison image;
• FIG. 9 is a schematic diagram illustrating a vehicle mounting the present range-finding system shown inFIG. 1;
• FIG. 10 is a fundamental view illustrating a general range-finding system including a stereo camera system;
• FIGS. 11A and 11B are image deviation examples caused by rotation of the camera; and
• FIG. 12 is a graph illustrating a lateral deviation of the image caused by the rotation of the cameras.
DETAILED DESCRIPTION
• In describing preferred embodiments illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the disclosure of this patent specification is not intended to be limited to the specific terminology so selected, and it is to be understood that each specific element includes all technical equivalents that have the same function, operate in a similar manner, and achieve a similar result.
• Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts throughout the several views thereof, and particularly toFIGS. 1 through 9, a range-finding system according to illustrative embodiments of the present disclosure is described.
• Next, a range-finding system1 is described below with reference toFIG. 1 according to the present disclosure.FIG. 1 is a block diagram illustrating a hardware configuration of the range-finding system1. InFIG. 1, the range-finding system1 includes astereo camera system11 in which aleft camera11aand aright camera11b(two imaging devices) are arranged in parallel, and aparallax calculator20. Theleft camera11aand theright camera11bcapture images from two different points. The captured images acquired by theleft camera11aand theright camera11bare input to theparallax calculator20, and theparallax calculator20 outputs parallax image data. Theparallax calculator20 is a general electronic computer constituted by a central processing unit (CPU), a dynamic random access memory (DRAM), and a non-volatile flash memory. The range-finding function (parallax calculation function) is memorized in the flash memory, which is implemented as a software program to be executed by the CPU and the DRAM. In the present embodiment, although thestereo camera system11 includes twocameras11aand11b;this is just example; alternatively, the stereo camera system can include at least three cameras.
First Embodiment
• FIG. 2 is a block diagram illustrating a software module executed by theparallax calculator20 according to a first embodiment. Theparallax calculator20 includes acamera controller21, a parallaximage generation processor22, a parallaxcorrection value calculator23, and aparallax correction processor24. Thecamera controller21 controls the twocameras11aand11band receives the image data based on the captured images. In addition, thecamera controller21 executes two-cameras synchronizing, camera initializing, and exposure controlling for outputting a stereo image and an image.
• Furthermore, it is desirable that, immediately after the image data is input, thecamera controller21 corrects the image (other than correcting the parallax offset) using processes that are generally considered effective for stereoscopic range-finding, such as correction of optical distortion, correction of image rotation, etc., if needed. The parallaximage generation processor22 generates pre-correction parallax images from the stereo image (reference image and comparison image) input from thecamera controller21, which plays a major role as the range-finding function. The parallaximage generation processor22, serving as the corresponding position difference calculator, receives the stereo image (reference image and the comparison image) for calculating parallax per pixels to generate a pre-correction parallax image. The operation in the parallaximage generation processor22 is described in further detail later.
• The parallax correctionvalue calculation processor23, serving as an estimation unit, has a function of a region parallax correctionvalue calculation processor23ato calculate the correction value of parallax by the regions. Then, using the reference image input from thecamera controller21 and the pre-correction parallax image input from the parallaximage generation processor22, the parallax correctionvalue calculation processor23 calculates the correction value of the parallax for each of X-coordinate. That is, assuming that the parallax correction value when the X-coordinate is a value “i” (X-coordinate=i), the parallax correctionvalue calculation processor23 generates table information [i. off(i)] for each pixel numbers of image width. The calculated parallax correction value is transmitted to theparallax correction processor24 and is used for correcting the parallax image, which is further described later.
• Theparallax correction processor24, serving as a correction unit, subtracts the correction value input from the parallax correctionvalue calculation processor23 from the pre-correction parallax value for each of pixels of the parallax image to correct the parallax image data. That is, assuming that a parallax value at a pixel position (i, j) is set at “d(i, j)” and the parallax correction value when the X-coordinate is the value “i” (X coordinate=i) is set at a value “off(i)”, theparallax correction processor24 outputs the value “O(i, j)=d(i, j)−off (i)” as the parallax value.
• Next, with reference toFIG. 3, the process of the parallaximage generation processor22 is described.FIG. 3 is a flow chart illustrating a process executed by the parallaximage generation processor22 according to the first embodiment.
• At step S1, the stereo image is input. At step S1, the parallaximage generation processor22 receives the stereo images (stereoscopic image) captured by the twocameras11aand11barranged in parallel simultaneously.
• At step S2, the parallaximage generation processor22 abstracts feature points whose shading is enormously changed from the reference images captured by thefirst camera11a.
• Then, at step S3, the parallaximage generation processor22 searches for the position of the corresponding points where the object detected as the feature point in the reference images is identical to the object shown in the comparison images, in a feature vicinity area (block) in the comparison images captured by thesecond camera11b.For the search of the corresponding points, various types of known method, such as Sum of Absolute Difference (SAD) and phase only correlation (POC) can be applied.
• At step S4, the parallaximage generation processor22 calculates the parallax (parallax data). More specifically, the parallaximage generation processor22 acquires the difference of the positions between the corresponding points in the comparison image and the feature point in the reference image to calculate the pre-correction parallax.
• At step S5, the parallaximage generation processor22 outputs the calculated pre-correction parallax images. By repeating above-described process, the parallax images corresponding to the area in front of the cameras can be always kept outputting.
• Next, the region parallax correctionvalue calculation processor23ais described below. The image is divided into three image regions having a first region E1, a second region E2, and a third region E3 as illustrated inFIG. 4. In parallax offset calculation process for each of the image region. That is, the region parallax correctionvalue calculation processor23 calculates the parallax offset based on collections (classes) of the parallax data pair of the feature point and the corresponding point where the same object is captured twice. Herein, the region parallaxcorrection calculation processor22 categorizes the three parts, depending on the region of which the feature points in the reference image is positioned in. More specifically, as illustrated inFIG. 4, assuming that the X-coordinate value of a border between the first region E1 and the second region E2 is X_th1, a X coordinate value of a border between the second region E2 and the third region E3 is X_th2, and the coordinate value of the feature is (X, Y), the region parallaxcorrection calculation processor22 categorizes the feature points as follows.
• When the feature point X is smaller than a threshold value X_th1, the feature point X is categorized as class 1. When the feature point X is greater than a threshold value X_th2, the feature point X is categorized asclass 3. When the feature point X is other case, the feature point X is categorized asclass 2.

• 	FUTURE POINTS	CATEGORIZED
  	OF COORDINATE	CLASS
  	X < Xth1	CLASS 1
  	Xth1 ≦ X≦ Xth2	CLASS	2
  	Xth2 <X	CLASS	3

• Executing statics processing for the respective classes independently, three items of parallax offset are estimated. Herein, if the number of feature points positioned in a certain image region is smaller than a threshold value determined in advance, the parallax offset in the certain region is not estimated, and the estimation result until a previous frame is directly adapted.
• Next, the parallax correction value process for each of the X-coordinate is described below. Herein, central positions of the first region E1, the second region E2, and the third region E3 in the X-coordinate are represented as X₁, X₂, and X₃. The results of the parallax offset calculation processing by the first though thee regions E1, E2, and E3 are regarded as the correction values at the three locations X₁, X₂, and X₃. Then, using linear interpolation, the correction values in entire X-coordinate are calculated. More specifically, the correction value in the range of lower than the value X₂(X<X₂) is acquired by the linear interpolation of the parallax offset between the coordinate points X₁and X₂. The correction value in the range of greater than the value (coordinate point) X₂(X₂, >X) is acquired by the linear interpolation of the parallax offset between of the values X₂and X₃. The table containing the values (X-coordinate, the correction value) for each 1 pixel in the image width is output to theparallax correction processor24.
• FIG. 5 is a graph illustrating calculation examples of the parallax correction values. InFIG. 5, a lateral axis represents the X-coordinate, and a vertical axis represents the parallax correction value. InFIG. 5, white circles represent the parallax offset (=parallax correction value) for each of the three regions, and a solid line A represents the correction value at the entire X coordinate acquired by the linear interpolation.FIG. 5, a broken line B represents an ideal correction value shown inFIG. 12 and an alternate long and short dash line C represents a correction value when a constant correction value (parallax) is used. As is clear fromFIG. 5, the correction value (solid line A) of the present embodiment is more likely to be near the ideal value (broken line B), compared to the correction value when the constant correction value is used (alternate long and short dash line C).
• In the first embodiment, the captured image is divided into three regions shown inFIG. 4 as the image regions, this is just one example. Various types of dividing method can be set depending on the characteristics of the change of thestereo camera system11 over time. For example, the number of divided regions and the divided direction can be changed. In addition, gaps or overlaps among the regions can be provided. In the first embodiment, although the region parallax correctionvalue calculation processor23athat calculates the parallax correction value by the multiple regions is described as one example of the parallax correctionvalue calculation processor23, other configuration can be applied for the parallax correctionvalue calculation processor23.
• In addition, although the correction value table for 1 pixel in X-coordinate is used as the correction value corresponding to the pixel position in the first embodiment, other types of data can express “X coordinate−correction value space”, using a polynomial expression, polygonal line, and a spline curve, etc. Further, in the first embodiment, although the parallax correction value calculation process is performed for 2 frames, the processing time can be changed by deleting the information in the frame whose reliability is low after evaluation of the reliability or accumulating the data in the frames, which can improve accuracy and stability in the calculation process.
• As described above, in the range-finding system of the first embodiment, using twocameras11aand11b,a great number of pairs of parallax data between the two images captured from the two different points are collected. Furthermore, in the region parallax correctionvalue calculation processor23a,the parallax offset is calculated by executing statics processing for the classes of the parallax data pairs. Since the feature points originally cover specific positions in the image, in the first embodiment, the pairs of parallax data are classified into the regions depending on the positions of the feature points, which detects the parallax offset for each image region.
• Referring back toFIG. 4, the entire image is divided into three regions having the first region E1, the second region E2, and the third region E3. Then, the pairs of parallax data are classified into three classes, depending on where the feature points are positioned among the image regions. As for the respective collection, by executing the statics process, the region parallax correctionvalue calculation processor23acan estimate the parallax offset value in the respective image regions.
• Accurately, what is estimated in the way is not direct the deviation amount of the captured image like that shown inFIGS. 11 and 12, but the change amount in the parallax between the two captured images. The parallax is changed depending on the distance to the object, and the change amount in the parallax is equal to the division amount of the position between the two images.
• However, when the change in the deviation amount is moderately changed relative to the pixel position as illustrated inFIG. 12, or when the target parallax is small relative to the rate of change (for example, when the parallax is lower than one-twentieth of the image width), the parallax offset can be treated as the deviation amount of the difference between the two images. Furthermore, as the parallax is smaller, the minimal image deviation is seriously influenced. For example, the deviation of 0.5 pixels corresponds to only 0.5% error for 100 pixels parallax but corresponds to the 50% deviation to 1 pixel parallax.
• In order to solve this problem, in the first embodiment, the parallax offset value depending on the pixel position is treated as the image deviation of the difference between the two images. Then, using the deviation of the difference, theparallax correction processor24, serving as the correction unit, corrects the pre-correction parallax data to output the parallax image. Accordingly, not uniform the image deviation in a lateral direction can be corrected without using the known objects, such as test chart in use, which can generate parallax image with higher degree of accuracy.
• In the first embodiment, the parallax generation processor (corresponding point difference calculator)22 searches for the corresponding point among the multiple images and calculates the pre-correction parallax data as the difference in the coordinate between the feature points and the corresponding points. Then, theparallax correction processor24, serving as the correction unit, corrects the parallax data based on the correction value from the parallax correction processor (estimation unit)23, which can achieve the correction of the pixel deviation in the lateral direction depending on the pixel position, with a simple calculation.
• Herein, as a comparative example in which the parallax is calculated based on the dividing line and the stop line on the road, the system can measure only an average parallax offset value in the entire image region where the dividing line is appeared. Then, the position at which the image captured by the vehicle front facing camera showing the white line on the road does not change in the lower area in the image, the offset measure by the region cannot be performed. By contrast, in the present embodiment, by using the parallax offset measuring method using the statics process on basis of the feature points, the offset measurement by the regions can be achieved.
Second Embodiment
• Next, a parallax calculation method according to a second embodiment is described below, with reference toFIGS. 6 through 8. Although the parallax data is corrected in the first embodiment, the comparison image used for generating the parallax image is corrected in the second embodiment. In the second embodiment, if image correction processes to correct optical distortion and assembly error are present, these correction processes are incorporated in the image correction process for the comparison image, thereby eliminating the need for a subsequent process to correct the parallax image. It is to be noted that, the entire configuration and the hardware configuration in the second embodiment is similar to the first embodiment thereof, therefore, the description thereof is omitted. Next, what is different from the first embodiment in the contents of the software process is described below.
• FIG. 6 is a block diagram illustrating the entire configuration of the software module executed by a parallax calculator20-A. InFIG. 6, components of the parallax calculator20-A identical to those of theparallax calculator20 are represented by identical numerals and the description thereof is omitted below. In the second embodiment shown inFIG. 6, the parallax calculator20-A includes thecamera controller21, a referenceimage correction processor31, an image correctionvalue calculation processor32, a comparisonimage correction processor33, and the parallaximage generation processor22A. Similarly to the first embodiment, thecamera controller21 controls the two cameras, and outputs the reference image and the comparison image.
• The referenceimage correction processor31, serving as a first image correction unit, compensates deformation of the reference image (modifies the reference image) depending on measured deviation in and distortion characteristics in manufacture of thestereo camera system11.FIG. 7 is a block diagram illustrating interior configuration of the referenceimage correction processor31; in other words,FIG. 7 is a flow chart illustrating a modification processing in the reference image executed by the referenceimage correction processor31. As a general image modification processing, a pixel position generation table (coordinate generation processor)41 that calculates the pixel position in the correlating the input image, for the entire pixel positions of the output, and animage interpolate unit42 that calculates the pixel values on the input image correlating to the calculated pixel position, are provided. Herein, the pixel value is not limited to integer.
• In the second embodiment, an X-Y coordinate for the entire pixels are memorized as a table in advance, then, referring the memorized table, the pixel on the input image is generated. The table41 is generated by capturing a test chart installed in known positions during manufacturing and interpolating the pixels between the feature points, and the table41 is memorized in thestereo camera system11. Although the table is used as the coordinate generation processor in the second embodiment, the position pixel can be generated other than using table, such as coordinate conversion by polynomial calculation.
• Similarly to the parallax correctionvalue calculation processor23, the image correctionvalue calculation processor32, serving as the estimation unit, divides the image into the multiple image regions (seeFIG. 4) and acquires the parallax offset by the respective image regions, and interpolates the parallax offset for output as the correction value (image correction value). Herein, in the second embodiment, the correction value output from the image correctionvalue calculation processor32 is used for correcting the comparison images.
• Similarly to the referenceimage correction processor31, the comparisonimage correction processor33, serving as a second image correction unit, compensates the deformation of the comparison image (modifies the comparison image). However, although the reference image is corrected based on the deviation characteristics measured in manufacturing thestereo camera system11, the comparison image is corrected based on the deviation calculated by the imagecorrection value processor32 in use, in addition to the characteristics measured and memorized in advance.
• FIG. 8 is a block diagram illustrating an interior configuration of the comparisonimage correction processor33; in other words,FIG. 8 is a flow chart illustrating a modification processing in the comparison image executed by the comparisonimage correction processor33. The comparisonimage correction processor33 includes the pixel position generation table41′, thepixel interpolator42′, and a pixelposition correction processor43. It is to be noted that, inFIG. 8, components of the comparisonimage correction processor33 identical to those of the referenceimage correction processor31 are represented by identical numerals and the description thereof is omitted below. The pixel position generation table41′ memorizes the characteristic measured in manufacturing thestereo camera system11, similar above.
• The pixelposition correction processor43 uses the image correction value for each of X-coordinate input from the image correctionvalue calculation processor32 to correct the pixel position data memorized in the table41′. More specifically, the pixelposition correction processor43 adds the correction value “off(i)” corresponding to the X-coordinate in the X-Y coordinate space (X, Y) read from the table41-A. Therefore, when the parallax is too large; that is, the parallax is positioned on a right side, compared to the reference image, the comparison image is modified so that the comparison image is moved to left side, and the parallax calculated in the subsequent process can be set smaller. Herein, when the X-coordinate is not integer, for example, the X-coordinate may round off to get the integer numeral.
• The parallaximage generation processor22A, serving as the corresponding position difference calculator, receives the corrected reference image and the corrected comparison image and generates the parallax image, similarly to process in the first embodiment. The input comparison image is already slid by the parallax offset calculated in the image correction value calculation process, and the noise for the parallax offset is minimized.
• Herein, although the results of the imagecorrection value processor32 is reflected only for the correction of the comparison image in the second embodiment, the results thereof can be reflected to the correction of the reference image side, or the results (correction value) can be distributed into the reference image and the comparison image. That is, not only the comparisonimage correction processor33, the referenceimage correction processor31, or both the comparisonimage correction processor33 and the referenceimage correction processor31 can function as the correction unit.
• As described above, in the second embodiment, the parallax offset value depending on the pixel position is treated as the deviation between the corrected comparison image and the corrected reference image, and the comparison image can be corrected using the parallax offset value. Accordingly, not-uniform image deviation in the lateral direction can be corrected, without using the known object, such as test chart, in use.
• Further, in the second embodiment, the comparison image correction processor (second image correction unit)33 functions as the correction unit to correct the comparison image based on the correction value from the image correctionvalue calculation processor32. Then, the parallaximage generation processor22A searches for the corresponding points between the corrected comparison image and the corrected reference image and calculates the parallax data as the difference in the coordinate between the corresponding points of the comparison image and the feature point of the reference image. Therefore, the pixel deviation in the lateral direction depending on the pixel position can be corrected with a simple calculation.
<Vehicle>
• FIG. 9 is a schematic diagram illustrating avehicle50 mounting the present range-finding system1. An image processing system in thevehicle50 includes animage pickup unit51 to acquire the image forward of thevehicle50 and animage analysis unit52 that calculates the distance from another vehicle driving in front of thevehicle50 based on the image acquired in theimage pickup unit51. Theimage pickup unit51 is positioned near a rearview mirror in thevehicle50 so that theimage pickup unit51 can capture the forward image containing another vehicle driving in front of thevehicle50 and calculate a distance to another vehicle from thevehicle50. The image of the vehicle in front acquired by theimage pickup unit51 is input to and converted into the image signal for output to theimage analysis unit52. Theimage analysis unit52 analyses the output image signal from theimage pickup unit51. As for theimage pickup unit51, the stereocamera system device11 of the present embodiment composed of thereference camera11aand thecomparison camera11bcan be used. As a part of function of theimage analysis unit52, theparallax calculator20 can be applied.
• A vehicledrive control unit53 can control steering and brake of thevehicle50 based on the distance from the vehicle in the front calculated by theimage analysis unit52.
• Numerous additional modifications and variations are possible in light of the above teachings. It is therefore to be understood that, within the scope of the appended claims, the disclosure of this patent specification may be practiced otherwise than as specifically described herein.

Claims (16)

1.-5. (canceled)

6. A method comprising:

calculating a disparity for each of multiple stereo images captured at different timing; and

calculating a correction amount corresponding to image coordinates based on the disparity calculated for each of the multiple stereo images.

7. The method ofclaim 6 further comprising:

calculating a disparity offset based on each disparity calculated for each of the multiple stereo images captured at different timing; and

calculating the correction amount corresponding to the image coordinates based on a profile of the disparity offset corresponding to image coordinates in the captured image.

8. The method ofclaim 6 further comprising correcting at least one of the disparity or the image based on the correction amount.

9. The method ofclaim 6, wherein the correction amount corresponding to the image coordinates is calculated for each of regions prepared by dividing the image in X direction.

10. The method ofclaim 9, wherein each of the regions have a gap between adjacent regions.

11. The method ofclaim 9, wherein the adjacent regions are overlapped.

12. The method ofclaim 6, wherein the correction amount corresponding to the image coordinates is calculated by performing an interpolation to each of the correction amount calculated for each of regions prepared by dividing the image in X direction.

13. The method ofclaim 12, wherein each of the regions have a gap between adjacent regions.

14. The method ofclaim 12, wherein the adjacent regions are overlapped.

15. The method ofclaim 6, wherein the correction amount corresponding to the image coordinates is calculated for each of regions prepared by dividing the image into two directions defining two dimensional space.

16. The method ofclaim 15, wherein each of the regions have a gap between adjacent regions.

17. The method ofclaim 15, wherein the adjacent regions are overlapped.

18. An apparatus comprising:

a correction unit to calculate a correction amount corresponding to image coordinates based on a disparity calculated for each of multiple stereo images captured at different timing.

19. The apparatus ofclaim 18 further comprising:

a calculation unit to calculate a disparity offset based on each disparity calculated for each of the multiple stereo images captured at different timing; and

a calculation unit to the correction amount corresponding to the image coordinates based on a profile of the disparity offset corresponding to image coordinates in the captured image.

20. The method ofclaim 18 further comprising

a calculation unit to correct the disparity based on the correction amount, or correct the image based on the correction amount.

Images