US20220208000A1 - Handheld laser-based vehicle speed measurement device incorporating an automatic number plate recognition (anpr) function - Google Patents

Abstract

Disclosed herein is a handheld speed gun incorporating automatic number plate recognition (ANPR) functionality. As the image size of a vehicle license plate varies depending on its distance from the speed gun, in the present invention the distance to the target vehicle is known through the functionality of the laser sensor so the size of the pixels for the number plate can be estimated. Since the image of the number plate may be tilted in a handheld device, the present invention compensates for this by use of the instrument's tilt sensor or through other compensation by use of the image itself.

Description

CROSS REFERENCE TO RELATED PATENT APPLICATIONS
• The present application is a divisional of U.S. patent application Ser. No. 16/688,633, filed on Nov. 19, 2019, which is a non-provisional of and claims priority to U.S. Provisional Patent Application Ser. No. 62/769,803 filed Nov. 20, 2018 for “Laser-Based Vehicle Speed Measurement Device Incorporating an Automatic Number Plate Recognition (ANPR) Function.” The present application is also related to U.S. patent application Ser. No. 15/473,307, filed Mar. 29, 2017, for “Camera Module and Folded Optical System for Laser-Based Speed Gun,” now U.S. Pat. No. 10,146,103, issued Dec. 4, 2018, which claims priority to U.S. Provisional Application Ser. No. 62/316,319, filed Mar. 31, 2016, for “Camera Module and Folded Optical System for Laser-Based Speed Gun.” The full disclosures of each of the foregoing patent applications are hereby incorporated by this reference in their entirety for all purposes.
COPYRIGHT NOTICE/PERMISSION
• A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document of the patent disclosure as it appears in the United States Patent and Trademark Office patent file or records, but otherwise, reserves all copyright rights whatsoever. The following notice applies to the software and data and described below, inclusive of the drawing figures where applicable: Copyright © 2019, Laser Technology, Inc.
BACKGROUND OF THE INVENTION
• The present invention relates, in general, to the field of handheld laser-based vehicle speed measurement devices and speed guns. More particularly, the present invention relates to a handheld laser-based vehicle speed measurement device incorporating an automatic number plate recognition (ANPR) function.
• Laser Technology, Inc. assignee of the present invention, has previously introduced the TruCAM® (a registered trademark of Laser Technology, Inc.) video laser-based vehicle speed measurement device which incorporates the industry's first video camera in a handheld form factor. In operation, it collects and stores a complete chain of video evidence for both speeding and tailgating violations along with a high-resolution image that identifies the vehicle make, model and license plate number. Representative of the technology embodied in the TruCAM devices is that disclosed in, for example, in U.S. Pat. No. 7,920,251 issuing Apr. 5, 2011 for: “Integrated Still Image, Motion Video and Speed Measurement System”; U.S. Pat. No. 8,311,343 issuing Nov. 13, 2013 for: “Vehicle Classification by Image Processing with Laser Range Finder” and U.S. Pat. No. 10,146,103 issuing Dec. 4, 2018 for: “Camera Module and Folded Optical System for Laser-Based Speed Gun”, the disclosures of which are herein specifically incorporated by this reference in their entirety as if fully set forth herein.
• Laser-based speed measurement devices, operate to calculate distance by measuring the time of flight of very short pulses of infrared light. That is, a measurement is made as to the time it takes one or more laser pulses to travel to a target vehicle and back with a precision time base. With knowledge of the constant speed of light, the distance the laser pulses have traveled can then be calculated. If the speed gun takes, for example, a thousand samples per second, its processor can compare the change in distance between successive samples and thereby calculate the speed of the target vehicle. By taking several hundred samples over the course of a fraction of a second or so, the accuracy can be extremely high.
• Conventional, fixed position, automatic number plate recognition (ANPR) is a technology that uses optical character recognition on images to read vehicle registration plates to create vehicle identification data. At present, it can be implemented using closed-circuit television, fixed traffic enforcement cameras, or other cameras specifically designed for the task. ANPR is used by police forces around the world for law enforcement purposes, including to check if a vehicle is currently being sought for whatever reason, registered and/or licensed. It is also used for electronic toll collection on pay-per-use roads and as a method of cataloguing the movements of traffic, for example by highways agencies.
• Automatic number plate recognition can be used to store the images captured by the cameras as well as the text from the license plate, with some configurable to store a photograph of the driver. Systems may also employ infrared lighting to allow the camera to take the picture at any time of day or night.
• Heretofore, ANPR functionality has not been able to be implemented in a handheld speed measurement device, whether implemented in conjunction with radar or laser-based speed measurement devices. Particularly, all existing ANPR equipment utilize and assume predetermined geometries between the instrument and the vehicle license plate (inclusive of optical parameters and distances to plate) in order to properly function. Moreover, such existing products are not capable of accurately functioning when tilting of the device must be compensated for as in the operation of a handheld device such as a laser-based speed measurement device as disclosed herein.
SUMMARY OF THE INVENTION
• Disclosed herein is a handheld speed gun incorporating ANPR functionality. As the image size of a vehicle license plate (or other uniquely identifiable alphanumeric indicia on the vehicle) varies depending on its distance from the speed gun, in the present invention the distance is known through the functionality of the laser sensor so the size of the pixels for the number plate can be estimated.
• In the use of a handheld speed gun, an image of the vehicle is generally not taken as orthogonal to the camera frame. In other words, the image may be somewhat rotated (or tilted) depending on whether the operator is left or right handed. The principles of the present invention allow for this to be compensated by use of the instrument's tilt sensor or through compensation by use of the image itself. As either technique has its advantages and disadvantages, in a representative embodiment of the present invention, both techniques may be employed.
• Typical low-cost cameras, such as those incorporated in “Body CAMs” and smart phones, utilize what are known as rolling shutters. Such cameras take an image line-by-line instead of the whole scene at once. On the other hand, with a conventional “wet film” a “global shutter” is employed wherein the camera shutter is mechanically opened. In contrast, modern cameras utilize an electronic shutter so a line-by-line technique is employed.
• In any event, when a vehicle (or the speed gun itself) is moving the image of the vehicle license plate is at least somewhat distorted. Consequently, the technique of the present invention incorporates rolling shutter compensation and pre-processing of the license plate image.
• As with any handheld device, power considerations are also important as most will be battery powered. Consequently, the processing algorithms are advantageously “lighter” and less computationally intensive. The principles of the present invention are readily implemented in a self-contained device such as an “off-line Body Cam” and may be coupled to a database to provide appropriate notice to a user of the speed gun when a “wanted” vehicle license plate is found through connection to a cellular network, WiFi connection or the like.
• Particularly disclosed herein is a handheld laser-based speed gun comprising a processor and laser signal transmitting and receiving sections coupled to the processor for determining a speed of a target vehicle based on changes in distance between the speed gun and the target vehicle over time. The speed gun further comprises a camera module coupled to the processor for capturing images of the target vehicle number plate and an automatic number plate recognition (ANPR) module is also coupled to the processor in operative association with said camera module.
• Also particularly disclosed herein is a method for identifying a vehicle number plate of interest with a handheld distance ranging device comprising an image sensor. The method comprises capturing an image of the vehicle number plate; compensating the captured image to account for a determined distance of the ranging device to the vehicle number plate to produce a compensated image; and determining if the compensated image resides in a database of the vehicle number plates of interest.
BRIEF DESCRIPTION OF DRAWINGS
• The aforementioned and other features and objects of the present invention and the manner of attaining them will become more apparent and the invention itself will be best understood by reference to the following description of a preferred embodiment taken in conjunction with the accompanying drawings, wherein:
• FIG. 1 is an isometric view of a representative laser-based speed gun incorporating a camera module and folded optical system incorporating an ANPR function;
• FIGS. 2A and 2B are representative functional block diagrams of a laser-based speed gun of the preceding figure in accordance with the principles of the present invention;
• FIG. 3 is a representative flow chart of one possible implementation of a laser-based speed gun incorporating an ANPR function in accordance with the principles of the present invention; and
• FIGS. 4A through 4I are illustrative of a representative ANPR function implemented in conjunction with a speed gun in accordance with the principles of the present invention.
DETAILED DESCRIPTION
• With reference now toFIG. 1, an isometric view of a representative laser-basedspeed gun100 is shown incorporating a camera module and foldedoptical system120 incorporating an ANPR function in accordance with the principles of the present invention. The laser-basedspeed gun100 comprises ahousing102 and associatedhandle104 for handheld operation. Atrigger106 is provided to initiate the transmission and reception of laser pulses toward a moving object, such as a vehicle, as well as initiate the recording of video of the object in conjunction with the camera module and foldedoptical system120. The laser-basedspeed gun100, as illustrated, includes adisplay108, user input andselection elements110 as well as targetvehicle sighting optics112. A representative laser-based speed gun may be implemented in accordance with the disclosure of commonly owned U.S. Pat. No. 10,146,103 issuing Dec. 4, 2018 for “Camera Module and Folded Optical System for Laser-Based Speed Gun”, the disclosure of which is specifically incorporated by this reference in its entirety as if fully set forth herein.
• With reference now toFIG. 2A, a representative functional block diagram of the laser-basedspeed gun200 of the preceding figure in accordance with the principles of the present invention is shown.
• Theexemplary speed gun100 comprises amicroprocessor202 or central processing unit (CPU) with an associated oscillator204 (where required) for providing clocking signals to themicroprocessor202. A battery andpower management section206 supplies operating power to themicroprocessor202 and various other speed gun subsystems (not shown) as well as the high voltage (HV)power supply208 which provides operating voltage to a laser transmitsection210 and associated laser diode as well as a laser receivesection212 and associated photodiode.
• The laser receivesection212 receives a portion of the laser energy transmitted by the laser transmitsection210 as reflected by a target vehicle to a photodiode and provides the return signals to a signal/noise (S/N)discriminator section214 in order to separate true return pulses from any associated noise. Atiming section216 accurately measures the time between the transmission of laser pulses from the laser transmitsection210 and the reception of the same target vehicle reflected pulses at the laser receivesection212 to determine, in conjunction with themicroprocessor202, the varying distance, and hence the speed, of the particular target vehicle towards which thespeed gun200 is aimed.
• Afire button222 is coupled to the battery andpower management section206 and is operable by a user of thespeed gun200 in conjunction with themicroprocessor202 to determine when to emit pulses toward a target vehicle from the laser transmitsection210.
• Thespeed gun200 may also incorporate a user viewable in-sight display218 implemented in conjunction with a novel and proprietary backlighting technique which may include a view of the target vehicle in conjunction with an aiming reticle as well as information regarding the range to, and/or speed of, the target vehicle, battery condition and other information. In certain embodiments, thespeed gun200 may also comprise a touchscreen display to allow user to provide inputs to thespeed gun200 in conjunction with, or as an alternative to, an input/output (I/O)section220.
• The I/O section220 may further comprise a keypad or other means of communicating information to or from themicroprocessor202 including wired connections such as a universal serial bus (USB) and the like as well as wireless connections such as an IEEE 802.11 (WiFi), or other wireless local area network (WLAN) transceiver; a Bluetooth transceiver or other personal area network (PAN) system for wirelessly exchanging data over short distances; and/or another near field communication (NFC) transceiver (inclusive of infrared (IR) coupling) for wirelessly coupling thespeed gun200 to external devices or data storage elements.
• As illustrated, thespeed gun200 may further include one or more of additional input modules such as aninclinometer224, accelerometer,226, magnetic sensor228 (e.g. a compass) and/orrate gyro230.
• As an exemplary utilization of a backlighting technique for LCDs and other display devices in electronic speed guns or the present invention, thespeed gun200 is illustrated as incorporating abacklight232. In a representative embodiment of thespeed gun200 of the present invention, thebacklight232 may be advantageously provided in accordance with the specification and teachings of commonly owned U.S. Pat. No. 9,964,805 issued on May 8, 2018 for: “Backlighting Technique for Liquid Crystal and Other Displays in Electronic Speed guns”, the disclosure of which is specifically incorporated by this reference in its entirety as if fully set forth herein.
• As further illustrated, thespeed gun200 may comprise areticle234 interposed between the backlight and the in-sight display218 as is more fully described in the aforementioned '805 patent. A global positioning satellite (GPS)module236 may also form a portion of thespeed gun200 to provide information to themicroprocessor202 as to the specific geographic position of thespeed gun200. In addition, and as previously noted, thespeed gun200 may further include anNFC module238 capable of enabling external bidirectional communication with thespeed gun200 via Bluetooth, WiFi and the like in conjunction with a smartphone, tablet device, computer laptop etc.
• In an alternative embodiment of the present invention, thespeed gun200 may further be configured to provide an augmented reality display to a user by the additional provision of an advanced in-scope display orcamera module240 andview screen242. In this manner, by angularly scanning thespeed gun200 about a target vehicle, other features and objects in the surrounding scene can be displayed in the view screen242 (or in-sight display218 and/or the screen of an associated smartphone, tablet device or laptop) to a user of thespeed gun200 along with the determined distances to such additional features and objects to provide additional terrain context over and above the speed of, or distance to, the desired target vehicle. Such features and objects might be, depending on the particular application of thespeed gun100 trees, highway overpasses, signs, buildings and the like. The in-scope display or camera module140 is then operational to log the surrounding features and objects, and their distances determined by the laser-basedspeed gun200 and this information displayed in a picture to a user of thespeed gun200, whether on the speed gun itself or on the screen of any associated device.
• With reference additionally now toFIG. 2B, an additional representative portion of the functional block diagram of the laser-basedspeed gun200 of the preceding figure is shown illustrative of an embodiment of the present invention which may further include a vibro-motor250 and one or more audio and/orvisual indicators252 to provide physical, haptic and audible and/or visible feedback to the user of a particular target vehicle number plate being of interest. A laser-basedspeed gun200 in accordance with the present invention will include an ANPR function block254 as illustrated and may further include acellular telephony block256 and/orWiFi block258, and/or NFC or other communications medium, to bidirectionally communicate data regarding a vehicle's license plate number as well as speed and other information to/from a location and database remote from the laser-basedspeed gun200. The ANPR function block254 is operative in conjunction with themicroprocessor202 and thecamera module240 as will be more fully disclosed hereinafter.
• A system comprising the laser-basedspeed gun200 may further include adatabase260 either resident in the laser-basedspeed gun200 itself or remotely therefrom in communication with said laser-basedspeed gun200. Thedatabase260 may, for example, comprise number plates of particular interest to authorities which can then be matched to the number plate of a target vehicle as determined by the ANPR functionality. This information can be added to thedatabase260 or communicated to the operator of the laser-basedspeed gun200 by haptic or aural and/or visual means by virtue of vibro-motor250 and the auralvisual indicator252.
• With reference additionally now toFIG. 3, a representative flow chart of one possible implementation of a laser-basedspeed gun200 incorporating anANPR function300 in accordance with the principles of the present invention is shown. The representative ANPR process includes determining the distance to the target vehicle license plate as determined by the laser rangefinder and as computed by theprocessor202 atstep302. Thespeed gun200 then computationally resizes the image of the license plate based on the computed distance atstep304.
• The height and location of the license plate is then obtained atstep306 and the segmented data is then copied into the work area of theprocessor202 along with tilt compensation information derived from, for example, the inclinometer224 (FIG. 2A) as shown instep308. Atstep310, the data is once again segmented and a new segment determined on the raw data atstep312. A further segment operation is performed atstep314 and the data is split, if required, atstep316 along with a merge operation atstep318 if also required.
• At this point a neural network is applied to each segmented zone in order to obtain the particular alpha or numerical character being considered atstep320. If another character of the license plate needs to be determined atdecision step322, then the ANPR process returns to step302 to continue theANPR function300.
• With reference now toFIGS. 4A through 4I, various views of an example license plate are shown illustrative of a representative ANPR function implemented in conjunction with a speed gun in accordance with the principles of the present invention.
• With reference specifically toFIG. 4A, a representative image of a license plate is shown. In general, the technique of the present invention performs well if the width of each character in the image is approximately 10 pixels or greater. InFIG. 4B, an average 3×3 filter is applied to the image. In the TruCam2® instrument available from Laser Technology, Inc., assignee of the present invention, this filter operation is performed by a Neon processor employing single instruction multiple data (SIMD), which employs and architecture for the ARM Cortex-A series and Cortex-R52 processors.
• With respect toFIG. 4C, edge detection of the image is performed in the X direction only and inFIG. 4D, adjacent segments or the characters are grouped.FIG. 4E shows the license plate character candidates being grouped and tilt compensation being applied.
• The representative license plate image is shown inFIG. 4F with anti-aliasing having been applied and, as shown inFIG. 4G, following a rolling shutter compensation operation. Anti-aliasing is then again applied as shown inFIG. 4H and the characters of the license plate segmented (after rolling shutter compensation) inFIG. 4I.
• Although not specifically illustrated, the ANPR function of the preceding figures may also advantageously incorporate steps such as the trimming of the license plate characters, the application of a neural network, the retrieval of data and the application of the process with respect to license plates having two rows of alpha and numeric characters. The technique of the present invention is also applicable to those number plates having, for example, Cyrillic, Chinese, Korean, Japanese characters or Arabic alphabet representations.
• While there have been described above the principles of the present invention in conjunction with specific apparatus, it is to be clearly understood that the foregoing description is made only by way of example and not as a limitation to the scope of the invention. Particularly, it is recognized that the teachings of the foregoing disclosure will suggest other modifications to those persons skilled in the relevant art. Such modifications may involve other features which are already known per se and which may be used instead of or in addition to features already described herein. Although claims have been formulated in this application to particular combinations of features, it should be understood that the scope of the disclosure herein also includes any novel feature or any novel combination of features disclosed either explicitly or implicitly or any generalization or modification thereof which would be apparent to persons skilled in the relevant art, whether or not such relates to the same invention as presently claimed in any claim and whether or not it mitigates any or all of the same technical problems as confronted by the present invention. The applicants hereby reserve the right to formulate new claims to such features and/or combinations of such features during the prosecution of the present application or of any further application derived therefrom.
• As used herein, the terms “comprises”, “comprising”, or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a recitation of certain elements does not necessarily include only those elements but may include other elements not expressly recited or inherent to such process, method, article or apparatus. None of the description in the present application should be read as implying that any particular element, step, or function is an essential element which must be included in the claim scope and THE SCOPE OF THE PATENTED SUBJECT MATTER IS DEFINED ONLY BY THE CLAIMS AS ALLOWED. Moreover, none of the appended claims are intended to invoke paragraph six of 35 U.S.C. Sect. 112 unless the exact phrase “means for” is employed and is followed by a participle.

Claims (16)

What is claimed is:

1. A handheld laser-based speed gun comprising:

a processor;

laser signal transmitting and receiving sections coupled to said processor configured to determine a speed of a target vehicle based on changes in distance between said speed gun and said target vehicle over time;

a camera module coupled to said processor configured to capture images of a number plate on said target vehicle, wherein said camera module comprises an autofocus mechanism; and

an automatic number plate recognition (ANPR) module coupled to said processor in operative association with said camera module.

2. The speed gun ofclaim 1 further comprising a folded optical system for redirecting an image pathway from said target vehicle to said camera module.

3. The speed gun ofclaim 1 wherein said distance between said speed gun and said target vehicle is utilized by said ANPR module and said processor to estimate the size of pixels in said captured images of alphanumeric characters of said target vehicle number plate.

4. The speed gun ofclaim 1 wherein said camera module is a rolling shutter device.

5. The speed gun ofclaim 4 wherein said camera module is operative to scan an image of said target vehicle number plate on a line-by-line basis.

6. The speed gun ofclaim 4 wherein said ANPR module and said processor are operative to perform rolling shutter compensation on captured images from said camera module.

7. The speed gun ofclaim 1 further comprising a database of vehicle number plates of interest to authorities for comparison to said captured images of said target vehicle number plate.

8. The speed gun ofclaim 7 wherein said database of vehicle number plates resides in a memory portion of said speed gun.

9. The speed gun ofclaim 7 wherein said database of vehicle number plates resides remotely from said speed gun and is bidirectionally coupled to said speed gun by a cellular network.

10. The speed gun ofclaim 8 wherein said database of vehicle number plates resides remotely from said speed gun and is bidirectionally coupled to said speed gun by at least one of a WiFi or NFC network.

11. The speed gun ofclaim 8 wherein a user of said speed gun is alerted to a match of said target vehicle number plate with a selected one of said database entries of interest to said authorities.

12. The speed gun ofclaim 11 wherein said user is alerted by at least one of haptic, aural or visual indicators.

13. A method for identifying a vehicle number plate of interest with a handheld distance ranging device comprising an image sensor, the method comprising:

capturing an image of said vehicle number plate;

compensating said captured image to account for a determined distance of said ranging device to said vehicle number plate to produce a compensated image; and

determining if said compensated image resides in a database of said vehicle number plates of interest.

14. The method ofclaim 13 wherein said database resides on said ranging device or at a remote location operationally coupled to said ranging device.

15. The method of claim17 further comprising:

indicating to a user of said ranging device that a match of said vehicle number plate to one of said database entries has been determined.

16. The method ofclaim 15 wherein said step of indicating is carried out by at least one of haptic, aural or visual indicators of said ranging device.

Images