Localization system for moving objects
FILED OF THE INVENTION
The invention relates to a localization system, a method for providing spatial information about an object in a surveillance area and a use of such localization system.
BACKGROUND OF THE INVENTION
From the US 6 133 946 a system is known that determines the height of an object in a sport arena, for example the height that a basketball player jumped, based on the evaluation of two or more video images. The height information is then indicated in a television image, a radio broadcasting or the like.
Moreover, the US 2006/0072012 Al discloses a security system with a plurality of spaced apart posts for detecting the intrusion of e.g. a person via an interruption of the line-of-sight between infrared emitters and corresponding detectors that are arranged on different posts. To avoid crosstalk between different light sources, the light of the infrared emitters is individually coded.
SUMMARY OF THE INVENTION Based on this situation it is an object of the present invention to provide means for gaining spatial information about at least one object in a surveillance area, wherein it is desirable that these means are comparatively simple to implement, precise, and flexible in their usage.
This object is achieved by a localization system for providing spatial information about at least one object in a surveillance area comprising: a) an array of light sources distributed in the surveillance area; b) an array of light detectors distributed in the surveillance area for detecting light that was emitted by a light source and reflected by the object and for providing an associated detection signal; c) a control unit for controlling the emission of light by the light sources; d) an evaluation unit for determining the spatial information about the object from the detection signals of the light detectors.
Here an object denotes any object of interest also comprising persons and animals. Preferred embodiments are disclosed in the dependent claims.
The localization system according to the present invention serves for providing spatial information about at least one object in a surveillance area, wherein said area may particularly be an (indoor or outdoor) sport arena, for example a field for soccer, football, handball, basketball, volleyball, field hockey, tennis, badminton or the like. Other surveillance areas may be the inside / outside of elderly homes, emergency rooms, security areas, areas of protection or traffic vehicles such as cars, airplanes etc., where the presence and status of persons (awake, asleep, alive etc.) may be determined by movement detection. The spatial information about the object may especially comprise the (one-, two- or three-dimensional) position, the orientation and/or the velocity of said object. In the field of sport, the object may particularly be a player or playing object like a ball. The localization system comprises the following components: a) An array of light sources that are distributed in the surveillance area. The term "array" shall denote in the context of the present application a single unit or any arbitrary one-, two- or three-dimensional arrangement of a plurality of units. Typically the units of such an array will be arranged in a regular pattern, for example a grid or matrix pattern. In a sport arena, the light sources may particularly be distributed in the floor, the wall(s) and/or the ceiling of the playing field. The light sources may preferably be realized by light emitting diodes (LEDs) which are easy to control, allow fast switching times, are mechanically robust, and have little power consumption, b) An array of light detectors that are distributed in the surveillance area for detecting light that was emitted by a light source and reflected by the object to be localized and for providing a detection signal that is associated to this detected light. The light detectors may particularly comprise photodiodes, and/or photocells. The detection signal will usually be an electrical signal like a voltage or a current that is indicative for the total amount of detected light. It should be noted that at least either the array of light sources or the array of light detectors will usually comprise more than one unit. c) A control unit for controlling the emission of light by the light sources. The control unit may comprise separate sub-units located at each light source and/or a central main-module that controls all light sources in parallel. It will usually comprise some logical module (e.g. realized by digital data processing hardware and software) for abstractly determining the emission pattern, and some driving module for actually generating or controlling the power supply to the light sources. d) An evaluation unit for determining the desired spatial or movement information about the object of interest from the aforementioned detection signals of the light detectors. The evaluation unit may optionally be realized by digital data processing hardware with associated software and/or by dedicated analogue electronic circuits. Several particular realizations of the evaluation unit will be discussed in connection with preferred embodiments of the invention. The proposed localization system has the advantage that it is comparatively simple to realize as it mainly requires the use of simple light sources and light detectors, wherein the accuracy and spatial resolution of the system can usually be tuned via the spatial density (number) of these elements. Moreover, the system has the advantage to make use of light that was reflected from an object, which is typically always and continuously available (in contrast to the on-off character of measurements that are based on an interruption of lines-of-sight).
The localization system may particularly comprise infrared light sources which emit in the invisible infrared light spectrum. This has the advantage that the measurements do not interfere with the observation of the surveillance area by human spectators. Moreover, the use of infrared light makes the system robust with respect to disturbances by visible light of the ambience.
In a preferred embodiment of the invention, the control unit is adapted to provide the light emission of each light source with an individual code pattern. Thus each light source may for example be switched on and off according to an associated individual binary pattern. Marking the light emissions with a code pattern allows to distinguish on the sensor side from which light source a measured light contribution comes. In another embodiment, the control unit may be adapted to selectively set the average intensity of the light emission for each light source. This property particularly allows to evaluate the measured intensity of light reflected from an object with respect to the desired spatial information, because the intensity of the originally emitted light is known. The adaptation of the average intensity is further of interest in the aforementioned embodiment for keeping the intensity of a light source at a desired level irrespective of the imprinted code pattern.
It was already mentioned that there are several possibilities for the evaluation unit to determine the desired spatial information based on the detection signals. In a first particular embodiment, the evaluation unit may be adapted to determine the individual contributions of different light sources to the detection signal of a light detector. This approach is based on the observation that the light received and determined by a particular light detector - and thus also the associated detection signal - will usually comprise a superposition of contributions from all (active) light sources. For some localization approaches it is however necessary to know the amount of reflected light that corresponds to a particular light source. This information can be obtained if the evaluation unit is adapted in the aforementioned way. The determination of individual contributions of the light sources may for example be based on different colors of the light sources, wherein the light detector should be able to provide spectrally resolved measurements in this case. Preferably, the distinction between different contributions is however based on individual code patterns of the kind mentioned above that are imprinted onto the emissions of the light sources. Thus the light sources may for example be modulated with different frequencies such that the evaluation unit can discriminate their contributions based on a Fourier analysis of a recorded detection signal. Knowing the spatial arrangement of the light sources and the light detector(s) as well as the individual contributions of the light sources to an observed detection signal allows in principle to determine the position of the object of interest (e.g. if all light sources emit with a known intensity and if the reflectivity of the object is known or at least the same for the light of all light sources, then the position of an object may be estimated with a triangulation like procedure.)
In another preferred embodiment of the invention, the evaluation unit is adapted to identify the light source with the largest normalized contribution to the detection signal. This approach is related to the aforementioned one, but requires only the identification of one particular contribution and not the (quantitative) determination of various contributions (if the latter are known, it is however readily possible to identify the largest contribution). The "normalization" refers in this context to the intensity of the initial light emission of the individual light sources, i.e. the absolute value of a contribution of a light source to the detection signal is normalized with the original emission intensity of said light source as the weakness/strength of the emissions is neutralized by the normalization. A nearby but weak light source will therefore not be surpassed by a remote but strong light source. Knowing the light source with the largest normalized contribution allows to locate the object of interest approximately "at" said light source (e.g. vertically above a light source embedded into the floor of a playfield). It should be noted that the normalization can be skipped if all light sources emit with the same intensity.
According to another approach, the evaluation unit may be adapted to determine the time-of- flight that a particular light ray needs from its emission by a light source via its reflection by the object to its detection by a light detector. Via the speed of light, the time-of-flight is related to the distance the light had to travel, which already provides a (coarse) positional information about the object with respect to the considered light source and light detector. In a further development of the aforementioned approach, the evaluation unit is further adapted to determine the desired spatial information from a triangulation of at least three different times-of- flight, which were determined as described above, or on using channel estimates. Thus the complete spatial coordinates of the object can be determined, wherein the accuracy increases with the number of considered times-of- flight.
According to still another approach, the evaluation unit is adapted to identify the light detector that received the highest amount of light which was emitted by (any number of) the light sources and reflected by the object. This approach is based on the fact that the total amount of light that stems from different light sources and is reflected by the object usually propagates with decaying intensity isotropically from the object in all directions. The light detector closest to the object will therefore see the highest intensity of this light, and its position can be taken as an estimation of the position of the object. An advantage of this approach is that the individual contributions of the light sources need not be separated from each other.
Two or more of the different approaches to determine spatial information about the object that were described above can of course be combined in order to increase the accuracy and robustness of the localization.
According to a further development of the invention, the localization system comprises a referee support module for evaluating the spatial information according to given rules of a game. The referee support module may particularly be realized by digital data processing hardware with associated software, and it may for example be integrated into the evaluation unit. The module may for instance detect if the object has crossed a marking of the playfϊeld which implies certain consequences according to the rules of the game (e.g. that a goal has been made or that the ball is offside). This allows to support or even to completely replace a human referee.
In another extension of the invention, the localization system comprises a display arrangement for indicating the spatial information that was determined and/or some information that was derived therefrom (e.g. decisions of the aforementioned referee support module). The display arrangement may comprise monitors or other display devices which are coupled by wire or wirelessly to the evaluation unit and which can be read by a single person and/or a large number of spectators. Moreover, the display arrangement may comprise markings of the playfϊeld that can be lighted, preferably in different colors, to indicate some information.
In another embodiment the localization system further comprises an alarming unit to provide audible and/or visible signals according to the determined spatial or movement information, e.g. an alarm signal in case vanishing movements of in- patients in hospitals or babies according to the so-called sudden baby death. Other situations, where an alarming signal may be required are drivers / pilots of traffic vehicles (e.g. cars or airplanes) in case of the so-called minute sleep. People skilled in the art are able to extend the application of alarming units to further applications not mentioned here. The invention further relates to a method for providing spatial information about at least one object in a surveillance area comprising the following steps: a) Emitting light from an array of light sources that are distributed in the surveillance area. b) Detecting light that was emitted by light sources and reflected by the object and providing an associated detection signal with an array of light detectors that are distributed in the surveillance area. c) Determining spatial or movement information about the object from the detection signals of the light detectors.
The method comprises in general form the steps that can be executed with a localization system of the kind described above. Therefore, reference is made to the preceding description for more information on the details, advantages and improvements of that method.
The invention related further to the use of a localization system according to claim 1 in one or more of the group of application comprising sport applications - referee support systems hospital applications elderly home applications emergency monitoring and/or controlling and/or alarming protection applications - security applications traffic vehicle applications
In particular, the use may be in an hospital / emergency / home application where a sudden breathing stop of small children shall be detected and which initiates an alarm, where a minute sleep of car drivers, pilots, etc. shall be detected and which initiates an alarm, where sudden movement changes or the complete absence of movements of persons shall be detected at intensive care units, elderly homes, etc., which initiates an alarm, or in traffic vehicle applications where a very accurate determination whether and which airbags have to be fired shall be detected.
These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. These embodiments will be described by way of example with the help of the accompanying drawings in which: BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 schematically shows a sport arena equipped with a localization system according to the present invention;
Figure 2 illustrates a localization system that is based on the determination of the closest active light source;
Figure 3 illustrates a localization system that is based on the determination of times-of- flight and a triangulation; Figure 4 illustrates a localization system that is based on the determination of the closest light detector;
Figure 5 illustrates a localization system that combines the aforementioned localization principles;
Figure 6 illustrates an On-Off-Keying modulation of the light emissions; Figure 7 illustrates a Duty-Cycle BiPhase modulation of the light emissions.
Like reference numbers or numbers differing by integer multiples of 100 refer in the Figures to identical or similar components.
DETAILED DESCRIPTION OF EMBODIMENTS
Though the invention will in the following be described with respect to an application in the field of sports, it is not restricted to this application and can favorably be applied in many different areas. Figure 1 illustrates schematically in a perspective view a playfield 10 with two goals 11 and with markings 12 according to the rules of the game that is played (e.g. handball or soccer). The sport arena 10 is equipped with a localization system 100 according to the present invention for determining spatial information about objects of interest, particularly about the ball 1 and/or about players 2. Said localization system 100 comprises
A matrix array of light sources 101 , particularly coded infrared (IR) LEDs, embedded in the floor of the playfield 10. A matrix array of light detectors or photodetectors 102 embedded in the floor of the playfield 10.
A control unit 103 for controlling the light emission of the light sources 101. - An evaluation unit 104 for evaluating the detection signals provided by the light detectors with respect to the desired spatial information about the objects 1, 2, wherein said evaluation module is preferably linked to the control unit 103. The evaluation module usually comprises digital data processing hardware and powerful software routines to process the information received in a manner which is relevant for each individual kind of sport.
An optional display arrangement 105 for displaying the determined spatial information and/or information derived from it, e.g. the number of goals made by the teams.
Optionally addressable visible LED modules, able to produce different colors, to produce field demarcation lines 12 and any other pattern. These modules should also be able to produce intensity-modulated light.
Optionally goals 11 equipped with such addressable visible LED modules. The described localization system 100 relies on coded IR LEDs 101 and fast photodetectors 102, wherein the modulated IR signal is detected. As the LEDs and photodetectors are arranged in a matrix array and each of the IR LEDs emits differently coded light, the movement of any object (player, playing object) can be detected at any time.
The use of IR LEDs 101 permits high switch frequencies, as ns switching times are possible, which allows coding in the μs range. Assuming for example a speed of any object of about 200 km/h (about 60 m/s), this implies a characteristic distance of 60 μm. In addition, any LED has the time to repeat the transmission of its individual code several times while an object passes by.
In the most convenient solution, the LEDs 101 and the photodetectors 102 are placed as shown in the floor. In case of outdoor sport arenas, optical fibers may be integrated in the sport field. Alternatively, the LEDs and photodetectors may be positioned in the walls around the arena and/or in the ceiling. On assuming a sport field of 50x25 m2 and a typical distance of LEDs and photodiodes, respectively, the number of LEDs and photodetectors may be about 30.000 each. Each of these LEDs has to be coded differently, or at least the LEDs in close proximity of each other. Using the coded LEDs and IR detectors, the position vectors and the velocity vectors of all players and the playing object can be traced at all times. When the density of coded LEDs and/or detectors is high enough, even the orientation of the players (and the playing object) can be calculated. In this way, also fouls, etc. can be detected automatically. This enables a complete electronic referee system, i.e. provides a very powerful referee decision support system, without sensors or transmitters to be carried by the players or to be integrated in the playing object. Such a system can detect a.o.:
The playing object 1 leaving the field 10 and the team which caused it. As the system also detects the players' movements, the system can detect such an event taking the game specific rules into account. a goal being made; offside;
30 s and 3 s rule in basketball; fouls being made. Moreover, the system can be used to:
Indicate field lines 12 for any sport in an indoor sport arena, enabling multiple use of indoor sport fields without permanent field demarcation lines.
Lit up the line where the playing object leaves the field in a relevant color (f.i. in the color of the team getting the ball). - Lit up the goal 11 after a scoring event.
Indicate which team gets the ball after a foul, f.i. by indication the color of this team or producing an arrow in a relevant direction.
Indicate information on a display unit 105.
Other applications of the localization system include water sports, indications of lines under water (requires low voltage), ice skating (f.i. to compare with previous races), and advertisement during breaks. In the following, various possible realizations for the localization system of Figure 1 will be described in more detail.
Figure 2 illustrates in this respect a first realization that is based on the determination of the closest firing LED 201. In this localization system 200, the photodiodes 202 embedded in the floor 10 measure the strength of the signals emitted by the different LEDs 201 and reflected from the object 1. The evaluation unit 204 that is coupled to the photodiodes 202 is able to extract the contributions from different LEDs 202 thanks to codes (e.g. code division multiple access (CDMA) like codes) that are imprinted onto their emissions by an associated LED control unit 203. The determination of the particular LED/LEDs that gives the largest contribution allows to localize the object 1 with respect to two dimensions (x-y coordinates). If the LEDs 201 do not emit with the same intensity, their contributions to a detection signal should be normalized with their emission intensity before being compared.
Advantages of this approach are: - There is a need for a small number of photodiodes. It is sufficient to have a number of LEDs large enough to cover the whole area. It is the (geographical) density of the LEDs that determines the resolution of the system: The larger the number of LEDs, the higher the resolution.
The method for finding out the position of an object is fairly easy. It is enough to make a comparison between the reflected lights from different LEDs to find the position of the object.
Figure 3 illustrates a localization system 300 that is based on time-of- flight determination and triangulation, alternatively based on triangulation on field strength. In principle, three couples "LED 301-photodiode 302" are required in this case for the exact determination of the three-dimensional position of the object 1. Controlled by the control unit 303, each LED 301 fires a signal with its individual code and the photodiode 302 coupled to it measures the delay of the signal reflected back from the object 1 (in a RADAR-like procedure). A triangulation is then enough to determine the exact x-y-z position of the object. Of course the use of more than three couples "LED- photodiode" improves the measure and increases the reliability.
An advantage of this approach is that a smaller number of LEDs is required. In fact, when the object 1 is located in between couples "LED-photodiode" while no couple "LED-photodiode" is located below the object, it is still possible to determine the position. A high resolution can thus be obtained with a comparatively small number of couples "LED-photodiode".
Figure 4 shows a localization system 400 that is based on the determination of the closest photodiode. The light emitted by the LEDs 401 is reflected by the object 1. The criterion to determine the x-y position of the object 1 is then to identify which particular photodiode 402 receives the largest amount of reflected light. The larger the number of photodiodes, the higher the resolution in this case. The function of the LEDs 401 is only to ensure that there is a full coverage of light in the surveillance area where the object shall be tracked. Their number should therefore be high enough to make sure that there is enough light to cover the whole floor 10. In general, there is however no need for the control unit 403 to embed a code in the light coming from the LEDs.
An advantage of this method is that it is fairly easy. It is enough to make a comparison between the light power measured by any photodiode to find out the position of the object.
Figure 5 shows a localization system 500 that is based on the processing of the complete information as described above in the single approaches. Here, each LED 501 continuously fires its code embedded by the control unit 503 in the light. Each photodiode 502 receives the light from all the (neighboring) LEDs and extracts the different contributions of the LEDs. An algorithm in the evaluation unit 504 processes all the data to determine the position of the object 1, which may particularly take the closest firing LED, the closest photodiode, and optionally also a triangulation of the time-of- flight into account. Thus a very reliable localization can be achieved. Figure 5 also indicates for one LED (the same holds for the photodiodes) the radiation pattern P. As usual, the length of a vector drawn from the position of the LED to the border of the pattern corresponds to the intensity I(α) that is emitted in the direction α of the vector. The radiation pattern is modeled here as a generalized lambertian with order n (the higher the order, the narrower the pattern). Its consideration helps in the definition of the density of LEDs and photodiodes.
Figures 6 and 7 illustrate two possible embodiments on how the control units 103-503 can embed a CDMA code in the light of the light sources 101-501, e.g. for visible light LEDs. Both approaches refer to a case where two different functions are desired:
1. controlling the intensity of the emitted light (for illumination applications)
2. embedding a CDMA code in the light in an invisible way. Figure 6 illustrates the different binary control signals ("0", "1") over one basic time unit T in case of a pulse width modulation (PWM) for data encoding. In order to guarantee the required illumination, one can modify the duration of pulses (i.e. the duty cycle of the signal) mo dify the amplitude A o f pulses . In this scheme, "0" and "1" will have different widths. Nevertheless, one can compensate this by using balanced codes (i.e. having the same number of bits 0 and 1). Therefore the width of pulses, averaged over a code-word, will be exactly the average value between the "0" and "1" widths.
Figure 7 illustrates the different control signals ("0", "1") for one basic time unit T in case of a generalization of BiPhase (BP) modulation, to allow an arbitrary duty cycle. When the duty cycle equals 50%, this "Duty Cycle BiPhase" (DC-BP) approach degenerates to BP modulation. In this case, the unique code that each LED is assigned is carried in the signal by transmitting "0" and "1" (as in Figure 7) accordingly.
In order to guarantee the required illumination, one can - modify the duration of pulses (i.e. the duty-cycle of the signal) mo dify the amplitude A o f pulses .
Different codes can be applied for IR and visible light LEDs. In the case IR LEDs are used and there is no interest in the illumination function, the LEDs could be directly on-off modulated with their identification code. In this way the code can be repeated more often resulting in an increased performance. Methods to modulate IR LED with data are known by people skilled in the art.
Finally it is pointed out that in the present application the term
"comprising" does not exclude other elements or steps, that "a" or "an" does not exclude a plurality, and that a single processor or other unit may fulfill the functions of several means. The invention resides in each and every novel characteristic feature and each and every combination of characteristic features. Moreover, reference signs in the claims shall not be construed as limiting their scope.