US20100285874A1 - Method and apparatus for detecting an image of a reflective object - Google Patents

Abstract

A method of detecting an image of a reflective object includes the steps of: capturing an exposure image by an image sensor while an LED is ON; down-sampling an exposure image signal from the image sensor to a first digital signal by comparing the level of the first image signal with a first reference level; capturing a dark image by the image sensor while the LED is OFF; down-sampling a second image signal from the image sensor to a second digital signal by comparing the level of the second image signal with a second reference level; computing an image of difference between the exposure image and the dark image by subtracting the second digital signal from the first digital signal; and specifying the image of the reflective object by locating a bright point remaining in the image of difference.

Description

• This Application is a Divisional of U.S. application Ser. No. 11/138,410 filed May 27, 2005 for METHOD AND APPARATUS FOR DETECTING AN IMAGE OF A REFLECTIVE OBJECT, the contents of which are herein incorporated by reference, which is a continuation-in-part of U.S. application Ser. No. 11/022,774 filed Dec. 28, 2004 for Method of detecting position of rectangular object and object detector, the contents of which are herein incorporated by reference.
BACKGROUND OF THE INVENTION
• 1. Field of the Invention
• The present invention generally relates to a method and an apparatus for detecting a position of an object and its angle to a specific reference and, more particularly, it relates to a method and an apparatus that precisely detects a position and its angle of a tool used in computer games.
• 2. Description of the Background Art
• The above materials are expressly incorporated by reference herein. Sports computer games directed to baseball, football, golf, tennis, table tennis, bowling, and so on form one of the categories of computer games. Most of these sports games require associated tools for playing. A bat for baseball, a racket for tennis or table tennis, a bowling ball for bowling, to name a few. A game program running on a CPU (Central Processing Unit) of a game apparatus creates a virtual game situation where a user is supposed to be a player, generates a video image of surroundings, and shows the image on a television set (TV). When a specific situation arises, the player is requested to take an action using the tool. In response to the player's action, the game program changes the virtual situation, and the player is requested to take a next action.
• Take a golf game as an example. At the start of the game, a golf game program creates a scene of a teeing ground. A green can be seen on a backside of the teeing ground and a virtual golf ball is placed at the center (or any other place) of the teeing ground. Then, the player “addresses” an image sensor unit placed on a floor and tries to hit the virtual ball with a club, i.e., swings the club above the image sensor unit.
• When the player swings the club, the image sensor picks up an image of the moving club head. The image of the club head is applied to the golf game program. In response, the golf game program computes the position of the club head on the basis of the image, computes the direction and the speed of the club head on the basis of the position, computes the resultant trajectory of the imaginary golf ball hit by the imaginary golf club in accordance with the direction and the speed of the club head, and creates a new game situation in accordance with the new position of the golf ball.
• Naturally, specific hardware is necessary for detecting the position of the club head. Japanese Patent Application Laying-Open (Tokkai) No. 2004-85524 discloses an apparatus for detecting such positions of a game tool. The apparatus is used in a computer golf game and includes a stroboscope having four LED's (light emitting diodes), a CMOS (Complementary Metal-Oxide-Silicon) image sensor (hereinafter “CIS”), and a processor. A reflector is attached to the bottom (sole) of a club head or a putter head. The reflector has a long rectangular shape with circular ends. The apparatus is connected to a TV monitor and a golf game program running on the processor generates the video image of a virtual golf course in response to player's action with the club or the putter.
• In operation, the CIS captures two kinds of images: images during the stroboscope LED's are on (emitting light); and images during the stroboscope LED's are off. The image signals are applied to the processor, where necessary computation is carried out.
• When LED's are emitting light, the reflector reflects that light to the CIS; therefore, the CMOS sensor forms the image of the reflector. Other light sources also form images on the CIS. When the LED's are off, the reflector does not reflect the light; the image is not formed. Only other light sources form their images. By computing the difference between these two kinds of images in the processor, therefore, the processor can detect the image of the reflector separate from other images.
• The processor detects two points farthest from each other in the image of the reflector. These two points indicate the two ends of the mid line of the reflector; by knowing the X and Y coordinates of these points, the processor can know the position of the club head or the putter head as an average of these two points. By computing the average of these two points for each of the captured images, the processor computes the direction and the speed of the movement of the club head. Also, the processor can compute the angle θ between the line connecting the two end points of the reflector and a prescribed reference line. From this angle θ, the angle of the clubface can be computed.
• The golf game program running on the processor processes these data, determines the trajectory of the virtual golf ball, and creates next virtual situation.
• However, in order to determine the two farthest points in the image of the reflector, the processor have to compute the distance of each combination of two points in the image of the reflector. This is relatively complicated operation and requires a considerable amount of computing time. Further, the CIS has a 32×32 pixel, 8 bits per pixel image plane. The data size of one image therefore amounts to 8192 bits=1024 bytes. The processor needs to receive the data from the CIS, store the data, and carry out the above-described computations on the stored data.
• Therefore, a processor with relatively high performance is necessary in order to carry out the computations necessary for the game in real time. Also, the processor needs to have storage with a capacity large enough to store the data output from the CIS. This results in a computer game machine with a relatively high cost. Because younger people are the main users of the computer game machines, the game machines should be inexpensive although they should have enough performance to fully operate in real time.
• Further, because it is basic demands that the game machine responses to the operation of the user, the image of the reflector should be precisely detected.
SUMMARY OF THE INVENTION
• Therefore, one of the objects of the present invention is to provide a method and an apparatus for detecting an image of a reflective object that detect a position of the reflective object with a simple operation.
• Another object of the present invention is to provide a method and an apparatus for detecting an image of a reflective object that detect a position of the reflective object with smaller amount of computation compared with the prior art.
• Still another object of the present invention is to provide a method and an apparatus for detecting an image of a reflective object that precisely detect a position of the reflective object with a simple operation.
• Yet another object of the present invention is to provide a method and an apparatus for detecting an image of a reflective object having simple structure that detect a position of the reflective object with smaller amount of computation compared with the prior art.
• In accordance with the first aspect of the present invention, a method of detecting an image of a reflective object comprising the steps of: capturing a first image by an image sensor while a light-emitting device is emitting light; down-sampling a first image signal outputted from the image sensor to a first digital signal by comparing the level of the first image signal with a predetermined first reference level; capturing a second image by an image sensor while the light-emitting device is not emitting light; down-sampling a second image signal outputted from the image sensor to a second digital signal by comparing the level of the second image signal with a predetermined second reference level; computing an image of difference between the first image and the second image by subtracting the second digital signal from the first digital signal; and specifying the image of the reflective object by locating a bright point remaining in the image of difference.
• The first and second images are down-sampled and the difference image is computed based on these images. Because the image data is in a digital format, the computation is simple and a relatively low-performance CPU or MCU can carry out the calculation within a period short enough to immediately respond to the movement of the reflective object.
• Preferably, the first reference level is the same as the second reference level. The step of down-sampling a first image signal includes the steps of: lowering the level of the first image by a predetermined amount; and down-sampling the first image signal having its level lowered by comparing the level of the first image signal having its level lowered with the second reference level.
• Alternatively, the first reference level is selected to be higher than the second reference level by a predetermined amount.
• Further preferably, the light emitting device includes a light emitting diode that emits light of a non-visible, for instance infrared, wavelength and the step of capturing a first image includes the step of capturing the first image by an image sensor with a filter that substantially blocks visible light while the light emitting device is emitting light, and the step of capturing a second image includes the step of capturing the second image by the image sensor with the filter while the light emitting device is not emitting light.
• The step of computing may include the step of computing the image of difference between the first image and the second image by subtracting the second digital signal from the first digital signal on a pixel-to-pixel basis.
• In accordance with the second aspect of the present invention, an apparatus for detecting an image of a reflective object comprising: a light-emitting device; an image sensor; a control device that causes the light emitting device and the image sensor such that the image sensor captures two successive images including a first image and a second image; the first image being captured while the light emitting device is emitting light, the second image being captured while the light emitting device is not emitting light; a down-sampling circuit connected to receive the output of the image sensor, for down-sampling the image signals of the first image and the second image outputted from the image sensor; a difference image computing unit for computing a difference image by subtracting the down-sampled image signal of the second image from the down-sampled image signal of the first image; and a means for specifying the image of the reflective object in the difference image by locating a bright point in the difference image.
• The device may further includes a light control circuit that outputs a light control signal that assumes a predetermined first level or a predetermined second level, indicating a period when the light emitting device is allowed to emit light and a period when the light emitting device is prohibited from emit light, respectively. The down-sampling circuit includes: a potential divider adjuster connected to receive the output of the image sensor, for lowering the level of the output of the image sensor by a predetermined amount while the light control signal is at the first level and for raising the level of the output of the image sensor by the predetermined amount while the light control signal is at the second level.
• Preferably, the potential divider adjuster includes: a first and second resistors connected in series between an output terminal of the image sensor and a ground potential thereby forming a potential divider, a third resistor having one end connected to a connecting node of the first and second resistors, a switching device connected between the other end of the third resistor and the ground potential for controlling the connection of the other end of the third resistor and the ground potential in response to the light control signal.
• In accordance with the third aspect of the present invention, a computer game program product includes: a computer readable recording medium storing a computer executable game program thereon. The game program includes: a first computer readable program code means for causing the computer to generate an image of imaginary object to be moved by the user in the game and a direction indicator indicating a desired direction in which the imaginary object is to be moved by the user in the game; a second computer readable program code means for causing the computer to receive a time-series data including position data of a controller of the game, the position data indicating a position of a representative point of the controller at a certain time with reference to a predetermined coordinate system; a third computer readable program code means for causing the computer to compute a trajectory of the imaginary object in response to the received time-series of position data, wherein the trajectory is computed such that a direction of movement of the imaginary object is determined using the desired direction as a reference based on the position data.
• The foregoing and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 illustrates an overall arrangement of agolf game system30 in accordance with a first embodiment of the present invention;
• FIG. 2 shows agame cassette76 including a CPU and a memory that stores a golf game program, and anadaptor46 for thegame cassette76 having TV connection capabilities and IR communication capability;
• FIG. 3 is a perspective view of aswing detector44 for detecting the direction and the speed of a club head as well as an angle of its face in accordance with the embodiment;
• FIG. 4 shows acontroller42 for a golf game used with theswing detector44 shown inFIG. 3;
• FIG. 5 shows a functional block diagram of theswing detector44 in accordance with the first embodiment;
• FIG. 6 schematically shows the image plane ofCIS146 of theswing detector44 shown inFIG. 5 and an image of areflector strip124 of thecontroller42 shown inFIG. 4;
• FIG. 7 is a waveform diagram that shows the timing settings of the FS, /STR, and VOUTS signals withinswing detector44 shown inFIG. 3;
• FIG. 8 is a waveform diagram of the signals withinswing detector44 shown inFIG. 3;
• FIG. 9 is waveform diagrams of an image signal outputted fromCIS146 to down-sampling comparator150 shown inFIG. 5 and an image signal down-sampled by down samplingcomparator150;
• FIG. 10 schematically shows how the image of the reflector is extracted by subtracting the dark data from the exposure data;
• FIGS. 11 to 14 show the overall control structure of golf club detecting program running onMCU148 ofswing detector44;
• FIG. 15 shows the status transition diagram of a golf game program that is stored ingame cassette76;
• FIG. 16 shows a main routine of the golf game program;
• FIG. 17 shows the control structure of the game mode selection process in the golf game program;
• FIG. 18 shows the control structure of the swing detection process in the golf game program;
• FIG. 19 shows how the angle of the clubface and the moving direction of the golf club are computed;
• FIG. 20 shows directions of a clubface that can be detected byswing detector44;
• FIG. 21 shows a detecteddirection344 of the movement of the golf club with reference to apredetermined reference direction542;
• FIG. 22 shows a conventional way of determining a direction of a golf ball movement hit by a golf club;
• FIG. 23 shows a novel way of determining a direction of a golf ball in accordance with the first embodiment;
• FIG. 24 shows the detected angle θ₂of the clubface in accordance with the first embodiment;
• FIG. 25 shows how the direction of a golf ball in the screen is determined in the embodiment of the present invention;
• FIG. 26 shows the waveform of an IR light source and the clock signal that defines the sampling timing ofCIS146;
• FIG. 27 shows how a light from the IR light source causes a problem in detectingreflector strip124 inCIS146;
• FIG. 28 shows a functional block diagram of aswing detector720 in accordance with the second embodiment;
• FIG. 29 shows a circuit diagram of apotential divider adjuster730 inswing detector720;
• FIG. 30 is a table showing the operation ofpotential divider adjuster730;
• FIG. 31 shows how the problem caused by light from the IR light source is handled inswing detector720 of the second embodiment;
• FIG. 32 shows a waveform diagram of the signals withinswing detector720;
• FIG. 33 shows how the problem caused by light from the IR light source may be handled in another embodiment;
• FIG. 34 shows an arrangement to handle the problem caused by light from the IR light source in still another embodiment; and
• FIG. 35 shows another arrangement to handle the problem.
DESCRIPTION OF THE PREFERRED EMBODIMENTSFirst Embodiment
• —Overall Arrangement of the System—
• FIG. 1 shows an overall arrangement of agolf game system30 in accordance with one embodiment of the present invention. Referring toFIG. 1,golf game system30 includes: anadaptor46 having connection facility toTV48 via acable52 and a wireless IR (Infrared) communication capability; and agame cassette76 that is to be mounted onadaptor46.
• Referring toFIG. 2,adaptor46 has ahousing72 and a receivingstage74 that moves up and down withinhousing72. A connector (not shown) is provided withinhousing72 ofadaptor46 and by pushing down receivingstage74, the connector is exposed.Adaptor46 further has anIR receiving window70 for IR communication.
• Game cassette76 has aconnector78 with connector pins Tn. Whengame cassette76 is put on receivingstage74, pushed down and slid toward the front face ofadapter46,connector78 will be coupled with the connector ofadaptor46. Although not illustrated,game cassette76 includes a processor and a memory that stores a golf game program. The processor ofgame cassette76 can utilize the IR communication capability ofadaptor46. The processor can also apply a video image signal of a golf game toTV48 shown inFIG. 1.
• Referring again toFIG. 1,golf game system30 further includes: a golf club-type controller42 which aplayer40 uses to play the golf game; and aswing detector44 for detecting the position of the head ofcontroller42 as well as the angle of the clubface ofcontroller42 with reference to a predefined reference direction.Swing detector44 also has a wireless IR communication capability and can transmit the detected position of the head ofcontroller42 as well as the angle of the clubface toadaptor46 throughIR light50.
• —Overview ofSwing Detector44—
• Referring toFIG. 3,swing detector44 includes a relativelyflat housing80.Swing detector44 further includes: anIR LED106 for transmitting data; apower switch90; fourbuttons98,100,102, and104 for adjusting functions ofswing detector44; aCIS146; and two IR LED's94 and96 for exposure provided on either side ofCIS146, all arranged on the upper surface ofhousing80. The arrangement of the circuitry withinswing detector44 will be described later with reference toFIG. 5.
• Referring toFIG. 4,controller42 includes ashaft120; and aclub head122 with aneck121 that is connected toshaft120. On the bottom (sole) ofclub head122, areflector strip124 having a rectangular shape is attached. Retro-reflector strip124 has two sets of edges; longer ones and shorter ones. Retro-reflector strip124 is attached toclub head122 so that its longer edges are parallel to the edge of the clubface.
• —Internal Structure ofSwing Detector44—
• Referring toFIG. 5, in addition toIR LED106, IR LED's94 and96 and fourbuttons98,100,102 and104,swing detector44 includes as its inner circuitry:CIS146 having 32H (Horizontal)×32V (Vertical) resolution and outputting VOUTS signal, which includes a series of analog pixel values; adown sampling comparator150 connected to receive the VOUTS signal fromCIS146 for down-sampling the VOUTS signal to a 1-bit binary signal; an MCU (Micro Controller Unit)148 that receives the output of downsampling comparator150 for computing the position of the center point of the club head as well as the angle of the clubface; and apower LED152 embedded withinpower key90 shown inFIG. 3 for the indication of power on and off. Although not shown,MCU148 has an internal memory, registers, and a CPU.
• Downsampling comparator150 includes a Schmidt trigger. In this embodiment, the positive going threshold and the negative going threshold of Schmidt trigger is the same V_TH. When the level of the input analog signal goes higher than the threshold V_TH, the output of downsampling comparator150 immediately goes High. If the level of the input analog signal falls to a level lower than the threshold V_TH, the output of downsampling comparator150 immediately falls to Low. Thus, the VOUTS signal, which is an analog signal, is converted into a 1-bit binary signal.
• Swing detector44 further includes: abattery box140 operatively coupled topower key90; avoltage regulator circuit142 for regulating the voltage outputted bybattery box140 and for supplying power to MCU148 and other circuits inswing detector44 via power lines; and apower control switch144 that, under control ofMCU148, supplies the power fromvoltage regulator circuit142 toCIS146.Power control switch144 andCIS146 receives control commands fromMCU148 via acontrol bus149. Outputs ofCIS146 and downsampling comparator150 are connected to the input ofMCU148 via adata bus151.
• Referring toFIG. 6,MCU148 finds the angle θ which one of the edges of animage182 ofreflector strip124 forms with one of the edges of an image plane ofCIS146 in the following manner. First,MCU148 scans animage180 captured byCIS146 row by row from the top to the bottom searching for theimage182 ofreflector strip124. The first bright point at a row with a y-coordinate T1yindicates the topmost corner190 of theimage182. For that purpose, a coordinate system is defined on the image180 (i.e., on the image plane of CIS146). Likewise,MCU148scans image180 column by column from the rightmost column until it finds the rightmost bright point. This point indicates the column with an x-coordinate value R1xof thecorner192 of theimage182. In a similar manner,MCU148 finds the leftmostbright point196 with an x-coordinate L1xand the bottombright point194 with a y-coordinate B1y.Here, T1y>B1yholds. Likewise, R1x>L1xholds. In other words, in this operation, the fourpoints190,192,194 and196 closest to respective edges ofimage180 from the respective edges of theimage182 are detected and then their x- or y-coordinate values are computed.
• Points190,192,194 and196 correspond to the four corners ofimage180 ofreflector strip124. The coordinates (X, Y) of thecenter point198 of theimage182 ofreflector strip124 are then computed by:
• 
  X=(L1x+R1x)/2
• 
  Y=(T1y+B1y)/2.
• The angle θ, which the longer edge ofimage182 ofreflector strip124 makes with the x-axis, is determined by:
• $\begin{array}{c}\tan \theta =\Delta y/\Delta x\\ =\left(B1y-T1y\right)/\left(R1x-L1x\right),\\ \therefore \theta ={\tan }^{-1}\frac{B1y-T1y}{R1x-L1x}.\end{array}$
• By the above-described simple computation, the position of thecenter point198 ofreflector strip124 and its angle θ between the x-axis can be computed. This requires a relatively small amount of computation compared with the prior art.
• FIG. 7 shows the waveforms of the signals amongCIS146,MCU148 and downsampling comparator150 shown inFIG. 5. Referring toFIG. 7, “FS” is a frame signal for synchronization of circuits external toCIS146. One cycle period of signal FS is predetermined by a clock signal (SCLK) and, in this embodiment, it equals to 12288 clock cycles. In this embodiment,CIS146 captures an image while signal FS is at the Low level. This period will be called an exposure time “Texp” hereinafter. WhenCIS146 is ready to output the captured image signal, signal FS is at the High level.
• A time period ofCIS146 for capturing an image (hereinafter “internal exposure time”) depends on the settings of a specific 8-bit register E0(7:0) internal toCIS146. The settings may be externally changed. The exposure time Texp is divided into 255 (=2⁸) parts.CIS146 determines the internal exposure time by the Texp times register value E0(7:0) divided by 255. Thus, if the register value E0(7:0) is 16, the internal exposure time will be Texp*16/255 as shown inFIG. 7.
• When signal FS is at the High level, i.e., signal FS indicates the data transfer period,CIS146 is ready to transfer the captured image data VOUTS. The falling edges of signal /STR show the timings of data hold and sampling of VOUTS at down samplingcomparator150. During the data transfer period, signal /STR includes 32×32+1 pulses. At each of the falling edges of these pulses, downsampling comparator150 samples theVOUTS signal220, compares the level ofVOUTS signal220 with thethreshold level V_TH221, and outputs the result as a 1-bit signal222. The first data ofVOUTS signal220 is a dummy and is discarded; therefore, downsampling comparator150outputs 32×32 pixel data within the data transfer period. Incidentally,VOUTS signal220 is the analog signal indicative of the intensity of the captured image. Each pixel data ofVOUTS signal220 is converted to the 1-bitdigital signal222 and is supplied toMCU148.
• Because the image signal is reduced to 32×32 pixel signals each of which is one bit, memory capacity ofMCU148 required for storing the image data is substantially reduced and an MCU with relatively low cost can be used.
• FIG. 8 shows IR-CTL, IR-LED, and EXP signals for controlling the exposure of theCIS146, in addition to FS, /STR, and VOUTS signals.
• The IR-CTL signal alternately assumes the High level and the Low level in synchronous with the falling edges of the FS. When the IR-CTL signal is at the High level, IR LED's94 and96 are allowed to emit IR lights. Otherwise, they are prohibited.
• The IR-LED signal assumes the High level when the FS signal is at the Low level and the IR-CTL signal is at the High level and drives the IR LED's94 and96. Otherwise, IR-LED signal remains at the Low level and the IR LED's94 and96 do not emit light.
• An EXP signal defines the internal exposure time. When the EXP is at the High level,CIS146 captures the image. The EXP attains the High level in response to each of the FS signal's falling edges. The internal exposure time is defined by the Texp times register value, as described above.
• In summary, for each cycle of the IR-CTL signal, two images are captured byCIS146; one is captured during IR LED's94 and96 are emitting IR light and the other is captured during IR LED's94 and96 are not emitting IR light. The former is called the exposure data and the latter is called the dark data.
• FIG. 9 shows the down sampling carried out by down samplingcomparator150. VOUTS outputted fromCIS146 is an analog signal as shown in waveforms220 (FIG. 9(b)). Downsampling comparator150 compares the level of VOUTS with the predeterminedthreshold level V_TH221 and outputs the resultant 1-bit binary signal as shown by the waveform222 (FIG. 9(a)).
• Note that, in this embodiment,CIS146 captures the IR images of the objects. The IR light emitted from IR LED's94 and96 will be reflected by thereflector strip124 of thecontroller42 and will be captured byCIS146. Thus, the exposure data will include the image of thereflector strip124.
• However, IR lights are abundant in the environment. For instance, the lighting facilities within a house, a hotel reception, or a convention hall will emit IR lights. Even the sun rays include IR and the sun rays incident uponCIS146 through a window may form an image on the image plane ofCIS146. Thus,CIS146 will receive IR lights from not onlyreflector strip124 but also IR light sources other thanreflector strip124.CIS146 must isolate the image ofreflector strip124 from those of other IR light sources.
• This may be done if one uses a high-performance CPU; however, such a high-performance CPU is expensive. Electronic game apparatus should be easy to purchase and therefore, it cannot include such an expensive CPU. Thus, it is essential to isolate the image ofreflector strip124 with an inexpensive CPU. TheCIS146 of the present embodiment deals with this challenge in the following way using the exposure data and the dark data.
• FIG. 10(A) shows an example of exposure data captured byCIS146. The image includes a number of images oflight sources230,232, and234, of which theimage234 is that ofreflector strip124.
• FIG. 10(B) shows an example of dark data captured byCIS146. The arrangement of the light sources is the same to that ofFIG. 10(A). Referring toFIG. 10 (B), the images oflight sources230 and232 are still present in the dark data; however, theimage234 ofreflector strip124 is not present. This is becausereflector strip124 is a passive light source. When IR LED's94 and96 do not emit IR light,reflector strip124 does not emit reflecting IR light; therefore, the dark data image does not include theimage234.
• FIG. 10(C) shows the difference between the images ofFIG. 10(A) andFIG. 10(B). As shown byFIG. 10(C), the difference data only includesimage234 ofreflector strip124. In this manner,CIS146 isolates theimage234 ofreflector strip124 from other images.
• —Program Structure ofSwing Detector44—
• FIGS. 11 to 14 show the overall control structure of the program running onMCU148 ofswing detector44 for controllingCIS146, capturing the image ofreflector strip124, and computing the position of its center point and its angle θ with reference to the x-axis.
• Referring toFIG. 11, after the power-on, the program starts atstep240 where registers ofMCU148 are initialized. Atstep242,MCU148 clears its RAM (random access memory). Then, atstep244, PIO (programmed input/output) setting ofMCU148 is carried out. Atstep246,MCU148 reads option code setting and resetsCIS146 and sets up registers ofCIS146 in accordance with the option code setting. Atstep248, watchdog timer is reset.
• Atstep250, it is determined whether the signal FS is Low or not. If not, the control returns to step250 and the determination is repeated until the signal FS is Low. When signal FS is Low,MCU148 turns on the exposure IR LED's94 and96 (seeFIGS. 3 and 5). Atstep254, exposure IR LED's94 and96 are kept on until the signal FS is High. When the signal FS is found to be High, exposure IR LED's94 and96 are turned off atstep256.
• Referring toFIG. 12,MCU148 waits until the signal /STR is at its falling edge atstep258. When the /STR is at its falling edge,MCU148 reads the VOUTS down-sampled by down samplingcomparator150 atstep260.
• Atstep262, it is determined whether all 32×32 data are received fromCIS146. If not, the control returns to step258. When all of the 32×32 data are received, the control goes to step266. The 32×32 data received atsteps258 to262 forms the exposure data.
• Atstep266,MCU148 tries to get key press data. Atstep268, a sleep counter (not shown) withinMCU148 is checked ant it is determined whether the sleep counter has overflowed or not. If overflowed, the control goes to step270; otherwise, it goes to step280 (FIG. 13).
• Atstep270,MCU148 controlspower control switch144 to stop the power supply toCIS146 and enters the sleep mode. Atstep272,MCU148 turns on the sleep LED, which ispower LED152 shown inFIGS. 3 and 5. Atstep274,MCU148 waits for a predetermined period by a delay loop. After the predetermined period,MCU148 turns on the sleep LED atstep276. Atstep278, it is determined whether key is pressed or not. If there is no key press, then control returns to step270 andMCU148 enters sleep mode again. If there is a key press, the control jumps back to step240 andMCU148 carries out thesteps240 et seq. again.
• When it is determined atstep268 that the sleep counter has not overflowed, control goes to step280 shown inFIG. 13. Referring toFIG. 13, atstep280,MCU148 waits until the signal FS is Low. When the signal FS is Low,MCU148 turns onpower LED152 atstep282 and waits until the signal FS is High atstep284. By turning onpower LED152,MCU148 indicates thatMCU148 andCIS146 are operating. When the signal FS is High,MCU148 turns offpower LED152. By turning off thepower LED152, power consumption is reduced.
• Next, atstep288,MCU148 waits until the signal /STR is at is falling edge. When the signal /STR is at its falling edge,MCU148 again reads VOUTS data atstep290.Steps288 and290 are repeated until it is determined that 32×32 data are received atstep292. The 32×32 data received atsteps288 to292 form the dark data. Then, the control goes to step294, whereMCU148 subtracts the dark data from the exposure data. By this operation, images of light sources other thanreflector strip124 are removed from the 32×32 exposure data. Control goes to step296 shown inFIG. 14.
• Atstep296, it is determined whether there is no bright point in the image or any key press. If there is a bright point or a key press, control goes to step298; otherwise, control goes to step318.
• Atstep298, it is determined whether there is no bright point in the image but a key press. If there is no bright point but a key press, control goes to step314; otherwise, control goes to step300.
• Atstep300,MCU148 scans the 32×32 image from top to bottom row until it gets the topmost bright point T1y. Atstep302,MCU148 scans the image from bottom to top row to get the bottommost bright point B1y. Atstep304,MCU148 scans the image from left to right column to get the leftmost bright point L1x. Finally, atstep306,MCU148 scans the image from right to left column to get the rightmost bright point R1x.
• Atstep308,MCU148 calculates center point (X, Y) of the image ofreflector strip124 by the following equations (1):
• 
  X=(L1x+R1x)/2
• 
  Y=(T1y+B1y)/2   (1)
• Atstep310, it is determined whether the game is in an angle mode where the angle of the clubface is considered in the golf game. If it is not in the angle mode, control goes to step314; otherwise, control goes to step312 where club angle θ is calculated by the following equation (2):
• 
  θ=tan⁻¹(B1y−T1y)/(R1x−L1x)   (2)
• Then control goes to step314. Atstep314,MCU148 sets up the IR output pattern for the IR communication toadaptor46 in accordance with the computed result.
• The data format of the position data and angle data for IR communication includes 22 bits. The first bit is a start bit, which is always is 1. The next thirteen bits represent the X and Y coordinates of the center point including parity bits. Because X and Y are in the range from 0 to 31 (32 pixels), it requires 5 bits to represent each of the X and Y coordinates. The parity bits include three bits.
• The next four bits represent the club angle. The angle computed atstep312 is rounded to the nearest 15 degrees)(15° as shown by the twelve angles θ₁to θ₁₂inFIG. 20. These angles are represented bynumbers 1 to 12, respectively. Thus, the club angle requires 4 bits in transmission. In this embodiment, θ₁to θ₁₂represent 0°, 15°, 30°, 45°, 60°, 75°, 90°, −75°, −60°, −45°, −30°, and −15°, respectively. The conversion is carried out in the game program.
• The next three bits indicates the pressed key. If no key is pressed, these three bits are not transmitted. The last bit is an end bit.
• Referring again toFIG. 14, atstep316,MCU148 resets the sleep mode counter. Atstep320,MCU148 outputs the IR data set up atstep314. The golf game program running ongame cassette76 can then utilize the data and change the game situation. Afterstep320, the control returns to step248 shown inFIG. 11.
• When it is determined atstep296 that there is no bright point in the 32×32 image or a key press, control goes to step318. Atstep318,MCU148 clears the IR output pattern. Then the control goes to step320 where the cleared IR output pattern is output toadaptor46.
• —Operation ofSwing Detector44—
• Swing detector44 of the present embodiment operates as follows. At the time of power-up,MCU148 ofswing detector44 initializes its registers (FIG. 11, step240), clears its RAM (step242), sets up PIO settings (step244), and reads option code setting and starts supplying power to CIS146 (step246). In response to the power supply,CIS146 starts capturing images. During the exposure period,CIS146 sets the signal FS at the Low level and during the transfer period,CIS146 sets the signal FS at the High level.
• Atstep248,MCU148 resets watchdog timer and waits for the signal FS fromCIS146 to be Low (FIG. 11, Step250). When the signal FS becomes Low, this indicates thatCIS146 is in the exposure period andCIS146 turns on IR LED's94 and96 for exposure.CIS146 captures the image during the exposure time.CIS146 waits for the signal FS to be High atstep254. WhenCIS146 is ready to output the VOUTS, it sets the signal FS to the Higher level andMCU148 turns off IR LED's94 and96 for exposure atstep256.
• Referring toFIG. 8, during the transfer period, the signal FS stays at the High level and the signal /STR alternately attains the Low level and the High level at a specific time period. At each of the falling edges of the signal /STR, downsampling comparator150 starts inputting data VOUTS showing the intensity of a pixel of the captured image as shown inFIG. 9(b).
• The output of downsampling comparator150 rises to the High level when the level of VOUTS is equal to or higher than the positive going threshold. It falls to the Low level when the level of VOUTS is lower than the negative going threshold. An example of the output of downsampling comparator150 is shown inFIG. 9(a). AS described above, the positive going threshold and the negative going threshold is the same V_TH.
• Referring again toFIG. 12, atsteps258 to262, at each of the falling edges of the signal /STR, with a prescribed delay long enough for VOUTS to settle,MCU148 reads VOUTS down-sampled by down-sampling comparator150. When 32×32 data are received,MCU148 tries to get key data. The received data forms the exposure data.
• If sleep counter is found to have overflowed atstep268,MCU148 enters the sleep mode until any of the keys is pressed. If sleep counter has not overflowed,MCU148 waits until the signal FS is Low at step280 (FIG. 13). When the signal FS is Low,CIS146 is again in the exposure period andMCU148 turn onpower LED152 at step282 (FIG. 13) indicating thatMCU148 andCIS146 are operating. Then,MCU148 waits until the signal FS is High atstep284. During this period,CIS146 captures the image without IR LED's94 and96 lighting. When the signal FS is High,CIS146 is now in transfer mode andMCU148 turns off power onLED152.
• Atsteps288 to292,MCU148 receives the 32×32 image VOUTS data outputted fromCIS146 and down-sampled by down samplingcomparator150. The image forms the dark data.
• Atstep294,MCU148 subtracts the dark data from the exposure data received atsteps288 to292 (FIG. 13). The resulting data includes, if any, only the exposure data ofreflector strip124.
• Atsteps296 and298 (FIG. 14),MCU148 determines whether the resulting image includes a bright point and if the image includes a bright point, referring toFIG. 6,MCU148 scans the image from top to bottom row to get the topmost bright point T1yat step300 (FIG. 14), from bottom to top to get the bottommost bright point B1yatstep302, from left to right to get the leftmost bright point L1xatstep304, and from right to left to get the rightmost bright point R1xatstep306.
• Atstep308,MCU148 calculates the coordinates (X, Y) of the image ofreflector strip124 by equations (1). If the game is in the angle mode,MCU148 calculates club angle by equation (2).
• Atstep314,MCU148 sets up IR output pattern. It resets the sleep mode counter atstep316 and outputs the IR data utilizingIR LED106 shown inFIGS. 3 and 5 toadaptor46 atstep320.
• By repeating the above-described operation,swing detector44 can detect the position of reflector strip124 (FIG. 4), i.e., the position of the head ofcontroller42, and the club angle and transmit the detected data toadaptor46. Theadapter46 receives the data, thegame cassette76 inserted in theadapter46 calculates the trajectory of the imaginary golf ball, and changes the game situation.
• —Use of the Club Angle—
• The golf game program running on the CPU ofgame cassette76 loaded onadaptor46 can use the information of the X and Y coordinates of center of the club head and the club angle ofcontroller42 as in the following manner. First, by computing the difference between the coordinates detected at different times, the game program can compute the position of the center point of the club head and the angle of the clubface. Using this information, the game program can compute the direction of the imaginary golf ball trajectory.
• In this connection, the golf game program running ongame cassette76 adopts a novel way of determining the direction of the golf ball trajectory. Referring toFIG. 21, assume that the movingdirection544 of the club head (the movement of the center point of reflector strip124) in the 32×32image plane540 makes an angle θ_swith areference line542, which is parallel to the y-axis ofimage plane540.
• In the prior art, as shown inFIG. 22, the golfgame program screen580 would show atarget arrow584 directed to the golf hole (not shown) and determine the trajectory of the imaginary golf ball in thedirection586 that makes the angle θ_swith thereference line582, which is parallel to the y-axis of thescreen580.
• In this arrangement, when thetarget arrow584 is directed downward, the player must swing the controller backward. In some situation, the player must address theswing detector44 facing the direction opposite to the golfgame program screen580. Further, when some obstacle is located between the imaginary golf ball and the target hole, the player might want to direct his/her ball to a direction other than the target direction. In such a situation, it will be hard to determine the right swing direction, because he/she must determine the swing direction with reference to thevertical reference line582, which is not shown on the golfgame program screen580.
• In contrast, the golf game program running ongame cassette76 determines the trajectory of the imaginary golf ball as in the following manner.
• Referring toFIG. 23, animaginary golf ball607 andarrow604 are displayed on thescreen600 ofTV48. Given the angle θ_s, the golf game program in this embodiment adds the angel θ_snot to thereference line602 of thescreen600 but to the direction of thearrow604 that is directed to the target golf hole, resulting in thedirection606.
• By this arrangement, if the player swingscontroller42 in a direction of the Y-axis of image plane ofCIS146, the angle θ_sis zero; therefore, the imaginary golf ball will go in the direction of the target hole. Even if thetarget arrow604 is directed downward, the player need not swing thecontroller42 backward. A usual, straight swing over theswing detector44 will make theimaginary golf ball607 go straight in the direction of thetarget arrow604. The player need not go around theswing detector44 to find the right place for the right angle. Therefore, in contrast to the prior art, the arrangement according to the present embodiment makes it much easier to play the golf game.
• Further, assume that an obstacle is located between the imaginary golf ball and the target hole. In this embodiment, it is easy for the player to determine the direction of the swing, because there is thetarget arrow604 shown on thescreen600, and what is needed is to see the deviation angle that the desired direction forms with thetarget arrow604. Once the deviation angle is known, the player can swing thecontroller42 such that the direction of the swing makes the deviation angle with the straight swing line on theswing detector44.
• In determining the trajectory of the imaginary golf ball, the club angle is further taken into consideration in a certain play mode (the “angle mode”) in this embodiment. In the angle mode, the trajectory of the golf ball is determined as shown inFIG. 25.
• Referring toFIG. 24, let us assume that the club angle detected byswing detector44 of the present embodiment is θ₂. This means that the angle that aclubface628 makes with theline626 corresponding to the X-coordinate ofCIS146 is θ₂. Further assuming that the player swings thecontroller42 in the direction of theline624 corresponding to the Y-coordinate ofCIS146 In the case ofFIG. 23, the trajectory of theimaginary golf ball620 is determined in the direction of thetarget arrow622. But the golf game program can determine the trajectory of theimaginary golf ball620 as in the following. Incidentally, if physical law is used in calculating the trajectory, theimaginary golf ball620 will move in the direction corresponding to the normal line of the clubface.
• Referring toFIG. 25, animaginary golf ball642 is displayed on thescreen640 ofTV48.Target arrow644 is also displayed directed to the target golf hole. Given the club angle θ₂and the deviation angle θ_sof the club head movement, the program first adds angle θ_sto the angle ofarrow644. This results in thedirection646. Further, the program adds the correction value α×θ₂(0≦a<1) of the clubface angle θ₂to thedirection646, resulting in adirection648 further deviated fromtarget arrow644. By this arrangement, the golf game will be much more amusing than the prior art golf games.
• As has been described,swing detector44 can detect the position of the center point of the club, and further the angle of the clubface. A sequence of these data is transmitted to adaptor46 (FIG. 1) via IR communication. Thus, the golf game program running on the CPU ofgame cassette76 mountedo adaptor46 can utilize these data and the resultant golf game will be more amusing than the prior art.
• —Control Structure of the Golf Game—
• The golf game program running on the CPU ofgame cassette76 ofadaptor46 uses the information fromswing detector44 to control the game process.FIG. 15 shows a state diagram of the golf game program.
• Referring toFIG. 15, the game starts at theinitialize state340. After completinginitialize state340, the game enters gamemode selection state342. Here, the game program shows a menu on the screen ofTV48. The menu will show a number of games of which settings are different. After the user selected a desired game, the game enters a “before swing”state344.
• In “before swing”state344, the game determines whether the user swingscontroller42 using the data fromswing detector44. If a swing is detected, the game enters aball moving state346, where the game computes the direction and speed of the imaginary ball and then compute the trajectory of the ball in the imaginary golf course. In this state, the golf game generates a sequence of images that show the movement of the imaginary golf ball together with the changing background scene of the golf course.
• When the imaginary ball comes to a halt, the game enters a scoring and resultdetermination state348. Here, the game determines where the imaginary golf ball lies and computes the score of the user.
• If it is determined that the game comes to an end, the game goes back to gamemode selection state342. If it is not, the game goes back to beforeswing state344.
• FIG. 16 shows overall control structure of the game program as described above. Referring toFIG. 16, when the game program is activated, it initializes the memory and registers with predetermined initial values and sets the game mode to “game mode selection” atstep360. Atstep362, the game updates the screen ofTV48 in accordance with the status of the game.
• Atstep364, the game checks the game state and goes to step366,368,370, and372, respectively, if the game is at the gamemode selection state342, beforeswing state344,ball moving state346, and scoring and resultdetermination state348, respectively. The details ofsteps366 and368 will be described later with reference toFIGS. 17 and 18, respectively.Steps370 and372 are not relevant to the present embodiment and therefore, the detailed description of them will not be given here.
• Aftersteps366,368,370, or372, the control goes to step374, where the program waits for a wait interrupt, which is issued by a timer (not shown). The timer will issue the wait interrupt at a predetermined period. Thus, the golf game repeats thesteps362 to374 for each timer interval.
• FIG. 17 shows the detailed control structure of gamemode selection step366 shown inFIG. 16. Referring toFIG. 17, atstep400, the program reads the output ofCIS146. Atstep402, the program determines whether the data indicates that any of the keys (buttons)98,100,102 and104 (FIG. 3) is pressed. If the data does not indicate that any key is pressed, the program exits this routine. If the data indicates that any of the keys is pressed, the control goes to step404.
• Atstep404, the program determines whether the key is the “UP” button98 (FIG. 3). If it is, the control goes to step414, where a variable “item_no” is decremented. The variable “item_no” indicates the selected item on the menu. If the user pushes the “UP”button98, the item that has been selected is deselected and the item just over the deselected one is newly selected. Afterstep414, the control exits this routine.
• If it is determined that the key is not the “UP”button98 atstep404, the control goes to step406 where the program determines whether the key is the “DOWN” button100 (FIG. 3). If it is, the control goes to step416; otherwise, the control goes to step408. Atstep416, the variable “item_no” is incremented. Afterstep416, the control exits this routine.
• Atstep408, the program determines whether the key is “ENTER” button104 (FIG. 3). If it is, the control goes to step418; otherwise, it goes to step410. Atstep418, the program sets the game mode, the game status, and a variable N₁to “game(item_no)”, “swing_detection”, and “0”, respectively. The game( )is an array that stores the game mode identification information. The variable “item_no” is an index into this array. By this step, the game mode is selected. Afterstep418, the control exits this routine. The variable N₁is used in the swing detection process atstep368, the details of which will be described later.
• Atstep410, the program determines whether the key is the “CANCEL” button102 (FIG. 3). If it is, the control goes to step412, where the program clears the program settings. Afterstep412, the control exits this routine. If it is determined atstep412 that the key is not the “CANCEL” button, the control exits this step immediately.
• FIG. 18 shows the control flow of the swing detection process at step368 (FIG. 16). This process determines whether the user swingscontroller42 over theswing detector44 or not.
• The process starts atstep430 where the program reads CIS data. Atstep432, the program determines whether the CIS data include key data that indicates that the user has pushed one of thebuttons98,100,102, and104. If the CIS data includes key data, the control goes to step468; otherwise, the control goes to step434.
• Atstep468, the program determines whether the key is the “CANCEL” button102 (FIG. 3). If it is, the control goes to step470 where the program sets the status of the game to “game_mode_selection” and exits this routine; otherwise, the program exits this routine immediately.
• If the CIS data does not contain the key data, the control goes to step434 where one is added to the variable N₁and the result is stored in another variable N₂. Both variables N₁and N₂are of 1-bit variables. Thus, if N₁is 0, N2 will be 1 and if N₁is 1, N₂will be 0.
• Atstep436, the coordinate X, Y contained in the CIS data are stored in array elements PX[N₂] and PY[N₂], respectively. The arrays PX[] and PY[] have two elements; PX[0] and PX[1], and PY[0] and PY[1], respectively.FIG. 19 shows an example of the movement of the center ofreflector strip124. The center position ofreflector strip124 at theprevious time552 is indicated by (PX[N₁], PY[N₁]) while the center position at the present554 is indicated by (PX[N₂], PY[N₂]).
• Atstep438, it is determined whether the game is the “angle_mode” game or not. The “angle_mode” game uses the angle θ of the clubface ofcontroller42. In other modes, the game does not use the angle θ. If the game is the “angle_mode” game, the control goes to step440, where the angle θ contained in the CIS data is stored in an array element θ[N₂] and the control goes to step442; otherwise the control immediately goes to step442. θ[] is a two-element array and its elements include θ[0] and θ[1].
• Atstep442, PX[N₂]−PX[N₁] is computed and the result is stored in an array element VX[N₂]. Likewise, the result of PY[N₁]−PY[N₂] is stored in VY[N₂]. The arrays VX[] and VY[] are two-element arrays and include VX[0] and VX[1], and VY[0] and VY[1], respectively.
• Atstep444, √{square root over ((VX[N₂])²+(VY[N₂])²)}{square root over ((VX[N₂])²+(VY[N₂])²)} is computed and the result is stored in an array element V[N₂]. The array V[] is a two-element array and includes V[0] and V[1]. V[N₂] indicates the distance the center ofreflector strip124 has moved from the last measurement. Therefore, V[N₂] also indicates the speed of the movement of the head ofcontroller42.
• Atstep446, the program determines whether the distance V[N₂] computed atstep444 is greater than a predetermined first threshold V_th1. If it is, the control goes to step452 where a flag_sthat indicates a swing has been detected is set to 1; otherwise the control goes to step448. The flag_sindicates that a swing is detected when it is 1 and indicates that the swing is not detected when it is 0.
• Atstep448, the program determines a line segmentP_N2P_N1 that connects the center point P_N2=(PX[N₂], PY[N₂]) detected this time and the center point P_N1=(PX[N₁], PY[N₁]) detected previous time. Further, the program determines whether the line segmentP_N2P_N1 crosses the preselected area at the center of the CIS image plane. If it does, the control goes to step450; otherwise, the control goes to step466.
• Atstep466, the flag_sis cleared to zero and the control goes to step464.
• Atstep450, the program determines whether the distance V[N₂] is greater than a predetermined second threshold V_th2. The second threshold V_th2is selected so that it is smaller than the first threshold V_th1used instep446. If V[N₂] is greater than the second threshold V_th2, the control goes to step452; otherwise the control goes to step466. Atstep452, the flag_sis set to 1 as described above.
• Thus, when the detected speed of the club head ofcontroller42 is greater than the first threshold V_th1, the program determines that a swing is detected while if the speed of the club head ofcontroller42 is smaller or equal to the first threshold V_th1, the program detects a swing only when the club head goes over the center point ofCIS146 and the speed is greater than the second threshold V_th2at the same time.
• Referring toFIG. 19, the program computes the slice angle θ_kin the following manner. Let us assume that the X- and Y-axes are taken as shown inFIG. 19 and that the line segmentP_N2P_N1 and thevertical line562 form an angle θ_i. Given the two coordinate data P_N1(PX[N₁], PY[N₁]) and P_N2(PX[N₂], PY[N₂]), the program can calculate the angle θ_jby
• ${\theta }_j={\tan }^{-1}\left(\frac{\mathrm{PX}\left[N_2\right]-\mathrm{PX}\left[N_1\right]}{\mathrm{PY}\left[N_1\right]-\mathrm{PY}\left[N_2\right]}\right).$
• Note that, inFIG. 19, angles are measured in a counterclockwise direction. Let us further assume thatline564 is an extension of the longer axis of theimage554 of reflector strip, and thatline560 is an extension of the line segmentP_N2P_N1. The face angle θ[N₂], which is formed byline564 and ahorizontal line566, is given byCIS146.
• The program further calculates an angle θ_kby θ_k=θ_j+α×θ[N₂], where θ_jand α×θ[N₂] correspond to θ_sand α×θ₂inFIG. 25, respectively, and α is a positive constant. Preferably, α is in the range of [0, 1). The program can use this angle θ_kto determine the trajectory of the imaginary golf ball in the angle_mode game, as described above.
• Referring again toFIG. 18, atstep454, the program computes the angle
• ${\theta }_j={\tan }^{-1}\left(\frac{\mathrm{PX}\left[N_2\right]-\mathrm{PX}\left[N_1\right]}{\mathrm{PY}\left[N_1\right]-\mathrm{PY}\left[N_2\right]}\right).$
• Atstep456, the program determines whether the game is in the “angle_mode”. If it is, the control goes to step458 where θ_kis computed by θ_k=θ_j+α×θ[N₂]; otherwise, the control goes to step460 where θ_kis set to θ_j.
• Next, atstep462, the initial velocity of the imaginary golf ball V₀is computed. Specifically, the imaginary golf ball will go in the direction that is deviated from the target direction by the angle θ_k=θ_j+α×θ[N₂].
• Atstep464, the program substitutes N₂for N₁. Afterstep464, the control exits this routine.
• In this embodiment, the angle θ_kis determined depending mainly on θ_j. Therefore, the imaginary golf ball will go substantially in the direction of the swing of thecontroller42. Thus, the player can enjoy the game without paying much attention to the club angle. If the club angle θ[N₂] is not zero, the course of the golf ball will deviate by the angle θ_kfrom the target arrow.
• Further, the term α×θ[N₂] will add a touch of sophistication to the game. However, the amount of deviation with reference to the term α×θ[N₂] will not be great and the game will not be ruined. Because the club angle will affect the course of the ball, the game will be slightly hard to play, but not too hard to be entertaining.
• Although the angle θ_kis determined by θ_k=θ_j+α×θ[N₂] in this embodiment, it may be computed based on θ_jand θ[N₂] in any way as far as the resulting game is reasonably interesting. For instance, the angle θ_kmay be determined by θ_k=0[N₂]+β×0_j(0≦β<1) instead of 0_k=0_j+α×0[N₂]. It is not necessary for the movement of the imaginary golf ball to strictly follow the physical law.
Second Embodiment
• In the first embodiment,CIS146 captures two images for each of the process cycles: the exposure data and the dark data. The outputs ofCIS146 are then down-sampled to 1-bit serial data. By computing the difference between the down-sampled images of the exposure data and the dark data,swing detector44 can detect the position and the angle of the image ofreflector strip124.
• By this arrangement, the image processing inswing detector44 is simplified and a relatively low-performance CPU or MCU can be used.
• However, in some environment where a fluctuating strong IR light source exists, there might be an error in the operation ofswing detector44 in accordance with the first embodiment. The present embodiment is an improvement to the first embodiment.
• Referring toFIG. 26, the problem caused in such an environment will be first described. Let us assume that aclock signal662 defines the sampling timings ofCIS146, and that an IR source nearCIS146 emits IR light with intensity that fluctuates as shown by thewaveform660. Let us assume thatCIS146 captures exposure data and dark data at time t₁and t₂, respectively, where the intensities of the IR light from the IR source are shown bypoints670 and672, respectively.
• Referring toFIG. 27(B), theexposure data700, when down-sampled, will include animage704 of the IR light source and animage708 ofreflector strip124. The down-sampled dark data will include animage706 of the IR light source as well.
• FIG. 27(A) shows an example of theintensity waveform682 of the CIS output of the pixels online680 shown inFIG. 27(B). Referring toFIG. 27(A),waveform682 will have twopeaks690 and692 caused by the IR light source. These peaks are unwanted and should be removed by subtracting the dark data from the exposure data.
• However, because the intensity of the IR light source is fluctuating, a problem will be caused. Theexposure data700 is captured when the intensity of the IR light is high while thedark data702 is captured when the intensity is low. Thus, the height ofpeak690 will be higher than that ofpeak692 and, accordingly, when down-sampled at thethreshold level694,image704 will be larger thanimage706. Under the circumstances, whendark data702 is subtracted fromexposure data700, the resulting image will be as shown byFIG. 27(C).
• Note that, in the first embodiment, subtraction of pixels is carried out in accordance with the rules as shown in the table 1 below. In table 1, “1” denotes a bright pixel and “0” denotes a dark pixel. Each cell shows the result of subtracting a dark data pixel value from exposure data pixel value.

• 	TABLE 1
  	Exposure
  	Data

  	0	1

  	Dark Data	0	0	1
  		1	0	0

• Referring toFIG. 27(C), the resultingimage710 includes: besidesimage708 ofreflector strip124, aresidual image714 that reflects the difference ofimages704 and706. The program might erroneously interpret theimage714 as that ofreflector strip124. If such an error occurs, the course of the swing ofcontroller42 will not be properly detected and the game will be ruined. The present embodiment deals with this problem by lowering the voltage level of the exposure data by a prescribed amount such that the image of the fluctuating IR light source in exposure data will not be larger than that of dark data.
• FIG. 28 shows a block diagram ofswing detector720 in accordance with the second embodiment of the present invention. Referring toFIG. 28,swing detector720 differs fromswing detector44 of the first embodiment in thatMCU148 ofswing detector720 outputs the IR-CTL signal shown inFIG. 8 and thatswing detector720 further includes apotential divider adjuster730 connected to receive the output VOUTS, /STR and FS signals fromCIS146, and IR-CTL signal fromMCU148 for adjusting the voltage level of VOUTS such that the voltage level of VOUTS supplied to down-sampling comparator150 is low while IR-CTL signal is at the High level, and returns to the normal level while IR-CTL signal is at the Low level.
• Referring toFIG. 29,potential divider adjuster730 of the present embodiment includes: two PNP transistors TR1 and TR3, a NPN transistor TR2, four resistors R1 (100 kΩ), R2 (100 kΩ), R3 (100 kΩ), and R4 (18 kΩ), and a capacitor C1.
• Transistor TR1 has its base connected to receive FS signal via transistor R1, its collector connected to receive IR-CTL, and its emitter connected to one end of resistor R2. Transistor TR2 has its base connected to the other end of resistor R2, its collector connected to one end of resistor R3, and its emitter grounded. The other end of resistor R3 is connected to a power supply Vcc (3.6V). Between the emitter of transistor TR1 and the ground, capacitor C1 is connected.
• Transistor TR3 has its base connected to the collector of transistor TR2, its collector grounded, and its emitter connected to one end of resistor R4. The other end of resistor R4 is connected to aline732.
• There are two resistors R6 (1 kΩ) and R5 (8.2 kΩ) having respective one ends connected toline732. Resistor R6 has its other end connected to receive VOUTS. Resistor R5 has its other end grounded.Line732 is further connected to an input of down-sampling comparator150 (FIG. 28).
• FIG. 30 describes combinations of potential levels of IR-CTL and FS, corresponding states of transistors TR1, TR2, and TR3, the potentials VA, VB, and VC, and the combined resistance RC of resistors R4 and R5 shown inFIG. 29. VA denotes the potential at the connection of the emitter of transistor TR1 and resistor R2. VB denotes the potential at the connection of the collector of transistor TR2 and resistor R3. VC denotes the potential at the connection of the emitter of TR3 and resistor R4.
• As is apparent fromFIG. 30, when IR-CTL is at the High level, transistor TR3 switches on the collector current. Current from VOUTS flows through transistors R4 and R5 to ground so that the potential VD (FIG. 29) becomes a relatively low level regardless of the level of FS. When IR-CTL is at the Low level, transistor TR3 switches off its collector current thereby causing the connecting node between resistor R4 and transistor TR3 at the high-impedance state. In this situation, the current from VOUTS flows through transistor R5 only, thereby the potential level VD will be relatively high. This change of the potential VD is shown inFIG. 31. Note that, inFIG. 31, “RC” denotes the combined resistance of resistors R4 and R5, which acts as a potential divider together with resistor R6.
• This leads to the following result. Referring toFIG. 32(A), the first half ofwaveform752 of output VOUTS ofCIS146 becomes lower than that ofwaveform682 shown inFIG. 27. The latter half will be at the same level with that ofwaveform682. Thus, thepeak760 caused by the IR light source will be lower than that ofpeak690 shown inFIG. 27, whereas thepeak692 will not change its height.
• Consequently, referring toFIG. 32(B), when the signal level is compared with thesame threshold694, theimage744 formed inexposure data740 due to peak760 will be smaller than theimage704 shown inFIG. 27 while theimage746 formed in thedark data742 remains the same in its size. If the resistances of resistors R4, R5 and R6 are appropriately selected, theimage746 will be always larger than theimage744. By subtractingimage746 fromimage744, due to the subtraction rule shown in Table 1, nothing remains as shown bydifference image770 inFIG. 32(C) thereby eliminating the problem caused by the fluctuating IR light source.
• In the alternative, thethreshold level694 with reference to the potential level of VOUTS may be raised in the former half of the IR-CTL cycles and lowered back in the latter half. This is shown by thewaveform682 and thevariable threshold780 inFIG. 33. Referring toFIG. 33, thethreshold780 is raised in the former half of the IR-CTL cycle and lowered back to its normal level in the latter half. The comparison between thewaveform682 and thethreshold780 is essentially the same with that of thewaveform752 and the fixedthreshold694 shown inFIG. 32(A). Consequently, theimage782 formed by thepeak690 will be smaller than theimage706 formed by thepeak692, thereby eliminating the problem caused by the fluctuating IR light source.
• The above-described second embodiment implements the adjustment of the potential of VOUTS by the hardware circuit (potential divider adjuster730); however, such adjustment may be implemented with a processor such as MCU and small amount of hardware.
• FIG. 34 shows another example of potential divider adjuster. InFIGS. 29 and 34, like elements are denoted with like reference numerals. Referring toFIG. 34, thispotential divider adjuster790 includes: a resistor R4 and a diode D1 connected in a series. The anode of diode D1 is connected to one end of resistor R4. The other end of resistor R4 is connected toline732. The cathode of diode D1 is connected to anoutput terminal794 ofMCU792 for adjusting the potential ofline732.
• Resistor R6 has its one end connected to receive the output of CIS146 (VOUTS) and the other end connected toline732. Resistor R5 is connected betweenline732 and the ground.
• When IR-CTL is at the High level,MCU792 outputs a Low level signal at itsoutput terminal794 to which diode D1 is connected. Diode D1 is forward biased and become conductive. Current flows through resistor R4 and diode D1, causing the combined resistance of resistors R4 and R5 to be smaller
• $\left(1/\left(\frac{1}{{\mathrm{<figure-callout\; label="R"\; filenames="US20100285874A1-20101111-D00004.png,US20100285874A1-20101111-D00022.png"\; state="\{\{state\}\}">R</figure-callout>}}_4}+\frac{1}{R_5}\right)\right),$
• thereby lowering the potential ofline730. When IR-CTL is at the Low level,MCU792 raises the potential at itsoutput794 connected to diode D1. Diode D1 will be reverse biased and will cease to be conductive. Current stops flowing through resistor R4 and diode D1, causing the complex resistance of resistors R4 and R5 to be larger (R5), thereby raising the potential ofline732 to its normal level.
• Another example of potential divider adjuster is shown inFIG. 35. InFIGS. 29,34 and35, like elements are denoted with like reference numerals. Referring toFIG. 35,potential divider adjuster800 of this example include; resistors R41 and R42, and diodes D11 and D12. The anodes of diodes D11 and D12 are connected to one ends of resistors R41 and R42, respectively. The other ends of resistors R41 and R42 are connected toline732. The cathodes of diodes D11 and D12 are connected tooutput terminals804 and806, respectively, ofMCU802 for controlling the combined resistance of resistors R41, R42, and R5. Note that resistors R41 and R42 are different from each other in this example.
• In this arrangement,MCU802 is capable of adjusting the potential ofline732 at any of four levels. WhenMCU802 outputs High level signals at itsoutput terminals804 and806, the combined resistance of resistors R41, R42 and R5 is the highest, causing the potential online732 to be the highest. WhenMCU802 outputs a Low level signals at itsterminals804 and806, the combined resistance of resistors R41, R42 and R5 will be at the lowest, thereby causing the potential ofline732 to be the lowest. WhenMCU802 outputs a High level signal at one of itsterminals804 and806, and outputs a Low level signal at the other, the potential ofline732 will be at one of two levels somewhere between the highest level and the lowest level depending on the resistance values of resistors R41, R42, and R5.
• Thus,potential divider adjuster800 is now field-programmable. Depending on the average level of the output VOUTS ofCIS146,potential divider adjuster800 can select a diode or diodes to be conductive so that error should be advantageously eliminated.
• Those skilled in the art will readily understand that any number of resistors and diodes may be used.
• Although the present invention has been described using the embodiment directed to a computer golf game, it is not limited thereto. The present invention can be applied to any kind of position detector as long as the image of the object is rectangular. Further, there is no need to use reflective strip. As long as the object can reflect a light and forms a rectangular image on the image plane of the image sensor, a detector in accordance with the present invention can detect the position and the angle of the object.
• The reflective object may be of any shape. Preferably, the reflective object may be of a symmetrical shape. The embodiments as have been described here are mere examples and should not be interpreted as restrictive. The scope of the present invention is determined by each of the claims with appropriate consideration of the written description of the embodiments and embraces modifications within the meaning of, and equivalent to, the languages in the claims.

Claims (21)

1-12. (canceled)

13. A computer game program product comprising:

a computer readable recording medium storing a computer executable game program thereon,

wherein the game program includes:

a first computer readable program code unit operable to cause the computer to generate an image of an imaginary object to be moved by a user in a game and a direction indicator indicating a target direction in which the imaginary object is to be moved by the user in the game;

a second computer readable program code unit operable to cause the computer to generate time-series position data items of a controller of the game, wherein the position data item indicates a position of a representative point of the controller with reference to a predetermined coordinate system; and

a third computer readable program code unit operable to cause the computer to compute a trajectory of the imaginary object such that a direction of movement of the imaginary object is determined using the target direction as a reference based on the time-series position data items.

14. The computer game program product as recited inclaim 13 wherein the controller has a representative surface for virtually hitting the imaginary object in the game,

wherein the game program further includes:

a fourth computer readable program code unit operable to cause the computer to generate angle data which indicates an angle of the representative surface of the controller,

wherein the third computer readable program code unit computes the trajectory of the imaginary object such that the direction of movement of the imaginary object is determined using the target direction as the reference based on the position data items and the angle data.

15. The computer game program product as recited inclaim 13 wherein the position data item is generated based on an image data by an image sensor having an image plane for capturing a series of images of the controller,

wherein the image plane has a predetermined reference axis that defines a direction of one axis of the coordinate system, and

wherein when a direction of movement of the representative point of the controller is in the direction of the predetermined reference axis, the trajectory is computed such that the imaginary object will go in the direction of the target direction.

16. The computer game program product as recited inclaim 15 wherein the position data item is generated based on a differential image between an image by the image sensor while a light emitting element is turned on and an image by the image sensor while the light emitting element is turned off.

17. The computer game program product as recited inclaim 16 wherein the controller has a retroreflective member and the position data item is generated based on an image of the retroreflective member contained in the differential image.

18. The computer game program product as recited inclaim 14 wherein the position data item and the angle data are generated based on a differential image between an image by an image sensor while a light emitting element is turned on and an image by the image sensor while the light emitting element is turned off.

19. The computer game program product as recited inclaim 18 wherein the controller has a retroreflective member, and the position data item and the angle data are generated based on an image of the retroreflective member contained in the differential image.

20. The computer game program product as recited inclaim 13 wherein an obstacle is located between a target indicated by the direction indicator and the imaginary object.

21. The computer game program product as recited inclaim 14 wherein an obstacle is located between a target indicated by the direction indicator and the imaginary object.

22. The computer game program product as recited inclaim 13 wherein the game is a golf game.

23. A game apparatus comprising:

an image generating unit operable to generate an image of an imaginary object to be moved by a user in a game and a direction indicator indicating a target direction in which the imaginary object is to be moved by the user in the game;

a controller of the game;

a position generating unit operable to generate time-series position data items of said controller, wherein the position data item indicates a position of a representative point of the controller with reference to a predetermined coordinate system; and

a trajectory computing unit operable to compute a trajectory of the imaginary object such that a direction of movement of the imaginary object is determined using the target direction as a reference based on the time-series position data items.

24. The game apparatus as claimed inclaim 23 wherein the controller has a representative surface for virtually hitting the imaginary object in the game,

wherein said game apparatus further comprising:

an angle generating unit operable to an angle data which indicates an angle of the representative surface of the controller, and

wherein said trajectory computing unit computes the trajectory such that the direction of movement of the imaginary object is determined using the target direction as the reference based on the time-series position data items and the angle data.

25. The game apparatus as claimed inclaim 23 further comprising:

an image sensor,

wherein said position generating unit generates the position data item based on an image data by said image sensor having an image plane for capturing a series of images of the controller,

wherein the image plane has a predetermined reference axis that defines a direction of one axis of the coordinate system, and

wherein when a direction of movement of the representative point of the controller is in the direction of the predetermined reference axis, the trajectory computing unit computes the trajectory such that the imaginary object will go in the direction of the target direction.

26. The game apparatus as recited inclaim 25 further comprising:

a light emitting element operable to emit light,

wherein the position generating unit generates the position data item based on a differential image between an image by the image sensor while the light emitting element emits the light and an image by the image sensor while the light emitting element does not emit the light.

27. The game apparatus as recited inclaim 26 wherein the controller has a retroreflective member, and

wherein said position generating unit generates the position data item based on an image of the retroreflective member contained in the differential image.

28. The game apparatus as recited inclaim 24 further comprising:

an image sensor; and

a light emitting element operable to emit light,

wherein the position generating unit generates the position data item based on a differential image between an image by the image sensor while the light emitting element emits the light and an image by the image sensor while the light emitting element does not emit the light, and

wherein the angle generating unit generates the angle data based on the differential image.

29. The game apparatus as recited inclaim 28 wherein the controller has a retroreflective member,

wherein the position generating unit generates the position data item based on an image of the retroreflective member contained in the differential image, and

wherein the angle generating unit generates the angle data based on the image of the retroreflective member contained in the differential image.

30. The game apparatus as recited inclaim 23 wherein an obstacle is located between a target indicated by the direction indicator and the imaginary object.

31. The game apparatus as recited inclaim 24 wherein an obstacle is located between a target indicated by the direction indicator and the imaginary object.

32. The game apparatus as recited inclaim 23 wherein the game is a golf game.

Images