CROSS REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. provisional application 60/681,007, Computer Interface Methods and Apparatuses,“filed May 13, 2005, incorporated herein by reference.
The present invention relates to methods and apparatuses related to user interaction with computer-simulated objects, and more specifically to methods and apparatuses related to force feedback in user interaction with different object behavior dependent on whether or not a user is holding or controlling the object.
Computing technology has seen a many-fold increase in capability in recent years. Processors work at ever higher rates; memories are ever larger and always faster; mass storage is larger and cheaper every year. Computers now are essential elements in many aspects of life, and are often used to present three-dimensional worlds to users, in everything from games to scientific visualization.
The interface between the user and the computer has not seen the same rate of change. Screen windows, keyboard, monitor, and mouse are the standard, and have seen little change since their introduction. Many computers are purchased with great study as to processor speed, memory size, and disk space. Often, little thought is given to the human-computer interface, although most of the user's experience with the computer will be dominated by the interface (rarely does a user spend significant time waiting for a computer to calculate, while every interaction must use the human-computer interface).
As computers continue to increase in capability, the human-computer interface becomes increasingly important. The effective bandwidth of communication with the user will not be sufficient using only the traditional mouse and keyboard for input and monitor and speakers for output. More capable interface support will be desired to accommodate more complex and demanding applications. For example, six degree of freedom input devices, force and tactile feedback devices, three dimensional sound, and stereo or holographic displays can improve the human-computer interface.
As these new interface capabilities become available, new interface methods are needed to fully utilize new modes of human-computer communication enabled. Specifically, new methods of interaction can use the additional human-computer communication paths to supplement or supplant conventional communication paths, freeing up traditional keyboard input and visual feedback bandwidth. The use of force feedback, or haptics, can be especially useful in allowing a user to feel parts of the interface, reducing the need for a user to visually manage interface characteristics that can be managed by feel. Users interfacing with non-computer tasks routinely exploit the combination of visual and haptic feedback (seeing one side of a task while feeling the other); bringing this sensory combination into human-computer interfaces can make such interfaces more efficient and more intuitive for the user. Accordingly, there is a need for new methods of human-computer interfacing that make appropriate use of haptic and visual feedback.
As a specific example, many contemporary computer games require the user to throw or otherwise propel an object. The games typically allow the user to specify a throw by pressing a button or combination of buttons. The timing of the button press, often relative to the timing of other actions occurring in the game, controls the affect of the throw (e.g., the accuracy or distance of the throw). Some games display a slider or power bar that indicates direction or force of a throw; the user must press the appropriate button when the slider or bar is at the right value for the desired throw. The user can thereby control aspects of the throw, but not with any of the skills learned in real world throwing. Specifically, the direction of the user's hand motion and the force applied by the user near the release of the throw generally are not significant to the throwing action in the game. Also, the object being thrown is generally represented within the game independent of whether it is being held or has been released, forcing the user to adjust the control of the object to the constraints of the simulations in the game. These limitations in current approaches can produce unrealistic and difficult to learn game interaction.
The present invention can provide a method of providing user interaction with a computer representation of a simulated object, where the user can control the object in three dimensions. The method can provide for two distinct states: a “holding” state, and a “released” state. The holding state roughly corresponds to the user holding the simulated object (although other metaphors such as holding a spring attached to the object, or controlling the object at a distance can also be suitable). The released state roughly corresponds to the user not holding the object. A simple example of the two states can include the holding, then throwing of a ball. While in the holding state, the method provides force feedback to the user representation of forces that the user might experience if the user were holding an actual object. The forces are not applied when in the released state.
The present invention can allow the user to direct transitions from the holding state to the released state (e.g., releasing the ball at the end of the throwing motion), from the released state to the holding state (e.g., picking up a ball), or both. The present invention can also provide forces that represent both pushing and pulling the simulated object. The present invention can also accommodate different haptic and visual expectations of the user by providing different interaction of the object within a simulated space in the two modes. For example, a ball can be simulated with a large mass when being held by the user, to provide significant force feedback communication to the user. Upon release, however, the ball's mass internal to the simulation can be adjusted to a smaller value, to allow the ball to move and interact with other objects on a scale more fitting to the visual display capabilities.
The present invention can also provide a method of representing a simulated object within a three dimensional computer simulation. The object can be represented within the simulation with a first set of properties when the user is directly controlling the object (e.g., holding the object), and with a second set of properties when the user is not directly controlling the object (e.g., after the user releases the object). The properties different between the two sets can comprise properties of the simulation (e.g., time or gravity forces), properties of the simulated object (e.g., mass or inertia), or a combination thereof. The simulation can comprise a simulation of real-world physics interactions, such as is supported by contemporary hardware accelerators, with the physics principles, the object properties, the environment properties, or a combination thereof, changed when the user initiates or terminates direct control of the object.
The present invention can be applied to computer game applications, where the present invention can provide for enhanced user experience in propelling an object. When the user is holding the object, the present invention can provide for a set of object and interaction forces that optimize the user experience of holding the object. When the user indicates a release of the object, the present invention can provide for a set of object and simulation properties that optimize the simulated object's behavior within the game environment. The present invention can be applied to games such as football, basketball, bowling, darts, and soccer.
The advantages and features of novelty that characterize the present invention are pointed out with particularity in the claims annexed hereto and forming a part hereof. However, for a better understanding of the invention and the methods of its making and using, reference should be made to the drawings which form a further part hereof, and to the accompanying descriptive matter in which there are illustrated and described preferred embodiments of the present invention. The description below involves specific examples; those skilled in the art will appreciate other examples from the teachings herein, and combinations of the teachings of the examples.
The present invention can provide a method of providing user interaction with a computer representation of a simulated object, where the user can control the object in three dimensions. The method can provide for two distinct states: a “holding” state, and a “released” state. The holding state roughly corresponds to the user holding the simulated object (although other metaphors such as holding a spring attached to the object, or controlling the object at a distance can also be suitable). The released state roughly corresponds to the user not holding the object. A simple example of the two states can include the holding, then throwing of a ball. While in the holding state, the method provides force feedback to the user representation of forces that the user might experience if the user were holding an actual object. The forces are not applied when in the released state.
The present invention can allow the user to direct transitions from the holding state to the released state (e.g., releasing the ball at the end of the throwing motion), from the released state to the holding state (e.g., picking up a ball), or both. The present invention can also provide forces that represent both pushing and pulling the simulated object. The present invention can also accommodate different haptic and visual expectations of the user by providing different interaction of the object within a simulated space in the two modes. For example, a ball can be simulated with a large mass when being held by the user, to provide significant force feedback communication to the user. Upon release, however, the ball's mass internal to the simulation can be adjusted to a smaller value, to allow the ball to move and interact with other objects on a scale more fitting to the visual display capabilities.
The present invention can also provide a method of representing a simulated object within a three dimensional computer simulation. The object can be represented within the simulation with a first set of properties when the user is directly controlling the object (e.g., holding the object), and with a second set of properties when the user is not directly controlling the object (e.g., after the user releases the object). The properties different between the two sets can comprise properties of the simulation (e.g., time or gravity forces), properties of the simulated object (e.g., mass or inertia), or a combination thereof. The simulation can comprise a simulation of real-world physics interactions, such as is supported by contemporary hardware accelerators, with the physics principles, the object properties, the environment properties, or a combination thereof, changed when the user initiates or terminates direct control of the object.
The present invention can be applied to computer game applications, where the present invention can provide for enhanced user experience in propelling an object. When the user is holding the object, the present invention can provide for a set of object and interaction forces that optimize the user experience of holding the object. When the user indicates a release of the object, the present invention can provide for a set of object and simulation properties that optimize the simulated object's behavior within the game environment. The present invention can be applied to games such as football, basketball, bowling, darts, and soccer.
Various terms may be referred to herein, and a discussion of their respective meanings is presented in order to facilitate understanding of the invention. Haptics is the field that studies the sense of touch. In computing, haptics refers to giving a User a sense of touch through a Haptic Device. A Haptic Device (or Device) is the mechanism that allows a User to feel virtual objects and sensations. The forces created from a Haptic Device can be controlled through motors or any other way of transferring sensations to a User. The position of a Device typically refers to the position of a handle on the Device that is held by User. Any of the algorithms described can vary depending on where the handle of the Device is within its workspace. Haptic Devices can have any number of Degrees of Freedom (DOF), and can have a different number of DOF for tracking than for forces. For example a Haptic Device can track 3 DOF (x, y, and z), and output forces in 3 DOF (x, y, and z), in which case the tracked DOF are the same as the forces DOF. As another example, a Haptic Device can track 6 DOF (x, y, and z, and rotation about x, y, and z), but only have 3 DOF (x, y, and z), in which case the tracked DOF are a superset of the forces DOF. Additionally, any of a Device's DOF can be controlled by direct movements not relative to a fixed point in space (like a standard computer mouse), controlled by direct movements relative to a fixed point in space (like a mechanical tracker, mechanically grounded to a table it is resting on, where it can only move within a limited workspace), or controlled by forces against springs, movements around pivot points, twisting or turning a handle, or another type of limited movements (joystick, spaceball, etc).
A User is a person utilizing a Haptic Device to play a game or utilize some other type of application that gives a sense of touch. The User can experience a simulation or game in ways that are consistent with a Character (described below) such that the User feels, sees, and does what the Character does. The User can also have any portion or all of the interactions with the simulation be separate from the Character. For example, the User's view (i.e. what is seen on the monitor) does not have to be lined up with a Character's view (i.e. what a Character would see given the environment and the location of the Character's eyes), whether the Character is currently being controlled or not. The User can directly feel what a Character feels, through the Haptic Device (for example, the weight of picking up an object), or the User can feel a representation of what the Character feels to varying degrees (for example a haptic representation of the Character spinning in the air). A Character is a person or object controlled by a User in a video game or application. A Character can also be a first person view and representation of the User. Characters can be simple representations described only by graphics, or they can have complex characteristics such as Inverse Kinematic equations, body mass, muscles, energy, damage levels, artificial intelligence, or can represent any type of person, animal, or object in real life in varying degrees of realism. A Character can be a complex system like a human, or it can simply be a simple geometric object like a marble in a marble controlling game. Characters and their information can be described and contained on a single computer, on multiple computers, and in online environments. Characters can interact with other Characters. A Character can be controlled by the position of a Device or a Cursor, and a Character can be any Cursor or any object.
A Cursor is a virtual object controlled by a User controlling a Haptic Device. As the User moves the Haptic Device, the Cursor can move in some relationship to the movement of the Device. Typically, though not always, the Cursor moves proportionally to the movement of the Haptic Device along each axis (x,y,z). Those movements, however, can be scaled, rotated, or skewed or modified by any other function, and can be modified in these ways differently along different axes. For example, a Cursor can be controlled through a pivot point, where a movement of the Haptic Device to the right would move the Cursor to the left (the amount of movement can depend on a simulation of where the pivot point is located with respect to the Cursor). A Cursor can be a point, a sphere, any other geometric shape, a polygonal surface, a volumetric representation, a solids model, a spline based object, or can be described in any other mathematical way. A Cursor can also be a combination of any number of those objects. The graphical, haptic, and sound representations of a Cursor can be different from each other. For example, a Cursor can be a perfect haptic sphere, but a polygonal graphical sphere.
A Cursor can be controlled directly, or can be controlled through the interactions of one or more virtual objects that interact with another virtual object or other virtual objects. For example, a Haptic Device can control a point that is connected to a sphere with a spring, where the sphere is used as the Cursor. A Cursor's movements can be constrained in the visual, haptic, audio, or other sensory domain, preventing the Cursor, or its use, from moving into a specified area. Cursor movements can be constrained by objects, algorithms, or physical stops on a Device as examples. The position of a Cursor and the forces created can be modified with any type of linear or non-linear transformation (for example, scaling in the x direction). Position can be modified through transformations, and the forces created can be modified through an inverse function to modify the perception of Cursor movements, to modify objects (such as scale, rotation, etc), or to give interesting effects to the User. A Cursor can have visual, haptic, and sound representations, and any properties of any of those three Cursor modalities can be different. For example, a Cursor can have different sound, graphic, and haptic representations. A Cursor does not need to be shown visually. Also, a Cursor can vary in any of the ways described above differently at different times. For example, a Cursor can have a consistent haptic and graphic position when not touching objects, but a different haptic and graphic position when objects are touched. A Cursor can be shown graphically when preparing to perform an action (like beginning a snowboard run, beginning a golf swing, or selecting an object), and then not shown graphically when performing the action (snowboarding, swinging a golf club, or holding an object, respectively).
A Cursor can also be a representation of a User's interaction with a Character or an action, rather than a specific object used to touch other objects. For example, a Cursor can be the object used to implement a golf swing, and control the weight and feel of the club, even though the User never actually sees the Cursor directly. A Cursor can change haptic and graphic characteristics as a function of time, or as a function of another variable. A Cursor can be any object, any Character, or control any part of a Character or object. A Cursor can be in the shape of a human hand, foot, or any other part or whole of a human, animal, or cartoon. The shape, function, and motion of a Cursor can be related to that of an equivalent or similar object in the real world. For example, a Cursor shaped like a human hand can change wrist, hand and finger positioning in order to gesture, grasp objects, or otherwise interact with object similar to how hands interact with real objects.
In video games and other computer applications that simulate throwing—such as propelling an object in an intended direction using a limb—there is a need to allow the computer User to actually feel a representation of the throwing process. This haptic experience of throwing can enable the User to more realistically control a virtual object or Character, can give a simulation a greater intensity or requirement of skill, and can make game-play more enjoyable. There are a number of applications, simulations, and games that simulate throwing, including overhand throwing (football, water polo, Quidditch, knife or axe throwing, javelin, darts, etc), sideways throwing (baseball, skipping stones, etc), underhand throwing (softball, egg toss, bowling, rugby, skeeball, horseshoes etc.), heaving (throwing a heavy boulder, log toss in celtic games, pushing another Character into the air, soccer throw-in, fish toss etc), overhand pushing (basketball, throwing something onto a roof, shotput, etc.), spinning and throwing (hammer throw, bolo, lasso, discus, etc), cross body throwing (frisbee, ring toss, etc), mechanical throwing (atlatl, lacrosse, fishing cast etc), dropping an object (tossing an object down to another person, jacks, etc), flipping an object off a finger (marbles, flipping a coin etc), and skilled tossing (juggling, baton twirling, etc). All of those applications are examples of what is meant herein by “to propel” an object.
Conventionally, the act of throwing is simulated by pressing buttons and/or moving joysticks using video game controllers. These types of interactions focus almost solely on the timing of game controller interactions and often provide the User only visual feedback. Although the timing of movements involves learning and can give a basic sense of the task, the lack of haptic feedback and the limited control buttons and joysticks offer limits the realism and more complex learning that this operation can have with3D force feedback interactions.
An interface in many games must provide the user with a method of indicating discharge of an object, for example release of a thrown ball. Conventional game interfaces use buttons or switches—unrelated to usual methods of releasing objects and consequently not a very realistic interface effect. In the real world, objects are thrown by imparting sufficient momentum to them. A haptic interface can accommodate interaction that allows intuitive release.
One or more force membranes can be presented to the user, where a force membrane is a region of the haptic space accessible by the user that imposes a force opposing motion toward the membrane as the user approaches the membrane. For example, a membrane placed in the direction of the intended target can discourage the user from releasing the object accidentally, but can allow intentional release by application of sufficient force by the user to exceed the membrane's threshold. As another example, consider a user throwing a football. The user brings the haptic interaction device back (as if to cock the arm back to throw) past a membrane, then pushes it forward (feeling the weight and drag of the football haptically), and if the user pushes the football forward fast enough to give it the required momentum, the football is thrown. Motion of the object in a throwing direction can be accompanied by a combination of dynamics forces and viscosity to guide the users movement. These forces can make directing the object thrown much easier. The forces related to the membrane can drop abruptly when the object is thrown, or can be decreased over time, depending on the desired interface characteristics. As examples, such a release mechanism can be used to throw balls or other objects (e.g., by pushing the object forward through a force barrier disposed between the user location and the target), to drop objects (e.g., by pushing the object downward through a force barrier between the user location and the floor), and to discharge weapons or blows (e.g., by pushing a representation of a weapon or character portion through a force barrier between the weapon or character portion and the target).
Other triggers can be used to effect release of the object. As an example, a membrane can apply an increasing force, and the object released when the user-applied force reaches a certain relation to the membrane's force (e.g., equals the maximum force, or is double the present force). Release can also be triggered by gesture recognition: a hand moving forward rapidly, then quickly slowing, can indicate a desired throw.
The direction of the object can be determined in various ways, some of which are discussed in more detail in U.S. provisional application 60/681,007, “Computer Interface Methods and Apparatuses,” filed May 13, 2005, incorporated herein by reference. As examples: the position, at release, pre-release, or both, can be used to set direction; the object can be modeled as attached by a spring to the cursor, and the direction of throw determined from the relative positions.
A visual indication can be combined with the haptic information to indicate status of the release; for example, an image of an arm about to throw can be displayed in connection with the image of the ball when the ball is ready to be thrown (pulled through the first membrane in the previous example). A haptic cue can also be used, for example a vibration in the device or perceptible bump in its motion.
Bimodal interface. The present invention can provide for two separate modes of object simulation, optionally selectable responsive to direction from the user. In a first mode, the user is directly controlling a simulated object. This can correspond to, as examples, the user holding the simulated object, the user holding a spring or other intermediate structure that in turn holds the object, or the user controlling the object at a distance (e.g., according to a magical levitation spell or a space-type tractor beam). In a second mode, the user is not directly controlling the simulated object. This can correspond to, as examples, the user having released a thrown object, the user dropping an object, some interaction with the simulated environment causing the user to lose control of the object, or the distance control being disrupted (e.g., the spell is broken). The object can be represented to the user and to the simulation with different interaction properties in the two states, where “interaction properties” that can be different are aspects of the user's perception of the object that change upon transition between the modes, including as examples the properties of the object in the simulation; the properties of the object as perceived by the user (e.g., texture, size); the rate of determining object characteristics (e.g., position or deformation); simulation properties (e.g., time scales, physics models); environment properties (e.g., gravitational constants, relative size of objects); and execution of a corresponding software program on a different thread or a different processor. The transition between the states can be at any point where the user associates the state transition with the release of the object, including simultaneous with the release of an object by the user or before or after such release.
“Holding” mode. In the first mode, the user can generally be allowed to hold the object, manipulate the object, and interact with the object and, through the object, with the simulated environment. After an object is touched or selected, a User can determine that the object should be held. This can be accomplished automatically (for example, an object can be automatically grabbed, when it is touched) or the object can be held or grabbed based on a User input such as a button press, a switch press, a voice input, a gesture that is recognized, or some other type of User input. When an object is held, it can have dynamics properties such as weight, inertia, momentum, or any other physical property. It can be implemented as a weight, and a User can feel the weight and reaction forces as it is moved. Objects that are held can have interactions with the environment. For example, a heavy object might need to be dragged across a virtual ground if it is too heavy to be picked up by the Character. In this case, as in real life, the User can feel the weight of the object, but that weight can be less than if the object was not touching the ground. As it is dragged across the ground, forces can be applied to the Cursor or object representing the friction or resistance of the object as it moves across the ground, bumps into things, or snags or gets caught on other virtual objects.
Objects that are held can also have forces applied to them (or directly to the Cursor) based on other virtual objects that interact with the object that is held. For example, a User might feed a virtual animal. As an apple is picked up the User might feel the weight of the apple, then, when a virtual horse bites the apple, the User can feel the apple being pulled and pushed, and the weight can be adjusted to reflect the material removed by the bite. Objects can be rotated while they are held. To rotate an object, a User can rotate the handle of the Device, or can modify the position of the Device so that the rotation of the object occurs based on a change in position of the Cursor.
Objects that are held can have other haptic characteristics. A User could hold an object that is spinning and feel the resulting forces. For example, a virtual gyroscope could create directional forces that the User would feel. A User can feel the acceleration of an object being held. For example, if a User holds a virtual firehose, the User might feel the force pushing back from the direction that firehose is spraying based on how much water is coming out, how close another virtual object is that is being sprayed, how tightly the hose is being held, how much pressure there is in the spraying, or other aspects of the simulation of the water being sprayed. If an object has its own forces, based on its representation, the User could feel them. For example, if a User is holding a popcorn popper, the User could feel the popcorn popping within it. Each individual force of a popcorn popping could be relayed to the User, or the forces could be represented through some other representation (such as a random approximation of forces that a popcorn popper would create). The forces that a User feels can have any form or representation which can represent any action or event. For example, a User might hold a virtual wand to cast spells. The User can feel any type of force through the wand, which can represent any type of spell.
The User can feel interactions of a virtual held object with another object that hits the object. For example, a User might feel the weight of a baseball bat, and a path constraint of how it can move. Then when the bat hits a baseball, that feeling can be felt by the User through a force applied to the bat or applied directly to the Cursor. If a User is holding onto a matador cape, and a virtual bull runs through the cape, the User would feel the pull of the cape against the Cursor as the bull pulls on the cape. The User might also feel the force adjusted if the cape were to rip, the feeling of the cape being pulled out of the Character's hands, the cape getting caught in the bull's horns and being pulled harder, or any other interaction with the object being held. A User might hold onto an object that is controlling the Character's movements. For example, the Character might grab onto the bumper of a moving car. Then, the User would feel the pull of the bumper relative to the movement of the Character. The feeling of pull could also be combined with other interactions such as maintaining balance. A User can feel an object interact with another object that is caught by the object. For example the User can hold a net and catch a fish. The User can feel the forces of the fish flopping within the net either directly through forces applied to the Cursor, or through forces applied to the net through the fish's actions (and therefore applied to the Cursor, through its interaction with the net).
A User can control a held object to interact with an environment. For example, A User could hold onto a fishing pole and swing it forward to cast a line out into a lake. Then a User might feel a tugging force representing a fish nibbling at the hook. The User might pull on the fishing pole to hook the fish, and then feel the weight of the fish along the line added to the weight of the pole as the fish is reeled in. While an object is held, it can be shaken. A User can feel the changing inertia of an object that is moved back and forth at different rates. A virtual object might have liquid inside it that can be felt by a User through a force representation that varies based on how the liquid moves within the object. A User might feel an object slipping in his hands. For example, if a User is pulling on a virtual rope, then the User might feel a force as the rope is pulled. Then the User could feel a lesser force if the rope slips through his hands until the rope is grabbed tighter, at which point the rope stops sliding and the User feels a larger force again. The User might also feel force variations representing the texture of an object that is sliding through his hands. An object that is held can be swung as well. The User can feel the weight of an object, and feel the force of the object change based on the relative positions of the Cursor (to which the object is attached) and the object.
“Released” mode. In the second mode, the User is no longer directly controlling or manipulating the object. The object can then interact with the remainder of the simulated environment as any other simulated object. In many applications, forces and motions that are most effective for efficient and intuitive User control or an object are not the same as forces and motions that are most suitable for the computer simulation. As an example, the physical constraints of the range of motion of an input device can be larger than the actual display, but smaller than the total simulated space. Accordingly, motions of the User to control an object that are directly mapped to motion in the simulated space can be ill-suited for efficient User interaction. As another example, forces applicable by the user and time over which such forces are applied can be different in interacting with a haptic input device than forces and times in the computer simulation. As a specific example, a user of a haptic input device might have only a few inches of motion to simulate shooting a basketball; while the simulation might represent a more complete range of travel for the shooting action. Also, the basketball experienced by the user might be more effective if the mass represented by the force feedback to the haptic device is larger than that in the simulation (moving a larger basketball mass a shorter distance, in this simplified example).
As another example, the time variable in a simulation can be different when the user is holding an object than when the object is released. Sophisticated user interaction experiences can be implemented by changing such parameters (e.g., mass, time, scale). The user can be provided with a sense of imparting energy to an object by applying a relatively large force; changing the time variable in the simulation when the user releases the object can interface the user's control of the object with the simulated environment in a manner that provides a unique user experience. Even if the properties of the simulated object are unchanged, the rate at which the object's position, velocity, or other characteristics can be different in the two modes. For example, in the holding mode, the object's position and force interactions can be updated at a rate amenable to haptics interaction (e.g., 1000 Hz). In a released state, the object's position can be updated at a rate amenable to the simulation or to the display characteristics desired (e.g., 60 Hz).
In many computer simulations, the behavior of an object is simulated by determining acceleration (a) by dividing the vector sum of forces determined to be acting on the object by the mass assigned to the object. The current velocity (V) of the object can be determined by adding the previously determined velocity (Vo) to the acceleration (a) times the time step (t). The position (P) of the object can be determined from the previously determined position (Po) by P=Po+Vo*t+½*a*t*t. In such a simulation, changing the effective time (t) on transition between modes can allow effective user interaction in some applications.
The separation of the two modes can be especially beneficial in a game or other simulation that relies on physics-based simulations of objects within the game. The physics simulation, sometimes accelerated with special purpose hardware, is often designed for efficient implementation of the desired game characteristics. These are often not directly translatable to characteristics desired for efficient and intuitive haptic interactions. By separation into two modes of interaction, the human-computer interface can optimize the haptic interaction; e.g., by presenting to the user's touch an object with mass, inertia, or other properties that give the user intuitive haptic control of the object. These properties can be effectively decoupled from the underlying physics simulation, so that the haptic interaction and physics simulation can be separately optimized.
The present invention can be especially useful in games where the user has experience or expectations in real world analogs to the game activity. For example, many users have experience throwing a football or shooting a basketball. The present invention can allow the haptic control of the throwing or shooting motion to be designed such that the user experiences forces and motions building on the real world experience, while the underlying game simulation can proceed at its slower iteration rate and with its game-based physics properties. Another example is a baseball game, where the speed and direction of motion of a haptic device, caused by the user in opposition to resistive forces, can be designed to provide a realistic throwing interaction. In all cases, the change from holding to released mode can be an occasion for acceleration, deceleration, error correction or amplification, or other changes suitable to the change in modes. Computer simulations of darts, bowling, and soccer throwins are also amenable for benefit from the present invention. Games with momentary contact between the user and an object can also benefit from the present invention; for example in a tennis game the time when the ball is in contact with the racket can be implemented as a holding mode; in a soccer game, the foot of a character can be holding the ball for a brief time to benefit from the present invention.
The particular sizes and equipment discussed above are cited merely to illustrate particular embodiments of the invention. It is contemplated that the use of the invention may involve components having different sizes and characteristics. It is intended that the scope of the invention be defined by the claims appended hereto.