US8279242B2 - Compensating for anticipated movement of a device - Google Patents

Abstract

Anticipating movement of a display device and compensating for the anticipated movement. As a user performs an activity while viewing the device, the device experiences movement. The movement is compared to pattern data associated with profiles. One of the profiles is selected based on the comparison. Based at least on compensation data associated with the profile, image data for display on the device is modified to compensate for the anticipated movement. In some embodiments, the image data is shifted horizontally and/or vertically relative to a display area of the device in the opposite direction of the anticipated movement.

Description

BACKGROUND

Mobile computing devices, such as mobile telephones and portable audio/video devices, have become increasingly popular in recent years. As the devices continue to get smaller and more portable, users have begun to operate the devices while performing various activities such as walking, jogging, running, and driving. For example, the users walk around while checking electronic mail messages. Unfortunately, the smaller form factor of the devices typically results in a smaller screen size on the device. Consequently, the effects of device movement or jitter during the activities are more pronounced, inducing eyestrain and negatively affecting operation and the user experience.

SUMMARY

Embodiments of the invention improve user interaction with a display device by anticipating movement of the device and compensating for the anticipated movement. Profiles describe patterns of movement of the device over time. Movement data generated from movement of the device is compared to the patterns. One of the profiles is selected based on the comparison. The selected profile includes compensation data for adjusting image data relative to a display area of the device in anticipation of the movement. The adjusted image data is provided to the device for display.

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an exemplary block diagram illustrating a user interacting with a computing device.

FIG. 2 is an exemplary block diagram illustrating a display device having one or more accelerometers.

FIG. 3A andFIG. 3B are exemplary block diagrams illustrating movement of a mobile computing device.

FIG. 4 is an exemplary flow chart illustrating the adjustment of image data based on the anticipated movement of the display device.

FIG. 5 is an exemplary diagram illustrating a user performing an activity while viewing the mobile computing device.

FIG. 6 is an exemplary block diagram illustrating a sequence of movement of the display device and the movement of text displayed on the device.

Corresponding reference characters indicate corresponding parts throughout the drawings.

DETAILED DESCRIPTION

Referring to the figures, embodiments of the invention automatically detect movement of adisplay device204 and stabilize an image displayed on thedisplay device204. Aspects of the invention operate to anticipate the movement and dynamically compensate for the movement to maintain a fixed line of sight between the device and auser101. For example, the image is shifted by a quantity of pixels rows or columns in a direction opposite from the anticipated movement. The compensation effectively stabilizes the image on thedisplay device204 relative to the movement to provide an improved user experience despite the movement.

While aspects of the invention are described with reference to thedisplay device204 being amobile computing device302 orother computing device102, embodiments of the invention are operable with any device with adisplay area206 subjected to movement or jitter. For example, aspects of the invention are operable with devices such as laptop computers, gaming consoles, and hand-held or vehicle-mounted navigation devices. Further, while some aspects of the invention are described with reference to one or moredigital accelerometers208 providing data on the movement of thedisplay device204, embodiments of the invention are operable with other means for determining movement data for thedisplay device204. Yet further, while aspects of the invention are described with reference to implementation in software, embodiments of the invention are operable with implementations in any form including hardware, software, firmware, or any combination thereof.

Referring again toFIG. 1, an exemplary block diagram illustrates theuser101 interacting with thecomputing device102. Thecomputing device102 includes aprocessor104 and amemory area106, or other computer-readable media. Thememory area106 stores one or more computer-executable components such as acustomization component114, aninterface component116, ajitter component116, arectifier component120, and adriver component122. Operation of these components is described with reference toFIG. 4 below.

Thememory area106 further stores one ormore profiles108 such asprofile #1 through profile #N, where N is a positive integer. Theprofiles108 describe possible movement of thecomputing device102 over time. Theprofiles108 may be associated with, for example, different activities to be performed by theuser101 such as walking, running, or jogging. Each of theprofiles108 includespattern data110 andcompensation data112. Thecompensation data112 describes a relationship betweenimage data202 and thedisplay area206 of thecomputing device102 during movement of thecomputing device102. Thecompensation data112 is applied to theimage data202 in anticipation of movement (e.g., display jitter) of thecomputing device102 to offset or compensate for the anticipated movement. Thepattern data110 includes, for example, a sequence of anticipated movements of thecomputing device102 including the direction and magnitude of each movement. Thecompensation data112 includes, for example, a sequence of adjustments to the image data202 (e.g., pixel row and/or column adjustments) to counteract the sequence of anticipated movements in thepattern data110.

Referring next toFIG. 2, an exemplary block diagram illustrates thedisplay device204 having one ormore accelerometers208. Theaccelerometers208 include, for example,accelerometer #1 through accelerometer #M, where M is a positive integer.Accelerometers208, including digital or analog accelerometers, sense the direction of tilt or a change in position of thedisplay device204 and provide acceleration data. Various types of accelerometers are known in the art such as a piezo-electric accelerometer, a piezo-resistive accelerometer, a strain gauge accelerometer, and a microelectromechanical accelerometer. Aspects of the invention are operable with these and other types ofaccelerometers208. Theaccelerometers208 are used to detect small disruptions indicating movement of thedisplay device204, such as those pathological to theuser101 walking around.

Thedisplay device204 further includes thedisplay area206 or screen. Thedisplay area206 may constitute any portion of a face of thedisplay device204. Thedisplay area206 is capable of rendering theimage data202, including video data.

Referring next toFIG. 3A andFIG. 3B, exemplary block diagrams illustrate movement of themobile computing device302.FIG. 3A illustrates a side-to-side, horizontal motion of themobile computing device302.FIG. 3B illustrates an up-and-down, vertical movement of themobile computing device302. In some embodiments (not shown), themobile computing device302 also moves towards and away from theuser101. In still other embodiments (not shown), themobile computing device302 has a yaw rotation. Embodiments of the invention are operable to compensate for motion in one or more of the three spatial dimensions or directions, including any combination of the directions within any of the dimensions. In some embodiments, there is oneaccelerometer208 providing data relating to movement in one of the three spatial dimensions. For example, to obtain three dimension movement data, themobile computing device302 includes threeaccelerometers208.

Referring next toFIG. 4, an exemplary flow chart illustrates the adjustment or re-positioning of theimage data202 based on the anticipated movement of thedisplay device204. The operations illustrated inFIG. 4 may be described with reference to the computer-executable components illustrated inFIG. 1. The components maintain a line of sight between theuser101 and thedisplay device204 during movement of thedisplay device204. Thecustomization component114 defines at402 one or more of theprofiles108. Theprofiles108 describe movement of thedisplay device204 over time for theuser101. In some embodiments, theprofiles108 are user-specific. For example, thecustomization component114 prompts theuser101 to provide physical characteristics of theuser101 on which to define theprofiles108. The physical characteristics may be used to alterdefault profiles108 stored on thedisplay device204. Alternatively, thecustomization component114 defines theprofiles108 via stop-motion cameras using tracking balls affixed theuser101 or other person. The physical characteristics include, but are not limited to, gait length, gait speed, length of arms, and length of legs.

If movement data is received or detected at404 by theinterface component116, the storedprofiles108 are accessed at406. The movement data is received from, for example, one or more of theaccelerometers208 associated with thedisplay device204. At408, the received movement data is compared to thepattern data110 associated with theprofiles108. One of theprofiles108 is identified at410 based at least on the comparison in408 (e.g., theuser101 has begun to walk around). For example, the identifiedprofile108 is theprofile108 whosepattern data110 most closely matches the movement data being received from thedisplay device204. In some embodiments, theinterface component116 calculates a time-based average of the received data. Thejitter component116 identifies theprofile108 based on the time-based average. Thejitter component116 may also calculate the anticipated movement based in part on the time-based average, and in part on thepattern data110 orcompensation data112 associated with the identifiedprofile108.

As theimage data202 is received at202, thecompensation data112 associated with the identifiedprofile108 is applied to theimage data202 to adjust theimage data202 at412. For example, thejitter component116 calculates anticipated movements of thedisplay device204 over time. The anticipated movements are measured in rows or columns of pixels, in some embodiments.

Therectifier component120 modifies theimage data202 at the corresponding times based on the anticipated movements to compensate or offset the anticipated movements to maintain the image stability relative to theuser101. In an example in which thedisplay device204 includes a standard display area, therectifier component120 modifies theimage data202 such that a portion of the modifiedimage data202 is displayed outside the standard display area. For example, the anticipated movement is calculated to be in a vertical direction. Theimage data202 is then shifted in a vertical direction (e.g., by one or more pixel rows) opposite the anticipated movement, as defined by thecompensation data112. In another example, the anticipated movement is calculated to be in a horizontal direction. Theimage data202 is then shifted in a horizontal direction (e.g., by one or more pixel columns) opposite the anticipate movement, as defined by thecompensation data112. Generally, thecompensation data112 is used to determine in which direction thedisplay device204 is moving and shift the display pixels opposite of that direction to stabilize display of the image.

Thedisplay device204 may include extra rows and columns of pixels around thedisplay area206 reserved for use with aspects of the invention (e.g., on each side of thedisplay area206—top, bottom, left, and right—for shifting the image data202). The movement data (e.g., from the accelerometers208) is used to re-position the image on thedisplay device204 making use of the extra rows and columns of pixels as appropriate.

Alternatively or in addition, existing rows and columns within thedisplay area206 are reserved for use. The quantity of rows and columns reserved varies based on the desired amount of compensation, the degree of the anticipated movements, and the degree of movement to which thedisplay device204 is prone. In some embodiments, the quantity of rows and columns is dynamically determined as the movement data is continually received by theinterface component116. For example, during vigorous activity, the displayed image may be scaled down or otherwise reduced in size to make available additional rows and columns of pixels to be used to compensate for the anticipated movements.

In some embodiments, thecompensation data112 associated with the identifiedprofile108 is altered based on the movement data continually received by the interface component116 (e.g., based on a calculated frequency of movement). For example, if thecompensation data112 dictates an image shift of three columns of pixels yet movement data received recently indicates that thedisplay device204 is generally moving less than that described by thepattern data110, theimage data202 may be shifted by less than three columns of pixels to accurately compensate for the anticipated movement. In addition, if thedevice204 has noted continuous deviations from the expected pattern data but the usage profile is indicated to remain the same, thedevice204 may update the profile with the new expected compensation data either by making permanent modifications to the stored profile data or by issuing temporary modifications that last only for the length of the current usage session.

In an embodiment in which thejitter component116 calculates an anticipated movement of thedisplay device204 away from theuser101 at a particular time, therectifier component120 modifies theimage data202 by magnifying theimage data202 to maintain a constant zoom factor for theimage data202.

Thecompensation data112 may be synchronized with the movement data from the display device204 (e.g., based on current movement data received by the interface component116) to enable the appropriate portions of thecompensation data112 to be applied to the anticipated movements of thedisplay device204. Thedriver component122 displays, or provides for display, the modifiedimage data202.

In an embodiment in which thedisplay device204 is themobile computing device302, data from at least one of theaccelerometers208 is continually received themobile computing device302. In embodiments in which power saving concerns are present, the power usage may be reduced by enabling an “auto-pilot” mode in which theaccelerometers208 are powered on for only short intervals, the data is processed, a profile selected, and then theaccelerometers208 are turned off and resources are not used evaluating the incoming data.

The received data is compared to thepattern data110 of each of theprofiles108, and one of theprofiles108 is identified (e.g., theprofile108 whosepattern data110 most closely matches the received data).Operations412 and414 inFIG. 4 are performed. Even after one of theprofiles108 has been selected, the data is continually received from theaccelerometers208 and compared to thepattern data110 to identify a change in theprofiles108. For example, theuser101 stops running and starts walking. Alternatively or in addition, thedisplay device204 receives notification of a change (e.g., via an event) or detects a change in the movement of the display device204 (e.g., as theuser101 changes the activity causing the movement of the display device204). For example, thedisplay device204 becomes connected to a vehicle or becomes disconnected from the vehicle. Another example of the event includes a speed of thedisplay device204 exceeding a threshold (e.g., theuser101 has entered a car and is driving), or a speed of thedisplay device204 falling below a threshold. Anotherprofile108 is then selected based on the received notification or detected change.

Referring next toFIG. 5, an exemplary diagram illustrates theuser101 performing an activity while viewing themobile computing device302. The activities include, for example, walking, running, jogging, or riding in a motor vehicle. Aspects of the invention operate to maintain a fixed point or line of sight between theuser101 and themobile computing device302.

Referring next toFIG. 6, an exemplary block diagram illustrates a sequence of movement of thedisplay device204. As thedisplay device204 moves upward, aspects of the invention anticipate the movement and determine a counteracting quantity of pixels by which to adjust the displayed Text downward. As thedisplay device204 moves downward, aspects of the invention anticipate the movement and determine a counteracting quantity of pixels by which to adjust the displayed Text upward.

Exemplary Operating Environment

By way of example and not limitation, computer readable media comprise computer storage media and communication media. Computer storage media store information such as computer readable instructions, data structures, program modules or other data. Communication media typically embody computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and include any information delivery media. Combinations of any of the above are also included within the scope of computer readable media.

Although described in connection with an exemplary computing system environment, embodiments of the invention are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with aspects of the invention include, but are not limited to, personal digital assistants, mobile computing devices, personal computers, server computers, hand-held or laptop devices, portable music players, multiprocessor systems, gaming consoles, microprocessor-based systems, set top boxes, programmable consumer electronics, mobile telephones, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

Embodiments of the invention may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. The computer-executable instructions may be organized into one or more computer-executable components or modules. Generally, program modules include, but are not limited to, routines, programs, objects, components, and data structures that perform particular tasks or implement particular abstract data types. Aspects of the invention may be implemented with any number and organization of such components or modules. For example, aspects of the invention are not limited to the specific computer-executable instructions or the specific components or modules illustrated in the figures and described herein. Other embodiments of the invention may include different computer-executable instructions or components having more or less functionality than illustrated and described herein.

The embodiments illustrated and described herein as well as embodiments not specifically described herein but within the scope of aspects of the invention constitute exemplary means for adjusting theimage data202 responsive to anticipated movement of themobile computing device302, and exemplary means for adjusting theimage data202 based on thecompensation data112 associated with the identified one of theprofiles108 and based on the continually received data from theaccelerometer208.

The order of execution or performance of the operations in embodiments of the invention illustrated and described herein is not essential, unless otherwise specified. That is, the operations may be performed in any order, unless otherwise specified, and embodiments of the invention may include additional or fewer operations than those disclosed herein. For example, it is contemplated that executing or performing a particular operation before, contemporaneously with, or after another operation is within the scope of aspects of the invention.

When introducing elements of aspects of the invention or the embodiments thereof, the articles “a,” “an,” “the,” and “said” are intended to mean that there are one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that there may be additional elements other than the listed elements.

Having described aspects of the invention in detail, it will be apparent that modifications and variations are possible without departing from the scope of aspects of the invention as defined in the appended claims. As various changes could be made in the above constructions, products, and methods without departing from the scope of aspects of the invention, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.

Claims (20)

1. A system for anticipating and compensating for display jitter on a mobile computing device, said system comprising:

a memory area for storing one or more profiles describing human gait, each of said profiles including pattern data and compensation data, wherein the pattern data includes a sequence of anticipated movements of the mobile computing device including a direction and magnitude of each of the anticipated movements, wherein the compensation data describes a sequence of adjustments to image data as a user of the mobile computing device walks with the mobile computing device; and

a processor programmed to:

continually receive data from at least one accelerometer in the mobile computing device;

compare the received data to the pattern data of the profiles;

identify one of the profiles based on the comparison;

receive the image data for display on the mobile computing device;

adjust the received image data based on the compensation data associated with the identified one of the profiles and based on the continually received data from the accelerometer to counteract the sequence of anticipated movements in the pattern data; and

provide the adjusted image data for display on the mobile computing device, wherein the adjusted image data compensates for anticipated jitter of the mobile computing device as the user walks with the mobile computing device.

2. The system ofclaim 1, wherein the processor is programmed to adjust the received image data by shifting the image data in a vertical direction opposite to the anticipated jitter by one or more pixel rows relative to a display area on the mobile computing device to maintain a line of sight between the user and the mobile computing device.

3. The system ofclaim 1, wherein the mobile computing device comprises a plurality of accelerometers providing data in three spatial dimensions.

4. The system ofclaim 1, wherein the processor is further programmed to calculate a frequency of movement of the mobile computing device based on the received data from the accelerometer, and wherein the processor is programmed to dynamically alter the compensation data based on the calculated frequency of movement.

5. The system ofclaim 1, further comprising means for adjusting the image data responsive to anticipated movement of the mobile computing device, and further comprising means for adjusting the image data based on the compensation data associated with the identified one of the profiles and based on the continually received data from the accelerometer.

6. A method comprising:

accessing a plurality of profiles describing anticipated movements of a display device over time, each of said profiles being associated with a different activity performed by a user, each of said profiles including pattern data and compensation data, wherein the pattern data includes a sequence of the anticipated movements of the display device, wherein the compensation data describes a relationship between image data and a display area of the display device as the display device moves over time;

receiving data from at least one accelerometer associated with the display device;

comparing the received data to the pattern data of the profiles;

identifying one of the profiles based on the comparison;

receiving image data intended for the display device;

applying the compensation data associated with the identified one of the profiles and based on the received data from the accelerometer to adjust the received image data relative to the display area to counteract the sequence of anticipated movements in the pattern data; and

providing the adjusted image data for display on the display device, wherein the adjusted image data dynamically compensates for anticipated movement of the display device over time.

7. The method ofclaim 6, wherein accessing the profiles comprises accessing the profiles describing movement of the display device comprising one or more of the following: a mobile computing device, a personal digital assistant, a portable music player, a portable gaming console, and a cellular telephone.

8. The method ofclaim 6, wherein receiving the data from the accelerometer comprises receiving the data from an accelerometer associated with a vehicle in which the display device is contained.

9. The method ofclaim 6, wherein receiving the data from the accelerometer comprises receiving data from at least three accelerometers, wherein the three accelerometers collectively provide acceleration data in three spatial dimensions.

10. The method ofclaim 6, wherein accessing the profiles comprises accessing profiles for one or more of the following activities: walking, running, jogging, and riding in a motor vehicle.

11. The method ofclaim 6, further comprising calculating an anticipated movement of the display device at a subsequent time based on the pattern data, compensation data, and received data from the accelerometer, and wherein applying the compensation data comprises applying the compensation data as a function of the calculated anticipated movement of the display device to adjust the received image data.

12. The method ofclaim 6, wherein applying the compensation data comprises vertically shifting the received image data by a quantity of rows of pixels relative to the display area in a direction opposite movement of the display device, said quantity of rows of pixels being defined by the compensation data.

13. The method ofclaim 6, wherein applying the compensation data comprises horizontally shifting the received image data by a quantity of columns of pixels relative to the display area in a direction opposite movement of the display device, said quantity of columns of pixels being defined by the compensation data.

14. The method ofclaim 6, further comprising:

receiving notification of an event associated with movement of the display device; and

selecting another one of the profiles responsive to the received notification.

15. The method ofclaim 14, wherein receiving the notification comprises receiving notification of one or more of the following: connection of the display device to a vehicle, disconnection of the display device from a vehicle, a speed of the display device exceeding a threshold, and a speed of the display device falling below a threshold.

16. One or more computer storage media having computer-executable components for maintaining a line of sight between a user and a display device during movement of the display device, said components comprising:

a customization component for defining a plurality of profiles describing movement of the display device over time for the user, each of said defined profiles being associated with a different activity performed by the user;

an interface component for receiving data describing movement of the display device;

a jitter component for identifying, based on the data received by the interface component, one of the profiles defined by the customization component and calculating an anticipated movement of the display device based on the identified profile and the data received by the interface component;

a rectifier component for modifying image data for display based on the anticipated movement calculated by the jitter component to compensate for the anticipated movement; and

a driver component for displaying the image data modified by the rectifier component.

17. The computer storage media ofclaim 16, wherein the jitter component calculates an anticipated movement of the display device away from the user, and wherein the rectifier component modifies the image data by magnifying the image data to maintain a constant zoom factor for the image data.

18. The computer storage media ofclaim 16, wherein the display device includes a standard display area, and wherein the rectifier component modifies the image data such that a portion of the modified image data is displayed outside the standard display area.

19. The computer storage media ofclaim 16, wherein the customization component prompts the user to provide physical characteristics of the user, and wherein the customization component defines the profiles based at least on the provided physical characteristics of the user.

20. The computer storage media ofclaim 16, wherein the interface component further calculates a time-based average of the received data, and wherein the jitter component calculates the anticipated movement based on the calculated time-based average.

Images