REFERENCE TO RELATED APPLICATIONSThe present application claims benefit of priority from U.S. Provisional App. No. 60/916,448, filed May 7, 2007, entitled “RIOT,” herein incorporated by reference in its entirety.
FIELD OF THE INVENTIONThe present invention relates generally to methods supporting predictive text entry in software applications.
BACKGROUND OF THE INVENTIONFor many consumer devices such as, for example, digital televisions or TVs attached to set-top boxes connected to cable, satellite, or fiber networks; phones or PDAs connected to wired or wireless networks; GPS navigation systems; automated teller machines; and kiosks; text entry is sometimes required by applications running on these devices. Such devices, however, frequently do not provide full keyboards and large display screens as typically provided by personal computers due to other design requirements, such as, for example, keeping the devices small, reliable, and low cost. As a result, text input methods are often slow, error prone, and not well accepted by users.
Some devices provide for entry of text using non-keyboard devices, often in conjunction with small-format displays or other alternative user feedback mechanisms. One class of techniques use small format keypads, most frequently 9-, 10-, or 12-key pads as found on phones wherein digit keys also map to multiple alpha characters, for example,digits 2, 3, 4, 5, 6, 7, 8, 9, and 0 map to characters ABC, DEF, GHI, JKL, MNO, PQRS, TUV, WXYZ, and “space” respectively. Some practitioners have explored similar use of five-key keypads, where four keys are each mapped to six or more characters and a fifth used as a space/next key. Still others have investigated use of five-key keypads, where four keys are used to navigate a cursor (i.e., highlighted symbol) over a grid of symbols, and a fifth key is used to enter the currently highlighted symbol.
The difficulty of entering text into devices where there is not a one-to-one correspondence between input keys and symbols has led to the use of algorithms that attempt to guess what the user is attempting to type, and thus minimize the number of physical keystrokes.
There is a need for a predictive text entry system and method which may provide one or more of the following: (a) The utilization of simple text entry devices, such as the pervasive 5-key directional keypad, in a way that minimizes or reduces the number of key activations required, (b) Usable by multiple unrelated applications (e.g., as a software widget, component, class, or shared function), (c) Tailoring the predictive capability of the widget for multiple distinct input contexts without knowledge of the input contexts when the widget is constructed, and (d) Enhancing the predictive accuracy of the widget in distinct input contexts over time by incorporating information about text previously entered into future predictions in the identical input context.
SUMMARY OF THE INVENTIONCertain embodiments according to the present disclosure address one or more of the aforementioned problems. For example, certain embodiments may address one or more of the aforementioned problems by providing a system and method for sharing a common, predictive, software text-entry widget or component within a consumer device across multiple software applications and input contexts. The method may include: a software application invoking an instance of a text-entry widget in a particular input context, the application optionally providing the widget a description of allowed symbols and a dictionary of expected symbol strings associated with the current context, the widget modifying a virtual keyboard display and predictive algorithm data according to the allowed symbols and dictionary, the user entering text via the widget, the widget returning the entered text to the application, and the application optionally including information derived from entered text in the associated dictionary to enhance the predictive capability of the widget on future invocations. Suitable dictionary and predictive algorithm data may consist of symbol frequency statistics and expected symbol strings maintained in a format appropriate to the prediction algorithm employed. The choice of dictionary and allowed symbols may be selected as applications are constructed and input contexts are defined. The storage, sharing, and modification of dictionaries may be performed by applications as appropriate to each input context, and may not be performed by the text-entry widget itself.
The text-entry widget may be a software function, class, or component provided in a dynamically linked shared library accessible to all applications executing on the consumer device, or it may be provided in a statically linked library, or may be provided as source code incorporated into the object code of applications executing on the device. A dictionary may be accessible to all applications executing on the device that require text input for the same context.
Features of a virtual keyboard are disclosed which may work in conjunction with various kinds of input devices to minimize the number of key activations required to enter text aided by underlying text prediction algorithms. Suitable input devices may include, but are not limited to, five-key directional keypads (located on a TV remote control for example), a twelve-key telephone keypad, an iPod®-style thumb wheel and selection switch, a touch sensitive switch network overlaying the virtual keyboard, a touch-sensitive mouse pad, and a microphone with speech recognition software responsive to spoken navigation and selection commands.
Consumer devices that may benefit from the method of the present invention may include, but are not limited to, digital televisions, mobile phones, Global Positioning System (“GPS”) navigation devices, personal computers, automated teller machines, and self-service kiosks.
BRIEF DESCRIPTION OF THE DRAWINGSIn order to facilitate a fuller understanding of the invention, reference is now made to the accompanying drawings, in which like elements are referenced with like numerals. These drawings should not be construed as limiting the present disclosure, but are intended to be exemplary only.
FIG. 1 is a diagram depicting a configuration of a Consumer Device in accordance with an embodiment of the invention.
FIG. 2 is a diagram depicting a Virtual Keyboard showing a set of allowed symbols and controls in accordance with an embodiment of the invention.
FIG. 3 is a diagram depicting a Virtual Keyboard showing another set of allowed symbols and controls in accordance with an embodiment of the invention.
FIG. 4 is a diagram depicting a Virtual Keyboard showing keys provided for containing the top five predicted strings and the top six predicted next symbols in accordance with an embodiment of the invention.
FIG. 5 is a diagram depicting the first in a series of exemplary screens illustrating an exemplary use of a Virtual Keyboard in accordance with an embodiment of the invention.
FIG. 6 is a diagram depicting the second in a series of exemplary screens illustrating an exemplary use of a Virtual Keyboard in accordance with an embodiment of the invention.
FIG. 7 is a diagram depicting the third in a series of exemplary screens illustrating an exemplary use of a Virtual Keyboard in accordance with an embodiment of the invention.
FIG. 8 is a diagram depicting the fourth in a series of exemplary screens illustrating an exemplary use of a Virtual Keyboard in accordance with an embodiment of the invention.
FIG. 9 is a diagram depicting the fifth in a series of exemplary screens illustrating an exemplary use of a Virtual Keyboard in accordance with an embodiment of the invention.
FIG. 10 is a diagram depicting the last in a series of exemplary screens illustrating an exemplary use of a Virtual Keyboard in accordance with an embodiment of the invention.
FIG. 11 is a flowchart depicting a method in accordance with an embodiment of the invention.
FIG. 12 is a flowchart depicting sub-steps ofstep740 depicted inFIG. 11, including the step where a Predictive Text-Entry Widget receives user input and returns an entered string to the requesting application in accordance with an embodiment of the invention.
DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTSThe following description is intended to convey an understanding of the invention by providing a number of specific embodiments and details involving various applications of the invention. It is understood, however, that the invention is not limited to these embodiments and details, which are exemplary only. It is further understood that one possessing ordinary skill in the art, in light of known systems and methods, would appreciate the use of the invention for its intended purposes and benefits in any number of alternative embodiments, depending upon specific design and other needs.
Referring now toFIG. 1, there is provided a diagram depicting a configuration of a Consumer Device in accordance with an embodiment of the invention.Consumer Devices100 appropriate to the present invention may include, but are not limited to, set-top boxes, mobile phones, personal digital assistants, GPS navigation devices, personal computers, and self-service kiosks. The present invention may be applicable when there are one ormore software Applications110 that use text entry executing via a computer or other electronic system provided by the device (e.g., a CPU, not depicted).Such Applications110 may include, but are not limited to, applications for accessing on-line yellow pages, receiving weather forecasts, managing Fantasy Sports teams, participating in on-line auctions, participating in votes or polls, searching the Internet, and engaging in email or chat. Components internal to theConsumer Device100 may be interconnected via an inter- and/orintra-application communication channel140, which may include, for example, physical interconnections on a chip, wiring in a computer circuit, or network interconnections between computing devices. ANetwork Interface175 may be provided to enableApplications110 to access data on anexternal network195.
The Text-Entry Widget120 is software that may provide predictive text entry services forApplications110. The Text-Entry Widget120 may also be known as a Text-Entry component. It may be implemented, for example, withinApplications110, within a library that is statically linked intoApplications110, within aLibrary130 that is dynamically linked into and shared byApplications110 as needed, or as a stand-alone application. The Text-Entry Widget120 may display a Virtual Keyboard on aDisplay Device170 connected via a wired orwireless interconnection155 to aDisplay Driver150. TheDisplay Device170 may be contained within theConsumer Device100 or may be provided as a separate unit. Exemplary embodiments of Virtual Keyboards are depicted inFIGS. 2,3,4,6,7,8, and9. The Text-Entry Widget120 may interpret key input from anInput Device180 connected via a wired orwireless interconnection165 to anInput Driver160. TheInput Device180 may be contained within theConsumer Device100, or may be provided as a separate unit. The Text-Entry Widget120 may receive a dictionary containing expected text strings and a description of allowed symbols from eachcalling Application110, and may use dictionaries and symbol descriptions to augment default information and statistics employed by a text prediction algorithm. The descriptions of allowed symbols may be, for example, lists of symbols, links to predefined named subsets of symbols (e.g., alpha, numeric, symbols, capital alphas, etc.), names represented as integer power of 2 codes wherein all subsets of symbols appropriate to a particular context are passed as the sum of the name codes, and other embodiments as appropriate to a particular embodiment of the invention. The Text-Entry Widget120 may apply the prediction algorithm as the user enters each symbol to predict the next symbols and strings most likely to be entered by the user. The symbols and/or strings may be assigned to keys displayed on the Virtual Keyboard by the Test-Entry Widget120 for easy activation by the user. An exemplary depiction of assignments is provided inFIG. 4, where the most likely five predicted strings410 (i.e.,Option 1 through Option 5) and most likely six predictednext symbols420 are provided by theVirtual Keyboard200.
AnInput Device180 may consist of or contain a 5-KeyDirectional Keypad185 that may provide four input keys associated with navigating a displayed cursor or highlight indicator UP, DOWN, LEFT, or RIGHT between keys on a Virtual Keyboard, and one key (depicted here in the center) to SELECT the highlighted item on the Virtual Keyboard. AnInput Device180 may also consist of or contain aNumeric Keypad190 which may provide, for example, a 4×3 grid of input keys associated with digits and symbols.Keys 2, 3, 6, 8, and 5 may be associated with UP, LEFT, RIGHT, DOWN, and SELECT functions or commands as another way of implementing a 5-Key Directional Keypad.Other Input Devices180 suitable for the present invention may include, but are not limited to, an iPod®-style thumb-wheel and selection switch, a touch sensitive switch network overlaying the Virtual Keyboard, a touch-sensitive mouse pad using touch-and-drag to navigate and tap to select and/or a dedicated selection switch, or a microphone with associated voice recognition software recognizing commands of UP, DOWN, LEFT, RIGHT, and ENTER. Any input device allowing navigation between keys on a Virtual Keyboard and selection of the highlighted key therein may be suitable for predictive text entry disclosed in the present invention.
Other navigation and selection paradigms may also be suitable for the method of the present invention. Another example is the use of aNumeric Keypad190 provided on telephones, wheredigit keys 2, 3, 4, 5, 6, 7, 8, and 9 are additionally labeled by letters of the alphabet (e.g., ABC, DEF, GHI, JKL, MNO, PQRS, TUV, and WXYZ respectively). In this embodiment, subsequent activations of the digit keys may be used by the Text-Entry Widget120 to make predictions of the most likely next symbols and strings, where the 3 or 4 letters associated with the activated key may be used to predict and display the next most likely symbols and strings to be selected by the user. In this embodiment, the user may activate the digit keys until the desired string is displayed on the Virtual Keyboard and the selection is made, or the user may use other keys on the Numeric Keypad (or other provided keys) to navigate and select between multiple suggested symbols and/or strings.
Referring now toFIGS. 2,3, and4, there are depicted three exemplary embodiments of aVirtual Keyboard200 according to an embodiment of the invention.FIG. 2 depicts an embodiment where the allowed set of symbols pertinent to anexemplary Application110 include the letters a-z, a space symbol, and numeric and other symbols referenced bylabels220, contained in the left and central key groups of the figure. Other depicted key types may include, for example: a “Done” key250 (which alternatively may be named “OK,” “Cancel,” “Quit,” etc.), Input Modifier keys240 (e.g., “Clear,” “Delete”), and Symbol Change keys230 (e.g., upper case/lower case Shift, 123-SYM). These are collectively referred to as “control keys.” Thecurrent text210 entered by the user may be contained in a textentry display area205. Acursor215 may denote the location where new symbols and strings are predicted by the Text-Entry Widget120 and where newly activated symbols or strings may be added upon the user activating the SELECT key. This exemplary embodiment may not provide separate keys containing the most likely strings and symbols. Instead, this exemplary embodiment may display only the most likely next symbol or string highlighted indisplay area205 at thecursor position215, and may allow the user to select that suggestion by activating the SELECT function or to select other legal symbols by navigating to the desired symbol if the predicted symbol or string is not correct.FIG. 3 depicts an embodiment of aVirtual Keyboard200 according to an embodiment of the invention where the allowed set of symbols that may be pertinent to anexemplary Application110 include the numeric and other symbols referenced bylabels220. (Alpha symbols are assumed not germane to theApplication110 embodied in this example.) Other keys that may be provided are “Done”250 (depicted in highlighted state), input modifiers “Delete” and “Clear”240, and symbol change key “ABC”230.FIG. 4 depicts an embodiment of aVirtual Keyboard200 where the allowed set of symbols pertinent to anexemplary Application110 may include the letters a-z and the space character, and keys for “Done,” “Delete,” “Clear,” “Shift,” and “123-SYM.” In addition, this embodiment may provide keys containing the top five predictedstrings410 and top six predictednext symbols420 pertinent to an exemplary application. The user may select any of these suggestions by navigating to and activating the corresponding key using theInput Device180.
Referring now toFIGS. 5 through 10, there are depicted a series of screens illustrating an exemplary use of aVirtual Keyboard200 according to an embodiment of the invention. These figures may depict interactions from a user interface perspective between anApplication110 requesting predictive text entry services and a Text-Entry Widget120 providing such services. In this example, theApplication110 may be an interactive application providing Yellow Pages on TV. InFIG. 5 there is depicted an element of theApplication Display500 wherein a user is presented with three text-entry fields to use for conducting a Yellow Pages search according to an embodiment of the invention. One such field,Category510, is highlighted in the figure. As the user prepares to enter text relevant to a Yellow Pages Category, the Text-Entry Widget120 may be invoked. Unseen by the user, the Yellow Pages application may have passed parameters to the Text-Entry Widget120 including, for example, a dictionary of common Yellow Pages category names and associated frequency information, and a description of allowed symbols, in this case alphanumeric characters. Upon receipt of these parameters, the Text-Entry Widget120 may select the appropriate display format to represent the set of allowed symbols and may integrate information derived from the provided dictionary into its internal predictive algorithm data, such as, for example, a trie data structure. Thereafter the Text-Entry Widget120 may present the selectedVirtual Keyboard200 to the user as depicted inFIG. 6. The text entry area may initially be empty, all appropriate (legal) symbols and controls may be provided, no string suggestions may be provided in Suggestions keys, and the most common six symbols appearing as the first character in Yellow Pages Category names (in this case: r, s, t, e, l, and n) may be mapped to the symbol keys along the top of theVirtual Keyboard200. InFIG. 7, the user may have activated the “a” key in the Virtual Keyboard. Upon this activation, the Text-Entry Widget120 may calculate the most likely strings and most likely second symbols the user may want to enter. These strings and symbols are shown populating the respective Suggestions keys. In this case, four strings are suggested: “auto,” “automobile,” “attorney,” and “aircraft,” and three next symbols are suggested: “u,” “t,” and “i.” The user may be free to activate any of these Suggestions keys or choose any other character from the symbol keys provided by theVirtual Keyboard200. InFIG. 8, the user may have activated the “t” Suggestions key410 twice. This may have caused thetext input area205 to contain the text “att”210 and now “attorney” is thestring Suggestion420. InFIG. 9, it is depicted the user may have activated the string suggestion key containing “attorney,” the text “attorney”210 may now appear in thetext entry area205, and the “Done” key250 may be highlighted assuming the user is ready to enter “attorney” as the desired category text. Finally inFIG. 10, the user may have activated the “Done” key and “attorney” may be passed to theYellow Pages Application110 by the Text-Entry Widget120 as the desired entry for theCategory field510 and thus may appear in the YellowPages Application Display500. A more detailed description of an exemplary process where a predictive Text-Entry Widget120 may receive user input and return it to anApplication110 is provided inFIG. 12.
Referring now toFIG. 11, there is depicted a flowchart depicting a method in accordance with an embodiment of the invention. The process may begin with anApplication110 executing instep700. Instep705, a test is made to determine if theApplication110 is terminating. If theApplication110 is terminating, processing stops. If the Application is not terminating,step710 may be executed. Instep710, a test may determine if theApplication110 requires text input. If this test determines that theApplication110 does not require text input, processing may continue withstep700. If the test instep710 determines that theApplication110 does require text input,step720 may be executed. Instep720, theApplication110 may select one or more dictionaries and one or more sets of allowed symbols for use in the Application's110 current input context. Current input context may refer to the text entry control (e.g., text box, window, etc.) that may have current focus in theApplication110. In the method of the invention, each such context may support a different dictionary, allowed set of symbols, and associated predictive model. Instep730, theApplication110 may invoke the predictive Text-Entry Widget120 by passing it the selected dictionary and allowed symbols. Instep740, the predictive Text-Entry Widget120 may process user input and may return one or more entered text string(s) to theApplication110. Instep750, a test may determine if enhancing the predictive capabilities of the predictive Text-Entry Widget120 may be beneficial in the input context. What may be beneficial may be predefined for each input context in theApplication110 when it is written. For example, if the input context is a field requiring entry of state abbreviations, the associated dictionary may consist of a set of 50 fixed strings. If the user enters something else, it may not be deemed beneficial and may not be added to the associated dictionary. In a Yellow Pages application example, remembering search strings entered by users in a dictionary may enhance future prediction. Thus such strings may be added to the associated dictionary, or frequency statistics may be modified for search strings already in the dictionary. Similarly, an IM application, as another example, may want to break up entered text into separate words and add new ones to the dictionary. Decisions about whether to allow additions to a dictionary, and what to add, may be defined when the application is written and may be independent of the Text-Entry Widget120. Continuing with the flow diagram inFIG. 11, if the test instep750 determines that enhancing the predictive capabilities of the predictive Text-Entry Widget120 may not be beneficial in the input context, processing may continue withstep700. If the test determines that enhancing the predictive capabilities of the predictive Text-Entry Widget120 may be beneficial in the input context, step760 may be executed wherein theApplication110 may update the selected dictionary with the text entered by the user and processing may continue withstep700.
Referring now toFIG. 12, there is depicted a flowchart illustrating sub-steps ofstep740 inFIG. 11 wherein a predictive Text-Entry Widget processes user input and returns the entered text string to the requesting application in accordance with an embodiment of the invention. The process may begin instep800 where the predictive Text-Entry Widget120 may be invoked by receiving the dictionary and allowed symbols provided by thecalling Application110. Instep805, the predictive Text-Entry Widget120 may draw an initialVirtual Keyboard200 on theDisplay Device170. Instep807, theWidget120 may wait for the user to activate an input key utilizing anInput Device180. In step810, a test may determine if the user has activated an input key. If the user has not activated an input key, processing may continue withstep807. If the user has activated an input key,step815 may be executed. Instep815, a test may determine if the input key is a navigation key, for example one of the UP, DOWN, LEFT, or RIGHT on a 5-KeyDirectional Keypad185. If the input key is a navigation key,step809 may be executed where a new key in theVirtual Keyboard200 may be highlighted, for example in the exemplary embodiments depicted inFIGS. 2 and 7, the “Done”key250 and “a” key are highlighted respectively. Then processing may continue withstep807. If the test instep815 determines that the input key is not a navigation key, it may be assumed that the input key activated by the user was theSELECT key820 and step830 may be executed. Instep830, a test may determine if a text symbol key was selected in theVirtual Keyboard200, for example,keys220 or410 inFIG. 8. If a text symbol key was selected in theVirtual Keyboard200,step855 may execute where theWidget120 may update the current input string and may redraw theVirtual Keyboard200 if necessary. Then step860 may be executed where theWidget120 may update predicted symbols for the currently entered text and may redraw theVirtual Keyboard200 if necessary, and processing may continue withstep807. If the test instep830 determines that a text symbol key was not selected in theVirtual Keyboard200,step835 may be executed. Instep835, a test may determine if one of the keys corresponding to one of the top predicted strings (or expected string) was selected in theVirtual Keyboard200, for example, one ofSuggestions keys410 inFIG. 4. If one of the keys corresponding to one of the top predicted strings was selected in theVirtual Keyboard200, processing may continue withstep855. If one of the keys corresponding to one of the top predicted strings was not selected in theVirtual Keyboard200,step840 may be executed. Instep840, a test may determine if a key corresponding to one of theInput Modifier Keys240 has been selected in theVirtual Keyboard200. If a key corresponding to one of theInput Modifier Keys240 has been selected in theVirtual Keyboard200,step865 may be executed where theWidget120 may update the current enteredtext210 according to the selectedInput Modifier Key240 and processing may continue withstep807. If the test instep840 determines that a key corresponding to one of theInput Modifier Keys240 has not been selected in theVirtual Keyboard200,step845 may be executed. Instep845, a test may determine if aSymbol Change Key230 has been selected in theVirtual Keyboard200. If aSymbol Change Key230 has been selected in theVirtual Keyboard200,step870 may be executed where theWidget120 may update theVirtual Keyboard200 to display selected subset of symbols and processing may continue withstep807. If the test instep845 determines that aSymbol Change Key230 has not been selected in theVirtual Keyboard200,step850 may be executed where the “Done” key may have been selected in theVirtual Keyboard200. TheWidget120 may erase theVirtual Keyboard200, may return the current entered text to thecalling Application110, and may terminate itself.