WO2001033839A1 - Fusion of media for information sources - Google Patents

Abstract

Using little more than a control device such as a handheld infrared remote control and a viewing device such as digital 'smart' television or a standard or high definition television with a set top box, a viewer is able to enjoy various pre-recorded or televised programming and is also able to easily access supplemental content that is 'something like' the programming currently being viewed or any accessible data element in that content. The supplemental content is sensitive to the present context of the programming being displayed and is also personalized for the viewer based on the viewer's profile. The personalized supplemental content is 'across media,' meaning that it is obtained or derived from various media sources and media types, including additional information through broadcasted programs as well as Internet sources, but is presented to the viewer in a easy-to-use, consistent, complementary and uniform way. The viewer selects only those items of personalized information of interest, and the selections appear merged with the program on the viewing device, thereby presenting the information to the viewer in a timely and convenient manner. The viewer's selections are used to update the viewer's profile. A collaborative filtering database may also be used in various ways to supplement the viewer's profile.

Description

Fusion of media for information sources.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to interactive television, and more particularly to the integration of interactive television with various data sources. Description of Related Art

Interactive TV merges the mass appeal of traditional TV with the rich, interactive power of the Internet. Rudiment forms of interactive TV are available from various vendors, including WebTV Networks, Inc. of Mountain View, California, which provides receivers for accessing MSNBC interactive television. More advanced forms of interactive TV are being developed. For example, as more fully described in Backer et al., "A Framework for Interactive Television Based on Internet Standards," http://www.microsoft.com/ atvef/wpaperl .asp, April 14, 1999 (last update date), several companies have been collaborating over the past months on a framework for supporting interactive television content based largely on the existing Internet standards. This framework in fact builds on a number of components, including Internet Protocol (IP) as the primary transport for all data; Session Announcement Protocol and Session Description Protocol (SAP/SDP) for announcing streams of interactive content; Hypertext Markup Language (HTML) and related data formats as mechanisms for describing the presentation of graphics, text, video, and other multimedia; Uniform Resource Identifiers (URIs) as the mechanism for referencing all multimedia elements, including television broadcasts and other content delivered over uni-directional links; ECMAScript and the Document Object Model (i.e., JavaScript™ and JScript™) for dynamic control of HTML content and the synchronization of HTML content to broadcast video; and Multipurpose Internet Mail Extensions / Hypertext Transport Protocol (MIME/HTTP)-style headers as the representation for content meta-data. The authors believe that these six elements provide virtually all of the services necessary to design and deliver compelling interactive television content, and that only a few elements of innovation are required. A specific framework related to this effort is expressed in the Advanced Television Enhancement Forum Specification, Draft Version 1.1 r26, February 2, While advances are being made in producing and delivering television enhancements, further enhancements are needed that use existing standards and a plethora of new component technologies to provide the viewer of television programming with greater access to a variety of different content types appropriate to the viewer in a convenient and intuitive way.

Summary of the Invention

One embodiment of the present invention is a method for displaying content from multiple information sources to a viewer, comprising establishing a viewer profile for the viewer based on various interests of the viewer; presenting a program to the viewer on a viewing device; generating items of personalized information based on the program and on the viewer profile; superimposing a visual selection aid over the program, the visual selection aid comprising index symbols for the items of personalized information, the index symbols on the visual selection aid being accessible to the viewer for selection; identifying a selection of one of the index symbols by the viewer to obtain a selected one of the items of personal information; and superimposing the selected item of personalized information over the program.

Another embodiment of the present invention is a receiver apparatus for obtaining content from multiple information sources for viewing by a viewer, comprising a program detector; an program-related additional information detector; a video output coupled to the program detector for driving a viewing device to present the program to the viewer; an adaptive viewer profile database; an implicit filter coupled to the adaptive viewer profile database, the implicit filter having an input coupled to the program-related additional information detector for generating items of personalized information based on the program- related additional information and on the adaptive viewer profile database; means for superimposing a visual selection aid over the program, the visual selection aid comprising index symbols for the items of personalized information and the index symbols on the visual selection aid being accessible to the viewer for selection; means for identifying a selection of one of the index symbols by the viewer to obtain a selected one of the items of personal information; and means for superimposing the selected item of personalized information over the program.

Yet another embodiment of the present invention is a computer program product comprising a computer readable medium having program logic recorded thereon for enabling a computer-enabled apparatus to display content from multiple information sources to a viewer on a viewing device, comprising: means for establishing a viewer profile for the viewer based on various interests of the viewer; means for presenting a program to the viewer on the viewing device; means for generating items of personalized information based on the program and on the viewer profile; means for superimposing a visual selection aid over the program, the visual selection aid comprising index symbols for the items of personalized information, the index symbols on the visual selection aid being accessible to the viewer for selection; means for identifying a selection of one of the index symbols by the viewer to obtain a selected one of the items of personal information; and means for superimposing the selected item of personalized information over the program. Brief Description of the Drawings Figure 1 through Figure 6 are pictorial representations of various viewing screen displays over time, in accordance with the present invention.

Figure 7 is a block schematic diagram of a media fusion system, in accordance with the present invention.

Figure 8 is a flowchart of an illustrative operating session of the media fusion system of Figure 7, in accordance with the present invention. Detailed Description of the Preferred Embodiment

Using little more than a control device such as a handheld infrared remote control and a viewing device, which includes, for example, digital "smart" television, a standard or high definition television with a set top box, a personal computer, or even a personal viewing device having a small screen or viewing glasses and a wireless or wired

Internet connection, a viewer is able to enjoy various pre-recorded or televised programming such as movies, sitcoms, theater, musicals, dance, sports, live action, hosted auctions, feature advertisements, and so forth, and is also able to view suggestions for supplemental content that is "something like" the programming currently being viewed or any accessible data element in that content. The supplemental content is sensitive to the present context of the programming being displayed and is also personalized for the viewer based on the viewer's profile. The personalized supplemental content is "across media," meaning that it is obtained or derived from various media sources and media types. The viewer may easily select any of the personalized supplemental content, which is then presented to the viewer on the viewing device in a consistent, complementary and uniform way.

In a standard broadcast television context, for example, the media fusion system enhances a viewer's experience by suggesting items of personalized information such as program information, biographical information on any actors and characters appearing in the program, information on the music and effects used in the program, background information on the making of the program, program related references, historical data on the program (e.g. movie data), sports statistics, interactive menus, advertising information, online shopping, other statistical or detailed information, or a combination thereof that is related both to the program and to the viewer's interests. The personalized information is obtained from among the plethora of information available from different media sources might interest the viewer. The media fusion system merges the screen presentation of the suggestions as well as, in some cases, the personalized information itself with the regular broadcast programming, thereby presenting the information to the viewer in a timely and convenient manner rather than requiring the viewer to access individual pieces of information as is necessary in a conventional Web-based environment.

An illustrative example of a viewer's experience with the media fusion system is shown in Figures 1 through 3, which pertain to receiving program related information during a standard broadcast television program. Figure 1 shows a broadcast program in progress after activation of the media fusion system. Activation of the media fusion may be by manual action of the viewer or automatically. Manual activation may be in any desired manner, as by pressing a function button on a remote control or by moving an on-screen cursor to a particular location on the screen. Automatic activation may be a periodic activation or activation at predefined times such as the beginning and end of a program and during breaks in the program, for example. A cube 5 appears in the upper right corner of the screen (it may be located in any desired place on the screen) to indicate that the media fusion system is active and that three items of personalized information are made available to the viewer, as indicated by symbols on three facets of the cube 5. If desired, the cube 5 may be programmed to rotate upon command, providing up to six facets to indicate the availability of items of personalized information. Alternatively, any three dimensional shape having a multiplicity of facets may be used. Multiple cubes or other three dimensional shapes may be used if more facets are required, or older items of personalized information may be retired as newer items of personalized information become available. Two dimensional matrices and one dimensional lists may be used as an alternative to three dimensional shapes, if desired, although these are most useful as nested menus to aid the viewer in making interactive choices once an item of personalized information has been accessed. The symbol appearing on a facet to indicate that an item of personalized information is available may be an arbitrary number such as shown in Figures 1 through 3, or may be an icon such as a musical note, talking head, motion picture camera, dollar sign, exclamation mark, and so forth that suggest the general nature of the item of personalized information referenced by the facet.

The cube 5 is just one specific type of a visual selection aid. While the visual selection aid may be one dimensioned or multi-dimensioned, it basically functions to provide the viewer with a convenient visual clue for selecting items of personalized information. The items of personalized information are indexed by any suitable symbols, which are displayed to the viewer on the visual selection aid.

Although the viewer may ignore the cube 5 and continue watching the television program, typically the viewer desires certain information or wishes to take certain actions related to the current television program. This is done by "selecting" a facet of the cube 5. How a selection is made depends on the type of controller used by the viewer. For example, a selection may be made with a system that supports an on-screen cursor by moving the cursor over the desired facet and pressing the select button, and selection may be made with a system that does not support an on-screen cursor by pressing a function button and then the number corresponding to the desired facet.

Figure 2 shows the result when the viewer selects the facet labeled "1." The item of personalized information represented by facet "1" is program related information identified by the title "1. On Safari (documentary)," which appears on the screen. Because the viewer of Figure 2 regularly watches wildlife and conservation programming, the media fusion system has presented an item of information that pertains to other conservation programming. The media fusion system accomplishes this by maintaining an adaptive profile of the viewer. The ability of the viewer's adaptive profile to generate relevant and valuable recommendations and links improves with experience.

If the viewer is not interested at the moment in more information on the "On Safari" documentary, the viewer may cancel the selection in any convenient manner, such as, for example, by pressing a "cancel" button on a remote control or moving a cursor over a "cancel" icon (not shown). The number "1" remains on a face of the cube to indicate that the item of personalized information is still available if the viewer later desires to access it. The normal advertisement stream and other ancillary content continues to be presented to the viewer. However, if the viewer is interested in more information, the viewer makes a further selection in any convenient manner, such as, for example, by again selecting the facet "1" or by selecting the displayed text. Additional program related information is displayed, illustratively "Documenting a team of explorers, we find them fighting for their lives while on safari in Kenya ... . Catch this program tonight: Discovery Channel: 8 PM" as shown in Figure 3. The viewer may use this additional program related information in any desired manner, including, for example, simply remembering to tune in at 8 PM to watch the program or by initiating the automatic programming of the viewer's VCR to record the program. Since the additional program related information is superimposed over the current broadcast program, the viewer simply cancels the selection to return to the program when her use of the additional program related information is complete.

Other facets of the cube represent other types of personalized information. An illustrative example of a viewer's receiving advertising information with the media fusion system during a standard broadcast television program is shown in Figure 4 and Figure 5. As the wildlife program progresses, the viewer selects the facet of cube 5 labeled "2." Figure 4 shows the result of this selection, which is a display on the screen of "2. Landrover (advertisement)." The advertisement is based on the current program content as well as the viewer's adaptive profile, so the advertising is more likely to be of specific value to the viewer. If the viewer is not interested at the moment in more information on the

"Landrover" advertisement, the viewer may cancel the selection. The number "2" remains on a face of the cube to indicate that the item of personalized information is still available if the viewer later desires to access it. The normal advertisement stream and other ancillary content continues to be presented to the viewer. However, if the viewer is interested in more information, the viewer makes a further selection in any convenient manner, such as, for example, by again selecting the facet "2" or by selecting the displayed text. The full Landrover advertisement appears superimposed over the program, as shown in Figure 5. The full advertisement is, illustratively, "The New 4x4, the best in its category, is ready for a new type of owner. Ready for an owner like you ... ." The viewer may use this advertisement in any desired manner depending on the capabilities of the viewer's set top box, ranging from simply taking note of it for general interest to automatically contacting an on-line vendor based on the displayed information to schedule an appointment for test driving the vehicle or even to directly purchase the vehicle.

An illustrative technique for using, for example, the related program information shown in Figure 3 or the Landrover advertisement shown in Figure 5 as a gateway to other viewer activities is to enable the viewer to evoke a menu or a nested set of interactive menus by again selecting the appropriate facet or by selecting the displayed advertisement. The menu may have a number of choices, including for the related program information example a "Program VCR" choice and for the Landrover advertisement example an e-commerce choice. For some activities, display of the program is discontinued at least temporarily so that the activity screen may be displayed.

The third item of personalized information represented by the third facet of cube 5 is, for example, an article on native African wildlife. This is because the viewer frequently consults on-line reference material while watching nature programming, a behavior captured in the viewer's adaptive profile.

Other examples to illustrate on-line shopping, biographical information, and interactive programming follow.

Example 1: On-Line Shopping. A viewer is watching an advertisement for a particular model of car. The viewer is interested in additional information about the car and "selects" the image of the car. The media fusion system responds with information relevant to the viewer's interests, such as the retail price of the car in the viewer's geographical region, Web sites or dealerships from which the car may be purchased, safety information about the car (the media fusion system having "learned" that the viewer is safety-conscious from previous viewing selections), and a video of the car being driven on a test track (the media fusion system having "learned" that the viewer likes action scenes from previous viewing selections). The relevant information is obtained from a variety of sources, both local and from the Web.

Example 2: Program Related References, On-line Shopping, and Enhanced Communications. The viewer of a movie enjoys motion picture soundtracks, a behavior captured in the viewer's adaptive profile along with the viewer's frequent use of e-commerce and email.

Finding that he or she enjoys the music, the viewer notices that one of the items of personalized information made available by the media fusion system is a "current audio" selection. The viewer selects this item, and the system responds by showing the composer, artist and title. The viewer selects "further options," and is presented with a screen menu of personalized options, including, for example, a music purchase option and an email option.

These options permit the viewer to effortlessly do a number of actions common to the viewer, such as purchasing the music for herself, or informing friends of the music, or purchasing the music for a friend and informing him that it is coming. Example 3: Movie Database. The viewer is watching a movie. As the movie ends, the viewer is presented with various items of personalized information, including a suggestion to watch another movie similar to the movie just viewed.

Example 4: Detailed Information. The viewer engages in frequent stock transactions with an on-line broker, a behavior captured in the viewer's adaptive profile. During a commercial advertisement for a product from a particular company, the viewer is presented with various items of personalized information, including a financial report on the company and a link to the viewer's on-line broker.

An illustrative media fusion system, which includes various computers, local area and global networks, enabling software, and databases, is shown in Figure 7. The viewer's portion of the media fusion system, the viewer's "receiving equipment," may be implement in any of a variety of ways, including contained in a separate housing such as a set top box, built into a broadcast television, or running as an application on a computer or another type of smart appliance having an attached monitor. Broadcast programming and Internet access is available by a variety of communications techniques, including radio, cable, satellite, and phone line. Set top boxes are presently available from a variety of vendors, including, for example, General Instruments, Inc. of Horsham, Pennsylvania, and Scientific Atlanta, Inc. of Atlanta, Georgia, and even more powerful set top boxes are in development and production. The set top box may be a Java console, for example. Where the media fusion system is implemented using the Internet, suitable Internet-resident computers include, for example, servers and clients. Suitable computers, communications networks, and set top boxes are well known in the art.

On-screen results 40 (Figure 7) are displayed on a television monitor connected to the set top box. It will be appreciated that the use of a television monitor is illustrative, and that many other techniques are well known to persons of ordinary skill in the relevant arts for displaying dynamic graphical data. In this example, the on-screen results 40 include three components, a TV program component 42, a enhanced behavior component 44, and a server-based content component 46. These components originate generally from data 20. It will also be appreciated that various circuits and methods for processing dynamic graphical data, including overlaying graphical data from different sources, are well known to persons of ordinary skill in the relevant arts.

The TV program component 42 originates from a primary broadcast 22, and typically is carried by FM transmission, satellite, or cable. The TV program component 42 may also be received in other ways, such as over DSL or ISDN or other types of advanced digital communications techniques, and from prerecorded materials such as video tape, digital tape, CD-ROMs, and DVDs. Program selection is in accordance with viewer selections 50.

The enhanced behavior component 44, which includes enhanced behavior information such as links related to the content of the primary broadcast 22 and other information designed to enhance the behavior of the viewer's receiving equipment, originates from additional information 24. The additional information 24 is coordinated with the primary broadcast 22 and preferably originates with it, is context sensitive by design, and illustratively accompanies the primary broadcast 22 in the vertical blanking interval (for analog television), for example. The additional information 24 is filtered by an explicit/implicit filter 14 and the resulting enhanced behavior information 44 is superimposed over the TV program 42. The explicit/implicit filter 14 also acquires information about the televised program context from the additional information 24.

The server based content component 46 originates from various remote data sources such as servers 28 and 29, based on requests, illustratively URL or CGI instructions, from the explicit/implicit filter 14. Servers 28 and 29 may reside on a local area network, on an enterprise network, or on the Internet, for example. In the event that servers 28 and 29 reside on the Internet, they are compliant with Web protocols. The Internet connection allows many different types of information and extended connections to be provided to the viewer while a program is being viewed. Examples of types of information that a viewer may requested and that may be displayed as server based content 46 are an article about African wildlife from the New York Times Web site and detailed advertising information and vehicle specifications from the Landrover Web site. An example of a type of extended connection supported is a purchase option, by which a viewer can conveniently purchase an advertised product or service on-line.

The explicit implicit filter 14 provides filtering for information from various sources, so that only context-sensitive information that is within the scope of the viewer's profile is furnished as the enhanced behavior component 44. The performance of the explicit/implicit filter 14 is dependent on the viewer's personal profile, as defined by information contained in the adaptive viewer profile database 12, and on the viewer's selections 50. The explicit/implicit filter 14 acts on the additional information 24 by blocking enhanced behavior information that is not within the scope of the viewer's personal profile and passing enhanced behavior information that is within the scope of the viewer's personal profile. The explicit/implicit filter 14 also requests information from the servers 28 and 29 as required by the viewer's selections 50 and in accordance with the viewer's personal profile. Through implicit filtering, the adaptive viewer profile database 12 captures the personality of a particular viewer (or several viewers, if more than one viewer uses the set top box), and is progressively refined and augmented to create an adaptive profile based on the viewer's history together with marketing statistics. The personal profile develops over time, in that the system "learns" what type of information to offer to the viewer based on various criteria such as what the viewer usually selects and the amount of time that the selection holds the viewer's interest, if relevant, and adapts accordingly. For example, if advertising information rather than reference material is usually selected, the system works out a ratio to reflect the viewer's behavior and uses the ratio in responding to the viewer's current programming choice or specific request.

Explicit filtering involves building or supplementing a viewer's profile based on her answers to specific questions. For example, detecting a new viewer, the explicit/implicit filter 14 presents a pre-programmed list of questions to the viewer to establish an initial profile, thereby allowing the media fusion system to function on at least a rudimentary level initially. Explicit filtering is also useful for confirming trends identified by the explicit/implicit filter 14, thereby saving time in updating the viewer's profile, and for resolving any ambiguity that may be detected in a viewer's developed profile by posing questions to the viewer to resolve the ambiguity. Implicit filtering involves building a viewer's profile based on her selections, including television program selections, items of personalized information accessed, Web links accessed, Web sites explored, purchases made, and so forth. The personal profile of each viewer using the media fusion system is stored in an adaptive viewer profile database 12, which preferably resides on the set top box along with the explicit/implicit filter 14 and is locally accessible to the explicit/implicit filter 14. Implicit filtering may, if desired, draw from collaborative data in building a viewer's profile. For example, Figure 7 shows that the explicit implicit filter 14 has a bidirectional path to a profile mirror 27, which is associated with a collaborative filtering database 26 and a server 29. The profile mirror 27 may be another computer or another process running on the same computer as the server 29. It is not part of the collaborative filtering database 26, but rather includes profile information derived from the collaborative filtering database 26 and, if desired, from the adaptive viewer profile database 12. The server 29 illustratively has "push" capabilities, which enable the server 29 to push meaningful information to the viewer based on information from the profile mirror 27.

The collaborative filtering database 26 is a sizeable database of profiles of viewers who visit the Web sites hosted by the server 29. A number of these viewers are likely to have profiles that are similar to the profile of the viewer whose profile is maintained in the adaptive viewer profile database 12. These similar profiles are identified by comparing the viewer's profile with the collaborative filtering database 26. Differences between the viewer's profile and the similar profiles in the collaborative filtering database 26 represent possible viewer interests, and as such are useful to the viewer in a variety of ways.

One way in which the differences between the viewer's profile and the similar profiles in the collaborative filtering database 26 is useful is as part of a response to one of the viewer selections 50. In this case, the differences are furnished by the profile mirror 27 to the server 29 which pushes information to the viewer as server based content 46.

Another way in which the differences between the viewer's profile and the similar profiles in the collaborative filtering database 26 is useful is to create additional items of personalized information under the assumption that the viewer might be interested in the same things as other viewers who have similar profiles. In this case, the differences are furnished by the profile mirror 27 to the explicit/implicit filter 14 which includes symbols that index the differences on the visual selection aid as items of personalized information. If the viewer selects any of them, the selection is used to update the adaptive viewer profile database 12. Another way in which the differences between the viewer's profile and the similar profiles in the collaborative filtering database 26 is useful is to augment the profile of the viewer whose profile is maintained in the adaptive viewer profile database 12, under the assumption that the viewer will be interested in the same things as other viewers who have similar profiles. The augmentation is performed in any convenient manner. For example, one technique involves copying the adaptive viewer profile database 12 to the profile mirror 27, updating the profile mirror with appropriate information from the collaborative filtering database 26, and writing the updated viewer profile back to the adaptive viewer profile database 12. Another technique involves copying the adaptive viewer profile database 12 to the profile mirror 27 so that the differences may be identified, then furnishing the differences to the explicit/implicit filter 14 which handles updating the adaptive viewer profile database 12. The new data in the viewer's profile is used by the explicit/implicit filter 14 to generate items of personalized information, and is evaluated over time by the explicit/implicit filter 14 just as is other data in the viewer's profile.

The collaborative filtering database 26 is also useful for establishing a "starter kit," or an initial profile of data for a new user. Preferably, the new viewer is asked a number of explicit start up questions as part of the explicit filtering process by the explicit/implicit filter 14, and then the adaptive viewer profile database 12 is augmented from the collaborative filtering database 26. Suitable algorithms and software for performing data collection and filtering and for maintaining databases such as the adaptive viewer profile database 12 and the collaborative filtering database 26 is well known to persons of ordinary skill in the art. Generally known as software agents, suitable software is based on any of a variety of techniques, including Bayesian probability-based recommendation models, decision tree models, neural network models, and distance metrics. The database itself may be organized in any suitable way, including flat file, relational, and object oriented. Examples of the Bayesian probability-based recommendation model are described in, for example, the following articles, which hereby are incorporated herein in their entirety by reference: Pazzani, M., Muramatsu, J., and Billsus, D., Syskill & Webert: Identifying Interesting Web Sites, in Proceedings of the National Conference on Artificial Intelligence, Portland, Oregon, 1996; Billsus, D. and Pazzani, M., Learning Probabilistic User Models, in Workshop Notes of Machine Learning for User Modeling, Sixth International Conference on User Modeling, Chia Laguna, Sardinia, 1997; and Pazzani, M. and Billsus, D., Learning and Revising User Profiles: The Identification of Interesting Web Sites, in Machine Learning 27, 1997, pp. 313- 331. An example of collaborative filtering is the technology being researched at the Media Laboratory of the Massachusetts Institute of Technology, Cambridge, MA and commercialized by such companies as Firefly Networks, Inc. of Cambridge, MA.

To enhance the seamlessness of media fusion, the primary broadcast 22 may be provided with embedded highlights. Embedded highlights are associated with specific objects on the screen, with the location of those objects being furnished as part of the additional information 24 or in any other suitable manner. A highlight is of any suitable form, such as, for example, a colored region or tag that tracks with the movement of its associated screen object. For example, the screen object may be a sweater worn by an actor, and the tag may be a colored rectangle that is superimposed on the sweater and moves with it. The highlight may be selected by the viewer in any convenient manner, in which event information is furnished about the item and other options, such as a purchase option, are presented. If the tag is not selected, it fades from the screen.

Figure 8 is a flowchart showing an operating session of the media fusion system of Figure 7. Figure 8 is merely illustrative, since the media fusion system may be programmed in a variety of ways depending on the experiences, skill and preferences of the system architect and computer programmers. When the viewer's receiver is turned on (step 100), the viewer's receiver determines using a viewer ID and password or other such mechanism whether the viewer is a new viewer or a previous viewer. If the viewer is new, the receiver (through the explicit/implicit filter 14 of Figure 7, which preferable is resident in the receiver) poses various questions to establish a start-up profile for the new viewer (step 112). Once having established a few general traits about the new viewer, the receiver (again through the explicit/implicit filter 14) may, if desired, augment the new viewer profile from the remote collaborative filtering database 26. Alternatively, a new viewer may adopt another viewer's profile or may choose an initial profile from among a number of pre-constructed "standard" profiles. Having created a viewer profile, the receiver then generates and makes available for selection new items of personalized information (step 114). Depending on the results of the decision step 110, either a newly created viewer profile or an existing viewer profile is used in subsequent operation.

As additional information 24 becomes available through the primary broadcast 22 (step 120), the receiver updates the viewer's profile in accordance therewith (step 122) and generates and makes available for selection new items of personalized information (step 124) based on the new additional information and the viewer's updated profile. Alternatively, the additional information 24 may be stored and the adaptive viewer profile database 12 updated periodically or when the storage is nearly full.

If the viewer makes a selection (step 130), the receiver responds in an appropriate manner. For example, if the selection is a channel selection or program change (step 140), the receiver changes the channel and updates the viewer's profile accordingly (step 142), and generates and makes available for selection new items of personalized information (step 144) based, for example, on the known characteristics of the new channel (news, weather, sports, general entertainment) and the viewer's updated profile.

If the viewer's selection is of an item of personalized information or an embedded tag (step 150), the receiver updates the viewer's profile accordingly (step 152), and generates and makes available for selection personalized information and/or options (step 154) based on the viewer's selection and the viewer's updated profile. A personalized option may itself have the characteristics of an item of personalized information, in that it may be selected in a subsequent execution of step 150 and acted upon in subsequent executions of steps 152 and 154. Personalized options may include such choices as a purchase option, a chat option, a comparison shopping option, and email option, a Web site browse option, and so forth. If the viewer's selection is of such a personalized option (step 160), the receiver updates the viewer's profile accordingly (step 162) and executes the option in the appropriate manner (step 164). The various methods and apparatus described herein in some aspects enhance a viewer's viewing experience by providing easy access to relevant information, and in some aspects provide a viewer of television programming with greater access to a variety of different content types appropriate to the viewer in a convenient and intuitive way. The description and applications as set forth herein are illustrative and are not intended to limit the scope of the invention, which is defined in the following claims. Variations and modifications of the embodiments disclosed herein are possible, and practical alternatives to and equivalents of the various elements of the embodiments are known to those of ordinary skill in the art. These and other variations and modifications of the embodiments disclosed herein may be made without departing from the scope and spirit of the invention, as set forth in the following claims.

Claims

CLAIMS:

1. A method of displaying content from multiple information sources to a viewer, comprising: establishing (100, 110, 112) a viewer profile for the viewer based on various interests of the viewer; presenting a program (22) to the viewer on a viewing device (40, 42); generating (114, 124, 144) items of personalized information based on the program and on the viewer profile; superimposing a visual selection aid (5) over the program, the visual selection aid comprising index symbols (1, 2, 3) for the items of personalized information, the index symbols on the visual selection aid being accessible to the viewer for selection; identifying (150) a selection of one of the index symbols by the viewer to obtain a selected one of the items of personal information; and superimposing the selected item of personalized information (44, 46) over the program.

2. A method as in claim 1 further comprising updating (152) the viewer profile in accordance with the viewer's selection of the selected item of personalized information.

3. A method as in claim 1 further comprising augmenting the viewer profile in accordance with a collaborative data base (26).

4. A method as in claim 3 wherein the personalized information generating step comprises: generating at least one of the items of personalized information (27, 29) based on user profiles in the collaborative database (26) similar to the viewer profile (12), at least one of the index symbols on the visual selection aid being an additional index symbol therefor; identifying a selection (50) of the index symbol for the item of personalized information based on the similar user profiles in the collaborative database; updating the viewer profile (12, 14) in accordance with the viewer's selection of the item of personalized information based on the similar user profiles in the collaborative database; and superimposing the item of personalized information (46) based on the similar user profiles in the collaborative database over the program.

5. A method as in claim 1 wherein the personalized information generating step comprises: detecting additional information (24) related to the program; filtering the additional information in accordance with the viewer profile (14) to obtain filtered additional information; and generating at least some of the items of personalized information (44) from the filtered additional information.

6. A method as in claim 1 wherein the personalized information generating step comprises generating at least one of the items of personalized information from information on content available from a Web site (28, 29).

7. A method as in claim 1 wherein the selected item of personalized information comprises a reference to a Web site, further comprising: identifying a selection of the Web site reference by the viewer (50); and displaying content from the Web site on the viewing device (28, 29, 46).

8. A method as in claim 7 wherein the Web site comprises a push server (29) and a collaborative filtering database (26), the Web site displaying step comprising: identifying differences (27) between the viewer profile (12) and similar user profiles in the collaborative database (26); and displaying content pushed from the push server on the viewing device in accordance with the differences (29, 46).

9. A method as in claim 8 wherein the Web site comprises a profile mirror (27) and wherein the difference identifying step comprises: downloading the viewer profile from an adaptive viewer profile database (12) to the profile mirror (27); updating the profile mirror from the collaborative filtering database (26) to obtain an updated profile mirror (27); and pushing content from the push server (29) in accordance with the updated profile mirror for display (46) on the viewing device.

10. A method as in claim 9 further comprising uploading the updated profile mirror from the profile mirror (27) to the adaptive viewer profile database (12).

11. A receiver apparatus for obtaining content from multiple information sources (20) for viewing (40) by a viewer, comprising: a program detector (22); an program-related additional information detector (24); a video output coupled to the program detector for driving a viewing device (40) to present the program (42) to the viewer; an adaptive viewer profile database (12); an implicit filter (14) coupled to the adaptive viewer profile database, the implicit filter having an input coupled to the program-related additional information detector for generating items of personalized information based on the program-related additional information and on the adaptive viewer profile database; means (44) for superimposing a visual selection aid (5) over the program, the visual selection aid comprising index symbols (1, 2, 3) for the items of personalized information and the index symbols on the visual selection aid being accessible to the viewer for selection; means (50) for identifying a selection of one of the index symbols by the viewer to obtain a selected one of the items of personal information; and means (44, 46) for superimposing the selected item of personalized information over the program.

12. A receiver apparatus as in claim 11 further comprising means (14) for updating the adaptive viewer profile database in accordance with the viewer's selection of the selected item of personalized information.

13. A receiver apparatus as in claim 12 wherein the implicit filter further comprises: means (27, 14) for generating at least one of the items of personalized information based on user profiles in a remote collaborative database (26) similar to the viewer profile, at least one of the index symbols on the visual selection aid being an additional index symbol therefor; and means (14, 50) for updating the viewer profile in accordance with a viewer's selection of the item of personalized information based on the similar user profiles in the collaborative database.

14. A receiver apparatus as in claim 11 further comprising means (27, 14) for augmenting the adaptive viewer profile database in accordance with a collaborative data base (26).

15. A computer program product comprising a computer readable medium having program logic recorded thereon for enabling a computer-enabled apparatus to display content from multiple information sources to a viewer on a viewing device, comprising: means (100, 110, 112) for establishing a viewer profile for the viewer based on various interests of the viewer; means for presenting a program (22) to the viewer on the viewing device (40, 42); means (114, 124, 144) for generating items of personalized information based on the program and on the viewer profile; means for superimposing a visual selection aid (5) over the program, the visual selection aid comprising index symbols (1, 2, 3) for the items of personalized information, the index symbols on the visual selection aid being accessible to the viewer for selection; means (150) for identifying a selection of one of the index symbols by the viewer to obtain a selected one of the items of personal information; and means for superimposing the selected item of personalized information (44, 46) over the program.

16. A computer program product as in claim 15 further comprising means (152) for updating the viewer profile in accordance with the viewer's selection of the selected item of personalized information.

17. A computer program product as in claim 15 wherein the means for generating personalized information comprises: means for generating at least one of the items of personalized information (27, 29) based on user profiles in the collaborative database (26) similar to the viewer profile (12), at least one of the index symbols on the visual selection aid being an additional index symbol therefor; means for identifying a selection (50) of the index symbol for the item of personalized information based on the similar user profiles in the collaborative database; means for updating the viewer profile (12, 14) in accordance with the viewer's selection of the item of personalized information based on the similar user profiles in the collaborative database; and means for superimposing the item of personalized information (46) based on the similar user profiles in the collaborative database over the program.

18. A computer program product as in claim 15 further comprising means for augmenting the viewer profile in accordance with a collaborative data base (26).