CN1784647A - Control architecture with scalable GUI for organizing, selecting and launching media items - Google Patents
Control architecture with scalable GUI for organizing, selecting and launching media itemsDownload PDFInfo
- Publication number
- CN1784647A CN1784647ACNA2004800124776ACN200480012477ACN1784647ACN 1784647 ACN1784647 ACN 1784647ACN A2004800124776 ACNA2004800124776 ACN A2004800124776ACN 200480012477 ACN200480012477 ACN 200480012477ACN 1784647 ACN1784647 ACN 1784647A
- Authority
- CN
- China
- Prior art keywords
- gui
- images
- media
- displaying
- screen
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
Description
Translated fromChinese背景技术Background technique
本发明描述了一种用于组织、选择和启动媒体项的架构(framework)。所述架构的一部分包括具有定位(point)、点击、滚动、停悬(hover)的缩放基本构件块(building block)的图形用户接口的设计和操作，更具体地说，涉及与能够与用自由空间定位遥控装置使用的媒体项关联的图形用户界面。This invention describes a framework for organizing, selecting and launching media items. Part of the architecture includes the design and operation of a graphical user interface with point, click, scroll, hover, and zoom basic building blocks, and more specifically relates to the ability to use freely A graphical user interface associated with the media item used by the remote control device for spatial positioning.
与信息的通信相关的技术在过去的几十年里有了迅速的发展。电视、蜂窝电话、互联网和光学通信技术(这里仅列出几样)结合起来将消费者淹没在可得到的信息和娱乐选择中。以电视为例，最近30年来已经引入了有线电视服务、卫星电视服务、按次计费电影和视频点播。然而，20世纪60年代的电视观众在他们的电视机上通常只能够接收到也许是四或五个无线TV频道，今天的电视观众有机会在数以百计和潜在的数以千计的节目和信息频道中选择。目前主要在旅馆等中使用的视频点播技术提供了一种用于从数以千计的电影节目单中进行家庭娱乐选择的潜能。例如TiVo股份有限公司(加利福尼亚州奥维索金街2160，95002)推出的数字视频记录(DVR)设备进一步扩大了可以得到的选择。Technologies related to the communication of information have developed rapidly over the past few decades. Television, cellular telephones, the Internet, and optical communication technologies (to name just a few) combine to overwhelm consumers with the information and entertainment options available. In the case of television, the last 30 years have introduced cable TV services, satellite TV services, pay-per-view movies, and video on demand. Whereas television viewers in the 1960s typically only had access to perhaps four or five over-the-air TV channels on their television sets, today's television viewers have the opportunity to view hundreds and potentially thousands of programs and Select from the information channel. Video-on-demand technology, currently used mainly in hotels and the like, offers a potential for home entertainment selection from thousands of movie listings. Digital video recording (DVR) devices such as TiVo, Inc. (2160 King Street, Overso, CA 95002 ) further expand the available choices.
为终端用户提供那么多的信息和内容的能力的技术给系统设计人员和服务供应商提供了机会和挑战。一个挑战是尽管终端用户通常更愿意拥有更多的选择而不愿意选择变小，但这种喜好与他们希望的即快速又简单的选择处理所相背。不幸的是，终端用户通过其来访问媒体节目的系统和界面的发展使得选择处理既不快又不简单。再次考虑电视节目的例子。在电视处于初期时，确定看哪一个节目是相对简单的事情，这是因为可选择的数目较少。人们可以查阅印制的节目指南，该节目指南被格式化为例如一系列的行和列，这些行和列显示出了在(1)邻近的电视频道、(2)被传输到这些频道的节目以及(3)日期和时间之间的对应关系。通过调节调谐钮，电视被调谐到所需的频道，并且观众观看到所选择的节目。在这之后，允许观众远距离调谐电视的遥控控制装置被引入。这种对用户-电视交互的增添产生了被称为“频道冲浪”的现像，借此，观众能够迅速地观看在大量频道中广播的短片段，从而可以快速得知在任意给定的时间什么节目是可看的。The technology's ability to provide end users with so much information and content presents both opportunities and challenges for system designers and service providers. One challenge is that while end users generally prefer to have more choices than smaller choices, this preference is at odds with their desire for quick and easy selection handling. Unfortunately, the evolution of the systems and interfaces through which end users access media programming has made the selection process neither quick nor simple. Consider again the TV show example. In the early days of television, deciding which show to watch was a relatively simple matter because the number of choices was small. One can consult a printed program guide formatted, for example, as a series of rows and columns showing programs on (1) adjacent television channels, (2) transmitted to those channels And (3) the corresponding relationship between date and time. By turning the tuning knob, the TV is tuned to the desired channel and the viewer watches the selected program. After this, remote controls were introduced that allowed viewers to tune their televisions from a distance. This addition to user-television interaction has led to a phenomenon known as "channel surfing," whereby viewers are able to quickly watch short episodes broadcast on a large number of channels, and thus quickly learn which channels are available at any given time. What programs are available to watch.
尽管存在频道的数量和可观看的内容急剧增加的事实，但通常可得到的用于电视的用户界面、控制设备选择和架构在过去的30年里没有改变太多。印制的节目指南仍然是用于传递节目信息的最流行的机制。具有上下箭头的多按钮遥控器仍然是最流行的频道/内容选择装置。设计和实现TV用户界面的人们对可得到的媒体内容的增加的反应已经成为现存的选择处理和界面对象的直接延伸。这样，印制的节目指南中的行数被增加以容纳更多的频道。遥控装置中的按钮的数目也被增加以支持附加的功能和内容处理，例如，像图1所示的那样。然而，这种方法大大增加了观众浏览可得到信息所需的时间和执行选择所需动作的复杂度。有争议的是，现存界面的麻烦的本质阻碍了一些服务(例如视频点播)的商业应用，这是由于消费者反对给在他们看来已经是太慢和太复杂的界面再增加复杂度的新服务。Despite the fact that the number of channels and viewable content has increased dramatically, the generally available user interfaces, control device options and architecture for televisions have not changed much over the past 30 years. Printed program guides remain the most popular mechanism for delivering program information. Multi-button remote controls with up and down arrows are still the most popular channel/content selection devices. The response of those designing and implementing TV user interfaces to the increase in available media content has been a direct extension of existing selection processes and interface objects. Thus, the number of lines in the printed program guide is increased to accommodate more channels. The number of buttons in the remote control is also increased to support additional functions and content handling, eg as shown in FIG. 1 . However, this approach greatly increases the time required for the viewer to browse the available information and the complexity of performing the actions required for selection. Arguably, the cumbersome nature of existing interfaces has hindered the commercial adoption of some services (such as video-on-demand) due to consumer resistance to new and complex additions to what they consider to be already too slow and complex interfaces. Serve.
除了在带宽和内容方面增加之外，技术的集成还加剧了用户界面的瓶颈问题。消费者正在积极地做出反应要选择购买集成系统而不是大量可分离组件。这种趋势的一个例子是电视/VCR/DVD的组合，其中的三个组成部分以前是三个独立的组件，如今则作为一个集成单元而被频繁购买。这种趋势会延续下去，潜在的最终结果是目前在家庭中可以找到的大部分(如果不是全部的话)通信设备将会组合在一起作为集成单元，例如，电视/VCR/DVD/互联网接入/收音机/立体声单元。即便是那些继续购买分离组件的人们也会期望上述单独组件的无缝控制和这些分立组件之间的互相交互。随着这种集成度的增长，产生了使用户界面更加复杂的潜在可能。例如，在所谓的“通用”遥控单元(例如TV遥控单元和VCR遥控单元功能的组合)被提出时，这些通用遥控单元上的按钮的数目通常地多于单个TV遥控单元或VCR遥控单元上的按钮数目。如果不能准确地找到该遥控装置中的正确按钮，这些增加了数目的按钮和功能使人除了只能在控制TV或VCR的最简单的方面外很难控制其它任何事情。许多时候，这些通用的遥控装置不能提供足够的按钮以访问某些TV所特有的许多层面上的控制或特性。在这种情况下，仍然需要原始设备的遥控单元，并且由于集成的复杂度导致的用户界面问题，所以处理多遥控的最初争论仍然存在。一些遥控单元通过增加可用专家命令编制的“软”按钮解决了这个问题。这些软按钮有时具有附随的LCD显示装置来指示它们的动作。这种方式也具有缺陷，即，如果不将视线从TV转移到遥控装置，它们就难以使用。这些遥控单元的另一个缺陷是采用了多模式以试图减少按钮个数。在这些“模式化”的通用遥控单元中，存在专门的按钮来选择该遥控装置是否与TV、DVD播放器、有线机顶盒和VCR等通信。这产生了许多使用性的问题，包括发送命令到错误的装置、迫使用户通过观看遥控装置来确信其是否处于正确的模式，并且它不能给多装置的集成提供任何的简化。这些通用遥控单元的最大好处是它可通过允许用户将用于多个装置的命令序列编制到遥控装置中来提供一定的集成度。这是一个较困难的任务，以至于许多用户雇佣职业的安装人员来对他们的通用遥控单元进行程序编制。In addition to increasing bandwidth and content, the integration of technologies has exacerbated user interface bottlenecks. Consumers are responding positively by choosing to purchase integrated systems rather than a multitude of separable components. An example of this trend is the combination TV/VCR/DVD, where the three components, formerly three separate components, are now frequently purchased as an integrated unit. This trend will continue, with the potential end result being that most, if not all, communication devices currently found in a home will be grouped together as an integrated unit, e.g. TV/VCR/DVD/Internet access/ Radio/stereo unit. Even those who continue to buy discrete components expect seamless control of the aforementioned individual components and the interplay between those discrete components. As this level of integration grows, comes the potential for more complex user interfaces. For example, when so-called "universal" remote control units (such as a combination of TV remote unit and VCR remote unit functionality) are being proposed, the number of buttons on these universal remote control units is usually more than that on a single TV remote control unit or VCR remote control unit. number of buttons. These increased numbers of buttons and functions make it difficult to control anything but the simplest aspects of controlling a TV or VCR without being able to find exactly the correct button in the remote. Many times, these universal remote controls do not provide enough buttons to access the many levels of control or features that are unique to certain TVs. In this case, the original equipment remote unit is still required, and the original argument for dealing with multiple remotes remains due to the complexity of integration resulting in user interface issues. Some remote control units solve this problem by adding "soft" buttons that can be programmed with expert commands. These soft buttons sometimes have an accompanying LCD display device to indicate their action. This approach also has the disadvantage that they are difficult to use without diverting the line of sight from the TV to the remote control. Another drawback of these remote units is the use of multiple modes in an attempt to reduce the number of buttons. In these "modular" universal remote control units, there are dedicated buttons to select whether the remote communicates with TVs, DVD players, cable set-top boxes, VCRs, etc. This creates a number of usability problems, including sending commands to the wrong device, forcing the user to look at the remote control to make sure it is in the correct mode, and it does not provide any simplification for the integration of multiple devices. The great benefit of these universal remote units is that it provides a level of integration by allowing the user to program command sequences for multiple devices into the remote. This is such a difficult task that many users hire professional installers to program their universal remote units.
人们还做出了一些尝试来使终端用户和媒体系统之间的屏幕界面适应现代需求。电子节目指南(EPG)被开发并被实现以取代前述的媒体指南。早期的EPG基本提供的是印制的媒体指南的电子复制品。例如，有线服务操作员提供了模拟的EPG，其中，专用的频道显示了频道的缓慢滚动格栅(grid)和它们在一些时间范围(例如，下两个小时)中相关的节目。利用这种方式即便是滚动通过一百个频道也是冗长缓慢的，并且不能够可行地升级以包括大量的附加内容安排(deployment)，例如，视频点播。更加精深复杂的数字EPG也被开发出来。在数字EPG中，节目表信息(以及可选的应用/系统软件)被传递到专用的EPG设备，例如，数字机顶盒(STB)。数字的EPG具有更大的灵活性来为媒体系统设计用户界面，这是因为它们能够提供局部交互，并能够在用户和将被观看的媒体节目之间置入一个或多个界面层。可在卡门(Kamen)等人的第6,421,067号美国专利中找到上述界面的一个实例，该文公开的内容被并入本文作为参考。图2示出了在’067号专利中描述的一种GUI(图形用户界面)。在图2中，根据卡门等人的专利，第一栏190列出了节目频道，第二栏191表示当前正在播放的节目，栏192表示在下半个小时播放的节目，第四栏193表示在半个小时之后播放的半小时节目。棒球棒图标121横跨栏191和192，以此表示预计棒球比赛继续到与栏192对应的时间档。然而，文本框111没有延伸通入栏192。这表示预计美式足球比赛不会延伸到与栏192对应的时间档。如图所示，象形图标194表示在美式足球比赛之后，ABC将播出赛马节目。图2中的图标能够使用游标(未示出)来启动(actuate)以执行各种操作，例如下载与所选择节目相关的信息。在美国第6,314,575、6,412,110和6,557,350号专利中还描述了其它的数字EPG和相关界面，其公开的内容也被并入本文作为参考。Attempts have also been made to adapt the screen interface between the end user and the media system to modern needs. An electronic program guide (EPG) is developed and implemented to replace the aforementioned media guide. Early EPGs basically provided electronic copies of printed media guides. For example, a cable service operator provides a simulated EPG in which dedicated channels display a slowly scrolling grid of channels and their associated programming over some time frame (eg, the next two hours). Scrolling through even a hundred channels using this approach is tedious and slow, and cannot feasibly scale up to include large additional content deployments, eg, video on demand. More sophisticated digital EPGs have also been developed. In a digital EPG, schedule information (and optionally application/system software) is delivered to a dedicated EPG device, such as a digital set-top box (STB). Digital EPGs have greater flexibility in designing user interfaces for media systems because they can provide partial interaction and place one or more interface layers between the user and the media program to be viewed. An example of such an interface can be found in US Patent No. 6,421,067 to Kamen et al., the disclosure of which is incorporated herein by reference. Figure 2 shows a GUI (Graphical User Interface) described in the '067 patent. In Fig. 2, according to the patents of Carmen et al., the first column 190 lists the program channels, the second column 191 represents the program currently being played, the column 192 represents the program played in the next half hour, and the fourth column 193 represents the program in the next half hour. A half-hour show that plays half an hour later. Baseball bat icon 121 spans columns 191 and 192 to indicate that baseball play is expected to continue into the time slot corresponding to column 192 . However, text box 111 does not extend into field 192 . This indicates that the football game is not expected to extend into the time slot corresponding to column 192 . As shown, pictogram 194 indicates that following the American football game, ABC will broadcast a horse racing program. The icons in FIG. 2 can be actuated using cursors (not shown) to perform various operations, such as downloading information related to a selected program. Other digital EPGs and related interfaces are described in US Patent Nos. 6,314,575, 6,412,110 and 6,557,350, the disclosures of which are also incorporated herein by reference.
然而，上述的界面具有(在其它的缺点中)不能够容易地在媒体项目的大集合和媒体项目的小集合之间调整比例(scale)的问题。例如，依赖于节目列表的界面可以为小的媒体节目集合很好地工作，但是对于浏览大的媒体节目集合却是冗长乏味的。与用于大媒体节目集合的列表界面相比，依赖于分等级的导航(navigation)(例如树结构)的界面可具有更快的遍历速度，但是不适于小的媒体节目集合。另外，用户趋向于失去在其中用户不得不在树结构中移动经过三层或更多层的选择处理的兴趣。对于所有的这些情况，目前的遥控单元通过迫使用户反复地按下上按钮和下按钮来遍历上述列表或分级结构以更冗长乏味地进行这些选择处理。在可以使用选择跳跃控制时(例如上翻页和下翻页)，用户经常不得不看该遥控装置以找到这些具体的按钮，或不得不经过训练以知道它们恰好存在。However, the interface described above has (among other disadvantages) the problem of not being able to easily scale between a large collection of media items and a small collection of media items. For example, an interface that relies on program listings may work well for small media program collections, but is tedious for browsing large media program collections. Interfaces that rely on hierarchical navigation (eg, a tree structure) may have faster traversal speeds than list interfaces for large media program collections, but are not suitable for small media program collections. In addition, the user tends to lose interest in the selection process in which the user has to move through three or more levels in the tree structure. In all of these cases, current remote control units make these selection processes more tedious by forcing the user to repeatedly press the up and down buttons to traverse the list or hierarchy described above. Where select skip controls are available (eg, page up and page down), the user often has to look at the remote to find these specific buttons, or has to be trained to know that they just exist.
因此，期望得到简化用户和媒体系统之间的控制和屏幕界面以及加速选择处理的组织架构、技术和系统，而同时允许服务提供商通过便利地向用户供应大量媒体节目和新的服务来利用对终端用户设备增加的可用带宽的益处。此外，还期望提供能够容易且快速地进行选择处理、而不管所被浏览的媒体节目集合的大小的界面。Accordingly, it would be desirable to have organizational structures, techniques and systems that simplify the control and screen interfaces between users and media systems and speed up the selection process, while at the same time allowing service providers to leverage the Benefits of increased bandwidth available to end-user devices. Furthermore, it is also desirable to provide an interface that enables easy and rapid selection processing regardless of the size of the collection of media programs being browsed.
发明内容Contents of the invention
根据本发明的系统和方法通过提供用于组织、选择和启动媒体项的总体控制架构满足了上述需要和其它需要，所述总体架构包括能够便捷且快速选择媒体项的用户界面架构。能够使用自由空间定位装置控制上述架构，所述自由空间定位装置包括最小集合的按钮和滚动轮，用于定位、点击和滚动以遍历相关图形用户界面中的选择。该示例性的图形用户界面(GUI)通过使用屏幕中的指示器、在所述指示器停悬在选择上时产生的图形动画，以及对选择进行放大和缩小以在全局视图和细节视图之间平滑地导航从而为用户提供了反馈。本发明的示例性实施方案使用图像以对增加/减少GUI对象(允许用户便利地导航)的细节和连续性的等级进行缩放。根据本发明的图形用户界面在虚拟的面上组织媒体项选择。相似的选择能够被组合在一起。初始地，该界面显示缩小的面视图，并且在大部分的情况下，实际的选择在该放大等级不会显示其全部的细节。随着用户逐渐的放大，关于媒体项组或选择的更多细节被显示出来。在不同的缩放等级上可使用不同的控制，从而使得用户能够播放选择组、单独选择，或到达虚拟面的其它部分以浏览其它相关的媒体项。Systems and methods in accordance with the present invention meet the above needs and others by providing an overall control architecture for organizing, selecting, and launching media items, including a user interface architecture that enables convenient and rapid selection of media items. The architecture described above can be controlled using a free-space pointing device comprising a minimal set of buttons and a scroll wheel for pointing, clicking and scrolling through selections in the associated graphical user interface. This exemplary graphical user interface (GUI) uses an on-screen pointer, graphical animation that occurs when the pointer hovers over a selection, and zooms in and out on the selection to navigate between the global view and the detail view. Navigating smoothly provides feedback to the user. Exemplary embodiments of the present invention use images to scale to increase/decrease the level of detail and continuity of GUI objects allowing easy navigation by the user. The graphical user interface according to the invention organizes media item selections on a virtual plane. Similar options can be grouped together. Initially, the interface displays a zoomed-out surface view, and in most cases, the actual selection does not reveal its full detail at this magnification level. As the user progressively zooms in, more details about the group or selection of media items are displayed. Different controls are available at different zoom levels, enabling the user to play selection groups, select individually, or reach other parts of the virtual surface to browse other related media items.
根据本发明的一个示例性实施方案，一种用于在图形用户界面中显示信息的方法，包括：以第一放大等级显示第一组图像；接收第一选择指示，所述第一选择指示识别所述第一组图像的子集；以第二放大等级显示所述第一组图像的所述选择子集的第一缩放版本；接收第二选择指示，所述第二选择指示识别所述图像的子集内的图像；以及显示所述被识别的图像的第二缩放版本，其中，所述被识别的图像的第二缩放版本包括至少一个用户可启动控制元素。在该实施例中的控制可以包括使用回转仪或其它机制来限定屏幕位置和位移矢量从而确定出所需的特殊命令的自由空间定位装置。还可以包括一组用于“点击”或选择事件的按钮以及“后退”按钮。According to an exemplary embodiment of the present invention, a method for displaying information in a graphical user interface includes: displaying a first set of images at a first magnification level; receiving a first selection indication, the first selection indication identifying a subset of the first set of images; displaying a first zoomed version of the selected subset of the first set of images at a second magnification level; receiving a second selection indication identifying images within the subset of images; and displaying a second scaled version of the identified image, wherein the second scaled version of the identified image includes at least one user-activatable control element. Control in this embodiment may include a free space pointing device using a gyro or other mechanism to define the screen position and displacement vector to determine the particular command required. You can also include a set of buttons for "click" or select events, and a "back" button.
根据本发明的另一个示例性实施方案，一种图形用户界面包括：第一GUI屏幕，它以第一放大等级显示第一组图像；用于接收第一选择指示的装置，所述第一选择指示识别所述第一组图像的子集；第二GUI屏幕，它以第二放大等级显示所述第一组图像的所述选择的子集的第一缩放版本；用于接收第二选择指示的装置，所述第二选择指示识别所述图像子集中的图像；以及第三GUI屏幕，它显示所述识别的图像的第二缩放版本，其中，所述被识别的图像的所述第二缩放版本包括至少一个用户可启动的控制元素。根据本发明示例性实施方案的缩放图形用户界面能够包含嵌套到任意深度的图像种类以及种类的种类。According to another exemplary embodiment of the present invention, a graphical user interface includes: a first GUI screen displaying a first set of images at a first magnification level; means for receiving a first selection indication, the first Selecting an indication to identify a subset of the first set of images; a second GUI screen that displays a first zoomed version of the selected subset of the first set of images at a second magnification level; for receiving a first zoomed version of the selected subset of the first set of images means for two selection indications, said second selection indication identifying an image in said subset of images; and a third GUI screen for displaying a second scaled version of said identified image, wherein all of said identified images The second zoomed version includes at least one user-activatable control element. A zoom GUI according to an exemplary embodiment of the present invention can contain image categories and categories of categories nested to an arbitrary depth.
附图简要说明Brief description of the drawings
相应的附图描述本发明的示例性实施方案，其中：The accompanying drawings depict exemplary embodiments of the invention, in which:
图1示出了用于娱乐系统的传统遥控单元；Figure 1 shows a conventional remote control unit for an entertainment system;
图2示出了传统的用于娱乐系统的图形用户界面；Figure 2 shows a conventional graphical user interface for an entertainment system;
图3示出了在其中能够实现本发明示例性实施方案(显示器和遥控器)的一个示例性媒体系统；Figure 3 illustrates an exemplary media system in which exemplary embodiments of the present invention (display and remote control) can be implemented;
图4详细地显示了图3中的系统控制器；Figure 4 shows the system controller in Figure 3 in detail;
图5-8示出了根据本发明一个示例性实施方案的用于媒体系统的图形用户界面；5-8 illustrate a graphical user interface for a media system according to an exemplary embodiment of the present invention;
图9描述了根据本发明一个示例性实施方案的示例性数据结构；Figure 9 depicts an exemplary data structure according to an exemplary embodiment of the present invention;
图10(a)和10(b)描述了利用图9所示的根据本发明一个示例性实施方案的数据结构建立的示例性GUI的一部分的缩小和放大版本；Figures 10(a) and 10(b) depict reduced and enlarged versions of a portion of an exemplary GUI built using the data structure shown in Figure 9 according to an exemplary embodiment of the present invention;
图11示出了用于产生根据本发明示例性实施方案的GUI显示的双链接的有序列表；Figure 11 shows an ordered list of double links used to generate a GUI display according to an exemplary embodiment of the present invention;
图12(a)和12(b)显示了另一示例性GUI的一部分的缩小和放大版本，它用于描述根据本发明示例性实施方案的节点监视算法的操作；Figures 12(a) and 12(b) show reduced and enlarged versions of a portion of another exemplary GUI for describing the operation of a node monitoring algorithm according to an exemplary embodiment of the present invention;
图13(a)和13(b)示出了根据本发明示例性实施方案的示例性数据结构，其被用来描述在GUI从图12(a)中的视图向图12(b)中的视图转换时节点监视算法的操作；Figures 13(a) and 13(b) show exemplary data structures according to an exemplary embodiment of the present invention, which are used to describe the transition from the view in Figure 12(a) to the view in Figure 12(b) in the GUI, according to an exemplary embodiment of the present invention. The operation of the node monitoring algorithm when the view transitions;
图14示出了根据本发明示例性实施方案的、包括实现分辨率一致的缩放的虚拟相机的示意性数据结构；FIG. 14 shows a schematic data structure of a virtual camera including scaling to achieve resolution consistency, according to an exemplary embodiment of the present invention;
图15(a)和15(b)显示了示例性GUI的一部分的缩小和放大版本，其示出了根据本发明示例性实施方案的语义缩放；Figures 15(a) and 15(b) show reduced and enlarged versions of a portion of an exemplary GUI illustrating semantic zooming according to an exemplary embodiment of the invention;
图16-20示出了根据本发明示例性实施方案的可缩放的图形用户界面；16-20 illustrate zoomable graphical user interfaces according to exemplary embodiments of the present invention;
图21示出了一组示例性的、能够被根据本发明示例性实施方案提供的覆盖控制(overlay control)；以及Figure 21 shows an exemplary set of overlay controls (overlay controls) that can be provided according to an exemplary embodiment of the present invention; and
图22描述了用于实现根据本发明的可缩放图形用户界面的示意性架构。Fig. 22 depicts a schematic architecture for implementing a scalable graphical user interface according to the present invention.
详细说明Detailed description
下面参照附图对本发明进行详细的说明。在不同的附图中，相同或相似的元件用同一参考标号表示。此外，以下对本发明的详细说明并不是对本发明的限制。相反，本发明的范围是由所附权利要求来限定的。The present invention will be described in detail below with reference to the accompanying drawings. In different drawings, the same or similar elements are indicated by the same reference numerals. In addition, the following detailed description of the present invention does not limit the present invention. Rather, the scope of the invention is defined by the appended claims.
为提供上下文以便于讨论，首先参考图3对其中可以实现本发明的一个示例性的集成媒体系统200进行详细描述。但是，本领域普通技术人员可以预见到，本发明并不仅限于实现这类媒体系统，而是可以在其中包括更多或更少的组件。在本系统中，输入/输出(I/O)总线210将媒体系统200中的各个组件连接在一起。该I/O总线210代表多种用于对媒体系统组件之间的信号进行传输的机制和技术中的任意一种。例如，该I/O总线210可以包括适当数量的用于传输音频信号的独立音频“插头”电缆、用于传输视频信号的同轴电缆、用于传输控制信号的双线式串行线路或红外线或射频收发器、光纤或任何用于传输其它类型的信号的其它传输机制。To provide context for discussion, an exemplary
在这个示例性的实施方案中，媒体系统200包括电视/监视器212、视频卡式记录器(VCR)214、数字化视频盘(DVD)记录/回放装置216、音频/视频调谐器218和小型碟播放机220，这些器件都被连接到I/O总线210上。VCR 214、DVD 216和小型碟播放机220可以是单光盘或单磁带式设备，也可以是多光盘或多磁带式设备。它们可以是独立的单元或者被集成在一起。此外，媒体系统200还包括一个麦克风/扬声器系统222、摄像机224和无线I/O控制装置226。根据本发明的示例性实施方案，无线I/O控制装置226是支持自由空间定位的媒体系统遥控单元，它具有最小数目的按钮以用于支持导航，并通过RF信号与娱乐系统200进行通信。例如，无线I/O控制装置226可以是利用回转仪(gyroscope)或其它机制来限定屏幕位置和位移矢量从而确定出所需的特殊命令的自由空间定位装置。在无线I/O控制装置226上还可包括将启动在下面描述的“点击”基元(primitive)的一组按钮，以及包括“后退”按钮。在另一个示例性的实施方案中，无线I/O控制装置226是通过IR信号与媒体系统200的组件进行通信的媒体系统遥控单元。在另一个实施方案中，无线I/O控制装置134可以在外观上类似于典型的媒体系统遥控装置，同时增加有可允许用户在媒体系统100的显示器中设置鼠标位置的跟踪球或其它导航机制的特性。In this exemplary embodiment,
媒体系统200还包括系统控制器228。根据本发明的一个示例性的实施方案所述，系统控制器228能够操作以存储和显示可从多个娱乐系统数据源获得的娱乐系统数据，并能控制与系统各组件相关的多种特征。如图3所示，必要时，系统控制器228可通过I/O总线210被直接或者间接地连接到系统的各个组件。在一个示例性的实施方案中，除I/O总线210外，或者用来替代I/O总线210，系统控制器228配备有无线通信发射机(或收发器)，它能够通过IR信号或RF信号与系统组件进行通信。无论控制媒质是什么，系统控制器228都被设置成能够通过下文所述的图形用户界面来控制媒体系统200中的各个媒体组件。The
如图3进一步所示，媒体系统200可以被设置用于接收来自多个媒体源和服务提供商的媒体节目。在这个示例性的实施方案中，媒体系统200从以下数据源中的任一个或全部接收信息并且可选择地向其发送信息：有线广播230，卫星广播232(例如通过卫星天线)，广播电视网234的甚高频(VHF)或超高频(UHF)射频通信(例如通过空中的天线)，电话网236和有线调制解调器238(或者其它的互联网内容源)。本领域技术人员可以预见到，图3所示的媒体组件和媒体源仅仅是一个示例，媒体系统200可以包括更多或者更少的这些组件。例如，对于上述系统的其它形式的输入包括AM/FM无线装置和卫星无线装置。As further shown in FIG. 3,
图4是示出根据本发明的示例性的系统控制器228的方框图。系统控制器228例如能够实现为机顶盒，并例如包括处理器300、存储器302、显示器控制器304、其它设备控制器(例如，与系统200的其它组件相关的装置)、一个或多个数据存储设备308以及I/O接口310。这些组件通过总线312与处理器300进行通信。本领域技术人员可以预见到，可使用一个或多个处理单元来实现处理器300。存储设备302可例如包括DRAM或SRM、ROM，这些存储设备中的一些可以被指定成存储待由处理器300运行的软件和/或这些程序使用的数据的高速缓冲存储器，这些软件和数据包括与在下面描述的图形用户界面相关联的软件和/或数据。显示器控制器304可由处理器300操作以控制监视器212的显示，以连通其它事情一起显示在下面描述的GUI屏幕和对象。根据本发明示例性实施方案的可缩放的GUI提供了与分辨率(resolution)无关的缩放，从而使得监视器212能够以任意的分辨率进行显示。设备控制器306提供了媒体系统200的其它组件和处理器300之间的接口。数据存储器308可包括一个或多个硬盘驱动器、软盘驱动器、CD-ROM设备或其它的大容量存贮器设备。输入/输出接口310可包括多种接口中的一个或多个，包括例如键盘接口、RF接口、IR接口和麦克风/语音接口。根据本发明示例性的实施方案，I/O接口310将包括用于接收与无线定位装置的运动关联的位置信息的接口。FIG. 4 is a block diagram illustrating an
根据本发明示例性实施方案的图形用户界面由系统控制器228响应处理器执行包含于存储器302中的指令序列来生成和控制，以显示媒体项选择信息。上述指令可从其它计算机可读媒介(例如数据存储设备308)或从外接于媒体系统200的计算机读入到存储器302中。对包含于存储器302中的指令序列的执行使得处理器在监视器212中连同其它的事情一起生成图形用户接口对象和控制。在可选的实施方案中，可使用硬连线电路来代替或与软件指令组合来实现本发明。如在背景技术部分中提到的那样，传统的与电视工业关联的界面架构在为用户提供简单且全面的选择体验能力方面具有严重的缺陷。因此，在本文中描述的控制架构克服了这些限制，并因此适于与电视一起使用，当然不排除与其它装置一起使用。还应该预见到在本文中描述的创新的控制架构、图形用户界面和/或各种算法可应用于与计算机和其它非电视设备使用的接口中。为了区别本发明的示例性实施方案的这些各种应用，在本说明书中使用术语“电视”和“TV”来指代显示器设备的子集，而术语“GUI”、“GUI屏幕”和“显示屏”是普通的，并指电视显示器、计算机显示器和其它任意显示器设备。更具体地说，术语“电视”和“TV”的意图是指在无需使用适配器就可将电视信号转换为其它格式(例如，计算机视频格式)的情况下能够显示电视信号(例如，NTSC信号、PAL信号或SECAM信号)的显示器设备的子集。此外，术语“电视”和“TV”指通常能从几英尺或更远的距离观看的显示器子集(例如沙发到家庭活动室TV)，而计算机显示器通常是近距离观看的(例如，从椅子到桌面的显示器)。A graphical user interface according to an exemplary embodiment of the present invention is generated and controlled by
在描述了能够用来实现根据本发明的包括可缩放图形界面的控制架构后，现在描述这些界面的几个实施例。根据本发明的示例性实施方案，用户界面显示了可按类别分组的可选择项。用户将遥控单元指向感兴趣的一个或多个种类，并且按下选择按钮以进行放大，或按下“后退”按钮缩放回原来的状态。用户的每次放大或缩放回原来状态的动作都使得通过用户界面显示于屏幕中的可选择的节目的放大等级和/或内容产生变化。根据示例性的实施方案，每次放大等级的改变可以是一致的，即以预定的级别来改变放大等级。本发明的示例性实施方案还提供了并入有几个可视化技术以实现放大到很大比例的用户界面。这些技术包括构件块和能够获得可缩放和易于使用的技术的组合，尤其是采用用户界面来提高用户视觉记忆以快速地重新访问用户界面对象的技术。Having described control architectures that can be used to implement zoomable graphical interfaces according to the invention, several embodiments of these interfaces are now described. According to an exemplary embodiment of the present invention, a user interface displays selectable items that may be grouped by category. The user points the remote control unit at one or more species of interest and presses the select button to zoom in, or the "back" button to zoom back in. Every time the user zooms in or zooms back to the original state, the zoom level and/or content of the selectable program displayed on the screen through the user interface changes. According to an exemplary embodiment, each change of the magnification level may be consistent, that is, the magnification level is changed at a predetermined level. Exemplary embodiments of the present invention also provide user interfaces that incorporate several visualization techniques to enable zoom-in to large scales. These techniques include a combination of building blocks and techniques that enable scalability and ease of use, especially techniques that employ user interfaces to improve user visual memory to quickly revisit user interface objects.
用户界面在很大程度上是视觉体验。在这样的环境中，本发明的示例性实施方案使用了用户能够记起该视觉环境中的对象位置的能力。这通过为用户界面选择项提供稳定的、相关的(dependable)位置来实现。每一个对象在可缩放布局中都具有相应的位置。一旦用户发现感兴趣的对象，就会本能地记起设置对象的方向。如果该对象是特别感兴趣的，则用户可能会不止一次地访问该项目，这会加强用户对到该对象的路径的记忆。根据本发明示例性实施方案的用户界面提供了视觉记忆系统(visual mnemonics)，它能够帮助用户记起所感兴趣的项目的位置。该视觉记忆系统包括移动(pan)和缩放动画、通过用户界面的虚拟面生成运动的地理感觉的转场效果、和一致的缩放功能性(functionality)，基于下面描述的实施例，它们连同其它的事情一起将变得更加显而易见。User interfaces are largely a visual experience. In such environments, exemplary embodiments of the present invention use the user's ability to recall the location of objects in the visual environment. This is achieved by providing a stable, dependable position for UI selections. Every object has a corresponding position in the scalable layout. Once users find an object of interest, they instinctively remember to set the orientation of the object. If the object is of particular interest, the user may visit the item more than once, which reinforces the user's memory of the path to the object. A user interface according to an exemplary embodiment of the present invention provides a visual memory system (visual mnemonics), which can help a user remember the location of an item of interest. The visual memory system includes pan and zoom animations, transition effects that generate a geographic sense of motion through virtual surfaces of the user interface, and consistent zoom functionality, based on the embodiments described below, among others Things will become more obvious together.
组织机制被提供以使得用户能够从极多的节目组中进行选择，而同时屏蔽与大量选择组关联的细节。根据本发明，可以使用各种类型的组织机制，在下面给出实施例。Organizational mechanisms are provided to enable users to select from a very large number of program groups, while at the same time masking the details associated with the large number of selected groups. Various types of organizational mechanisms can be used in accordance with the present invention, examples of which are given below.
首先参照图5到图8，其中描述了根据本发明示例性实施方案的包括可缩放图形用户界面的示例性控制架构在显示和选择音乐媒体节目中的使用。图5描绘了处于最大缩小状态的可缩放的GUI。其中，界面显示了一组形状(shape)500。显示在各个形状500中的是描述通过GUI的这些部分可对媒体节目选择组进行操作的文本502和/或图片504。如图5所示，形状500为矩形的，文本502和/或图片504描述了媒体的种类。然而，本领域的技术人员应该认识到，该首先看到的GUI组能够表示用户可得到的媒体选择的其它方面，例如，艺术家、出品年份、艺术家的居住区域、节目的长度或选择的其它特征。而且用来在GUI中概略描述各种分组的形状不一定要是矩形的。除了形状组500中的文本500和/或图片504外，或作为对文本500和/或图片504的替代，专辑封面(album cover)的缩小版本和其它图标能够用来为用户提供进一步的浏览提示。GUI 506的背景部分可以显示为纯色，或可以是例如映射图的图片的一部分，以帮助用户记起种类的空间位置，从而在以后需要较少阅读就能使用该界面。选择定位器(游标)508跟随输入设备的运动，并在用户按下该设备中的按钮(在图5中未示出)时指出将要放大的位置。Referring first to Figures 5 through 8, there is described the use of an exemplary control architecture including a zoomable graphical user interface for displaying and selecting music media programs according to exemplary embodiments of the present invention. Figure 5 depicts the zoomable GUI in a maximum zoom out state. Wherein, the interface displays a group of shapes (shapes) 500 . Displayed within each
根据本发明的一个示例性实施方案，输入设备可以是无线鼠标，例如，由Gyration inc.(回转有限公司)，(位于加利福尼亚的萨拉托加林荫道12930，萨拉托加大厦C，95070(12930Saratoga Avenue，Bldg.C，Saratoga，CA 95070))制造的无线鼠标，它与支持定位、点击、滚动、停悬和缩放构件块(将在下面进行更加详细的描述)的图形用户接口耦合。在和本发明一起使用时会带来有益效果的该示例性输入设备的一个特征是，其仅具有两个按钮和一个滚动轮，也就是三个输入启动对象。这些按钮中的一个能够被配置成放大(选择)按钮，另一个按钮能够被配置为缩小(后退)按钮。与例如图1所示的传统遥控控制单元相比，本发明通过大大减少按钮的个数等来简化用户在选择他或她的媒体节目时所面对的GUI的这些方面。根据本发明示例性实施方案的输入设备的附加优选(但不是必需)特征是它们为用户提供了“自由空间定位”的能力。在该说明书中使用的短语“自由空间定位”指用户在显示屏前空中的三维(或更多)空间自由移动输入设备的能力，以及用户界面将这些运动直接转换为游标在屏幕上的运动的相应能力。因此，“自由空间定位”不同于传统的计算机鼠标定位技术，传统的计算机鼠标定位技术使用不同于显示屏的表面(例如桌子表面或鼠标垫)作为代理(proxy)表面，来自于这些表面的鼠标相对运动被转化成计算机显示屏中的游标运动。使用自由空间定位来控制根据本发明示例性实施方案的架构进一步简化了用户的选择过程，而同时提供了将姿势(gesture)作为可区别的输入引入到上述界面的机会。姿势可被看作是能够随时间变化的可识别运动模式，该模式可被转换为GUI命令，例如以x、y、z、平摆(yaw)、俯仰(pitch)和摇摆维度的或它们任意的子组合的运动的函数。然而，本领域的技术人员应该认识到任意适合的输入设备能够与根据本发明的可缩放的GUI一起使用。其它适合的输入设备的例子包括但不限于跟踪球、触摸垫(touchpad)、传统的TV遥控设备、语音输入设备、任意能够将用户的姿势传递/转换成GUI命令的设备，或它们的任意组合。其意图是在本文中进行描述的GUI功能的每个方面都能够使用至少一个姿势和语音命令在根据本发明的架构中启动。可选的实现包括使用游标和/或其它遥控键或甚至是语音输入来识别选择的节目。According to an exemplary embodiment of the present invention, the input device may be a wireless mouse, such as that provided by Gyration inc. (Gyration Inc.), (located at 12930 Saratoga Boulevard, Saratoga Building C, 95070 (12930 Saratoga, California). Avenue, Bldg. C, Saratoga, CA 95070)) manufactures a wireless mouse coupled with a graphical user interface that supports positioning, clicking, scrolling, hovering, and zooming building blocks (described in more detail below). One feature of the exemplary input device that is beneficial when used with the present invention is that it has only two buttons and a scroll wheel, ie, three input-enabled objects. One of these buttons can be configured as a zoom in (select) button and the other button can be configured as a zoom out (back) button. The present invention simplifies these aspects of the GUI that a user faces when selecting his or her media programs, by greatly reducing the number of buttons, etc., compared to a conventional remote control unit such as that shown in FIG. 1 . An additional preferred (but not required) feature of input devices according to exemplary embodiments of the present invention is that they provide the user with "free space pointing" capabilities. As used in this specification, the phrase "free-space positioning" refers to the ability of the user to freely move the input device in three (or more) spaces in space in front of the display, and the ability of the user interface to translate these movements directly into movements of a cursor on the screen Corresponding ability. Thus, "free-space pointing" differs from traditional computer mouse pointing techniques, which use a surface other than a display screen (such as a table surface or a mouse pad) as a proxy surface from which the mouse The relative motion is translated into cursor motion on the computer display. Using free-space positioning to control the architecture according to exemplary embodiments of the present invention further simplifies the user's selection process, while at the same time providing the opportunity to introduce gestures as distinguishable inputs to the above-mentioned interface. Gestures can be thought of as recognizable motion patterns that can change over time, which can be translated into GUI commands, e.g. in x, y, z, yaw, pitch and roll dimensions, or any of them A function of the motion of subcombinations of . However, those skilled in the art will recognize that any suitable input device can be used with a zoomable GUI according to the present invention. Examples of other suitable input devices include, but are not limited to, trackballs, touchpads, conventional TV remote control devices, voice input devices, any device capable of communicating/converting user gestures into GUI commands, or any combination thereof . It is intended that every aspect of the GUI functionality described herein can be activated in the framework according to the invention using at least one gesture and voice command. Alternative implementations include using cursors and/or other remote control keys or even voice input to identify selected programs.
图6显示了如果用户从图5中选择种类3(例如，通过在显示器212中将游标508在矩形环绕种类3围绕的区域上移动，并在输入设备中按下按钮)的话将被显示的种类3进行放大的图。本发明的界面能够产生从图5到图6的缩放，从而使得用户清楚产生了缩放。上述产生的缩放/转场效果的实施例将在下面进行描述。一旦包含种类3的形状516占用了大部分的显示器212中的屏幕，上述界面显示了在上述种类中具有专辑的艺术家。在该实施例中，显示了七个不同的艺术家和/或它们的作品。与种类3相邻的未选择的种类515(其在图5中为缩小视图)仍然与在视图上放大的种类3相邻，但是它在显示器212的边缘处于被部分剪切的状态。这些未选择的种类能够通过选择指针508对它们的选择进行快速浏览。然而应该认识到本发明的其它示例性实施方案能够忽略被剪切相邻对象，相反仅显示未被剪切的选择。艺术家组(例如，艺术家组512)中的每一个都能够包含缩小专辑封面的图像、艺术家的照片或可由用户定制的艺术作品(如果上述分类包含用户建立的艺术家列表的话)。Figure 6 shows the categories that would be displayed if the user selected
用户则可以选择一个艺术家组用于进一步浏览和/或选择。图7显示了响应用户经由对游标508进行定位、并启动输入设备而选择了艺术家3的进一步放大的视图，其中，可以看到专辑封面520的图片。如图从图5和图6的GUI屏幕转变一样，未被选择的、邻近的艺术家(在该实施例中是艺术家#2、6和7)被显示为朝向被放大的显示中的侧边，用户能够使用选择指针508对它们进行点击，从而看到这些艺术家的全部视图。除了专辑封面的图像520外，在界面的这部分中，艺术家信息524能够作为艺术家组中的节目而显示。这些信息可包含例如艺术家的照片、传记、琐事、作品目录、影响、链接网址和其它有关数据。相册图片520中的每一个可包含相册封面的图片，以及可选地包含有文本数据。在相册图片520包括用户创建的艺术家列表的情况下，图形用户界面能够显示通过该界面自动选择或用户预先选择的图片。The user can then select an artist group for further browsing and/or selection. FIG. 7 shows a further zoomed-in view in response to user selection of
最后，在用户从组521中选择专辑封面图片520时，上述界面缩放到如图8所示的专辑封面。随着缩放的继续，该专辑封面能够退为或变为包含这样一些项目的视图，例如，专辑530的艺术家和标题、曲目列表532、与专辑相关的进一步信息536、专辑封面的更小版本528、以及用于回放内容、修改分类、链接到艺术家网页，或找出与选择有关的任意其它信息的控制信息534。相邻的专辑538显示出那些能够通过使用选择指针508选择，以使得界面显示它们的内容。如上所提到的那样，本发明可选择的实施方案能够例如放大到仅显示被选择的对象(例如，专辑5)，并忽略未被选择的对象(例如，专辑4和6)的剪切部分。该最终的缩放提供了语义性缩放(semantic zooming)的一个例子，其中某些GUI元件被显示出来，而先前的缩放等级中它则是不可见的。各种用于实现根据本发明示例性实施方案的技术在下面进行描述。Finally, when the user selects the
如图5-8以及说明书中示出的那样，该示例性实施方案的图形用户界面提供了对音乐集的浏览。根据本发明的界面还能够用于视频集，例如用于DVD、VHS带、其它记录的媒体、视频点播、视频片段和家庭影院。其它的音频使用包括对无线电节目、教学磁带、文史资料和声音剪接的浏览。还能够使用本发明来组织和访问印制的或文本媒体，例如新闻故事和电子书。As shown in FIGS. 5-8 and in the description, the graphical user interface of the exemplary embodiment provides browsing of a music collection. The interface according to the invention can also be used for video collections, eg for DVD, VHS tapes, other recorded media, video on demand, video clips and home theater. Other audio uses include browsing of radio programs, instructional tapes, literary and historical sources, and sound clips. The invention can also be used to organize and access printed or text media such as news stories and e-books.
根据前面的描述，本领域的技术人员应该认识到，根据本发明的可缩放的图形用户界面为用户提供了快速且容易地浏览大量(或较少)媒体项的能力。这种能力可归因于本发明示例性实施方案的界面的许多特性，这些特性包括但不限于：(1)使用图像作为用于特殊媒体项的全部或部分选择信息，(2)使用缩放来快速提供与用户进行选择所需的尽可能多或尽可能少的信息，以及(3)使用几种GUI技术，它们被组合起来给用户所有界面都位于同一平面的感觉，从而能够完成GUI的浏览，并通过用户的方向感觉而被记住。根据本发明的GUI的后一方面能够连同其它事情一起将各种GUI屏幕“地理性地”连接起来，它通过使尽可能多的GUI对象从一个GUI屏幕到下一个GUI屏幕保持连续而实现，例如，通过显示相邻的、环绕于当前GUI屏幕边沿的未被选择的对象的边缘。可选地，如果期望得到清楚的视图以及其它的GUI技术提供地理性的反馈，则可以忽略被剪切的对象。如在该文种使用的那样，短语“GUI屏幕”指在同一时间呈现于一个或多个显示器单元中的一组GUI对象。GUI屏幕可被呈现于输出媒体项的同一显示器中，或者它可被呈现在不同的显示器中。显示器可以是TV显示器，计算机监视器或任意其它合适的GUI输出设备。From the foregoing description, those skilled in the art will appreciate that a zoomable graphical user interface according to the present invention provides a user with the ability to quickly and easily browse through a large number (or fewer) of media items. This capability is attributable to a number of features of the interface of the exemplary embodiments of the present invention, including but not limited to: (1) using images as all or part of the selection information for a particular media item, (2) using zoom to Quickly provide as much or as little information as the user needs to make a selection, and (3) use several GUI techniques that are combined to give the user the feeling that all interfaces are on the same plane, enabling browsing of the GUI , and are remembered by the user's sense of direction. The latter aspect of the GUI according to the present invention enables, among other things, the various GUI screens to be connected "geographically" by keeping as many GUI objects as possible contiguous from one GUI screen to the next, For example, by displaying the edges of adjacent unselected objects that wrap around the edge of the current GUI screen. Optionally, clipped objects can be ignored if a clear view is desired and other GUI techniques provide geographic feedback. As used in this text, the phrase "GUI screen" refers to a group of GUI objects that are presented to one or more display units at the same time. The GUI screen can be presented in the same display that outputs the media item, or it can be presented in a different display. The display may be a TV display, computer monitor or any other suitable GUI output device.
另一个提高用户感觉到GUI屏幕是连接性的GUI效果是在执行缩放时、或在用户选择与当前被选择的对象的缩放等级相等的邻近对象时产生的移动动画(panning animation)效果。返回到图5所示的实施例，在用户最初观看该GUI屏幕时，他或她的观察点集中在点550附近。然而，在他或她选择种类3进行放大时，他或她的观察点转移到点552。根据本发明示例性实施方案，放大处理被激励以将POV中心从点550转移到点552。这种移动动画能够被用来改变各种GUI，例如从缩放等级的改变，或将一个对象改变到同一GUI放大等级中的另一个对象的改变。因此，如果(例如)位于图6的GUI屏幕中的用户选择最左侧的未被选择的种类515(种类2)，则将发生移动动画，这将给用户在视觉上以左或西“移”的影响。本发明的示例性实施方案使用上述技术来提供GUI屏幕之间的定向运动是一致的感觉，从而使用户更加快速地在放大等级之间和在相同放大等级的媒体项之间浏览GUI。Another GUI effect that enhances the user's perception that the GUI screen is connected is the panning animation effect that occurs when zooming is performed, or when the user selects an adjacent object that has the same zoom level as the currently selected object. Returning to the embodiment shown in FIG. 5 , when the user initially views the GUI screen, his or her point of view is centered around
能够使用各种数据结构和算法来实现根据本发明的可缩放的GUI。例如，用于在显示照片的图像浏览器中进行移动和缩放的数据结构和算法在例如下面的论文中已经被描述：“Quantum Treemaps andBubblemaps for a Zoomable Image Browser”，Benjamin B.Bederson，UIST2001，ACM Symposium on User Interface Software and Technology，CHI Letters，3(2)，pp.71-80(本杰明·B·比得森的题为“用于可缩放图像浏览器的量子数图和冒泡图”，UIST2001，用户界面软件和技术的ACM论坛，CHI学报，3(2)，第71-80页)，其被并入本文作为参考。然而，为了提供用于媒体选择的、能够以高的等级来在大量的应用中进行切换，并且能够使用户以较低的等级控制所选择的图像以执行各种媒体选择功能的GUI，需要附加的结构和算法。A scalable GUI according to the present invention can be implemented using various data structures and algorithms. For example, data structures and algorithms for panning and zooming in an image browser displaying photographs have been described in, for example, the following paper: "Quantum Treemaps and Bubblemaps for a Zoomable Image Browser", Benjamin B. Bederson, UIST2001, ACM Symposium on User Interface Software and Technology, CHI Letters, 3(2), pp.71-80 (Benjamin B. Peterson entitled "Quantum graphs and bubble graphs for scalable image browsers", pp. UIST2001, ACM Forum on User Interface Software and Technology, Acta CHI, 3(2), pp. 71-80), which is incorporated herein by reference. However, in order to provide a GUI for media selection that can switch among a large number of applications at a high level and enable the user to control a selected image to perform various media selection functions at a low level, additional structure and algorithm.
可缩放的GUI可被概念化为能够在显示设备的视区中的用户界面组件的场景周围支持移动和缩放动画。为了完成这种影响，根据本发明示例性实施方案的可缩放的GUI能够使用场景图数据结构(scenegraph data structure)来实现。在场景图中的每一个节点表示用户界面组件的某些部分，例如按钮或文本标签或一组界面组件。节点内部的子节点表示图形元素(线、文本、图像等)。例如，在场景图中的一个应用可以被表示为具有用于其界面中的图形元素的各种子节点的节点。两种特殊类型的节点在本文中被称为相机和分层。相机是通过着眼于分层节点而将视区提供到场景图的其它部分的节点。在这些分层节点的下面是用户界面元素。用于可缩放界面的控制逻辑通过编程调整相机视线变化，以提供移动或缩放效果。A scalable GUI can be conceptualized as being able to support movement and scaling animations around a scene of a user interface component in the viewport of a display device. To accomplish this effect, zoomable GUIs according to exemplary embodiments of the present invention can be implemented using a scenegraph data structure. Each node in the scene graph represents some part of a user interface component, such as a button or a text label or a set of interface components. Child nodes inside a node represent graphical elements (lines, text, images, etc.). For example, an application in a scene graph may be represented as a node with various child nodes for graphical elements in its interface. Two special types of nodes are referred to in this paper as cameras and layers. Cameras are nodes that provide viewports to other parts of the scene graph by looking at hierarchical nodes. Below these hierarchical nodes are user interface elements. Control logic for zoomable interfaces programmatically adjusts for changes in the camera's line of sight to provide a panning or zooming effect.
图9显示了包含基本可缩放的界面元素的场景图，这些元素能够用来实现本发明示例性实施方案，具体地讲，该场景图包含了一个相机节点900和一个分层节点902。相机节点900和分层节点902之间的点划线表示相机节点900被配置成将分层节点902的子节点绘制(render)到相机的视区中。被联结的显示器设备904使用户看到相机的视区。分层节点具有描绘圆和一对椭圆的三个子节点904。该场景图还进一步指出，通过节点912-918在上述圆中画出了矩形，以及在三个三角形中画出了矩形。场景图通过根节点920被联系入其它数据结构的其它场景图。节点906-918中的每一个具有通过使用局部坐标变换矩阵使其相对于其父节点进行比例缩放和定位的能力。图10(a)和10(b)分别示出了通过相机以第一放大等级缩小、以及以第二放大等级放大的场景图的样子。FIG. 9 shows a scene graph containing substantially zoomable interface elements that can be used to implement an exemplary embodiment of the present invention. Specifically, the scene graph includes a
绘制场景图可实现如下。当显示器904需要更新时，例如用户触发了从图(a)的视图到图(b)的视图的放大，则重画事件调用联结到显示器904的相机节点900以绘制相机节点900自身。这样反过来使得相机节点900通知分层节点902绘制相机视区内的区域。分层节点902通过通知它的子节点绘制这些子节点自身而将其本身绘制，等等。用于上述区域更新的当前变换矩阵和外接长方形在各个步骤被传递(pass)，并且可选地得到修改以通知各个节点它们应该用来进行绘制的适当比例和偏移。由于在根据本发明的可缩放的GUI内操作的应用的场景图可包括数以千计的节点，所以各个节点能够检测变换矩阵以及要被更新的区域，以确保它们的绘制操作真正能够被用户看到。尽管前面描述的实施例描述了包括一个相机节点和一个分层节点的场景图，但是应该预见到，本发明的示例性实施方案能够嵌入多个相机和分层。这些被嵌入的相机能够为用户提供用户节点元素，例如指示在整个可缩放界面中用户当前视线区域的小的缩小图，并且能够允许用户界面组件可独立地缩放和移动。Drawing a scene graph can be achieved as follows. When the
在使用可缩放界面来协调多应用(例如像将在下面参照图14-18描述的示例性电影浏览器一样)的操作时，用于每个应用的存储和资源需求可能会超过在媒体系统中的总有效存储。这就暗示在用户不再看它们时，需将这些应用的一些或全部代码和数据卸载。然而，在根据本发明的可缩放的GUI中，提供这样的外观是期望的，即，使一些或所有应用对于用户总是看起来是激活的。为了满足上述两个对抗的目标，对于用户视线来说是“幕外(off-screen)”的应用能够进入到临时挂起的状态。为了在根据本发明的可缩放的GUI中实现这个功能，将事件发送到应用以指示它们何时进入和退出视区。实现上述事件的一个方式是在绘制组件的代码中增加逻辑，从而使得在用户进入视区时进行检测。然而，这意味者该通知逻辑在每个绘制事件被调用，并且更重要的是在用户浏览过远离组件的视区时，它不能够容易地检测。另一种用于将事件发送到应用的方法是将通知逻辑并入到GUI浏览元件(例如超级链接和按钮)中，从而使得在它们改变相机视区以将所感兴趣的组件包括进来时，将通知发送到上述组件。然而，这需要程序员来警惕地(vigilantly)将通知代码增加到所有可能的浏览UI元素中。When using a scalable interface to coordinate the operation of multiple applications (such as the exemplary movie browser described below with reference to FIGS. 14-18 ), the storage and resource requirements for each application may exceed total available storage. This implies uninstalling some or all of the code and data of these apps when the user is no longer looking at them. However, in a zoomable GUI according to the present invention, it is desirable to provide an appearance such that some or all applications always appear to be active to the user. In order to meet the above two conflicting goals, applications that are "off-screen" to the user's line of sight can enter a temporarily suspended state. To implement this functionality in a zoomable GUI according to the present invention, events are sent to applications to indicate when they enter and exit the viewport. One way to implement the above events is to add logic to the code of the drawing component to detect when the user enters the viewport. However, this means that the notification logic is called on every paint event, and more importantly it cannot be easily detected when the user navigates away from the component's viewport. Another method for sending events to applications is to incorporate notification logic into GUI browsing elements such as hyperlinks and buttons so that when they change the camera viewport to include components of interest, the Notifications are sent to the above components. However, this requires the programmer to vigilantly add notification codes to all possible browsing UI elements.
根据一个示例性实施方案，能够使用有计算效率节点监视算法来通知应用GUI组件和/或应用在何时进入和退出相机视区。在高的等级上，该节点监视算法具有三个主要处理级：(1)初始化，(2)视区改变评估，以及(3)场景图改变评估。初始化级计算由视区改变评估级使用的节点量，并且初始化正确的数据结构。视区改变评估级在视区改变时被调用，并通知所有进入或退出视区的监视节点。最后，场景图改变评估级更新在初始级产生的计算，这些计算由于在场景图中的变化而变得无效。例如，如果监视节点的先辈节点(ancestor node)在场景图中改变了位置，则在初始级中进行的计算需要重新计算。According to one exemplary embodiment, a computationally efficient node monitoring algorithm can be used to notify applications when GUI components and/or applications enter and exit the camera viewport. At a high level, the node monitoring algorithm has three main processing stages: (1) initialization, (2) viewport change evaluation, and (3) scene graph change evaluation. The initialization stage calculates the amount of nodes used by the viewport change evaluation stage and initializes the correct data structures. The viewport change evaluation class is called when the viewport changes and notifies all watch nodes that enter or exit the viewport. Finally, scene graph changes to the evaluation stage update computations produced at the initial stage that became invalid due to changes in the scene graph. For example, calculations made in the initial stage need to be recomputed if the ancestor node of the monitor node changes position in the scene graph.
在这些级中，视区改变评估级驱动节点监视算法的其余部分。为了描绘节点在何时进入和退出视区，初始化步骤确定期望节点的边界矩形，并将其从其局部坐标系统转变到视区的局部坐标系统。按照这种方式，检测节点进入不需要在各个视区改变处进行一系列座标转换。由于该节点的父节点可具有转换矩阵，所以该初始步骤需要从该节点直到相机节点来遍历场景图。如下所述，如果在场景图数据结构中使用嵌入的相机节点，则需要多个边界矩形来容纳在多个位置出现的节点。Of these stages, the viewport change evaluation stage drives the rest of the node monitoring algorithm. To delineate when a node enters and exits the viewport, the initialization step determines the desired node's bounding rectangle and transforms it from its local coordinate system to the viewport's local coordinate system. In this way, detecting node entry does not require a series of coordinate transformations at each viewport change. Since this node's parent node may have a transformation matrix, this initial step requires traversing the scene graph from this node up to the camera node. As described below, if embedded camera nodes are used in the scene graph data structure, multiple bounding rectangles are required to accommodate the node occurring in multiple locations.
在视区坐标系统中计算出用于各个监视节点的边界矩形后，初始化级将边界矩阵增加到视区改变评估数据结构中。上述节点监视算法使用基本的构件块用于屏幕中的每一维度。在根据一些示例性实施方案的可缩放界面中，这包括x维度、y维度以及比例维度。然而，如在下面所述的那样，其它示例性实施方案可具有附加的或不同的维度。比例维度描述了在视区的节点的放大等级，并且通过下式来描述：After computing the bounding rectangles for each monitor node in the viewport coordinate system, the initialization stage adds the bounding matrix to the viewport change evaluation data structure. The node monitoring algorithm described above uses basic building blocks for each dimension in the screen. In zoomable interfaces according to some example embodiments, this includes an x-dimension, a y-dimension, and a scale dimension. However, other exemplary embodiments may have additional or different dimensions, as described below. The scale dimension describes the magnification level of the node in the viewport and is described by the following formula:
其中，s是比例，d是从该节点的一个点到该节点的局部坐标中的另一个点的距离，以及d’是在该视区中上述点到另一点的距离。where s is the scale, d is the distance from one point of the node to another point in the node's local coordinates, and d' is the distance from the above point to another point in the viewport.
图11显示了用于检测一个维度的场景进入和退出的示例性构件块。下面描述在x维度的处理，但是本领域的技术人员应该认识到能够以相似的方式处理其它的维度。区域块1100包含被转换边界矩形坐标的基准(references)。这包括矩形的左和右(顶部和底部或最小和最大比例)偏移量。左和右偏移量分别存储在转换块1102和1104中，转换块1102和1104本身置于双链接的有序列表中，从而使得较低编号的偏移量朝向起始位置。当前的视区边界存储于视线边界区块1106。区块1106具有指向刚好超出视图的左侧和右侧的转换块的指针，例如，直接位于由视图左侧指向的区块的右侧的转换块出现在视图中，除非上述由视图左侧指向的区块由视区右侧指向。Figure 11 shows exemplary building blocks for detecting scene entry and exit in one dimension. Processing in the x-dimension is described below, but those skilled in the art will recognize that other dimensions can be processed in a similar manner. Region block 1100 contains references to the coordinates of the transformed bounding rectangle. This includes the left and right (top and bottom or min and max scale) offsets of the rectangle. The left and right offsets are stored in translation blocks 1102 and 1104 respectively, which are themselves placed in a doubly-linked ordered list such that lower numbered offsets are towards the start position. The current view boundary is stored in view boundary block 1106 . Block 1106 has pointers to transition blocks just beyond the left and right sides of the view, e.g., transition blocks that are directly to the right of the block pointed to by the left of the view appear in view unless the above The block of is pointed from the right side of the viewport.
在视区改变时，对于每一维度发生下述处理。首先，检测视图左侧和视图右侧指针以确认它们是否需要移动以包括或排除转换块。接下来，如果上述指针中的一个或两个需要移动，则它们略过(slide over)上述转换块以到达它们新的位置。接着，对于由左侧和视图右侧指针经过的每个转换块，上述节点监视算法执行在下面将描述的转换块通知编码。该通知编码确定其各自的节点是否有可能进入或退出的视区。如果具有，则该节点被添加到后处理列表(post processing list)中。最后，在用于各个维度的该处理的结尾，检测上述后处理列表中的各个节点其视区状态是否真实地发生了改变(与先改变然后变回相反)。如果真的发生了改变，则上述算法发送事件到上述组件。应该注意到，如果视区快速跳转到可缩放的界面的新区域，则上述算法可检测到更多伪进入和退出事件。As the viewport changes, the following processing occurs for each dimension. First, the ViewLeft and ViewRight pointers are checked to see if they need to be moved to include or exclude transition blocks. Next, if one or both of the above pointers need to be moved, they slide over the above transition block to get to their new positions. Next, for each transition block passed by the left and view right pointers, the node monitoring algorithm described above performs the transition block notification encoding described below. The notification code determines whether its respective node is likely to enter or exit the viewport. If so, the node is added to the post processing list. Finally, at the end of this processing for each dimension, it is checked whether each node in the above-mentioned post-processing list has actually changed its viewport state (as opposed to first changing and then changing back). If a change does occur, the above algorithm sends an event to the above component. It should be noted that the algorithm described above can detect more spurious entry and exit events if the viewport quickly jumps to a new area of the zoomable interface.
转换块通知编码能够被实现成检查表，该检查表确定移入和移出用于该维度的节点是否被检测。下面显示了示例性的上述表。
表1-转换通知表 Form 1 - Conversion Notification Form
第1、2和3栏是转换通知表的输入。具体地说，节点监视算法使用节点侧、视图侧和视图移动方向的组合来编址该表，以确定该评估的节点是否被进入、退出或没有影响。栏1指由视区指针经过的转换块所表征的节点侧。栏2指视区侧，栏3指在视区通过节点的转换块时该视区侧移动的方向。输出栏4或5是在上述节点部分或完全可视时根据该节点是否应该被通知来进行选择的。例如，在一些实现方案中，理想的情况是仅在应用(例如视频流窗口)完全可视之后才通知该应用，这是因为将部分可视的视频窗口加载到可缩放的GUI中在视觉上是分裂性的。
在该表的输出表示进入和退出时，该节点监视算法将该节点添加到后处理列表。表1中的输出栏基于下面的规则来填充(populate)。如果该节点在所有的维度相交，则将进入通知发送到上述后处理步骤。如果该节点在视图中并且现在一个或多个维度停止交叉，则将发送退出通知。为了降低后处理列表中的节点个数，转换块通知编码在将该节点添加到列表之前检测与其它维度的交叉。这在所有数目的维度(例如，三个或更多交叉)中仅有一个或两个维度交叉时，免去了后处理步骤。在用户界面对象(例如，应用)要被通知其在GUI中的视区状态时，它用上述节点监视算法登记一个函数(function)。在上述应用进入或退出视图时，节点监视算法用指示发生什么情况的参数来调用该应用所登记的函数。作为一种选择，能够使用消息传递来执行通知。在这种情况下，各个应用具有事件队列。上述应用告诉节点监视算法如何与其事件队列来通信。例如，它能够指出队列的地址。接着，在节点监视检测出转换时，它建立包含通知起因的数据结构，并将其置于该应用的队列中。The node monitoring algorithm adds the node to the postprocessing list when the output of the table indicates entry and exit. The output columns in Table 1 are populated based on the following rules. If the node intersects in all dimensions, an incoming notification is sent to the post-processing step described above. If the node is in view and one or more dimensions have now stopped intersecting, an exit notification will be sent. To reduce the number of nodes in the postprocessing list, the transformation block informs the code to detect intersections with other dimensions before adding the node to the list. This eliminates a post-processing step when only one or two dimensions intersect among all number of dimensions (eg three or more intersect). When a user interface object (eg, an application) is to be notified of its viewport state in the GUI, it registers a function with the node monitoring algorithm described above. When an application as described above enters or exits a view, the node monitoring algorithm calls a function registered by the application with parameters indicating what happened. As an option, notification can be performed using message passing. In this case, each application has an event queue. The above application tells the node monitoring algorithm how to communicate with its event queue. For example, it can indicate the address of the queue. Then, when node monitoring detects a transition, it builds a data structure containing the cause of the notification and places it in the application's queue.
除了使用用于应用存储管理的节点监视通知，该算法还能够用于根据本发明的可缩放GUI中的其它功能。例如，节点监视算法能够用来基于用户视图的焦点来改变应用行为，例如通过将音频输入焦点切换到当前可视的应用中。其它用于节点监视算法的应用在放大等级改变时，将载入或卸载较高的分辨率，并将图像进行合成。这通过使绘图实施者(renderer)绘制较少的、分辨率更加接近匹配显示器的对象，来降低了其计算负载。除了节点监视算法监视相机的视区外，使它监视浏览编码(navigation code)也是有用的，该浏览编码告诉视区在动画制造之后该视区所结束的位置。这较早地通知组件它们将出现在视图中，并且还能够使得根据本发明实施方案的可缩放的GUI避免将通知发送到由于移动动画而被溢出的节点。In addition to using node monitoring notifications for application storage management, this algorithm can also be used for other functions in the scalable GUI according to the present invention. For example, node monitoring algorithms can be used to change application behavior based on the focus of the user's view, such as by switching audio input focus to the currently visible application. Other applications for node monitoring algorithms will load or unload higher resolutions and composite images when the magnification level is changed. This reduces the computational load on the renderer by enabling it to draw fewer objects whose resolution more closely matches the display. In addition to the node watch algorithm watching the camera's viewport, it is also useful to have it watch the navigation code, which tells the viewport where the viewport ends after animation. This notifies components earlier that they will appear in view, and also enables zoomable GUIs according to embodiments of the present invention to avoid sending notifications to nodes that are overflowed due to movement animations.
为了更好地理解节点操作监视算法的操作，下面将参照图12(a)、12(b)、13(a)和13(b)来描述实施例。图12(a)和12(b)示出了两个不同放大等级的、可缩放的图像用户界面的部分。在图12(a)的较低放大等级处可以看见三个节点：圆形、三角形和椭圆。在图12(b)中，该视图被放大，从而使得椭圆和圆形部分可见，三角形完全不可见。这些节点可例如表征依赖于有效事件通知的应用或用户界面组件，并因此由根据本发明示例性实施方案的节点监视算法跟踪。在该实施例中，在图12(a)和12(b)明确示出了各个节点的边界矩形，尽管本领域的技术人员应该认识到在GUI中典型地不会显示这些边界矩形。在12(a)和12(b)中对各个边界矩形的各边进行了标注，这些标注将用来显示边界矩形的边与上面描述的转换块数据结构之间的对应关系。In order to better understand the operation of the node operation monitoring algorithm, embodiments will be described below with reference to Figures 12(a), 12(b), 13(a) and 13(b). Figures 12(a) and 12(b) show portions of a zoomable graphical user interface at two different magnification levels. Three nodes can be seen at the lower magnification levels of Figure 12(a): a circle, a triangle and an ellipse. In Figure 12(b), the view is zoomed in such that the ellipse and circle are partially visible and the triangle is completely invisible. These nodes may, for example, represent applications or user interface components that rely on active event notifications, and thus are tracked by node monitoring algorithms according to exemplary embodiments of the present invention. In this embodiment, the bounding rectangles of the various nodes are explicitly shown in Figures 12(a) and 12(b), although those skilled in the art will recognize that these bounding rectangles are typically not displayed in the GUI. Each side of each bounding rectangle is labeled in 12(a) and 12(b), and these labels will be used to show the correspondence between the sides of the bounding rectangle and the transformation block data structure described above.
图13(a)示出了用于图(12)的缩小视图的水平维度的示例性节点监视数据结构。在该图中，节点边界矩形的各边使用转换块来表征。水平转换块以它们在GUI中出现的从左到右的顺序显示于图13(a)中。例如，圆形的左边C左首先出现，接着三角形的左边T左，等等，直到椭圆的右边E右出现为止。该列表的两端用空标志转换块来标记。在图13(a)中还显示了用于各个节点和指向它们相应的到它们边界矩形的水平转换块的指针的区域块。在图13(a)的底部是视图边界数据结构，它包含指向刚好位于当前视图之外的转换块的指针。对于该缩小的视图，所有节点是完全可见的，并因此它们所有的转换块位于由视图边界数据结构指向的转换块之间。Fig. 13(a) shows an exemplary node monitoring data structure for the horizontal dimension of the zoomed-in view of graph (12). In this figure, the sides of a node's bounding rectangle are represented using transition blocks. The horizontal transition blocks are shown in Figure 13(a) in the order they appear in the GUI from left to right. For example, the left side of the circle, Cleft, comes first, followed by the left side of the triangle, Tleft, and so on, until the right side of the ellipse, Eright, appears. Both ends of the list are marked with empty flag transition blocks. Also shown in Figure 13(a) are area blocks for the individual nodes and pointers to their corresponding horizontal transition blocks to their bounding rectangles. At the bottom of Figure 13(a) is the view bounds data structure, which contains pointers to transition blocks that lie just outside the current view. For this zoomed out view, all nodes are fully visible, and thus all their transition blocks are located between the transition blocks pointed to by the view bounds data structure.
图13(b)显示了用于图12(b)的放大视图的节点监视数据结构。在该图中，可以看出数据结构的视图边界部分被改变，从而使得它指向用于三角形的右边T右以及椭圆的右边E右的转换块，这是因为上述两个边界矩形边正好位于当前(放大)视图之外。Figure 13(b) shows the node monitoring data structure for the enlarged view of Figure 12(b). In this figure, it can be seen that the View Bounds portion of the data structure has been changed so that it points to the transition blocks for the right T Right of the triangle and the right E Right of the ellipse, since both of these bounding rectangle sides are located exactly at the current (zoom in) out of view.
在给出上述示例性数据结构和GUI场景后，在进行缩放转换时节点监视算法内的相关处理如下所述。首先从视图的左侧开始，节点监视算法将视图左侧指针向右移动，直到到达刚好处于左侧视图之外的转换块。如图13(b)所示，视图左侧指针首先通过C左转换块。例如，假设圆形节点表征与可缩放的GUI关联的应用或其它用户界面对象(在其完全在视图中可视时，需要通知)。给出到节点监视算法的上述输入后，表1示出该圆形节点对于水平维度应该接收退出通知。当然，节点监视算法在通知该节点之前会典型地集合来自所有维度的通知，以避免发送多余的退出通知。接下来，视图左侧指针通过三角形的左边T左。如果该三角形节点在其完全离开视图时请求通知，则该节点监视算法在每个表1中指出没有通知是必要的。然而，在视图指针通过T右时，表1示出该三角形完全退出视图，并且应该被通知。视图指针停止在这里，这里因为圆形边界矩形的右边C右在视图中仍然可见。After the above-mentioned exemplary data structure and GUI scene are given, the relevant processing in the node monitoring algorithm when scaling and transforming is performed is as follows. Starting from the left side of the view first, the node monitoring algorithm moves the view left pointer to the right until it reaches a transition block just outside the left view. As shown in Figure 13(b), the view left pointer first passes through the C left transition block. For example, suppose a circular node represents an application or other user interface object associated with a zoomable GUI that requires notification when it is fully visible in view. Given the above inputs to the node monitoring algorithm, Table 1 shows that the circular node should receive exit notifications for the horizontal dimension. Of course, node monitoring algorithms typically aggregate notifications from all dimensions before notifying the node, to avoid sending redundant exit notifications. Next, the view left pointer passes the left side of the triangle T left. If the triangle node requests a notification when it is completely out of view, the node monitoring algorithm indicates in each Table 1 that no notification is necessary. However, when the view pointer passes Tright, Table 1 shows that the triangle is completely out of view and should be notified. The view pointer stops here because the right side of the circular bounding rectangle C is still visible in the view.
节点监视算法从该右侧的处理是相似的。试图右侧指针左移到椭圆的右边E右。根据椭圆是否请求全部或部分的通知，该节点监视算法会或不会按照表1对椭圆发送通知。能够使用相似的数据结构以及顶和底边界矩形值来以相似的方式处理垂直维度。本领域的技术人员还应该认识到在需要更精确的通知时，多个边界矩形能够用来逼近非矩形的节点。此外，本发明设想能够通过节点监视算法来跟踪并处理通过其它维度的运动，例如，第三几何(深度或比例)维度，以及非几何维度，例如时间、内容定级(成人，PG-13等)以及内容类型(戏剧、喜剧等)。根据使用的维度的个数，该算法更精确地检测边界段、矩形和n维超立方体的交叉。The processing of the node monitoring algorithm from this right side is similar. Try to move the right pointer left to the right side of the ellipse. Depending on whether the ellipse requests full or partial notifications, the node monitoring algorithm may or may not send notifications to ellipses according to Table 1. The vertical dimension can be handled in a similar manner using similar data structures and top and bottom bounding rectangle values. Those skilled in the art will also recognize that multiple bounding rectangles can be used to approximate non-rectangular nodes when more precise notification is required. Furthermore, the present invention contemplates the ability to track and process motion through other dimensions, e.g., a third geometric (depth or scale) dimension, as well as non-geometric dimensions, such as time, content ratings (adult, PG-13, etc.) ) and content type (drama, comedy, etc.). Depending on the number of dimensions used, the algorithm more accurately detects intersections of boundary segments, rectangles, and n-dimensional hypercubes.
除了上面描述的节点监视算法外，本发明的示例性实施方案提供了能够在根据本发明示例性实施方案的可缩放的GUI中使用的分辨率一致语义缩放算法。语义缩放是指根据组件的放大等级对可缩放的GUI中的组件进行增加、去除或改变组件的细节。例如，在下面描述的电影浏览器界面中，当用户将镜头足够接近影片的图像时，该浏览器改变以显示影片元数据以及回放控制。该放大等级的计算是基于组件在显示器设备中使用的像素个数的。该可缩放的GUI能够存储门限放大等级，该门限放大等级指示将在何时进行切换以例如从没有影片元数据和回放控制的视图切换至具有影片元数据和回放控制的视图。In addition to the node monitoring algorithm described above, an exemplary embodiment of the present invention provides a resolution consistent semantic zooming algorithm that can be used in a zoomable GUI according to an exemplary embodiment of the present invention. Semantic zooming refers to adding, removing or changing details of components in a zoomable GUI according to the magnification level of the components. For example, in the movie browser interface described below, when the user brings the camera close enough to the image of the movie, the browser changes to display movie metadata as well as playback controls. This magnification level is calculated based on the number of pixels the component uses in the display device. The zoomable GUI can store a threshold magnification level indicating when a switch is to be made, eg, from a view without movie metadata and playback controls to a view with movie metadata and playback controls.
电视和计算机显示器具有变化范围很大的显示器分辨率。一些监视器具有足够高的分辨率，从而使得可呈现在低分辨率显示器中的图片和文本太小以至于完全不可阅读。这还对于使用语义缩放、尤其是在高分辨率显示器(例如，HDTV)中的应用产生问题。在这种情况下，基于显示的像素个数绘制的语义缩放码在可阅读更详细的视图前将改变图像。程序性地修改语义缩放在其处改变组件的门限仅能够对一个分辨率起到作用。Television and computer monitors have a wide range of display resolutions. Some monitors have a high enough resolution that pictures and text that can be rendered on a low resolution display are too small to be read at all. This also creates problems for applications using semantic scaling, especially in high-resolution displays (eg, HDTV). In this case, the semantic zoom code drawn based on the number of pixels displayed will change the image before a more detailed view can be read. Programmatically modifying semantic scaling at which to change the component's threshold can only work for one resolution.
理想的结果是所有的监视器分辨率都进行一致的语义缩放。一种解决方法是在高分辨率的监视器中使用较低分辨率的显示模式，以使得在所有显示器中的分辨率是一致的。然而，如果语义缩放仍然如期望的那样进行的话，高分辨率的监视器用户更喜欢以它们最佳的分辨率来绘制图片。因此，本发明示例性实施方案提供了一种支持所述不同分辨率显示、而不存在前述语义观看问题的语义缩放技术。这能够例如通过在场景内建立虚拟显示器来实现。通过使用内嵌的虚拟相机节点1200和增加逻辑来补偿显示器分辨率从而在图14中显示。该虚拟的相机节点1200定义了尺寸映射到用户视线距离和监视器尺寸的视区。例如，大的虚拟相机视区表示用户要么是非常近地坐在监视器前，要么具有足够大的监视器来分辩许多细节。可选地，小的视区表示用户远离监视器，并需要大的字体和图像。该可缩放的GUI编码能够是基于在上述虚拟相机中看到的，并使用用户最佳观看状态的组件的放大等级中的语义缩放转换。The ideal result is consistent semantic scaling across all monitor resolutions. One solution is to use a lower-resolution display mode on a high-resolution monitor so that the resolution is consistent across all displays. However, users of high-resolution monitors prefer to draw pictures at their optimal resolution if semantic scaling still works as expected. Accordingly, exemplary embodiments of the present invention provide a semantic zoom technique that supports such different resolution displays without the aforementioned semantic viewing issues. This can be achieved, for example, by establishing a virtual display within the scene. This is shown in Figure 14 by using the embedded
附着到显示器设备1204的主相机节点1202具有通过配置以使其显示虚拟相机1200正在显示的各种事物的视区。由于图示影像和文本在该主相机1202之前没有映射成像素，所以不会从虚拟相机中发生质量下降。这种结果是高精确度的监视器显示器显示高质量的图像，并且不会触发使显示难于阅读的语义缩放改变。The
根据本发明的示例性实施方案，上述处理实现如下。在场景图中的各相机和节点具有相关的转换矩阵(Ti到Tn)。这些矩阵将节点的局部坐标系统转换至针对显示器的下一节点的局部坐标系统。在图中，T1从其视区将坐标转换到显示器坐标。同样，T2将其局部坐标系统转换到相机的视区。如果叶节点1206需要在显示器中绘制一些内容，则它计算下面的转换矩阵：According to an exemplary embodiment of the present invention, the above processing is implemented as follows. Each camera and node in the scene graph has an associated transformation matrix (Ti to Tn ). These matrices transform the local coordinate system of a node to the local coordinate system of the next node for the display. In the diagram,T1 transforms coordinates from its viewport to display coordinates. Likewise,T2 transforms its local coordinate system to the camera's viewport. If the
A＝T1T2…TnA＝T1 T2 ...Tn
该计算能够在通过场景图时被执行。由于改变组件来支持语义缩放是基于虚拟相机1200的，所以执行下面的计算：This calculation can be performed while passing through the scene graph. Since changing components to support semantic zoom is based on the
B＝T4T5…TnB＝T4 T5 ...Tn
通常可通过询问监视器的分辨率和检查场景图而事先确定T1到T3。因此，从A确定B是通过下式对这些矩阵求逆和相乘实现的：T1 to T3 can usually be determined in advance by interrogating the resolution of the monitor and examining the scene map. Therefore, determining B from A is achieved by inverting and multiplying these matrices:
B＝(T1T2T3)-1AB＝(T1 T2 T3 )-1 A
对于事先计算T1到T3时存在问题的这种情况，例如，如果图形API隐藏了附加的转换的话，可将逻辑增加到虚拟相机，从而拦截(intercept)它将用来绘制到显示器的转换矩阵。这些被拦截的转换矩阵接着被求逆并进行相乘(如上所述)，以计算语义缩放门限。For cases where there is a problem with computingT1 toT3 beforehand, for example, if the graphics API hides the additional transformation, logic can be added to the virtual camera to intercept the transformation it will use to draw to the display matrix. These intercepted transformation matrices are then inverted and multiplied (as described above) to compute the semantic scaling threshold.
根据本发明的示例性实施方案的可缩放的界面的一个有利特征是它具有能够在导航界面时维护上下文的能力。所有的界面组件看来似乎存在于可缩放的世界中，用户仅需要移动和缩放来达到任意的它们。上述的语义缩放技术根据缩放或放大等级改变上述组件的外观。图15(a)和15(b)提供了用于组件的语义缩放的实施例，其中该组件的缩小的版本(图15(a))是图片，放大版本(图15(b))包括相同的图片以及一些控制和细节。对此更详细的一些实施例在下面给出。与语义缩放有关的一个挑战是在视图之间会突然发生改变，以及例如α混合的转换技术在两个上述视图之间转换时不会在视觉上提供满意的结果。One advantageous feature of a zoomable interface according to an exemplary embodiment of the present invention is its ability to maintain context while navigating the interface. All interface components appear to exist in a zoomable world, the user just needs to move and zoom to reach any of them. The above-mentioned semantic zoom technology changes the appearance of the above-mentioned components according to the zoom or magnification level. Figures 15(a) and 15(b) provide examples for semantic zooming of components, where the scaled-down version of the component (Figure 15(a)) is an image and the scaled-up version (Figure 15(b)) includes the same pictures and some controls and details. Some more detailed examples of this are given below. One challenge related to semantic zooming is that changes can occur abruptly between views, and transition techniques such as alpha blending do not provide visually pleasing results when transitioning between two such views.
因此，本发明的示例性实施方案在所有的组件视图中提供一些公用(common)的图像和文本，以在执行语义缩放时对转场效果提供焦点。例如，在图15(a)和15(b)中，公用元素是图片。在放大版本和缩小版本之间的转场效果能够例如使用上述节点监视算法通过以下方式来触发。首先，在主相机的视区从组件的缩小版本转换到放大版本时，用上述节点监视执行登记以接收事件。接着，在事件发生时，能够显示展示从它们缩小版本位置缩放和转换到它们放大版本位置的公用元素的动画。同时，相机视区继续推进到上述组件。Therefore, an exemplary embodiment of the present invention provides some common images and text in all component views to provide focus for transition effects when performing semantic zoom. For example, in Figures 15(a) and 15(b), the common element is a picture. The transition effect between the zoomed-in version and the zoomed-out version can be triggered in the following way, for example using the node monitoring algorithm described above. First, the node watch described above performs a registration to receive events when the main camera's viewport transitions from the zoomed-in version of the component to the zoomed-in version. Then, animations showing common elements scaling and transitioning from their scaled-down position to their scaled-up position can be displayed when the event occurs. Meanwhile, the camera viewport continues to advance to the aforementioned components.
在浏览下面参照图16-20描述的其它示例性实施方案后，根据本发明的图形用户界面的这些能力将变得更加显而易见。在图16-20中，启始GUI屏幕1400显示多个像媒体组一样操作的组织对象。完全是示例性的家庭影院、影片、TV、体育、收音机、音乐和音乐的媒体组当然能够包括不同的、更多或更少的媒体组。在通过用户启动这些图标中的一个时，根据本发明的该示例性实施方案的GUI将显示多个图像，这些多个图像中的每一个被分组成特殊的类别和种类。例如，如果用户启动图16中的“影片”图标，则能够显示图17的GUI屏幕。在图17中显示了大量的(例如120个或更多的)选择对象。这些选择对象能够被分类为特殊的组，例如，动作、古典、喜剧、戏剧、家庭剧和新发行影片。本领域的普通技术人员应该认识到能够提供更多或更少的种类。在该示例性实施方案中，媒体项的图像可以是与各影片选择关联的特征封面。尽管在图17中的区块太小而不能允许显示该选择项图像的相对多的组的详细描述，但是在应用中，这些图像的放大等级可使通过与影片关联的图像辨别这些影片的身份，即便是这些文本中的一些或全部可以是太小以至于不能容易地阅读。These capabilities of the graphical user interface according to the present invention will become more apparent after reviewing the other exemplary embodiments described below with reference to FIGS. 16-20. In FIGS. 16-20, a
可将游标(在图17中未示出)置于一组影片图像上并驱动输入设备为该组中的一个提供选择指示。在该示例性实施例中，用户选择戏剧组，则用户图形界面显示戏剧图像组中的缩放版本，如图18所示。和前述实施方案一样，在GUI从图17中的GUI屏幕切换到图18中的GUI屏幕时，还会显示转场效果，例如，在缩放之前或在缩放过程当中，该GUI可以将视图从图17中的GUI屏幕的中心移动到戏剧组图像的中心。应该注意到，尽管图18的戏剧组的缩放版本仅显示了戏剧组中的所有图像的子集，但是该缩放版本能够可选地包含被选择组中的所有图像。在任意给定的GUI屏幕的放大版本中显示或不显示被选择组中的所有图像能够基于(例如)成组的媒体项的个数，以及用于特殊缩放等级的媒体项的最小期望放大等级来进行。根据本发明的GUI的后一特征能够通过系统设计员/服务提供商预先确定，或通过用户经由GUI中的软件设置来定制。例如，可通过服务提供商和终端用户中的一个和两个来配置成组的媒体项的个数以及最小和/或最大放大等级。上述特征使得那些例如具有不好的视力的用户能够增加被显示的媒体项的放大等级。相反，视力非常好的用户可减小放大等级，以及增加在任意给定事件显示在GUI屏幕中的媒体项个数，并减少浏览时间。A cursor (not shown in Figure 17) can be positioned over a set of cine images and the input device driven to provide a selection indication for one of the set. In this exemplary embodiment, the user selects a drama group, and the graphical user interface displays a scaled version of the drama image group, as shown in FIG. 18 . Like the preceding embodiments, when the GUI switches from the GUI screen in FIG. 17 to the GUI screen in FIG. The center of the GUI screen in 17 moved to the center of the drama group image. It should be noted that although the zoomed version of the drama group of FIG. 18 only shows a subset of all the images in the drama group, the zoomed version can optionally contain all the images in the selected group. Showing or not showing all images in a selected group in the zoomed-in version of any given GUI screen can be based on, for example, the number of media items grouped, and the minimum desired zoom level of the media item for a particular zoom level to proceed. The latter feature of the GUI according to the present invention can be predetermined by the system designer/service provider, or customized by the user via software settings in the GUI. For example, the number of media items grouped and the minimum and/or maximum magnification levels may be configured by one or both of the service provider and the end user. The above-described features enable those users, for example, with poor eyesight to increase the magnification level of displayed media items. Conversely, very sighted users can reduce the magnification level and increase the number of media items displayed in the GUI screen at any given event, and reduce browsing time.
能够在根据本发明的图形用户界面中使用的一个示例性的转场效果被称为“鞋到细节(shoe-to-detail)”视图效果。在被启动时，该转场效果获取缩小的图像并同时将该缩小的图像收缩和转换到更小的视图，即，下一更高等级的缩放。从在图17中的GUI屏幕中使用的放大等级转换到在图18中的GUI屏幕中使用的更大的放大等级，这导致GUI为显示于图18的放大版本中的图像披露出附加的细节。该GUI根据这些细节在当前被选择的缩放等级是否很好地被显示，来选择性地显示或隐藏各种缩放等级的细节。和试图分辨细节而不论它们对于肉眼的可视能力如何的相机缩放不同，本发明的示例性实施方案提供了可配置的缩放等级参数，这些参数规定了在何时显示全部图像和何时显示具有被扣留细节的图像版本之间的转换点。该转换点能够基于独立于图像描述的内部分辨率，而不是基于TV/监视器212的分辨率。利用这种方式，根据本发明的GUI是一致的，而不论在媒体系统中使用的显示设备的分辨率是多小。One exemplary transition effect that can be used in a graphical user interface according to the present invention is called a "shoe-to-detail" view effect. When activated, the transition effect takes a zoomed out image and simultaneously shrinks and transitions the zoomed out image to a smaller view, ie, the next higher level of zoom. Switching from the magnification level used in the GUI screen in Figure 17 to the larger magnification level used in the GUI screen in Figure 18 causes the GUI to reveal additional detail for the image shown in the enlarged version of Figure 18 . The GUI selectively shows or hides details at various zoom levels depending on whether those details are well displayed at the currently selected zoom level. Unlike camera zoom, which attempts to resolve details regardless of their visibility to the naked eye, exemplary embodiments of the present invention provide configurable zoom level parameters that dictate when to display the full image and when to display images with The point of transition between image versions with withheld details. This transition point can be based on an internal resolution independent of the image description rather than the TV/
在该示例性实施方案中，能够通过将游标经过特殊图像之上来提供用于特殊图像的附加量的放大。这种特征在图19中显示，其中，游标在影片“阿波罗13”的图像上滚动。尽管在图19中未示出，与图18中GUI屏幕中处于较低放大等级的相应图像相比，上述附加的放大例如能够使实现与相关媒体项的特征封面中的引文“休斯顿，我们遇到了麻烦”变得更加清晰可读。用户对于该图像的选择，例如通过在输入设备上按下按钮，能够进一步缩放以显示图20中显示的细节。这还提供了另一个如前所述的语义缩放的实施例，这是因为在图19的GUI屏幕中不可见的各种信息和控制元素被显示在图20的GUI屏幕中。例如，关于影片“阿波罗13”的主要包括影片放映时间、价格和演员的信息被显示。本领域的技术人员应该认识到，其它类型的信息能够在这里提供。该GUI附加地包括GUI控制对象，GUI控制对例如包括用于购买该影片、观看电影预告片或返回到前一GUI屏幕(其还能够在输入设备中按下放大按钮来实现)的按钮控制对象。还能够使用超级链接来允许用户例如跳转到与相关影片(在图20的GUI屏幕的右下角处标志)关联的GUI屏幕，或与该影片中的演员相关的信息。在该实施例中，在标题“影片目录(Filmography)”下的一些或全部电影名能够实现为超级链接，在用户通过输入设备启动时，这些链接将使得GUI为被指示的电影显示与图20中的GUI屏幕对应的GUI屏幕。In this exemplary embodiment, an additional amount of magnification for a particular image can be provided by passing a cursor over the particular image. This feature is shown in Figure 19, where the cursor scrolls over the image of the movie "Apollo 13". Although not shown in FIG. 19 , the additional magnification described above enables, for example, the realization of a quote “Houston, we met In trouble" became more readable. User selection of the image, for example by pressing a button on the input device, enables further zooming to reveal the details shown in FIG. 20 . This also provides another embodiment of semantic zoom as previously described, because various information and control elements that are not visible in the GUI screen of FIG. 19 are displayed in the GUI screen of FIG. 20 . For example, information about the movie "Apollo 13" mainly including movie show time, price, and actors is displayed. Those skilled in the art will recognize that other types of information can be provided herein. The GUI additionally includes GUI control objects including, for example, button control objects for purchasing the movie, watching a movie trailer, or returning to a previous GUI screen (which can also be achieved by pressing the zoom button in the input device) . Hyperlinks can also be used to allow the user, for example, to jump to a GUI screen associated with a related movie (marked at the bottom right corner of the GUI screen of FIG. 20 ), or information related to actors in that movie. In this embodiment, some or all of the movie names under the heading "Filmography" can be implemented as hyperlinks that, when activated by the user through an input device, will cause the GUI to display the same as in Figure 20 for the indicated movie. The GUI screen in the corresponding GUI screen.
在用户启动超级链接时还能够使用转场效果。由于超级链接可以高放大等级来产生，因此，简单地跳转到被链接的媒体项可使用户失去他或她对媒体项选择“映射图”的跟踪。相应地，本发明的示例性实施方案提供了转场效果以在启动超级链接时，帮助维持用户的地理性位置感觉。为此而使用的一个示例性的转场效果是跳跃转换。在该转场效果的初始阶段，GUI缩小并以超级链接指向的项的方向移动。缩小和移动在目的图像和原始图像都被用户看见之前持续进行。再次使用图20的实施例，如果用户选择超级链接用于“拯救大兵瑞恩”，则该超级链接的跳跃转场效果的第一阶段将包括缩小并朝图像“拯救大兵瑞恩”移动，直到用于“拯救大兵瑞恩”的图像和“阿波罗13”对于用户是可视的位置。在此，该转场效果给用户带来以弧形朝向目的图像向上移动的视觉印象。一旦目标图像出现在视图中，转场效果的第二阶段给用户带来放大和例如以上述弧形的另一半移动到目的图像的视觉印象。跳跃时间，即显示第一阶段和第二阶段的时间量，在任意两个超级链接图像项之间是可以是固定的。可选地，跳跃时间可以变化，例如基于在GUI上前进的距离。例如，跳跃时间能够被参数化为：跳跃时间(HopTime)＝A log(放大比例等级/跳跃顶点比例等级)+B(超级链接媒体项之间的距离)+C，其中，A、B、和C被合适地选择为恒定值。Transition effects can also be used when the user initiates a hyperlink. Since hyperlinks can be generated at high magnification levels, simply jumping to the linked media item can cause the user to lose track of his or her media item selection "map". Accordingly, exemplary embodiments of the present invention provide transition effects to help maintain a user's sense of geographic location when a hyperlink is activated. An exemplary transition effect used for this purpose is a jump transition. During the initial phase of this transition effect, the GUI shrinks and moves in the direction of the item pointed to by the hyperlink. Zooming out and moving continues until both the destination image and the original image are seen by the user. Using the example of FIG. 20 again, if the user selects a hyperlink for "Saving Private Ryan," the first stage of the hyperlink's jump transition effect would consist of zooming out and moving toward the image "Saving Private Ryan" until The images for "Saving Private Ryan" and "Apollo 13" are visible to the user. Here, the transition effect gives the user the visual impression of moving upwards towards the target image in an arc. Once the target image is in view, the second stage of the transition effect gives the user the visual impression of zooming in and moving to the target image eg in the other half of the arc described above. The jump time, ie the amount of time to display the first stage and the second stage, can be fixed between any two hyperlinked image items. Optionally, the jump time may vary, for example based on the distance traveled on the GUI. For example, Hop Time can be parameterized as: Hop Time (HopTime)=A log (magnification scale level/jump vertex scale level)+B (distance between hyperlink media items)+C, where A, B, and C is suitably chosen as a constant value.
上面参照图9-13(b)描述的节点监视算法还能够用来帮助在图19的示例性GUI屏幕和图20的示例性GUI屏幕中描述的缩放等级之间进行转换。与较低缩放等级的图像相比，包含文本和/或控制元素(它们在被选择的图像的其它缩放等级是看不见的)的GUI屏幕的绘制需要更强的计算和/或存储。相应地，该节点监视算法能够在本发明的示例性实施方案中使用，以通过监视GUI的导航节点以更加快速地识别被缩放的媒体项，从而帮助预先载入GUI屏幕，例如在图20中显示的GUI屏幕。The node monitoring algorithm described above with reference to FIGS. 9-13(b) can also be used to facilitate switching between the zoom levels described in the example GUI screen of FIG. 19 and the example GUI screen of FIG. 20 . Rendering of GUI screens containing text and/or control elements that are invisible at other zoom levels of the selected image requires more computation and/or storage than images at lower zoom levels. Accordingly, the node monitoring algorithm can be used in exemplary embodiments of the present invention to help preload GUI screens by monitoring the GUI's navigation nodes to more quickly identify zoomed media items, such as in FIG. 20 GUI screen displayed.
被包含在本发明的示例性实现中的是屏幕位置和基于语义的导航控制。在游标接近用户位置时，或者游标处于在与屏幕中的这些控制关联的区域(在其中这些控制是正确的，如图21所示)中时，这些控制区域出现。例如，在播放影片时，快进、后退、暂停、停止等这些所谓的技巧(trick)功能在语义上是适当的。在该示例性实施方案中，被分配以这些功能的屏幕区域在右下角，并且在游标位于这些区域附近或位于其中时，用于这些技巧功能的图标集显示。这些图标在被使用的功能明确结束或在游标重新位于屏幕中的其它位置时消失。还能够使用同样的技术来覆盖如同文本搜索和室内屏幕选择的其它导航特征。在该示例性的实现中，这些控制在所有的屏幕上是语义相关的，并且配与它们的区域位于右上角。在游标位于这些区域或这些区域附近时，用于这些导航控制的图标集显示。这些图标在功能被启动或游标重新位于屏幕的其它位置时消失。应该注意到，处于用户训练的目的，关联的控制图标可初始可选地暂时显示(例如5秒)在一些或所有的相关屏幕中，以将引起不熟练的用户注意到它们的存在。Included in an exemplary implementation of the present invention are screen position and semantic based navigation controls. These control areas appear when the cursor is close to the user's position, or when the cursor is in the area associated with these controls in the screen where they are true, as shown in FIG. 21 . For example, when playing a movie, the so-called trick functions such as fast forward, rewind, pause, stop, etc. are semantically appropriate. In this exemplary embodiment, the areas of the screen assigned to these functions are in the lower right corner, and when the cursor is positioned near or within these areas, the icon sets for these trick functions are displayed. These icons disappear when the function being used is explicitly terminated or when the cursor is positioned elsewhere on the screen. The same technique can also be used to cover other navigation features like text search and room screen selection. In this exemplary implementation, these controls are semantically related across all screens, and the area assigned to them is located in the upper right corner. The icon sets for these navigation controls are displayed when the cursor is positioned at or near these areas. These icons disappear when the function is activated or the cursor is repositioned elsewhere on the screen. It should be noted that for user training purposes, the associated control icons may initially optionally be temporarily displayed (eg, for 5 seconds) in some or all of the relevant screens, to draw attention to their presence by unskilled users.
在提供了根据本发明的可缩放的图形用户界面的一些实施例后，现在描述使用上述界面的示例性架构和结构(infrastructure)。图22提供了架构图，其中，与各种高级应用1900(例如，影片、电视、收音机和体育)关联的可缩放的界面由基元(primitive)1902(在图中被称为“原子”)扮演。在该示例性实施方案中，基元1902包括定位、点击、缩放、停悬和滚动，当然本领域的技术人员应该认识到其它的基元也能够包含在该组中，例如，移动和拖拽。如上所述，定位和点击基元的操作确定游标的位置，并且在例如用户启动手持输入设备中的放大或缩小按钮时触发事件。这些基元简化了导航并去除了用于重复上、下、左、右按钮的需要。如上所述，缩放基元用来纵览可能的选择，并在用户缩小他或她的选择时给予用户上下文。这种构思使该界面能够按照大量的媒体选择和任意的显示大小来规定。滚动基元处理来自示例性手持输入设备中的滚动轮输入设备的输入，并能够用来例如加速线性菜单导航。停悬按钮基元动态地放大指针下面的选择(和/或改变选择的内容)，以使得用户能够浏览潜在的选择而无需实际执行。能够以大量不同的方式在根据本发明的GUI中启动前述各个基元的操作。例如，定位、点击、停悬、停悬和缩放可以与用户能够执行的不同姿势关联。这些姿势能够经由输入设备(不管它是否是自由空间指针、跟踪球、触摸垫等)被传递到系统，并被转换成适当基元的启动。同样，各个基元能够与各自的语音命令关联。Having provided some embodiments of a zoomable graphical user interface according to the present invention, an exemplary architecture and infrastructure for using the above interface is now described. 22 provides an architectural diagram in which zoomable interfaces associated with various high-level applications 1900 (e.g., movies, television, radio, and sports) are represented by primitives 1902 (referred to as "atoms" in the diagram). play. In this exemplary embodiment,
在低级的基元1902和高级的应用1900之间存在各种软硬件结构1904，它们用来产生与根据本发明的可缩放的GUI关联的图像。如在图22中看到的那样，该结构1904能够包括手持输入设备/指针、应用程序接口(API)，可缩放的GUI屏幕、开发工具等。Between the low-
前面的示例性实施方案在本质上是完全描述性的。以各等级提供到用户的缩放等级个数以及特殊的信息和控制是变化的。本领域的技术人员应该认识到本发明提供了革命性的技术用于使用可缩放的界面来呈现大和小组的媒体项，从而使得用户能够容易地搜索、浏览、组织和回放媒体项，例如影片和音乐。根据本发明的图形用户界面在虚拟的表面中组织媒体项选择，从而使得相似的选择被组合在一起。最开始，该界面传送上述表面的缩小视图，并且在大部分情况下，在该等级是看不见实际的选择的，而仅仅看到它们的组名。随着用户逐渐向内缩放，显示关于媒体项组或选择的更多细节。在各个缩放等级可得到不同的控制，从而使得用户能够播放选择的组或单个的选择、或者到达虚拟面的其它部分以浏览其它相关的媒体项。根据本发明示例性实施方案的缩放图形用户界面能够包含嵌套到任意深度的图像种类以及种类的种类。媒体项能够包括局部存储的内容、广播提供商的广播、经由来自内容提供商的直接连接或在对等基础上接收的信息。媒体项能够以日程安排的格式提供，其中，日期/时间信息在一些等级的GUI提供。另外，根据本发明示例性实施方案的架构和GUI还能够应用到在其中将用于选择的项卖给用户的电视商业中。The foregoing exemplary embodiments are entirely descriptive in nature. The number of zoom levels and specific information and controls provided to the user at each level varies. Those skilled in the art will appreciate that the present invention provides a revolutionary technique for presenting large and small media items using a zoomable interface, thereby enabling users to easily search, browse, organize and playback media items such as movies and music. A graphical user interface according to the present invention organizes media item selections in a virtual surface so that similar selections are grouped together. Initially, the interface delivers a zoomed-out view of the above-mentioned surfaces, and in most cases the actual selections are not visible at this level, only their group names. As the user progressively zooms in, more detail is displayed about the group or selection of media items. Different controls are available at various zoom levels, enabling the user to play selected groups or individual selections, or to reach other parts of the virtual surface to browse other related media items. A zoom GUI according to an exemplary embodiment of the present invention can contain image categories and categories of categories nested to an arbitrary depth. Media items can include locally stored content, broadcast by a broadcast provider, information received via a direct connection from a content provider, or on a peer-to-peer basis. Media items can be provided in a scheduled format where date/time information is provided at some level of GUI. In addition, the architecture and GUI according to an exemplary embodiment of the present invention can also be applied to a television commercial in which items for selection are sold to users.
上述示例性的实施方案在所有方面趋于是用来描述而不是限制本发明。因此，本发明能够在具体的实现中具有许多变种，本领域的技术人员能够通过包含在本文中的描述得到这些变种。所有的这些变种和修改被看作是落入本发明在下面权利要求中定义的范围和精神内。除非是明确说明，否则在本申请的说明书中使用的元素、动作或指令不能解释为对本发明是关键或必要的。而且，如在本文中使用的那样，冠词“一(a)”的意图包括了一个或多个。The above-described exemplary embodiments are intended in all respects to illustrate rather than limit the invention. Thus the present invention is capable of many variations in specific implementation which can be derived from the description contained herein by a person skilled in the art. All such variations and modifications are considered within the scope and spirit of the invention as defined in the following claims. No element, act, or instruction used in the description of the present application should be construed as critical or essential to the invention unless explicitly stated otherwise. Also, as used herein, the article "a" or "a" is intended to include one or more.
Claims (159)
Translated fromChineseApplications Claiming Priority (7)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US46883003P | 2003-05-08 | 2003-05-08 | |
| US60/468,830 | 2003-05-08 | ||
| US60/495,998 | 2003-08-18 | ||
| US10/768,432 | 2004-01-30 | ||
| US10/768,422 | 2004-01-30 | ||
| US10/768,234 | 2004-01-30 | ||
| US10/768,460 | 2004-01-30 |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2008101811003ADivisionCN101430631B (en) | 2003-05-08 | 2004-05-07 | Method for interaction with multiple images and user interface device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1784647Atrue CN1784647A (en) | 2006-06-07 |
| CN100454220C CN100454220C (en) | 2009-01-21 |
Family
ID=36773853
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2008101811003AExpired - LifetimeCN101430631B (en) | 2003-05-08 | 2004-05-07 | Method for interaction with multiple images and user interface device |
| CNB2004800124776AExpired - LifetimeCN100454220C (en) | 2003-05-08 | 2004-05-07 | Control architecture for organizing, selecting and launching media items |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2008101811003AExpired - LifetimeCN101430631B (en) | 2003-05-08 | 2004-05-07 | Method for interaction with multiple images and user interface device |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN101430631B (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102077162A (en)* | 2008-06-27 | 2011-05-25 | 微软公司 | Semantic zoom in a virtual three-dimensional graphical user interface |
| CN102165403A (en)* | 2008-09-25 | 2011-08-24 | 微软公司 | User interface having zoom functionality |
| CN102279863A (en)* | 2010-06-08 | 2011-12-14 | 索尼计算机娱乐公司 | Information providing apparatus and information providing method |
| CN102460370A (en)* | 2009-06-08 | 2012-05-16 | 苹果公司 | User Interface for Multiple Displays |
| CN103235643A (en)* | 2013-04-09 | 2013-08-07 | 青岛旲天下智能科技有限公司 | Man-machine interactive device adopting voice control type |
| CN103345350A (en)* | 2012-01-30 | 2013-10-09 | 英特尔公司 | One-click tagging user interface |
| CN101794207B (en)* | 2009-01-01 | 2013-12-04 | 英特尔公司 | Pose to device mapping |
| US8656312B2 (en) | 2007-09-07 | 2014-02-18 | Canon Kabushiki Kaisha | Content display apparatus and display method thereof |
| CN103853809A (en)* | 2012-12-04 | 2014-06-11 | Sap股份公司 | Efficient space allocation for business objects |
| CN104793868A (en)* | 2007-09-07 | 2015-07-22 | 苹果公司 | GUI applications for use with 3D remote controller |
| CN107396154A (en)* | 2011-08-05 | 2017-11-24 | 三星电子株式会社 | Electronic equipment and the method that its user interface is provided |
| CN108134960A (en)* | 2017-12-21 | 2018-06-08 | 四川迪佳通电子有限公司 | Digital TV terminal method for previewing programs and system |
| CN109885273A (en)* | 2019-03-08 | 2019-06-14 | 南昌黑鲨科技有限公司 | Image display system and display methods |
| CN110275744A (en)* | 2018-03-14 | 2019-09-24 | Tcl集团股份有限公司 | It is a kind of for making the method and system of scalable user interface |
| US10992995B2 (en) | 2013-09-10 | 2021-04-27 | Opentv, Inc. | Systems and methods of displaying content |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102385443B (en)* | 2010-08-30 | 2014-02-12 | 联想(北京)有限公司 | Adjusting method and controlling device for output contents of electronic device |
| EP2643743A4 (en)* | 2010-11-22 | 2014-06-25 | Hillcrest Lab Inc | 3d pointing device with up-down-left-right mode switching and integrated swipe detector |
| EP2656627A1 (en)* | 2010-12-22 | 2013-10-30 | Thomson Licensing | Method and system for providing media recommendations |
| EP2807541B1 (en)* | 2012-01-26 | 2020-09-02 | EntIT Software LLC | Image-based application automation |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW355774B (en)* | 1995-10-12 | 1999-04-11 | Konami Co Ltd | TV game device, its loading support method and control method |
| US6154723A (en)* | 1996-12-06 | 2000-11-28 | The Board Of Trustees Of The University Of Illinois | Virtual reality 3D interface system for data creation, viewing and editing |
| US6005578A (en)* | 1997-09-25 | 1999-12-21 | Mindsphere, Inc. | Method and apparatus for visual navigation of information objects |
| US6426761B1 (en)* | 1999-04-23 | 2002-07-30 | Internation Business Machines Corporation | Information presentation system for a graphical user interface |
| US20020112237A1 (en)* | 2000-04-10 | 2002-08-15 | Kelts Brett R. | System and method for providing an interactive display interface for information objects |
| JP3436253B2 (en)* | 2001-03-01 | 2003-08-11 | 松下電器産業株式会社 | Resin-sealed semiconductor device and method of manufacturing the same |
- 2004
- 2004-05-07CNCN2008101811003Apatent/CN101430631B/ennot_activeExpired - Lifetime
- 2004-05-07CNCNB2004800124776Apatent/CN100454220C/ennot_activeExpired - Lifetime
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8656312B2 (en) | 2007-09-07 | 2014-02-18 | Canon Kabushiki Kaisha | Content display apparatus and display method thereof |
| CN104793868B (en)* | 2007-09-07 | 2018-08-31 | 苹果公司 | Method and apparatus for controlling media application operation |
| CN104793868A (en)* | 2007-09-07 | 2015-07-22 | 苹果公司 | GUI applications for use with 3D remote controller |
| CN102077162A (en)* | 2008-06-27 | 2011-05-25 | 微软公司 | Semantic zoom in a virtual three-dimensional graphical user interface |
| CN102165403A (en)* | 2008-09-25 | 2011-08-24 | 微软公司 | User interface having zoom functionality |
| US9591118B2 (en) | 2009-01-01 | 2017-03-07 | Intel Corporation | Pose to device mapping |
| CN101794207B (en)* | 2009-01-01 | 2013-12-04 | 英特尔公司 | Pose to device mapping |
| US9223465B2 (en) | 2009-06-08 | 2015-12-29 | Apple Inc. | User interface for multiple display regions |
| CN102460370A (en)* | 2009-06-08 | 2012-05-16 | 苹果公司 | User Interface for Multiple Displays |
| US10579204B2 (en) | 2009-06-08 | 2020-03-03 | Apple Inc. | User interface for multiple display regions |
| US9081474B2 (en) | 2009-06-08 | 2015-07-14 | Apple Inc. | User interface for multiple display regions |
| US9720584B2 (en) | 2009-06-08 | 2017-08-01 | Apple Inc. | User interface for multiple display regions |
| CN102279863A (en)* | 2010-06-08 | 2011-12-14 | 索尼计算机娱乐公司 | Information providing apparatus and information providing method |
| CN102279863B (en)* | 2010-06-08 | 2015-02-04 | 索尼计算机娱乐公司 | Information providing apparatus and information providing method |
| CN107396154A (en)* | 2011-08-05 | 2017-11-24 | 三星电子株式会社 | Electronic equipment and the method that its user interface is provided |
| CN103345350A (en)* | 2012-01-30 | 2013-10-09 | 英特尔公司 | One-click tagging user interface |
| US10254919B2 (en) | 2012-01-30 | 2019-04-09 | Intel Corporation | One-click tagging user interface |
| CN103853809A (en)* | 2012-12-04 | 2014-06-11 | Sap股份公司 | Efficient space allocation for business objects |
| CN103235643A (en)* | 2013-04-09 | 2013-08-07 | 青岛旲天下智能科技有限公司 | Man-machine interactive device adopting voice control type |
| US11363342B2 (en) | 2013-09-10 | 2022-06-14 | Opentv, Inc. | Systems and methods of displaying content |
| US12219219B2 (en) | 2013-09-10 | 2025-02-04 | Opentv, Inc. | Systems and methods of displaying content |
| US11825171B2 (en) | 2013-09-10 | 2023-11-21 | Opentv, Inc. | Systems and methods of displaying content |
| US10992995B2 (en) | 2013-09-10 | 2021-04-27 | Opentv, Inc. | Systems and methods of displaying content |
| CN108134960A (en)* | 2017-12-21 | 2018-06-08 | 四川迪佳通电子有限公司 | Digital TV terminal method for previewing programs and system |
| CN110275744B (en)* | 2018-03-14 | 2021-11-23 | Tcl科技集团股份有限公司 | Method and system for making scalable user interface |
| CN110275744A (en)* | 2018-03-14 | 2019-09-24 | Tcl集团股份有限公司 | It is a kind of for making the method and system of scalable user interface |
| CN109885273A (en)* | 2019-03-08 | 2019-06-14 | 南昌黑鲨科技有限公司 | Image display system and display methods |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101430631B (en) | 2012-05-30 |
| CN101430631A (en) | 2009-05-13 |
| CN100454220C (en) | 2009-01-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101233504B (en) | Distributed software construction for user interfaces | |
| US8046705B2 (en) | Systems and methods for resolution consistent semantic zooming | |
| US8601396B2 (en) | Systems and methods for node tracking and notification in a control framework including a zoomable graphical user interface | |
| US8555165B2 (en) | Methods and systems for generating a zoomable graphical user interface | |
| US7834849B2 (en) | Control framework with a zoomable graphical user interface for organizing selecting and launching media items | |
| US8432358B2 (en) | Methods and systems for enhancing television applications using 3D pointing | |
| KR100817394B1 (en) | A control framework with a zoomable graphical user interface for organizing, selecting and launching media items | |
| US7386806B2 (en) | Scaling and layout methods and systems for handling one-to-many objects | |
| US8521587B2 (en) | Systems and methods for placing advertisements | |
| CN100454220C (en) | Control architecture for organizing, selecting and launching media items |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | Effective date of registration:20180604 Address after:Delaware Patentee after:IDHL HOLDINGS, Inc. Address before:American Maryland Patentee before:HILLCREST LABORATORIES, Inc. | |
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right | Effective date of registration:20231102 Address after:Delaware Patentee after:TAG-COMM Inc. Address before:Delaware Patentee before:IDHL HOLDINGS, Inc. | |
| TR01 | Transfer of patent right | ||
| CX01 | Expiry of patent term | Granted publication date:20090121 | |
| CX01 | Expiry of patent term |