CN110741652A - Display device with intelligent user interface - Google Patents

Abstract

A display device including user input circuitry for receiving user commands and a display for outputting video content and a user interface, the video content including metadata, the display device further including a processor in communication with the user input circuitry and the display, and a non-volatile computer readable medium in communication with the processor and storing instruction code, the instruction code being executed by the processor and causing the processor to receive a scene command from the user input circuitry, the scene command to search for scenes in the video content that belong to a scene type, the processor determining or more scenes in the video content that belong to the scene type based on the metadata, the processor then updating the user interface to show or more scene images related to or more scenes that belong to the scene type.

Description

Translated fromChinese

具有智能用户界面的显示设备Display device with smart user interface

相关申请的交叉引用CROSS-REFERENCE TO RELATED APPLICATIONS

本申请要求于2018年5月21日提交的、申请号为15/985,206的美国专利申请、于2018年5月21日提交的、申请号为15/985,292的美国专利申请、于2018年5月21日提交的、申请号为15/985,251的美国专利申请、于2018年5月21日提交的、申请号为15/985,273的美国专利申请、于2018年5月21日提交的、申请号为15/985,303的美国专利申请、于2018年5月21日提交的、申请号为15/985,338的美国专利申请、及于2018年5月21日提交的、申请号为15/985,325的美国专利申请的优先权，上述申请的全文以引用的方式并入本文中。This application claims US Patent Application No. 15/985,206, filed May 21, 2018, US Patent Application No. 15/985,292, filed May 21, 2018, filed May 21, 2018 US Patent Application No. 15/985,251, filed May 21, 2018 US Patent Application No. 15/985,273, filed May 21, 2018, Application No. US Patent Application 15/985,303, US Patent Application No. 15/985,338, filed May 21, 2018, and US Patent Application No. 15/985,325, filed May 21, 2018 of priority, the entirety of the above application is incorporated herein by reference.

技术领域technical field

本申请主要涉及诸如电视机的显示设备。具体地，本申请描述了一种具有智能用户界面的显示设备。This application primarily relates to display devices such as televisions. Specifically, this application describes a display device with an intelligent user interface.

背景技术Background technique

当前种类的高端电视通常包括网络连接功能，以便于从诸如

等的内容服务器流式传输视频内容。在一些情况下，电视使用诸如

的操作系统，所述操作系统便于执行用于其他目的的app。Current varieties of high-end TVs often include network connectivity to facilitate

etc. to stream video content from a content server. In some cases, televisions use

an operating system that facilitates the execution of apps for other purposes.

访问数量不断增加的新特征要求改变用户界面。不幸的是，访问这些较新特征常常导致用户界面令人沮丧地复杂且难以导航。The ever-increasing number of accesses to new features requires changes to the user interface. Unfortunately, accessing these newer features often results in user interfaces that are frustratingly complex and difficult to navigate.

发明内容SUMMARY OF THE INVENTION

在第一方面，一种显示设备，包括用于接收用户命令的用户输入电路和用于输出视频内容和用户界面的显示器。视频内容包括元数据。所述显示设备还包括与用户输入电路和显示器通信的处理器，以及与所述处理器通信且存储指令代码的非易失性计算机可读介质。所述指令代码由所述处理器执行，并且使所述处理器从所述用户输入电路接收第一场景命令，所述第一场景命令用以搜索视频内容中属于场景类型的场景。所述处理器从所述元数据确定视频内容中属于场景类型的一个或多个场景。然后所述处理器更新所述用户界面以示出与属于所述场景类型的一个或多个场景相关的一个或多个场景图像。In a first aspect, a display device includes user input circuitry for receiving user commands and a display for outputting video content and a user interface. Video content includes metadata. The display device also includes a processor in communication with the user input circuitry and the display, and a non-transitory computer readable medium in communication with the processor and storing instruction code. The instruction code is executed by the processor and causes the processor to receive a first scene command from the user input circuit, the first scene command to search the video content for a scene of a scene type. The processor determines from the metadata one or more scenes in the video content that are of a scene type. The processor then updates the user interface to show one or more scene images associated with one or more scenes belonging to the scene type.

在第二方面，一种用于控制显示设备的方法，包括：通过用户输入电路接收用户命令；并通过显示器输出视频内容和用户界面。视频内容包括元数据。所述方法包括从所述用户输入电路接收第一场景命令，所述第一场景命令用以搜索所述视频内容中属于场景类型的场景；从所述元数据确定视频内容中属于场景类型的一个或多个场景；以及更新所述用户界面以示出与属于所述场景类型的一个或多个场景相关的一个或多个场景图像。In a second aspect, a method for controlling a display device includes: receiving user commands through a user input circuit; and outputting video content and a user interface through a display. Video content includes metadata. The method includes receiving a first scene command from the user input circuit, the first scene command to search the video content for a scene of a scene type; determining from the metadata one of the scene types in the video content and updating the user interface to show one or more scene images associated with one or more scenes belonging to the scene type.

在第三方面，提供了存储用于控制显示设备的指令代码的非易失性计算机可读介质。所述指令代码可由计算机执行，用于使所述计算机从用户输入电路接收第一场景命令，所述第一场景命令用以搜索视频内容中属于场景类型的场景；从视频内容的元数据确定视频内容中属于场景类型的一个或多个场景；以及更新用户界面以示出与属于所述场景类型的一个或多个场景相关的一个或多个场景图像。In a third aspect, a non-transitory computer readable medium storing instruction code for controlling a display device is provided. The instruction code is executable by a computer for causing the computer to receive a first scene command from a user input circuit, the first scene command for searching the video content for a scene of a scene type; determining the video from metadata of the video content one or more scenes in the content belonging to a scene type; and updating the user interface to show one or more scene images associated with the one or more scenes belonging to the scene type.

在第四方面，一种显示设备，包括用于接收用户命令的用户输入电路；用于显示视频内容和用户界面的显示器。所述设备还包括与所述用户输入电路、所述显示器和搜索历史数据库通信的处理器；和与所述处理器通信且存储指令代码的非易失性计算机可读介质，当由所述处理器执行时，所述指令代码使所述处理器：从所述用户输入电路接收第一搜索命令。所述处理器从所述第一搜索命令确定与所述第一搜索命令相关的一个或多个候选的搜索命令。所述处理器随后更新所述用户界面以示出所述候选的搜索命令中的一个或多个；并从所述用户输入电路接收第二搜索命令，所述第二搜索命令对应于所述一个或多个候选的搜索命令之一。所述处理器确定与所述第一搜索命令和所述第二搜索命令相关联的视频内容；并更新所述用户界面以示出一个或多个控件，每个所述控件都与所确定的视频内容的不同视频内容相关联。In a fourth aspect, a display device includes user input circuitry for receiving user commands; a display for displaying video content and a user interface. The apparatus also includes a processor in communication with the user input circuitry, the display, and the search history database; and a non-transitory computer-readable medium in communication with the processor and storing instruction code, when used by the processor When executed by the processor, the instruction code causes the processor to: receive a first search command from the user input circuit. The processor determines from the first search command one or more candidate search commands related to the first search command. the processor then updates the user interface to show one or more of the candidate search commands; and receives a second search command from the user input circuit, the second search command corresponding to the one or one of several candidate search commands. the processor determines video content associated with the first search command and the second search command; and updates the user interface to show one or more controls, each of the controls related to the determined Different video contents of the video content are associated.

可选的，所述指令代码使所述处理器：更新所述用户界面以在所述一个或多个控件的每个控件上示出唯一标识符；从所述用户输入电路接收指定所述唯一标识符之一的第三搜索命令；并显示与所指定的唯一标识符相关联的视频内容。Optionally, the instruction code causes the processor to: update the user interface to show a unique identifier on each of the one or more controls; receive from the user input circuit specifying the unique identifier; a third search command for one of the identifiers; and display the video content associated with the specified unique identifier.

可选的，所述第一搜索命令和第二搜索命令对应于语音命令，所述指令代码使所述处理器：实现自然语言处理器；并通过所述自然语言处理器确定所述语音命令的含义。Optionally, the first search command and the second search command correspond to voice commands, and the instruction code enables the processor to: implement a natural language processor; and determine the voice command by the natural language processor. meaning.

可选的，所述指令代码使所述处理器：更新所述搜索历史数据库以反映所述第二搜索命令被选择的事实，从而增加在后续搜索期间预测所述第二搜索命令的可能性。Optionally, the instruction code causes the processor to: update the search history database to reflect the fact that the second search command was selected, thereby increasing the likelihood of predicting the second search command during subsequent searches.

可选的，所述指令代码使所述处理器至少部分基于所述搜索历史数据库中存储的由所述用户指定的搜索命令的历史记录预测所述一个或多个候选的搜索命令。Optionally, the instruction code causes the processor to predict the one or more candidate search commands based at least in part on a history of search commands specified by the user stored in the search history database.

可选的，所述指令代码使所述处理器更新所述用户界面以示出对应于所述第一搜索命令和所述第二搜索命令的短语，所述短语随所述用户指定不同搜索命令而实时更新。Optionally, the instruction code causes the processor to update the user interface to show phrases corresponding to the first search command and the second search command, the phrases being different search commands specified by the user And real-time updates.

在第五方面，一种用于控制显示设备的方法，包括：通过用户输入电路接收用户命令；显示视频内容和用户界面；从所述用户输入电路接收第一搜索命令；从所述第一搜索命令确定与所述第一搜索命令相关的一个或多个候选的搜索命令；更新所述用户界面以示出所述候选的搜索命令的一个或多个；从所述用户输入电路接收第二搜索命令，所述第二搜索命令对应于所述一个或多个候选的搜索命令之一；确定与所述第一搜索命令和所述第二搜索命令相关联的视频内容；以及更新所述用户界面以示出一个或多个控件，每个所述控件都与所确定的视频内容的不同视频内容相关联。In a fifth aspect, a method for controlling a display device, comprising: receiving a user command through a user input circuit; displaying video content and a user interface; receiving a first search command from the user input circuit; Commands determine one or more candidate search commands related to the first search command; update the user interface to show one or more of the candidate search commands; receive a second search from the user input circuit command, the second search command corresponding to one of the one or more candidate search commands; determining video content associated with the first search command and the second search command; and updating the user interface to illustrate one or more controls, each associated with a different video content of the determined video content.

可选的，所述方法还包括：更新所述用户界面以在所述一个或多个控件的每个控件上示出唯一标识符；从所述用户输入电路接收指定所述唯一标识符之一的第三搜索命令；以及显示与所指定的唯一标识符相关联的视频内容。Optionally, the method further comprises: updating the user interface to show a unique identifier on each of the one or more controls; receiving from the user input circuit specifying one of the unique identifiers and displaying the video content associated with the specified unique identifier.

可选的，所述第一搜索命令和第二搜索命令对应语音命令，所述方法还包括：实现自然语言处理器；并通过所述自然语言处理器确定所述语音命令的含义。Optionally, the first search command and the second search command correspond to voice commands, and the method further includes: implementing a natural language processor; and determining the meaning of the voice commands through the natural language processor.

可选的，所述方法还包括：更新所述搜索历史数据库以反映所述第二搜索命令被选择的事实，从而增加在后续搜索期间预测所述第二搜索命令的可能性。Optionally, the method further comprises: updating the search history database to reflect the fact that the second search command was selected, thereby increasing the likelihood of predicting the second search command during subsequent searches.

可选的，所述方法还包括：至少部分基于存储在所述搜索历史数据库中的由所述用户指定的搜索命令的历史记录预测所述一个或多个候选的搜索命令。Optionally, the method further includes predicting the one or more candidate search commands based at least in part on a history of search commands specified by the user stored in the search history database.

可选的，所述方法还包括：更新所述用户界面以示出对应于所述第一搜索命令和第二搜索命令的短语，其中所述短语随所述用户指定不同搜索命令而实时更新。Optionally, the method further includes updating the user interface to show phrases corresponding to the first search command and the second search command, wherein the phrases are updated in real time as the user specifies different search commands.

在第六方面，提供了一种非易失性计算机可读介质，其存储用于控制显示设备的指令代码。所述指令代码能由计算机执行以使所述计算机：从所述计算机的用户输入电路接收第一搜索命令；从所述第一搜索命令确定与所述第一搜索命令相关的一个或多个候选的搜索命令；更新所述计算机的用户界面以示出所述候选的搜索命令中的一个或多个；从所述用户输入电路接收第二搜索命令，所述第二搜索命令对应于所述一个或多个候选的搜索命令之一；确定与所述第一搜索命令和所述第二搜索命令相关联的视频内容；以及更新所述用户界面以示出一个或多个控件，每个所述控件都与所确定的视频内容的不同视频内容相关联。In a sixth aspect, there is provided a non-volatile computer-readable medium storing instruction code for controlling a display device. The instruction code is executable by a computer to cause the computer to: receive a first search command from a user input circuit of the computer; determine from the first search command one or more candidates related to the first search command updating a user interface of the computer to show one or more of the candidate search commands; receiving a second search command from the user input circuit, the second search command corresponding to the one one or more candidate search commands; determining video content associated with the first search command and the second search command; and updating the user interface to show one or more controls, each of the The controls are all associated with different video content of the determined video content.

可选的，所述指令代码使所述计算机：更新所述用户界面以在所述一个或多个控件中的每个控件上示出唯一标识符；从所述用户输入电路接收指定所述唯一标识符之一的第三搜索命令；及显示与所指定的唯一标识符相关联的视频内容。Optionally, the instruction code causes the computer to: update the user interface to show a unique identifier on each of the one or more controls; receive from the user input circuit specifying the unique identifier; a third search command for one of the identifiers; and displaying video content associated with the specified unique identifier.

可选的，所述第一搜索命令和第二搜索命令对应于语音命令，所述指令代码使所述计算机：实现自然语言处理器；并通过所述自然语言处理器确定所述语音命令的含义。Optionally, the first search command and the second search command correspond to voice commands, and the instruction code enables the computer to: implement a natural language processor; and determine the meaning of the voice command through the natural language processor .

可选的，所述指令代码使所述计算机更新所述搜索历史数据库以反映所述第二搜索命令被选择的事实，从而增加在后续搜索期间预测所述第二搜索命令的可能性。Optionally, the instruction code causes the computer to update the search history database to reflect the fact that the second search command was selected, thereby increasing the likelihood of predicting the second search command during subsequent searches.

可选的，所述指令代码使所述计算机至少部分基于存储在所述历史数据库中的由所述用户指定的搜索命令的历史记录预测所述一个或多个候选的搜索命令。Optionally, the instruction code causes the computer to predict the one or more candidate search commands based at least in part on a history of search commands specified by the user stored in the historical database.

可选的，所述指令代码使所述计算机更新所述用户界面以示出对应于所述第一搜索命令和所述第二搜索命令的短语，所述短语随所述用户指定不同搜索命令而实时更新。Optionally, the instruction code causes the computer to update the user interface to show phrases corresponding to the first search command and the second search command, the phrases changing as the user specifies different search commands. Live Update.

在第七方面，一种显示设备，包括：用于接收用户命令的用户输入电路和用于输出视频内容和用户界面的显示器。所述视频内容包括元数据。所述设备还包括与所述用户输入电路和所述显示器通信的处理器；及与所述处理器通信且存储指令代码的非易失性计算机可读介质，当由所述处理器执行时，所述指令代码使所述处理器：从所述用户输入电路接收对于当前在所述显示器上显示的视频内容的图像的查询；基于所述元数据确定所述图像的与所述查询相关联的一个或多个对象；更新所述用户界面以示出一个或多个控件，每个控件都与所确定的一个或多个对象之一相关联；接收对所述控件之一的选择；以及更新所述用户界面以示出与所述选择相关的信息。In a seventh aspect, a display device includes a user input circuit for receiving user commands and a display for outputting video content and a user interface. The video content includes metadata. The apparatus also includes a processor in communication with the user input circuitry and the display; and a non-transitory computer-readable medium in communication with the processor and storing instruction code that, when executed by the processor, The instruction code causes the processor to: receive, from the user input circuitry, a query for an image of video content currently displayed on the display; determine an image of the image associated with the query based on the metadata one or more objects; updating the user interface to show one or more controls, each control associated with the determined one or more objects; receiving a selection of one of the controls; and updating the user interface to show information related to the selection.

可选的，所述指令代码使所述处理器：确定与所述第一查询和所确定的一个或多个对象相关的一个或多个候选的第二查询；更新所述用户界面以示出所述一个或多个候选的第二查询中的一个或多个；从所述用户输入电路接收第二查询，所述第二查询对应于所述一个或多个候选的第二查询之一；基于所述元数据确定所述图像的与所述第一查询和所述第二查询相关联的一个或多个对象；更新所述用户界面以示出一个或多个控件，每个控件都与所确定的一个或多个对象之一相关联；接收对所述控件之一的选择；以及更新所述用户界面以示出与所述选择相关的信息。Optionally, the instruction code causes the processor to: determine one or more candidate second queries related to the first query and the determined object or objects; update the user interface to show one or more of the one or more candidate second queries; receiving a second query from the user input circuit, the second query corresponding to one of the one or more candidate second queries; determining one or more objects of the image associated with the first query and the second query based on the metadata; updating the user interface to show one or more controls, each control associated with One of the determined one or more objects is associated; a selection of one of the controls is received; and the user interface is updated to show information related to the selection.

可选的，所述指令代码使所述处理器：更新所述用户界面以在所述一个或多个控件中的每个控件上示出唯一标识符；从所述用户输入电路接收指定所述唯一标识符之一的命令；及显示与选择相关联的信息，所述选择与所指定的唯一标识符相关联。Optionally, the instruction code causes the processor to: update the user interface to show a unique identifier on each of the one or more controls; receive from the user input circuit specifying the a command for one of the unique identifiers; and displaying information associated with the selection associated with the specified unique identifier.

可选的，所述查询和所述选择对应于语音命令，所述指令代码使所述处理器：实现自然语言处理器；及通过所述自然语言处理器确定所述语音命令的含义。Optionally, the query and the selection correspond to voice commands, and the instruction code causes the processor to: implement a natural language processor; and determine the meaning of the voice command by the natural language processor.

可选的，所述元数据定义查询的层级。Optionally, the metadata defines the level of the query.

可选的，所述一个或多个控件中的每个控件对应于与所确定的一个或多个对象中的对象相关联的图像。Optionally, each of the one or more controls corresponds to an image associated with an object of the determined one or more objects.

可选的，所述指令代码使所述处理器更新所述用户界面以示出对应于所述第一查询和所述第二查询的短语，其中，所述短语随所述用户指定不同查询而实时更新。Optionally, the instruction code causes the processor to update the user interface to show phrases corresponding to the first query and the second query, wherein the phrases change as the user specifies a different query. Live Update.

可选的，当所述显示器示出所述一个或多个控件和与所述选择相关的信息时，所述视频内容继续进行流式传输。Optionally, the video content continues to be streamed while the display shows the one or more controls and information related to the selection.

在第八方面，一种用于控制显示设备的方法，包括：通过用户输入电路接收用户命令；及显示视频内容和用户界面。所述视频内容包括元数据。所述方法包括从所述用户输入电路接收对于当前显示的所述视频内容的图像的查询；基于所述元数据，确定所述图像的与所述查询相关联的一个或多个对象；更新所述用户界面以示出一个或多个控件，每个控件都与所确定的一个或多个对象之一相关联；接收对所述控件之一的选择；及更新所述用户界面以示出与所述选择相关的信息。In an eighth aspect, a method for controlling a display device comprising: receiving user commands through a user input circuit; and displaying video content and a user interface. The video content includes metadata. The method includes receiving a query for an image of the video content currently displayed from the user input circuit; determining one or more objects of the image associated with the query based on the metadata; updating the the user interface to show one or more controls, each control associated with one of the determined one or more objects; receiving a selection of one of the controls; and updating the user interface to show a the selection-related information.

可选的，所述的方法还包括：确定与所述第一查询及所确定的一个或多个对象相关的一个或多个候选的第二查询；更新所述用户界面以示出所述一个或多个候选的第二查询的中的一个或多个；从所述用户输入电路接收第二查询，所述第二查询对应于所述一个或多个候选的第二查询之一；基于所述元数据确定所述图像的与所述第一查询和所述第二查询相关联的一个或多个对象；更新所述用户界面以示出一个或多个控件，每个控件都与所确定的一个或多个对象之一相关联；接收对所述控件之一的选择；及更新所述用户界面以示出与所述选择相关的信息。Optionally, the method further comprises: determining one or more candidate second queries related to the first query and the determined object or objects; updating the user interface to show the one or more one or more of the one or more candidate second queries; receiving a second query from the user input circuit, the second query corresponding to one of the one or more candidate second queries; based on the the metadata determines one or more objects of the image associated with the first query and the second query; updating the user interface to show one or more controls, each control related to the determined is associated with one of the one or more objects of the control; receives a selection of one of the controls; and updates the user interface to show information related to the selection.

可选的，所述的方法还包括：更新所述用户界面以在所述一个或多个控件中的每个控件上示出唯一标识符；从所述用户输入电路接收指定所述唯一标识符之一的命令；显示与选择相关联的信息，所述选择与所指定的唯一标识符相关联。Optionally, the method further comprises: updating the user interface to show a unique identifier on each of the one or more controls; receiving from the user input circuit specifying the unique identifier A command for one of the; displays information associated with the selection associated with the specified unique identifier.

可选的，所述查询和所述选择对应于语音命令，其中所述方法还包括：实现自然语言处理器；及通过所述自然语言处理器确定所述语音命令的含义。Optionally, the query and the selection correspond to voice commands, wherein the method further comprises: implementing a natural language processor; and determining, by the natural language processor, the meaning of the voice command.

可选的，所述元数据定义查询的层级。Optionally, the metadata defines the level of the query.

可选的，所述一个或多个控件中的每个控件对应于与所确定的一个或多个对象中的对象相关联的图像。Optionally, each of the one or more controls corresponds to an image associated with an object of the determined one or more objects.

可选的，所述的方法还包括：示出对应于所述第一查询和所述第二查询的短语，其中，所述短语随所述用户指定不同查询而实时更新。Optionally, the method further includes: displaying phrases corresponding to the first query and the second query, wherein the phrases are updated in real time as the user specifies different queries.

可选的，在示出所述一个或多个控件及与所述选择相关的信息的同时，所述视频内容继续进行流式传输。Optionally, the video content continues to be streamed while the one or more controls and information related to the selection are shown.

在第九方面，提供了一种非易失性计算机可读介质，其存储用于控制显示设备的指令代码。所述指令代码能由计算机执行以使所述计算机：从所述计算机的用户输入电路接收对于当前在所述计算机的显示器上示出的视频内容的图像的查询；基于元数据确定所述图像的与所述查询相关联的一个或多个对象；更新所述计算机的用户界面以示出一个或多个控件，每个控件与所确定的一个或多个对象之一相关联；接收对所述控件之一的选择；及更新所述用户界面以示出与所述选择相关的信息。In a ninth aspect, a non-volatile computer-readable medium storing instruction code for controlling a display device is provided. The instruction code is executable by a computer to cause the computer to: receive, from a user input circuit of the computer, a query for an image of video content currently shown on a display of the computer; one or more objects associated with the query; updating a user interface of the computer to show one or more controls, each control associated with one of the determined one or more objects; receiving feedback on the selection of one of the controls; and updating the user interface to show information related to the selection.

可选的，所述指令代码使所述计算机：确定与所述第一查询及所确定的一个或多个对象相关的一个或多个候选的第二查询；更新所述用户界面以示出所述一个或多个候选的第二查询中的一个或多个；从所述用户输入电路接收第二查询，所述第二查询对应于所述一个或多个候选的第二查询之一；基于所述元数据确定所述图像的与所述第一查询和所述第二查询相关联的一个或多个对象；更新所述用户界面以示出一个或多个控件，每个控件都与所确定的一个或多个对象之一相关联；接收对所述控件之一的选择；及更新所述用户界面以示出与所述选择相关的信息。Optionally, the instruction code causes the computer to: determine one or more candidate second queries related to the first query and the determined object or objects; update the user interface to show the receiving one or more of the one or more candidate second queries; receiving a second query from the user input circuit, the second query corresponding to one of the one or more candidate second queries; based on The metadata determines one or more objects of the image associated with the first query and the second query; the user interface is updated to show one or more controls, each control related to the is associated with one of the determined one or more objects; receives a selection of one of the controls; and updates the user interface to show information related to the selection.

可选的，所述指令代码使所述计算机：更新所述用户界面以在所述一个或多个控件中的每个控件上示出唯一标识符；从所述用户输入电路接收指定所述唯一标识符之一的命令；以及显示与选择相关联的信息，所述选择与所指定的唯一标识符相关联。Optionally, the instruction code causes the computer to: update the user interface to show a unique identifier on each of the one or more controls; receive from the user input circuit specifying the unique identifier; a command for one of the identifiers; and displaying information associated with the selection associated with the specified unique identifier.

可选的，所述查询和所述选择对应于语音命令，所述指令代码使所述计算机：实现自然语言处理器；并通过所述自然语言处理器确定所述语音命令的含义。Optionally, the query and the selection correspond to voice commands, and the instruction code causes the computer to: implement a natural language processor; and determine the meaning of the voice command through the natural language processor.

在第十方面，一种显示设备，包括：用于接收用户命令的用户输入电路，及用于输出视频内容和用户界面的显示器。所述视频内容包括元数据。所述设备还包括与所述用户输入电路和所述显示器通信的处理器；和与所述处理器通信且存储指令代码的非易失性计算机可读介质，当由所述处理器执行时，所述指令代码使所述处理器：从所述用户接收暂停命令，从而暂停所述视频内容使得所述显示器示出静止图像；随后基于所述元数据确定所述静止图像中的一个或多个对象；更新所述用户界面以示出一个或多个控件，每个控件都与所确定的一个或多个对象之一相关联；接收对所述控件之一的选择；及更新所述用户界面以示出与所述选择相关的信息。In a tenth aspect, a display device includes user input circuitry for receiving user commands, and a display for outputting video content and a user interface. The video content includes metadata. The apparatus also includes a processor in communication with the user input circuitry and the display; and a non-transitory computer-readable medium in communication with the processor and storing instruction code that, when executed by the processor, The instruction code causes the processor to: receive a pause command from the user to pause the video content such that the display shows still images; then determine one or more of the still images based on the metadata object; update the user interface to show one or more controls, each control associated with the determined one or more objects; receive a selection of one of the controls; and update the user interface to show information related to the selection.

可选的，所述一个或多个控件的每个控件对应于与所确定的一个或多个对象中的对象相关联的图像。Optionally, each of the one or more controls corresponds to an image associated with an object of the determined one or more objects.

可选的，所述控件包括以下的至少一个：与所述对象之一相关的广告、分享所述视频内容的分享控件、对所述视频内容进行评分的评分控件、以及用于显示与所述对象之一相关的信息的信息控件。Optionally, the control includes at least one of the following: an advertisement related to one of the objects, a sharing control for sharing the video content, a scoring control for scoring the video content, and a An information control for information about one of the objects.

可选的，所示出的与所述选择相关的信息包括与URL相关联的QR代码，所述URL链接到与所述选择相关的信息。Optionally, the information shown related to the selection includes a QR code associated with a URL that links to the information related to the selection.

在第十一方面，一种用于控制显示设备的方法，包括：通过用户输入电路接收用户命令；并显示视频内容和用户界面。所述视频内容包括元数据。所述方法还包括：从用户接收暂停命令，从而暂停所述视频内容以示出静止图像；随后基于所述元数据确定所述静止图像中的一个或多个对象；更新所述用户界面以示出一个或多个控件，每个控件都与所确定的一个或多个对象之一相关联；接收对所述控件之一的选择；以及更新所述用户界面以示出与所述选择相关的信息。In an eleventh aspect, a method for controlling a display device comprising: receiving a user command through a user input circuit; and displaying video content and a user interface. The video content includes metadata. The method further includes: receiving a pause command from a user to pause the video content to show a still image; then determining one or more objects in the still image based on the metadata; and updating the user interface to show the still image. generating one or more controls, each control being associated with the determined one of the one or more objects; receiving a selection of one of the controls; and updating the user interface to show a selection associated with the selection information.

可选的，所述一个或多个控件中的每个控件对应于与所确定的一个或多个对象中的对象相关联的图像。Optionally, each of the one or more controls corresponds to an image associated with an object of the determined one or more objects.

可选的，所述控件包括以下的至少一个：与所述对象之一相关的广告、分享所述视频内容的分享控件、对所述视频内容进行评分的评分控件以及用于显示与所述对象之一相关的信息的信息控件。Optionally, the control includes at least one of the following: an advertisement related to one of the objects, a sharing control for sharing the video content, a scoring control for scoring the video content, and a display related to the object. One of the information controls for related information.

可选的，所示出的与所述选择相关的信息包括与URL相关联的QR码，所述URL链接到与所述选择相关的信息。Optionally, the displayed information related to the selection includes a QR code associated with a URL that links to the information related to the selection.

第十二方面，提供了一种非易失性计算机可读介质，其存储用于控制显示设备的指令代码。所述指令代码能由计算机执行以使所述计算机：从用户接收暂停命令，从而暂停视频内容以使所述计算机的显示器示出静止图像；随后基于所述视频内容的元数据确定所述静止图像中的一个或多个对象；更新所述计算机的用户界面以示出一个或多个控件，每个所述控件与所确定的一个或多个对象之一相关联；接收对所述控件之一的选择；及更新所述用户界面以示出与所述选择相关的信息。A twelfth aspect provides a non-volatile computer-readable medium storing instruction codes for controlling a display device. The instruction code is executable by a computer to cause the computer to: receive a pause command from a user to pause video content to cause a display of the computer to show a still image; then determine the still image based on metadata of the video content one or more objects in; updating the user interface of the computer to show one or more controls, each of the controls being associated with one of the determined one or more objects; receiving a response to one of the controls and updating the user interface to show information related to the selection.

可选的，所述一个或多个控件的每个控件对应于与所确定的一个或多个对象的对象相关联的图像。Optionally, each of the one or more controls corresponds to an image associated with the determined object of the one or more objects.

可选的，所述控件包括以下的至少一个：与所述对象之一相关的广告、用于分享所述视频内容的分享控件、用于对所述视频内容进行评分的评分控件以及用于显示与所述对象之一相关的信息的信息控件。Optionally, the control includes at least one of the following: an advertisement related to one of the objects, a sharing control for sharing the video content, a scoring control for scoring the video content, and a display for displaying An information control for information related to one of the objects.

可选的，所示出的与所述选择相关的信息包括与URL相关联的QR码，所述URL链接到与所述选择相关的信息。Optionally, the displayed information related to the selection includes a QR code associated with a URL that links to the information related to the selection.

在第十三方面，一种显示设备，包括：用于检测所述显示设备附近的个体的存在检测电路；用于显示视频内容和用户界面的显示器；与所述存在检测电路和所述显示器通信的处理器；以及与所述处理器通信且存储指令代码的非易失性计算机可读介质，当由所述处理器执行时，所述指令代码使所述处理器：从所述存在检测电路确定用户是否在所述显示设备附近；当确定所述用户不在所述显示设备附近时，暂停所述视频内容；及当随后确定所述用户在所述显示设备附近时，恢复所述视频内容。In a thirteenth aspect, a display device comprising: a presence detection circuit for detecting individuals in the vicinity of the display device; a display for displaying video content and a user interface; in communication with the presence detection circuit and the display a processor; and a non-transitory computer-readable medium in communication with the processor and storing instruction code that, when executed by the processor, causes the processor to: determining whether a user is near the display device; pausing the video content when it is determined that the user is not near the display device; and resuming the video content when it is subsequently determined that the user is near the display device.

可选的，所述存在检测电路包括用于拍摄所述显示设备前面的图像的成像器，所述指令代码使所述处理器：定期使所述成像器捕捉图像；分析所捕捉的图像以识别面部数据；以及将所述面部数据和与所述用户相关联的面部数据进行比较，以确定所述用户是否在所述显示设备附近。Optionally, the presence detection circuit includes an imager for capturing an image in front of the display device, and the instruction code causes the processor to: periodically cause the imager to capture an image; analyze the captured image to identify facial data; and comparing the facial data to facial data associated with the user to determine whether the user is near the display device.

可选的，在初始状态中，包括主要用户在内的多个用户在所述显示设备附近，当随后确定所述主要用户不在所述显示设备附近时，暂停所述视频内容并更新所述用户界面以指示所述视频内容已被暂停；及当随后确定所述主要用户在所述显示设备附近时，恢复所述视频内容并且更新所述用户界面以指示所述视频内容已被恢复。Optionally, in the initial state, multiple users including the main user are near the display device, and when it is subsequently determined that the main user is not near the display device, the video content is paused and the user is updated. an interface to indicate that the video content has been paused; and when it is subsequently determined that the primary user is in the vicinity of the display device, resuming the video content and updating the user interface to indicate that the video content has been resumed.

可选的，所述存在检测电路包括用于执行与所述显示设备附近的设备进行近场通信的近场通信电路，所述指令代码使所述处理器通过检测来自与所述用户相关联的便携式设备的近场通信，确定所述用户是否在所述显示设备附近。Optionally, the presence detection circuit includes a near-field communication circuit for performing near-field communication with a device near the display device, and the instruction code causes the processor to detect an incoming call from a device associated with the user. Near field communication of a portable device to determine if the user is in the vicinity of the display device.

可选的，当所述视频内容被暂停时，所述处理器更新所述显示设备上的用户界面以指示所述视频内容已被暂停；及当所述视频内容被恢复时，所述处理器更新所述显示设备上的用户界面以指示所述视频内容已被恢复。Optionally, when the video content is paused, the processor updates a user interface on the display device to indicate that the video content has been paused; and when the video content is resumed, the processor A user interface on the display device is updated to indicate that the video content has been restored.

可选的，当所述用户界面指示所述视频内容被暂停时，更新所述用户界面以示出与所述视频的内容相关的信息。Optionally, when the user interface indicates that the video content is paused, the user interface is updated to show information related to the content of the video.

可选的，与所述视频的内容相关的信息包括与所述内容相关的广告信息。Optionally, the information related to the content of the video includes advertisement information related to the content.

在第十四方面，一种用于控制显示设备的方法，包括：显示视频内容和用户界面；从存在检测电路确定用户是否在所述显示设备附近，所述存在检测电路被配置为检测所述显示设备附近存在的个体；当确定所述用户不在所述显示设备附近时，暂停所述视频内容；及当随后确定所述用户在所述显示设备附近时，恢复所述视频内容。In a fourteenth aspect, a method for controlling a display device, comprising: displaying video content and a user interface; and determining from a presence detection circuit whether a user is near the display device, the presence detection circuit configured to detect the displaying an individual present in the vicinity of a device; pausing the video content when it is determined that the user is not in the vicinity of the display device; and resuming the video content when it is subsequently determined that the user is in the vicinity of the display device.

可选的，所述存在检测电路包括用于捕获所述显示设备前面的图像的成像器，所述方法还包括：定期使所述成像器捕捉图像；分析所捕捉的图像以识别面部数据；以及将所述面部数据和与所述用户相关联的面部数据进行比较，以确定所述用户是否在所述显示设备附近。Optionally, the presence detection circuit includes an imager for capturing an image in front of the display device, the method further comprising: periodically causing the imager to capture an image; analyzing the captured image to identify facial data; and The facial data is compared to facial data associated with the user to determine whether the user is in the vicinity of the display device.

可选的，在初始状态中，包括主要用户在内的多个用户在所述显示设备附近，所述方法还包括：当随后确定所述主要用户不在所述显示设备附近时，暂停所述视频内容并更新所述用户界面以指示所述视频内容被暂停；及当随后确定所述主要用户在所述显示设备附近时，恢复所述视频内容并更新所述用户界面以指示所述视频内容已被恢复。Optionally, in the initial state, multiple users including the main user are near the display device, and the method further includes: when it is subsequently determined that the main user is not near the display device, pausing the video content and updating the user interface to indicate that the video content was paused; and when it is subsequently determined that the primary user is in the vicinity of the display device, resuming the video content and updating the user interface to indicate that the video content has been was restored.

可选的，所述存在检测电路包括用于执行与在所述显示设备附近的设备进行近场通信的近场通信电路，所述方法还包括通过检测与所述用户相关联的便携式设备的近场通信确定所述用户是否在所述显示设备附近。Optionally, the presence detection circuit includes a near-field communication circuit for performing near-field communication with a device in the vicinity of the display device, and the method further includes detecting the proximity of a portable device associated with the user by detecting the proximity of the user. Field communication determines whether the user is in the vicinity of the display device.

可选的，所述方法还包括：当所述视频内容被暂停时，更新所述显示设备上的用户界面以指示所述视频内容被暂停；及当所述视频内容被恢复时，更新所述显示设备上的用户界面以指示所述视频内容被恢复。Optionally, the method further comprises: when the video content is paused, updating a user interface on the display device to indicate that the video content is paused; and when the video content is resumed, updating the A user interface on the device is displayed to indicate that the video content is restored.

可选的，当所述用户界面指示所述视频内容被暂停时，所述方法包括更新所述用户界面以示出与所述视频的内容相关的信息。Optionally, when the user interface indicates that the video content is paused, the method includes updating the user interface to show information related to the content of the video.

可选的，与所述视频的内容相关的信息包括与所述内容相关的广告信息。Optionally, the information related to the content of the video includes advertisement information related to the content.

在第十五方面，提供了一种非易失性计算机可读介质，该介质存储用于控制显示设备的指令代码。所述指令代码可由计算机执行以使所述计算机执行：通过所述计算机的存在检测电路确定用户是否在所述显示设备附近，所述存在检测电路被配置为检测所述显示设备附近的个体；当确定所述用户不在所述显示设备附近时，暂停视频内容；及当随后确定所述用户在所述显示设备附近时，恢复所述视频内容。In a fifteenth aspect, a non-volatile computer-readable medium storing instruction code for controlling a display device is provided. The instruction code is executable by a computer to cause the computer to: determine whether a user is in the vicinity of the display device by a presence detection circuit of the computer, the presence detection circuit being configured to detect an individual in the vicinity of the display device; when When it is determined that the user is not in the vicinity of the display device, the video content is paused; and when it is subsequently determined that the user is in the vicinity of the display device, the video content is resumed.

可选的，所述存在检测电路包括用于拍摄所述显示设备前面的图像的成像器，其中所述指令代码使所述计算机执行：定期使所述成像器捕获图像；分析所捕获的图像以识别面部数据；及将所述面部数据和与所用户相关联的面部数据进行比较，以确定所述用户是否在所述显示设备附近。Optionally, the presence detection circuit includes an imager for capturing an image in front of the display device, wherein the instruction code causes the computer to: periodically cause the imager to capture an image; analyze the captured image to identifying facial data; and comparing the facial data to facial data associated with the user to determine whether the user is near the display device.

可选的，在初始状态中，包括主要用户在内的多个用户在所述显示设备附近，当随后确定所述主要用户不在所述显示设备附近时，所述指令代码使所述计算机暂停所述视频内容并更新所述用户界面以指示所述视频内容已被暂停；及当随后确定所述主要用户在所述显示设备附近时，所述指令代码使所述计算机恢复所述视频内容并更新所述用户界面以指示所述视频内容已被恢复。Optionally, in the initial state, multiple users including the main user are near the display device, and when it is subsequently determined that the main user is not near the display device, the instruction code causes the computer to suspend all operations. the video content and updating the user interface to indicate that the video content has been paused; and when the primary user is subsequently determined to be in the vicinity of the display device, the instruction code causes the computer to resume the video content and update the user interface to indicate that the video content has been restored.

可选的，所述存在检测电路包括用于执行与所述显示设备附近的设备进行近场通信的近场通信电路，所述指令代码使所述计算机通过检测与所述用户相关联的便携式设备的近场通信，确定所述用户是否在所述显示设备附近。Optionally, the presence detection circuit includes a near field communication circuit for performing near field communication with devices near the display device, and the instruction code causes the computer to detect a portable device associated with the user by detecting the near field communication to determine whether the user is in the vicinity of the display device.

可选的，当所述视频内容被暂停时，所述指令代码使所述计算机更新所述显示设备上的用户界面以指示所述视频内容被暂停；且当所述视频内容被恢复时，所述指令代码使所述计算机更新所述显示设备上的用户界面以指示所述视频内容被恢复。Optionally, when the video content is paused, the instruction code causes the computer to update a user interface on the display device to indicate that the video content is paused; and when the video content is resumed, the The instruction code causes the computer to update a user interface on the display device to indicate that the video content is restored.

可选的，当所述用户界面指示所述视频内容被暂停时，所述指令代码使所述计算机更新所述用户界面以示出与所述视频的内容相关的信息。Optionally, when the user interface indicates that the video content is paused, the instruction code causes the computer to update the user interface to show information related to the content of the video.

在第十六方面，一种显示设备，包括：用于检测所述显示设备附近的个体的存在检测电路；用于显示视频内容和用户界面的显示器；与用户输入电路、所述显示器和搜索历史数据库通信的处理器；和与所述处理器通信且存储指令代码的非易失性计算机可读介质，当由所述处理器执行时，所述指令代码使所述处理器：a)从所述存在检测电路确定在所述显示设备附近的用户；b)确定与所述用户相关联的一个或多个节目类型；c)确定与所确定的一个或多个节目类型匹配的可获得的节目；及d)更新所述用户界面以示出与所确定的一个或多个节目类型匹配的一个或多个所述可获得节目的列表。In a sixteenth aspect, a display device comprising: presence detection circuitry for detecting individuals in the vicinity of the display device; a display for displaying video content and a user interface; and user input circuitry, the display, and a search history a processor in communication with a database; and a non-transitory computer-readable medium in communication with the processor and storing instruction code that, when executed by the processor, causes the processor to: a) obtain an the presence detection circuitry determines a user in the vicinity of the display device; b) determines one or more program types associated with the user; c) determines available programs that match the determined one or more program types ; and d) updating the user interface to show a listing of one or more of the available programs that match the determined one or more program types.

可选的，所述指令代码使所处理器：接收所述用户的开机命令以使所述显示设备进入观看状态；及在接收所述开机命令之后，但在接收所述用户的任何后续命令之前，执行上述方面中的操作a)-d)。Optionally, the instruction code causes the processor to: receive a power-on command from the user to cause the display device to enter a viewing state; and after receiving the power-on command, but before receiving any subsequent commands from the user , perform operations a)-d) in the above aspects.

可选的，所述指令代码使所述处理器：从所述存在检测电路确定在所述显示设备附近的多个用户；基于存储在所述搜索历史数据库中的由所述多个用户先前观看的节目类型的历史记录，预测与所述多个用户相关联的一个或多个节目类型；从所预测的一个或多个节目类型确定对于所述多个用户中的各个用户共有的共有节目类型；确定与所述共有节目类型匹配的可获得节目；及更新所述用户界面以示出与所述共有节目类型匹配的一个或多个所述可获得节目的列表。Optionally, the instruction code causes the processor to: determine from the presence detection circuitry a plurality of users in the vicinity of the display device; based on previous viewing by the plurality of users stored in the search history database the history of the program types, predicting one or more program types associated with the plurality of users; determining a common program type common to each of the plurality of users from the predicted one or more program types ; determining available programs that match the common program type; and updating the user interface to show a listing of one or more of the available programs that match the common program type.

可选的，所述存在检测电路包括用于捕获所述显示设备前面的图像的成像器，所述指令代码使所述处理器：定期使所述成像器捕获图像；分析所捕获的图像以识别面部数据；以及将所述面部数据和与所述用户相关联的面部数据进行比较，以确定所述用户是否在所述显示设备附近。Optionally, the presence detection circuit includes an imager for capturing an image in front of the display device, and the instruction code causes the processor to: periodically cause the imager to capture an image; analyze the captured image to identify facial data; and comparing the facial data to facial data associated with the user to determine whether the user is near the display device.

可选的，所述存在检测电路包括用于执行与所述显示设备附近的设备进行近场通信的近场通信电路，其中，所述指令代码使所述处理器通过检测与所述用户相关联的便携式设备的近场通信确定所述用户是否在所述显示设备附近。Optionally, the presence detection circuit includes a near field communication circuit for performing near field communication with devices near the display device, wherein the instruction code causes the processor to associate with the user by detecting The near field communication of the portable device determines whether the user is in the vicinity of the display device.

可选的，所述显示设备还包括用于接收用户命令的所述用户输入电路，其中，所述指令代码使所述处理器：接收命令，所述命令用以选择所述可获得节目之一；以及使与所选择的可获得节目相关联的视频内容显示在所述显示器上。Optionally, the display device further includes the user input circuit for receiving a user command, wherein the instruction code causes the processor to: receive a command for selecting one of the available programs ; and causing video content associated with the selected available program to be displayed on the display.

可选的，所述命令对应于语音命令，所述指令代码使所述处理器：实现自然语言处理器；以及通过所述自然语言处理器确定所述语音命令的含义。Optionally, the command corresponds to a voice command, and the instruction code causes the processor to: implement a natural language processor; and determine the meaning of the voice command through the natural language processor.

可选的，与所述用户相关联的一个或多个节目类型的确定基于存储在与所述显示设备通信的所述搜索历史数据库中的由所述用户先前观看的节目类型的历史记录。Optionally, the determination of one or more program types associated with the user is based on a history of program types previously viewed by the user stored in the search history database in communication with the display device.

可选的，所述指令代码使所述处理器：接收用户的断电命令，从而使所述显示设备进入低功率状态并停用所述显示器；在接收所述断电命令之后，但在接收所述用户的任何后续命令之前执行上述方面所述的操作a)-d)；及在没有检测到对所述显示设备开机的用户指示预定时间之后，停用所述显示器。Optionally, the instruction code causes the processor to: receive a power-off command from a user, so that the display device enters a low-power state and disables the display; after receiving the power-off command, but after receiving the power-off command performing operations a)-d) of the above aspects prior to any subsequent commands by the user; and deactivating the display after a predetermined period of time when no user instruction to power on the display device is detected.

可选的，在停用所述显示器之后并且在所述预定时间之前，所述指令代码使所述处理器：预测与所述用户相关联的一个或多个信息类型；以及更新所述用户界面以示出属于所预测的一个或多个信息类型的信息。Optionally, after deactivating the display and before the predetermined time, the instruction code causes the processor to: predict one or more types of information associated with the user; and update the user interface to show information belonging to the predicted information type or types.

在第十七方面，一种用于控制显示设备的方法，包括：a)提供用于检测所述显示设备附近的个体的存在检测电路；b)显示视频内容和用户界面；c)从所述存在检测电路确定在所述显示设备附近的用户；d)确定与所述用户相关联的一个或多个节目类型；e)确定与所确定的一个或多个节目类型匹配的可获得节目；及f)更新所述用户界面以示出与所确定的一个或多个节目类型匹配的一个或多个所述可获得节目的列表。In a seventeenth aspect, a method for controlling a display device, comprising: a) providing a presence detection circuit for detecting an individual in the vicinity of the display device; b) displaying video content and a user interface; presence detection circuitry determines a user in the vicinity of the display device; d) determines one or more program types associated with the user; e) determines available programs matching the determined one or more program types; and f) updating the user interface to show a listing of one or more of the available programs that match the determined one or more program types.

可选的，所述方法还包括：从所述用户接收开机命令以使所述显示设备进入观看状态；及在接收所述开机命令之后，但在从所述用户接收任何后续命令之前，执行操作c)-f)。Optionally, the method further comprises: receiving a power-on command from the user to cause the display device to enter a viewing state; and after receiving the power-on command, but before receiving any subsequent commands from the user, performing an operation c)-f).

可选的，所述方法还包括：从所述存在检测电路确定在所述显示设备附近的多个用户；基于存储在搜索历史数据库中的由所述多个用户先前观看的节目类型的历史记录预测与所述多个用户相关联的一个或多个节目类型；从所预测的一个或多个节目类型确定所述多个用户中的各个用户共有的共有节目类型；确定与所述共有节目类型匹配的可获得节目；及更新所述用户界面以示出与所述共有节目类型匹配的一个或多个可获得节目的列表。Optionally, the method further comprises: determining from the presence detection circuit a plurality of users in the vicinity of the display device; based on a history of program types previously viewed by the plurality of users stored in a search history database predicting one or more program types associated with the plurality of users; determining a common program type common to each of the plurality of users from the predicted one or more program types; determining a common program type associated with the plurality of users matching available programs; and updating the user interface to show a listing of one or more available programs matching the common program type.

可选的，所述存在检测电路包括：用于捕获所述显示设备前面的图像的成像器，所述方法还包括：定期使所述成像器捕获图像；分析所捕获的图像以识别面部数据；以及将所述面部数据和与所述用户相关联的面部数据进行比较，以确定所述用户是否在所述显示设备附近。Optionally, the presence detection circuit includes: an imager for capturing an image in front of the display device, the method further comprising: periodically causing the imager to capture an image; analyzing the captured image to identify facial data; and comparing the facial data to facial data associated with the user to determine whether the user is in the vicinity of the display device.

可选的，所述存在检测电路包括用于执行与在所述显示设备附近的设备进行近场通信的近场通信电路，所述方法还包括通过检测与所述用户相关联的便携式设备的近场通信确定所述用户是否在所述显示设备附近。Optionally, the presence detection circuit includes a near-field communication circuit for performing near-field communication with a device in the vicinity of the display device, and the method further includes detecting the proximity of a portable device associated with the user by detecting the proximity of the user. Field communication determines whether the user is in the vicinity of the display device.

可选的，所述的方法还包括：通过用户输入电路接收用户命令，接收命令，所述命令用以选择一个可获得节目；以及使与所选择的可获得节目相关联的视频内容显示在所述显示设备上。Optionally, the method further includes: receiving a user command through the user input circuit, receiving the command, the command is used to select an available program; and causing the video content associated with the selected available program to be displayed on the selected available program. on the display device described above.

可选的，所述命令对应于语音命令，所述方法还包括：实现自然语言处理器；以及通过所述自然语言处理器确定所述语音命令的含义。Optionally, the command corresponds to a voice command, and the method further includes: implementing a natural language processor; and determining the meaning of the voice command by the natural language processor.

可选的，与所述用户相关联的所述一个或多个节目类型的确定基于存储在与所述显示设备通信的所述搜索历史数据库中的由所述用户先前观看的节目类型的历史记录。Optionally, the determination of the one or more program types associated with the user is based on a history of program types previously viewed by the user stored in the search history database in communication with the display device .

可选的，所述方法还包括：接收所述用户的断电命令，从而使所述显示设备进入低功率状态并停用显示器；在接收所述断电命令之后，但从所述用户接收任何后续命令之前，执行操作c)-f)；以及在没有检测到对所述显示设备开机的用户指示预定时间之后，停用所述显示器。Optionally, the method further comprises: receiving a power-off command from the user, thereby causing the display device to enter a low power state and deactivating the display; after receiving the power-off command, but receiving any Before subsequent commands, operations c)-f) are performed; and the display is deactivated after a predetermined period of time after no user instruction to power on the display device is detected.

可选的，在停用所述显示器之后并在所述预定时间之前，所述方法还包括：预测与所述用户相关联的一个或多个信息类型；以及更新所述用户界面以示出属于所预测的一个或多个信息类型的信息。Optionally, after deactivating the display and before the predetermined time, the method further comprises: predicting one or more types of information associated with the user; and updating the user interface to show belonging to Information of one or more types of information predicted.

在第十八方面，一种显示设备，包括：用于显示视频内容和用户界面的显示器；与存在检测电路和所述显示器通信的处理器；以及与所述处理器通信且存储指令代码的非易失性计算机可读介质，当由所述处理器执行时，所述指令代码使所述处理器：接收将智能家电状态与显示设备使用情况关联的数据；确定当前显示设备使用情况；基于所接收的数据确定与所述当前显示设备使用情况对应的建议的智能家电状态；以及将智能家电调整至所确定的状态。In an eighteenth aspect, a display device comprising: a display for displaying video content and a user interface; a processor in communication with a presence detection circuit and the display; and a non-operating device in communication with the processor and storing instruction code A volatile computer-readable medium, when executed by the processor, the instruction code causes the processor to: receive data associating smart appliance status with display device usage; determine current display device usage; The received data determines a suggested state of the smart home appliance corresponding to the current display device usage; and adjusts the smart home appliance to the determined state.

可选的，所述智能家电状态限定所述智能家电的激活状态，所述显示设备使用情况限定以下的一个或多个：所述显示设备的使用时间、在所述显示设备上观看的节目类型以及所述显示设备的特定用户。Optionally, the status of the smart home appliance defines the activation status of the smart home appliance, and the usage of the display device defines one or more of the following: the usage time of the display device, the type of programs watched on the display device and the specific user of the display device.

可选的，所述显示设备包括所述存在检测电路，其用于检测在所述显示设备附近的特定用户，所述存在检测电路包括用于捕获所述显示设备前面的图像的成像器，所述指令代码使所述处理器：定期使所述成像器捕获图像；分析所捕获的图像以识别面部数据；以及将所述面部数据和与多个用户相关联的面部数据进行比较以确定所述特定用户是否在所述显示设备附近。Optionally, the display device includes the presence detection circuit for detecting a particular user in the vicinity of the display device, the presence detection circuit including an imager for capturing an image in front of the display device, the The instruction code causes the processor to: periodically cause the imager to capture images; analyze the captured images to identify facial data; and compare the facial data to facial data associated with a plurality of users to determine the Whether a particular user is in the vicinity of the display device.

可选的，所述显示设备包括：通信电路，其用于从各智能家电接收新状态信息，以及数据库，其用于存储所述各智能家电的新状态信息和定义所述显示设备的新显示设备使用情况信息，所述指令代码使所述处理器：用所述各智能家电的新状态信息和所述显示设备的新显示设备使用情况信息不断更新所述数据库；及将所述各智能家电的新状态信息和与所述显示设备相关联的新显示设备使用情况信息相关联以形成所述智能家电状态和所述显示设备使用情况之间的关系。Optionally, the display device includes: a communication circuit for receiving new status information from each smart home appliance, and a database for storing the new status information for each smart home appliance and defining a new display of the display device device usage information, the instruction code causes the processor to: continuously update the database with the new status information of the smart home appliances and the new display device usage information of the display device; and update the smart home appliances The new state information of the device is associated with the new display device usage information associated with the display device to form a relationship between the smart home appliance state and the display device usage.

在第十九方面，一种用于控制显示设备的方法，包括：显示视频内容和用户界面；接收将智能家电状态与显示设备使用情况相关联的数据；确定当前显示设备使用情况；基于所接收的数据确定与所述当前显示设备使用情况相对应的建议的智能家电状态；以及将智能家电调整至所确定的状态。In a nineteenth aspect, a method for controlling a display device, comprising: displaying video content and a user interface; receiving data associating smart appliance status with display device usage; determining current display device usage; Determine the suggested state of the smart home appliance corresponding to the current usage of the display device; and adjust the smart home appliance to the determined state.

可选的，所述智能家电状态限定所述智能家电的激活状态，所述显示设备使用情况限定以下的一个或多个：所述显示设备的使用时间、在所述显示设备上观看的节目类型以及所述显示设备的特定用户。Optionally, the status of the smart home appliance defines the activation status of the smart home appliance, and the usage of the display device defines one or more of the following: the usage time of the display device, the type of programs watched on the display device and the specific user of the display device.

可选的，所述显示设备包括：用于检测所述显示设备附近的特定用户的存在检测电路，所述存在检测电路包括用于捕获所述显示设备前面的图像的成像器，所述方法还包括：定期使所述成像器捕获图像；分析所捕获的图像以识别面部数据；及将所述面部数据和与多个用户相关联的面部数据进行比较以确定所述特定用户是否在所述显示设备附近。Optionally, the display device includes a presence detection circuit for detecting a specific user in the vicinity of the display device, the presence detection circuit including an imager for capturing an image in front of the display device, the method further comprising: periodically causing the imager to capture images; analyzing the captured images to identify facial data; and comparing the facial data to facial data associated with multiple users to determine whether the particular user is on the display near the device.

可选的，所述显示设备包括：通信电路，其用于从各智能家电接收新状态信息和数据库，其用于存储所述各智能家电的新状态信息和定义所述显示设备的新显示设备使用情况的信息，所述方法还包括：用所述各智能家电的新状态信息和所述显示设备的新显示设备使用情况信息不断更新所述数据库；及将所述各智能家电的新状态信息和与所述显示设备相关联的新显示设备使用情况信息相关联以形成所述各智能家电状态和所述显示设备使用情况之间的关系。Optionally, the display device includes: a communication circuit for receiving new status information from each smart home appliance and a database for storing the new status information for each smart home appliance and defining a new display device for the display device The method further includes: continuously updating the database with the new status information of the smart home appliances and the new display device usage information of the display device; and updating the new status information of the smart home appliances It is associated with new display device usage information associated with the display device to form a relationship between the states of the smart home appliances and the display device usage.

在第二十方面，提供了一种非易失性计算机可读介质，其上存储用于控制显示设备的指令代码。所述指令代码能由计算机执行以使所述计算机：接收将智能家电状态与显示设备使用情况相关联的数据；确定当前显示设备使用情况；基于所接收的数据确定与所述当前显示设备使用情况相对应的建议的智能家电状态；以及将智能家电调整至所确定的状态。In a twentieth aspect, there is provided a non-transitory computer-readable medium having stored thereon instruction code for controlling a display device. The instruction code is executable by a computer to cause the computer to: receive data associating smart appliance status with display device usage; determine current display device usage; determine a correlation with the current display device usage based on the received data corresponding suggested state of the smart home appliance; and adjusting the smart home appliance to the determined state.

可选的，所述智能家电状态限定所述智能家电的激活状态，所述显示设备使用情况限定以下的一个或多个：所述显示设备的使用时间、在所述显示设备上观看的节目类型及所述显示设备的特定用户。Optionally, the status of the smart home appliance defines the activation status of the smart home appliance, and the usage of the display device defines one or more of the following: the usage time of the display device, the type of programs watched on the display device and the specific user of the display device.

可选的，所述显示设备包括用于检测所述显示设备附近的特定用户的存在检测电路，所述存在检测电路包括用于捕获所述显示设备前面的图像的成像器，所述指令代码使所述计算机：定期使所述成像器捕获图像；分析所捕获的图像以识别面部数据；及将所述面部数据和与多个用户相关联的面部数据进行比较以确定所述特定用户是否在所述显示设备附近。Optionally, the display device includes a presence detection circuit for detecting a specific user in the vicinity of the display device, the presence detection circuit includes an imager for capturing an image in front of the display device, the instruction code causes the The computer: periodically causes the imager to capture images; analyzes the captured images to identify facial data; and compares the facial data to facial data associated with multiple users to determine whether the particular user is near the display device described above.

可选的，所述显示设备包括通信电路，其用于从各智能家电接收新状态信息，以及数据库，其用于存储所述各智能家电的新状态信息和定义所述显示设备的新显示设备使用情况的信息，所述指令代码使所述计算机：用所述各智能家电的新状态信息和所述显示设备的新显示设备使用情况信息不断更新所述数据库；以及将所述智能家电的新状态信息和与所述显示设备相关联的新显示设备使用情况信息相关联以形成所述智能家电状态和所述显示设备使用情况之间的关系。Optionally, the display device includes a communication circuit for receiving new status information from each smart home appliance, and a database for storing the new status information for each smart home appliance and defining a new display device for the display device. information of usage, the instruction code causes the computer to: continuously update the database with the new status information of each smart home appliance and the new display device usage information of the display device; and update the new status information of the smart home appliance The state information is associated with new display device usage information associated with the display device to form a relationship between the smart appliance state and the display device usage.

附图说明Description of drawings

图1示出了示例性环境，显示设备在所述示例性环境中操作；FIG. 1 illustrates an exemplary environment in which a display device operates;

图2示出了用于增强视频内容的导航的示例性操作。FIG. 2 illustrates exemplary operations for enhancing navigation of video content.

图3A-图3C示出了在图2的操作期间可以呈现给用户的示例性用户界面；3A-3C illustrate exemplary user interfaces that may be presented to a user during the operation of FIG. 2;

图4示出了便于定位特定类型的视频内容的示例性操作；4 illustrates exemplary operations for facilitating locating specific types of video content;

图5示出了在图4的操作期间可以呈现给用户的示例性用户界面；FIG. 5 illustrates an exemplary user interface that may be presented to a user during the operation of FIG. 4;

图6示出了用于确定与视频内容中的图像相关的信息的示例性操作。6 illustrates example operations for determining information related to images in video content.

图7A和图7B示出了在图6的操作期间可以呈现给用户的示例性用户界面；7A and 7B illustrate exemplary user interfaces that may be presented to a user during the operation of FIG. 6;

图8示出了用于确定与视频内容中的图像相关的信息的可选示例性操作；8 illustrates optional exemplary operations for determining information related to images in video content;

图9A和图9B示出了在图8的操作期间可以呈现给用户的示例性用户界面；9A and 9B illustrate exemplary user interfaces that may be presented to a user during the operation of FIG. 8;

图10示出了用于自动暂停视频内容的可选示例性操作；Figure 10 illustrates optional exemplary operations for automatically pausing video content;

图11A和图11B示出了在图10的操作期间可以呈现给用户的示例性用户界面；11A and 11B illustrate exemplary user interfaces that may be presented to a user during the operations of FIG. 10;

图12示出了用于自动暂停视频内容的可选示例性操作；Figure 12 illustrates optional exemplary operations for automatically pausing video content;

图13A-图13D示出了在图12的操作期间可以呈现给用户的示例性用户界面；13A-13D illustrate exemplary user interfaces that may be presented to a user during the operations of FIG. 12;

图14示出了用于基于所检测到的用户使用习惯调整各种智能家电的示例性操作；14 illustrates exemplary operations for adjusting various smart home appliances based on detected user usage habits;

图15A-图15B示出了在图14的操作期间可以呈现给用户的示例性用户界面；以及15A-15B illustrate exemplary user interfaces that may be presented to a user during the operations of FIG. 14; and

图16示出了可以形成附图或以下段落中描述的系统的一部分或实现这些系统的示例性计算机系统。16 illustrates an exemplary computer system that may form part of, or implement, the systems described in the accompanying figures or in the following paragraphs.

具体实施方式Detailed ways

以下描述的实施方案涉及便于以智能、易于使用的方式访问电视特征的各种用户界面实施方式。通常，所述用户界面依赖于各种机器学习技术，这些技术便于以最少的步骤访问这些特征和其它信息。所述用户界面被配置为是直观的且只需要最低学习时间就可以熟练地导航用户界面。The embodiments described below relate to various user interface implementations that facilitate access to television features in an intelligent, easy-to-use manner. Typically, the user interface relies on various machine learning techniques that facilitate access to these features and other information with a minimum of steps. The user interface is configured to be intuitive and requires minimal learning time to navigate the user interface proficiently.

图1示出了显示设备在其中操作的示例性环境。示出了显示设备100、一组移动设备105、GPS网络110、计算机网络115、一组社交媒体服务器120、一组内容服务器125、支持服务器127以及可以观看显示设备100和/或与显示设备100交互的一个或多个用户。显示设备100、社交媒体服务器120、内容服务器125和支持服务器127可以经由诸如因特网、有线网络、卫星网络等的网络107彼此通信。FIG. 1 illustrates an exemplary environment in which a display device operates. Shown is adisplay device 100 , a set ofmobile devices 105 , a GPS network 110 , acomputer network 115 , a set ofsocial media servers 120 , a set ofcontent servers 125 , asupport server 127 , anddisplay device 100 may be viewed and/or associated withdisplay device 100 One or more users who interact.Display device 100,social media server 120,content server 125, andsupport server 127 may communicate with each other via network 107, such as the Internet, a cable network, a satellite network, and the like.

社交媒体服务器120一般对应于承载可与显示设备100的用户130相关的公开可用信息的计算机系统。例如，社交媒体服务器120可以是

等。社交媒体服务器120可以包括博客、论坛和/或可以从其获得与用户130相关的信息的任何其他系统或网站。Social media server 120 generally corresponds to a computer system that hosts publicly available information that may be relevant touser 130 ofdisplay device 100 . For example,social media server 120 may be

Wait.Social media server 120 may include blogs, forums, and/or any other system or website from which information related touser 130 may be obtained.

移动设备105可以对应于由用户130中的一者或多者携带的移动电话、平板电脑等。移动设备105可以包括有助于与显示设备100直接通信的短程通信电路。例如，移动设备105可以包括

电路、近场通信电路等。当给定的移动设备105在显示设备100附近时，通信电路有助于检测给定的移动设备105。这反过来可以便于由显示设备100确定在显示设备100的观看距离内存在用户130。Mobile device 105 may correspond to a mobile phone, tablet, etc. carried by one or more ofusers 130 .Mobile device 105 may include short-range communication circuitry that facilitates direct communication withdisplay device 100 . For example,mobile device 105 may include

circuits, near field communication circuits, etc. The communication circuitry facilitates the detection of a givenmobile device 105 when the givenmobile device 105 is in the vicinity of thedisplay device 100 . This in turn may facilitate determination bydisplay device 100 thatuser 130 is present within the viewing distance ofdisplay device 100 .

GPS网络110和计算机网络115可以将信息传送至显示设备100，所述信息反过来可以便于显示设备100确定显示设备100的大致位置(general location)。例如，GPS网络110可以传送有助于确定显示设备100的相对精确位置的信息。计算机网络115可以向显示设备100分配IP地址，所述IP地址可以与诸如城市或其他地理区域的大致位置相关联。GPS network 110 andcomputer network 115 may communicate information to displaydevice 100 , which in turn may facilitatedisplay device 100 in determining the general location ofdisplay device 100 . For example, the GPS network 110 may transmit information useful in determining the relatively precise location of thedisplay device 100 .Computer network 115 may assigndisplay device 100 an IP address, which may be associated with an approximate location such as a city or other geographic area.

内容服务器125通常对应于承载视频内容的计算机系统。例如，内容服务器125可以对应于由有线电视提供商、网络提供商等运营的前端设备。在一些情况下，内容服务器125可以存储视频内容，例如电影、电视节目、体育节目等。Content server 125 generally corresponds to a computer system that hosts video content. For example,content server 125 may correspond to head-end equipment operated by a cable television provider, network provider, or the like. In some cases, thecontent server 125 may store video content, such as movies, television programs, sports programs, and the like.

在一些情况下，视频内容可以包括定义视频内容的各个方面的元数据。例如，与体育比赛相关联的元数据可以包括与比赛的各种事件，例如进球、点球等，相关的信息时间戳(information timestamp)、静止图像等。元数据可以包括与视频内容中示出的不同个体相关联的信息，例如选手、教练等的姓名。In some cases, the video content may include metadata that defines various aspects of the video content. For example, metadata associated with a sports game may include information timestamps, still images, etc. associated with various events of the game, such as goals, penalties, etc. The metadata may include information associated with the different individuals shown in the video content, such as the names of players, coaches, and the like.

视频内容中的元数据可以包括便于确定视频内容是否为特定类型(例如喜剧、戏剧、体育、冒险等)的信息。元数据可包括与视频内容中所示出的不同个体相关联的信息，例如视频内容中所示出的演员的姓名。元数据可以包括与视频内容中示出的不同物体，诸如个人所穿的服装、个人所携带的个人物品以及在视频内容中示出的各种物体相关联的信息。Metadata in video content may include information that facilitates determining whether the video content is of a particular genre (eg, comedy, drama, sports, adventure, etc.). The metadata may include information associated with different individuals shown in the video content, such as the names of actors shown in the video content. The metadata may include information associated with different objects shown in the video content, such as clothing worn by the individual, personal items carried by the individual, and various objects shown in the video content.

元数据可以已经通过用于识别视频内容中的个体、场景、事件等的各种机器学习技术预先自动生成。另外或可选地，机器学习技术可以在做出所述确定时使用某种形式的人工辅助。The metadata may have been automatically generated in advance by various machine learning techniques for identifying individuals, scenes, events, etc. in the video content. Additionally or alternatively, machine learning techniques may use some form of human assistance in making the determination.

支持服务器127通常对应于配置为向显示设备100提供高级服务的计算机系统。例如，支持服务器127可以对应于高端计算机，所述高端计算机被配置为执行各种机器学习技术，以确定语音命令的含义、预测对语音命令的响应等。支持服务器127可以从显示设备100接收语音命令和其他类型的命令，并且将与所述命令相关联的响应传送回显示设备。Thesupport server 127 generally corresponds to a computer system configured to provide advanced services to thedisplay device 100 . For example,support server 127 may correspond to a high-end computer configured to perform various machine learning techniques to determine the meaning of voice commands, predict responses to voice commands, and the like.Support server 127 may receive voice commands and other types of commands fromdisplay device 100 and transmit responses associated with the commands back to the display device.

显示设备100可以对应于具有增强的用户界面能力的电视或其他观看设备。显示设备100可以包括CPU 150、视频处理器160、I/O接口155、AI处理器165、显示器175、支持数据库153和指令存储器170。Display device 100 may correspond to a television or other viewing device with enhanced user interface capabilities. Thedisplay device 100 may include aCPU 150 , a video processor 160 , an I/O interface 155 , anAI processor 165 , adisplay 175 , asupport database 153 and aninstruction memory 170 .

CPU 150可对应于处理器，例如基于

等的处理器。CPU150可以执行操作系统，例如

或适于在显示设备内执行的其它操作系统。与操作系统相关联并用于控制显示设备100的各个方面的指令代码可以存储在指令存储器170内。例如，存储在指令存储器170中的指令代码可以有助于控制CPU 150向I/O接口155传送信息以及从I/O接口155接收信息。CPU 150可以处理从I/O接口155接收的视频内容，并且将处理后的视频内容传送至显示器175。CPU 150可以生成便于控制显示设备的不同方面的各种用户界面。CPU 150 may correspond to a processor, such as based on

etc. processor.CPU 150 may execute an operating system such as

or other operating system suitable for execution within the display device. Instruction code associated with the operating system and used to control various aspects of thedisplay device 100 may be stored within theinstruction memory 170 . For example, instruction code stored ininstruction memory 170 may assist in controllingCPU 150 to transmit and receive information to and from I/O interface 155 . TheCPU 150 may process video content received from the I/O interface 155 and transmit the processed video content to thedisplay 175 . TheCPU 150 may generate various user interfaces that facilitate controlling different aspects of the display device.

I/O接口155被配置为与各种类型的硬件连接，并将从硬件接收的信息传送至CPU。例如，I/O接口155可以耦联至一个或多个有助于从移动终端105、GPS网络110、计算机网络115、智能家电117等接收信息的天线。I/O接口可以与布置在显示设备100的面上的成像器151连接，以便于拍摄显示设备附近的个体的图像。I/O接口可以与布置在显示设备100上的一个或多个麦克风152连接，以便于捕获可由用户130传达的语音指令。The I/O interface 155 is configured to connect with various types of hardware, and transmit information received from the hardware to the CPU. For example, I/O interface 155 may be coupled to one or more antennas that facilitate receiving information frommobile terminal 105, GPS network 110,computer network 115,smart appliances 117, and the like. The I/O interface may be connected to theimager 151 arranged on the face of thedisplay device 100 in order to take images of individuals in the vicinity of the display device. The I/O interface may be connected to one ormore microphones 152 disposed on thedisplay device 100 to facilitate capturing voice commands that may be conveyed by theuser 130 .

AI处理器165可以对应于专门配置为执行诸如自然语言处理、静止和运动图像处理、语音处理等AI操作的处理器。例如，AI处理器165可被配置为执行语音识别以识别通过麦克风接收的语音命令。AI处理器165可包括面部识别功能以识别由成像器拍摄的图像中的个体。在一些实施方式中，AI处理器165可被配置为分析从一个或多个内容服务器传送来的内容以识别所述内容内的对象。TheAI processor 165 may correspond to a processor specially configured to perform AI operations such as natural language processing, still and moving image processing, speech processing, and the like. For example, theAI processor 165 may be configured to perform speech recognition to recognize voice commands received through a microphone.AI processor 165 may include facial recognition functionality to identify individuals in images captured by the imager. In some embodiments, theAI processor 165 may be configured to analyze content delivered from one or more content servers to identify objects within the content.

下面说明在提供智能用户界面时由CPU 150和/或显示设备100的其它模块执行的示例性操作。在这点上，所述操作可以经由存储在位于子系统内的非易失性计算机可读介质170中的指令代码实现，所述指令代码被配置成使相应的子系统执行附图中示出的并在本文中讨论的操作。Exemplary operations performed by theCPU 150 and/or other modules of thedisplay device 100 in providing an intelligent user interface are described below. In this regard, the operations may be implemented via instruction code stored in a non-transitory computer-readable medium 170 located within the subsystems, the instruction code being configured to cause the respective subsystems to perform the operations shown in the figures and the operations discussed in this article.

图2示出了用于增强视频内容的导航的示例性操作。参考图3A-图3C，可以更好地理解图2的操作。FIG. 2 illustrates exemplary operations for enhancing navigation of video content. The operation of FIG. 2 may be better understood with reference to FIGS. 3A-3C .

在步骤200处，显示设备100可以正在显示视频内容，诸如足球比赛，如图3A所示。然后用户130可以向显示设备100发出第一场景命令305，以使显示设备100搜索视频内容中的场景。例如，用户130可以仅大声地说出，“向我显示所有进球(Show me all thegoals)”。在这种情况下，由CPU 150单独或与AI处理器165协作实现的自然语言处理器可以确定语音命令的含义。另外或可选地，与语音命令相关联的数据可被传送至支持服务器127，然后支持服务器127可确定语音命令的含义并将所确定的含义回传至显示设备。At step 200, thedisplay device 100 may be displaying video content, such as a football game, as shown in Figure 3A. Theuser 130 may then issue afirst scene command 305 to thedisplay device 100 to cause thedisplay device 100 to search for a scene in the video content. For example,user 130 may simply say out loud, "Show me all the goals." In this case, a natural language processor implemented byCPU 150 alone or in cooperation withAI processor 165 can determine the meaning of the voice command. Additionally or alternatively, data associated with the voice command may be communicated to supportserver 127, which may then determine the meaning of the voice command and transmit the determined meaning back to the display device.

如图3A所示，在一些实施方式中，用户界面300可包括被实时更新以示出与用户发出的命令相关联的文本的短语控件310。As shown in FIG. 3A, in some implementations, theuser interface 300 may include aphrase control 310 that is updated in real time to show text associated with commands issued by the user.

在步骤205处，响应于第一场景命令305，显示设备100可以确定视频内容中属于场景类型的场景，所述场景类型与所述第一场景命令305相关联。在这点上，CPU 150单独地或与AI处理器165协作可以实现各种机器学习技术，所述机器学习技术利用与视频内容相关联的元数据确定视频内容中的属于场景类型的场景。另外或可选地，第一场景命令305可以被传送至支持服务器127，支持服务器127可以确定场景类型并传送所述场景类型至显示设备100。At step 205 , in response to thefirst scene command 305 , thedisplay device 100 may determine a scene in the video content that belongs to the scene type associated with thefirst scene command 305 . In this regard, theCPU 150, alone or in cooperation with theAI processor 165, may implement various machine learning techniques that utilize metadata associated with the video content to determine scenes in the video content that are of a scene type. Additionally or alternatively, thefirst scene command 305 may be transmitted to thesupport server 127 , which may determine the scene type and transmit the scene type to thedisplay device 100 .

在步骤210处，可以更新显示设备100的用户界面300，以示出与所确定的场景相关联的场景图像320。例如，可以在用户界面300上显示来自与场景相关联的视频内容元数据的图像320。图像320可以对应于与场景相关联的静止图像和/或图像序列或视频。At step 210, theuser interface 300 of thedisplay device 100 may be updated to show thescene image 320 associated with the determined scene. For example, animage 320 from video content metadata associated with a scene may be displayed on theuser interface 300 .Image 320 may correspond to a still image and/or sequence of images or video associated with the scene.

在一些实施方式中，用户界面300可以被更新以在每个图像上或每个图像附近显示唯一标识符325。在一些实施方式中，标识符以显著的方式在每个图像上的一部分上叠加展示。In some implementations, theuser interface 300 may be updated to display theunique identifier 325 on or near each image. In some embodiments, the identifiers are displayed in a prominent manner overlaid on a portion of each image.

在步骤215处，用户130可以指定第二场景命令，所述第二场景命令指定唯一标识符325之一。例如，用户130可以指定“1”以选择与第一图像320相关联的场景，该唯一的标识符与关联的场景对应。在一些实施例中，该唯一的标识符采用图3A所示的诸如阿拉伯数字之类的易于用户说出并且易于显示设备本身或者服务器识别的识别符。在一些实施例中，用户通过说出阿拉伯数字(例如，说出“1”)作为第二场景命令。在一些实施例中，用户通过按压控制装置(例如遥控器)上相应的数字按键(例如，1)来发出第二场景命令。At step 215 , theuser 130 may specify a second scene command that specifies one of theunique identifiers 325 . For example,user 130 may specify "1" to select the scene associated withfirst image 320, the unique identifier corresponding to the associated scene. In some embodiments, the unique identifier employs an identifier such as Arabic numerals shown in FIG. 3A that is easy for the user to speak and easy to display for identification by the device itself or the server. In some embodiments, the user commands the second scene by speaking an Arabic numeral (eg, speaking "1"). In some embodiments, the user issues the second scene command by pressing a corresponding number key (eg, 1) on a control device (eg, a remote control).

在步骤220处，可以在用户界面300上显示与所指定的唯一标识符325(例如，“1”)相关联的视频内容，如图3C所示。At step 220, the video content associated with the specified unique identifier 325 (eg, "1") may be displayed on theuser interface 300, as shown in Figure 3C.

返回至步骤200，在一些实施方式中，用户130可以通过指定额外信息细化场景命令。例如，响应于在步骤200处接收第一场景命令305，在步骤225处，可以确定与第一场景命令305相关的一个或多个可能的场景命令315。由CPU 150、AI处理器165和/或支持服务器127实施的机器学习技术可用于确定与第一场景命令305相关的可能场景命令。在这点上，视频内容中的元数据可以定义在确定与给定的第一场景命令305相关的可能的场景命令时由机器学习技术使用的场景命令的层级。Returning to step 200, in some embodiments,user 130 may refine the scene command by specifying additional information. For example, in response to receiving thefirst scenario command 305 at step 200 , at step 225 one or more possible scenario commands 315 related to thefirst scenario command 305 may be determined. Machine learning techniques implemented byCPU 150 ,AI processor 165 , and/orsupport server 127 may be used to determine possible scene commands related tofirst scene command 305 . In this regard, metadata in the video content may define a hierarchy of scene commands used by machine learning techniques in determining possible scene commands related to a givenfirst scene command 305 .

在步骤230处，用户界面300可以被更新以示出一个或多个候选的场景命令315，如图3A所示。例如，响应于第一场景命令305“向我显示所有进球”，可确定并示出可能的场景命令“在上半场(in the first half)”、“皇家马德里队(Real Madrid)”等。At step 230, theuser interface 300 may be updated to show one or more candidate scene commands 315, as shown in Figure 3A. For example, in response to thefirst scene command 305 "show me all goals", possible scene commands "in the first half", "Real Madrid", etc. may be determined and shown .

在步骤235处，用户130可以发布候选的场景命令315之一，以指示显示设备100搜索视频内容中的场景，如图3B所示。例如，用户130可以简单地大声地说出“在上半场”。短语控件310可以被实时更新以示出与第一场景命令305和第三场景命令330相关联的文本。At step 235,user 130 may issue one of the candidate scene commands 315 to instructdisplay device 100 to search for scenes in the video content, as shown in Figure 3B. For example,user 130 may simply say "in the first half" out loud.Phrase control 310 may be updated in real time to show the text associated withfirst scene command 305 andthird scene command 330 .

操作可以从步骤205处开始重复。例如，响应于第三场景命令330，显示设备100可以确定视频内容中属于场景类型的场景，所述场景类型与第一场景命令305和第三场景命令330相关联。另外或可选地，第一场景命令305和第三场景命令330可以被传送至支持服务器127，支持服务器127可以将定义相关场景的信息传送至显示设备。Operations may be repeated starting at step 205 . For example, in response to thethird scene command 330 , thedisplay device 100 may determine a scene in the video content that belongs to the scene type associated with thefirst scene command 305 and thethird scene command 330 . Additionally or alternatively, thefirst scene command 305 and thethird scene command 330 may be communicated to thesupport server 127, which may communicate information defining the relevant scene to the display device.

应当理解，可以指定除第一场景命令和第三场景命令之外的额外场景命令，以便于缩小所期望的内容的范围。例如，在发布第三场景命令330之后，可以示出另一组候选的场景命令315，等等。It should be understood that additional scene commands other than the first scene command and the third scene command may be specified in order to narrow down the desired content. For example, after issuing thethird scene command 330, another set of candidate scene commands 315 may be shown, and so on.

图4示出了便于定位特定类型的视频内容的示例性操作。参考图5可以更好地理解图4的操作。FIG. 4 illustrates example operations that facilitate locating specific types of video content. The operation of FIG. 4 may be better understood with reference to FIG. 5 .

在步骤400处，显示设备100可以正在显示视频内容，诸如情景喜剧(sitcom)，如图5所示。用户130可以向显示设备100发布第一搜索命令505，以使显示设备100搜索特定类型的视频内容。例如，用户130可以简单地大声地说出“表演(show)”。在这种情况下，由CPU150单独或与AI处理器165协作实现的自然语言处理器可以确定语音命令的含义。另外或可选地，与语音命令相关联的数据可被传送至支持服务器127，然后支持服务器127可确定语音命令的含义并将所确定的含义回传至显示设备。Atstep 400 , thedisplay device 100 may be displaying video content, such as a sitcom, as shown in FIG. 5 .User 130 may issue afirst search command 505 to displaydevice 100 to causedisplay device 100 to search for a particular type of video content. For example,user 130 may simply say "show" out loud. In this case, a natural language processor implemented byCPU 150 alone or in cooperation withAI processor 165 can determine the meaning of the voice command. Additionally or alternatively, data associated with the voice command may be communicated to supportserver 127, which may then determine the meaning of the voice command and transmit the determined meaning back to the display device.

在步骤405处，显示设备100可以确定与第一搜索命令505相关的视频内容。在这点上，CPU 150单独地或与AI处理器165协作可以实施各种机器学习技术，这些机器学习技术利用与视频内容相关联的元数据确定与搜索命令相关的视频内容。另外或可选地，第一搜索命令505可以被传送至支持服务器127，支持服务器127可以确定与视频内容相关的信息并将所述信息回传至显示设备100，所述视频内容反过来与第一搜索命令相关。Atstep 405 , thedisplay device 100 may determine video content related to thefirst search command 505 . In this regard,CPU 150, alone or in cooperation withAI processor 165, may implement various machine learning techniques that utilize metadata associated with video content to determine video content relevant to a search command. Additionally or alternatively, thefirst search command 505 may be communicated to thesupport server 127, which may determine and transmit back to thedisplay device 100 information related to the video content, which in turn is associated with the first A search command is related.

在步骤410处，可以更新用户界面500以示出便于选择视频内容的控件520。每个控件可以包括便于通过语音选择控件的控件520上或控件520附近的唯一标识符525。例如，具有唯一标识符“1”的第一控件可以对应于表示显示设备100的输入源的图像，所述图像便于从输入源选择视频内容。具有唯一标识符“2”的第二控件可以对应于演员的图像，当所述图像被选择时，便于选择包括所述演员的视频内容。具有唯一标识符“4”的第四控件可对应于来自用户频繁观看的电影的场景或属于用户130观看的表演类型的场景。Atstep 410,user interface 500 may be updated to showcontrols 520 that facilitate selection of video content. Each control may include aunique identifier 525 on or nearcontrol 520 that facilitates selection of the control by voice. For example, the first control with the unique identifier "1" may correspond to an image representing an input source of thedisplay device 100 that facilitates selection of video content from the input source. A second control having the unique identifier "2" may correspond to an image of an actor, which, when selected, facilitates selection of video content including the actor. The fourth control with the unique identifier "4" may correspond to a scene from a movie frequently watched by the user or a scene belonging to the type of show watched by theuser 130 .

机器学习技术可以至少部分基于由用户指定的搜索命令和选择的历史记录，确定要显示的控件的类型，所述历史记录可以被存储在显示设备100的支持数据库153中或者被维护在支持服务器127内。在一些实施方式中，动态地更新支持数据库153以反映用户的选择，从而改进用于后续请求的向用户显示的控件的相关性。Machine learning techniques may determine the type of controls to display based at least in part on a history of search commands and selections specified by the user, which may be stored in thesupport database 153 of thedisplay device 100 or maintained at thesupport server 127 Inside. In some embodiments, thesupport database 153 is dynamically updated to reflect the user's selection, thereby improving the relevance of the controls displayed to the user for subsequent requests.

在步骤415处，用户130可以指定第二搜索命令，所述第二搜索命令指定唯一标识符之一。例如，用户130可以指定“4”以选择与第四图像520相关联的场景。Atstep 415, theuser 130 may specify a second search command that specifies one of the unique identifiers. For example,user 130 may specify "4" to select the scene associated withfourth image 520 .

在步骤420处，可以在显示设备100的用户界面500上示出与所指定的唯一标识符(例如，“4”)相关联的视频内容。Atstep 420 , the video content associated with the specified unique identifier (eg, "4") may be displayed on theuser interface 500 of thedisplay device 100 .

返回至步骤400，在一些实施方式中，用户130可以通过指定额外信息细化搜索命令。例如，响应于在步骤400处接收第一搜索命令，在步骤425处，可以确定与第一搜索命令505相关的一个或多个可能的第三搜索命令515。由CPU 150、AI处理器165和/或支持服务器127实施的机器学习技术可用于确定与第一搜索命令505相关的可能命令。如前所述，视频内容中的元数据可以包括便于确定视频内容是否与特定类型的视频内容(例如喜剧、戏剧、体育等)相关联的信息。在确定与给定的第一搜索命令相关的可能的第三搜索命令中可由机器学习技术使用所述元数据。Returning to step 400, in some embodiments,user 130 may refine the search command by specifying additional information. For example, in response to receiving the first search command atstep 400, atstep 425, one or more possible third search commands 515 related to thefirst search command 505 may be determined. Machine learning techniques implemented byCPU 150 ,AI processor 165 and/orsupport server 127 may be used to determine possible commands related tofirst search command 505 . As previously discussed, metadata in video content may include information that facilitates determining whether the video content is associated with a particular type of video content (eg, comedy, drama, sports, etc.). The metadata may be used by machine learning techniques in determining possible third search commands related to a given first search command.

在步骤430处，用户界面500可以被更新以示出可能的搜索命令515的一个或多个，如图5所示。例如，响应于第一场景命令“演出”，可以确定并显示候选的搜索命令515“游戏”、“动作电影”等。Atstep 430 , theuser interface 500 may be updated to show one or more of the possible search commands 515 , as shown in FIG. 5 . For example, in response to the first scene command "show", candidate search commands 515 "game", "action movie", etc. may be determined and displayed.

如前所述，在一些实施方式中，用户界面500可包括短语控件510，所述短语控件510被实时更新以示出与用户发出的命令相关联的文本。As previously mentioned, in some embodiments, theuser interface 500 may include aphrase control 510 that is updated in real time to show text associated with commands issued by the user.

在步骤435处，用户130可以发出的候选的搜索命令515之一，以指示显示设备100搜索各种类型的视频内容。例如，用户130可以简单地大声地说出“动作电影”。短语控件510可以被实时更新以示出与第一搜索命令505和第三搜索命令515相关联的文本(例如，“显示动作电影(show action movies)”)。Atstep 435,user 130 may issue one of the candidate search commands 515 to instructdisplay device 100 to search for various types of video content. For example,user 130 may simply say "action movie" out loud.Phrase control 510 may be updated in real time to show text associated withfirst search command 505 and third search command 515 (eg, "show action movies").

操作可以从步骤405开始重复。例如，响应于第三搜索命令，显示设备100可以确定与第一搜索命令和第三搜索命令相关的视频内容，并显示适当的控件以供用户选择。Operations may repeat fromstep 405 onwards. For example, in response to the third search command, thedisplay device 100 may determine video content related to the first search command and the third search command, and display appropriate controls for selection by the user.

图6示出了用于确定与视频内容中的图像相关的信息的示例性操作。参考图7A和图7B，可以更好地理解图6的操作。6 illustrates example operations for determining information related to images in video content. The operation of FIG. 6 may be better understood with reference to FIGS. 7A and 7B .

在步骤600处，显示设备100可以正在示出视频内容，诸如电影，如图7A所示。用户130可以向显示设备100发出第一查询(query)705，以使显示设备100提供与所述查询相关的信息。例如，用户130可以简单地大声地说出“屏幕上是谁(who is on screen)”。在这种情况下，由CPU 150和/或AI处理器165实现的自然语言处理器可以确定语音命令的含义。另外或可选地，与语音命令相关联的数据可被传送至支持服务器127，然后支持服务器127可确定语音命令的含义并将所确定的含义回传至显示设备100。Atstep 600, thedisplay device 100 may be showing video content, such as a movie, as shown in Figure 7A.User 130 may issue afirst query 705 to displaydevice 100 to causedisplay device 100 to provide information related to the query. For example,user 130 may simply say "who is on screen" out loud. In this case, a natural language processor implemented byCPU 150 and/orAI processor 165 can determine the meaning of the voice command. Additionally or alternatively, data associated with the voice command may be communicated to supportserver 127 , which may then determine the meaning of the voice command and transmit the determined meaning back todisplay device 100 .

在步骤605处，响应于第一查询705，显示设备100可确定图像中与查询705相关联的一个或多个对象。在这点上，CPU 150单独地或与AI处理器165协作可以实施各种机器学习技术，这些机器学习技术利用与视频内容相关联的元数据确定正在显示设备100的用户界面700上示出的不同对象。另外或可选地，第一查询705可被传送至支持服务器127，支持服务器127可确定与在用户界面700上示出的不同对象相关的信息并将所述信息传送至显示设备100。Atstep 605 , in response to thefirst query 705 , thedisplay device 100 may determine one or more objects in the image that are associated with thequery 705 . In this regard,CPU 150 alone or in cooperation withAI processor 165 may implement various machine learning techniques that utilize metadata associated with video content to determine what is being shown onuser interface 700 ofdisplay device 100 different objects. Additionally or alternatively, thefirst query 705 may be transmitted to thesupport server 127 , which may determine information related to the different objects shown on theuser interface 700 and transmit the information to thedisplay device 100 .

在步骤610处，可以更新显示设备100的用户界面700以示出便于选择不同对象的控件720。每个控件可以包括每个控件720上或控件720附近的便于通过语音选择控件的唯一标识符725。例如，可以在用户界面700上示出针对每个演员的控件。Atstep 610, theuser interface 700 of thedisplay device 100 may be updated to showcontrols 720 that facilitate selection of different objects. Each control may include aunique identifier 725 on or near eachcontrol 720 that facilitates selection of the control by voice. For example, controls for each actor may be shown on theuser interface 700 .

在步骤615处，用户130可以选择唯一标识符725之一。例如，用户130可以指定“2”以选择特定演员。Atstep 615 , theuser 130 may select one of theunique identifiers 725 . For example,user 130 may specify "2" to select a particular actor.

在步骤620处，用户界面700可以被更新以示出与所述选择相关的信息。例如，如图7B所示，可以提供具有与所选演员相关的信息的信息控件730。Atstep 620,user interface 700 may be updated to show information related to the selection. For example, as shown in FIG. 7B, aninformation control 730 may be provided with information related to the selected actor.

返回至步骤600，在一些实施方式中，用户130可以通过指定额外信息细化查询。例如，响应于在步骤600接收第一查询，在步骤625处，可以确定与第一查询705相关的一个或多个可能的第二查询715。由CPU 150和/或支持服务器127实施的机器学习技术可用于确定与第一查询705相关的可能查询。视频内容中的元数据可以由机器学习技术使用以确定与给定的第一查询相关的可能查询。Returning to step 600, in some implementations,user 130 may refine the query by specifying additional information. For example, in response to receiving the first query atstep 600, atstep 625, one or more possiblesecond queries 715 related to thefirst query 705 may be determined. Machine learning techniques implemented byCPU 150 and/orsupport server 127 may be used to determine possible queries related tofirst query 705 . The metadata in the video content can be used by machine learning techniques to determine possible queries related to a given first query.

在步骤630处，如图7A所示，用户界面700可被更新以示出一个或多个候选查询715。例如，响应于第一查询“屏幕上是谁”，可以确定并示出候选查询“某某(John Doe)的其它电影”、“在哪里拍摄的”等。Atstep 630 , as shown in FIG. 7A ,user interface 700 may be updated to show one or more candidate queries 715 . For example, in response to the first query "who is on the screen", candidate queries "other movies by John Doe", "where was it filmed", etc. may be determined and shown.

如前所述，在一些实施方式中，用户界面700可包括被实时更新以示出与用户发出的查询相关联的文本的短语控件710。As previously mentioned, in some implementations, theuser interface 700 may include aphrase control 710 that is updated in real-time to show text associated with the query issued by the user.

在步骤635处，用户130可以指示与候选查询715中的一个候选查询相对应的第二查询，以指示显示设备100示出与所述查询相关的信息。短语控件710可以被实时更新以示出与第一查询705和第二查询相关联的文本。Atstep 635,user 130 may indicate a second query corresponding to one of candidate queries 715 to instructdisplay device 100 to show information related to the query.Phrase control 710 may be updated in real time to show the text associated withfirst query 705 and second query.

在步骤640处，可确定与第二查询相关的对象并且与第二查询相关的对象可包括于先前所确定的对象中或可替换先前所确定的对象。然后，操作可以从步骤605开始重复。At step 640, objects related to the second query can be determined and the objects related to the second query can be included in the previously determined objects or can replace the previously determined objects. Operations may then repeat fromstep 605 onwards.

图8示出了用于确定与视频内容中的图像相关的信息的可选示例性操作。参考图9A和图9B，可以更好地理解图8的操作。FIG. 8 illustrates optional exemplary operations for determining information related to images in video content. The operation of FIG. 8 may be better understood with reference to FIGS. 9A and 9B .

在步骤800处，显示设备100可以正在示出视频内容，诸如情景喜剧，如图9A所示。用户130可以向显示设备100发布命令以暂停视频内容，从而在用户界面900上示出静止图像。At step 800, thedisplay device 100 may be showing video content, such as a sitcom, as shown in Figure 9A.User 130 may issue a command to displaydevice 100 to pause the video content, thereby showing still images onuser interface 900 .

在步骤805处，显示设备100可以确定图像的一个或多个对象。在这点上，CPU 150单独地或与AI处理器165协作可以实施各种机器学习技术，这些机器学习技术利用与视频内容相关联的元数据确定正在静止图像中示出的不同对象。另外或可选地，静止图像可以被传送至支持服务器127，支持服务器127可以确定静止图像中示出的不同对象并将其传送至显示设备100。At step 805,display device 100 may determine one or more objects of the image. In this regard,CPU 150, alone or in cooperation withAI processor 165, may implement various machine learning techniques that utilize metadata associated with video content to determine the different objects being shown in still images. Additionally or alternatively, the still images may be transmitted to thesupport server 127 , which may determine and transmit the different objects shown in the still images to thedisplay device 100 .

在步骤810处，可以更新显示设备100的用户界面以示出便于选择不同对象的控件920，如图9A所示。例如，可以提供控件920，用于选择与静止图像中的对象之一相关的广告、以分享视频内容、对视频内容进行评分、显示与所述对象之一相关的信息。可以提供用于其它方面的控件920。At step 810, the user interface ofdisplay device 100 may be updated to showcontrols 920 that facilitate selection of different objects, as shown in Figure 9A. For example, controls 920 may be provided for selecting an advertisement related to one of the objects in the still image to share video content, rate the video content, display information related to one of the objects.Controls 920 may be provided for other aspects.

每个控件920可以包括控件920上或控件920附近的唯一标识符，所述标识符便于通过语音选择控件。Eachcontrol 920 may include a unique identifier on or near thecontrol 920 that facilitates selection of the control by voice.

在步骤815处，用户130可以选择所述唯一标识符之一。例如，用户130可以指定与示出手提包的控件相关联的唯一标识符，所述示出的手提包对应于静止图像中所示的手提包。At step 815,user 130 may select one of the unique identifiers. For example,user 130 may specify a unique identifier associated with a control showing a handbag that corresponds to the handbag shown in the still image.

在步骤820处，用户界面900被可以更新以示出与所述选择相关的信息。例如，如图9B所示，可提供具有与所述选择相关的信息的信息控件925。在一实施方式中，信息控件925可以示出与URL相关联的QR码，所述URL可以用于找出与所述选择相关的更多信息。通过用例如移动设备上的适当应用程序扫描QR码，QR码便于导航至URL。At step 820,user interface 900 may be updated to show information related to the selection. For example, as shown in FIG. 9B, an information control 925 may be provided with information related to the selection. In one embodiment, the information control 925 can show a QR code associated with a URL that can be used to find out more information related to the selection. The QR code facilitates navigation to the URL by scanning the QR code with an appropriate application on, for example, a mobile device.

图10示出了用于自动暂停视频内容的可选示例性操作。参考图11A和图11B可以更好地理解图10的操作。Figure 10 illustrates optional exemplary operations for automatically pausing video content. The operation of FIG. 10 may be better understood with reference to FIGS. 11A and 11B .

在步骤1000处，显示设备100可以确定用户是否在显示设备100附近。例如，在一实施方案中，显示设备100的成像器151可拍摄显示设备前方的图像。CPU 150可单独或与AI处理器165协作控制成像器151以拍摄图像、分析所拍摄的图像以识别图像中的面部数据，并将面部数据和与用户130相关联的面部数据进行比较以确定用户130是否在显示设备附近。在这点上，显示设备100可以在例如初始设置例程期间预先已经拍摄与用户130相关联的面部数据。面部数据可以已经存储至支持数据库153。Atstep 1000 , thedisplay device 100 may determine whether the user is in the vicinity of thedisplay device 100 . For example, in one embodiment, theimager 151 of thedisplay device 100 may capture an image of the front of the display device.CPU 150, alone or in cooperation withAI processor 165, may controlimager 151 to capture images, analyze the captured images to identify facial data in the images, and compare the facial data to facial data associated withuser 130 to determine theuser 130 is near the display device. In this regard, thedisplay device 100 may have captured facial data associated with theuser 130 in advance, eg, during an initial setup routine. Facial data may have been stored to supportdatabase 153 .

在另一实施方式中，显示设备100的近场通信电路可用来检测显示设备附近的、由用户130携带且具有近场通信能力的设备的存在。所述设备可以预先已经在显示设备100上被注册为特定用户。可以将注册信息存储至支持数据库153。In another embodiment, the near field communication circuitry ofdisplay device 100 may be used to detect the presence of a near field communication capable device carried byuser 130 in the vicinity of the display device. The device may have been registered as a specific user on thedisplay device 100 in advance. The registration information may be stored to thesupport database 153 .

在步骤1005处，如果确定用户不在显示设备100附近，则在步骤1010处，如果视频内容还没有被暂停，则可以暂停视频内容，如图11A所示。参照图11A，在用户界面1100上可以示出状态控件1105，以指示视频内容已被暂停。Atstep 1005, if it is determined that the user is not near thedisplay device 100, atstep 1010, if the video content has not been paused, the video content may be paused, as shown in FIG. 11A. Referring to Figure 11A, astatus control 1105 may be shown on theuser interface 1100 to indicate that the video content has been paused.

在一些实施方式中，用户界面1100可以示出与在用户界面1100上显示的静止图像相关的额外细节，诸如以上关于图9A和图9B描述的信息。In some implementations,user interface 1100 may show additional details related to still images displayed onuser interface 1100, such as the information described above with respect to Figures 9A and 9B.

如果在步骤1005处，确定用户130在显示器附近，则在步骤1015处，如果视频内容尚未被恢复，则可以恢复视频内容，如图11B所示。参考图11B，状态控件1105可以被更新以指示视频内容将被恢复。If, atstep 1005, it is determined that theuser 130 is near the display, then atstep 1015, if the video content has not been restored, the video content may be restored, as shown in FIG. 11B. Referring to Figure 11B, thestatus control 1105 may be updated to indicate that the video content is to be resumed.

在一些实施方式中，即使当其他用户130在显示设备100附近时，显示设备100也可以执行上述操作。例如，在初始状态中，包括主要用户130的多个用户130可以在显示设备附近。当后来确定主要用户不在显示设备附近时，视频内容可以被暂停，如上所述。当后来确定主要用户在显示设备附近时，视频内容可以被恢复。In some embodiments, thedisplay device 100 may perform the above-described operations even whenother users 130 are in the vicinity of thedisplay device 100 . For example, in the initial state,multiple users 130 including theprimary user 130 may be in the vicinity of the display device. When it is later determined that the primary user is not in the vicinity of the display device, the video content may be paused, as described above. When it is later determined that the primary user is in the vicinity of the display device, the video content can be restored.

图12示出了用于自动暂停视频内容的可选示例性操作。参考图13A-图13D，可以更好地理解图12的操作。Figure 12 illustrates optional exemplary operations for automatically pausing video content. 13A-13D, the operation of FIG. 12 may be better understood.

在步骤1200处，显示设备100可以确定用户是否在显示设备附近。例如，在一实施方案中，显示设备100的成像器151可拍摄显示设备前方的图像。CPU 150可单独或与AI处理器165协作控制成像器151以拍摄图像、分析所拍摄的图像以识别图像中的面部数据，并将面部数据和与用户相关联的面部数据进行比较以确定用户是否在显示设备100附近。如上所述，显示设备100可以在例如初始设置例程期间预先已经拍摄与用户130相关联的面部数据。Atstep 1200, thedisplay device 100 may determine whether the user is in the vicinity of the display device. For example, in one embodiment, theimager 151 of thedisplay device 100 may capture an image of the front of the display device. TheCPU 150, alone or in cooperation with theAI processor 165, may control theimager 151 to capture images, analyze the captured images to identify facial data in the images, and compare the facial data to facial data associated with the user to determine whether the user is near thedisplay device 100 . As described above, thedisplay device 100 may have captured facial data associated with theuser 130 in advance, eg, during an initial setup routine.

在另一实施方式中，如上所述，可以基于用户130所携带的设备的近场通信电路确定用户130的存在。In another embodiment, the presence ofuser 130 may be determined based on the near field communication circuitry of the device carried byuser 130, as described above.

在步骤1205处，如果确定用户在显示设备100附近，则确定与用户130相关联的一个或多个节目类型。在这点上，CPU 150单独地或与AI处理器165协作可以实施各种机器学习技术以确定与用户130相关联的节目类型。另外或可选地，标识用户130身份的信息可以被传送至支持服务器127，支持服务器127可以确定与用户相关联的节目类型。机器学习技术可以通过例如分析用户130所观看的节目的历史记录、通过从社交媒体服务器120接收与用户的喜好和厌恶相关的信息、和/或通过其他方式确定与用户130相关联的节目类型。Atstep 1205, if the user is determined to be in the vicinity of thedisplay device 100, then one or more program types associated with theuser 130 are determined. In this regard,CPU 150 alone or in cooperation withAI processor 165 may implement various machine learning techniques to determine the type of programming associated withuser 130 . Additionally or alternatively, information identifying the identity of theuser 130 may be communicated to thesupport server 127, which may determine the type of programming associated with the user. Machine learning techniques may determine the type of programming associated withuser 130 by, for example, analyzing a history of programming viewed byuser 130, by receiving information fromsocial media server 120 related to the user's likes and dislikes, and/or by other means.

在步骤1210处，可以确定在检测到用户时或在稍后的预定时间(例如，30分钟)内可供观看的节目。例如，可以分析与可用视频内容相关联的元数据以确定是否有任何视频内容属于如上所确定的用户相关联节目类型。Atstep 1210, programs available for viewing when the user is detected or at a later predetermined time (eg, 30 minutes) may be determined. For example, metadata associated with available video content may be analyzed to determine whether any video content is of the user-associated program type determined above.

在步骤1215处，用户界面1300可以被更新以呈现匹配用户相关联节目类型的与可获得节目相关的信息1305。用户界面1300可以包括便于观看可获得节目之一、记录可获得节目等的控件。Atstep 1215, theuser interface 1300 may be updated to presentinformation 1305 related to available programs matching the user's associated program type.User interface 1300 may include controls to facilitate viewing one of the available programs, recording available programs, and the like.

在一些实施方式中，显示设备100附近可以检测到一组用户130，并且在步骤1205处所确定的节目类型可以基于与两个或更多个用户130相关联的节目类型的交集。用户界面1300可被更新以示出与可获得节目相关的信息1305，所述可获得节目与用户相关联节目类型的交集匹配。In some implementations, a group ofusers 130 may be detected near thedisplay device 100 and the program type determined atstep 1205 may be based on the intersection of program types associated with two ormore users 130 .User interface 1300 may be updated to showinformation 1305 related to available programs that match the intersection of user-associated program types.

在某些实施方式中，当检测到用户130时，以上操作可以自发地被执行。例如，当第二用户接近显示设备时，第一用户130可能正在观看显示设备100上的视频内容。如上所执行的操作可以在检测到第二用户之后发生。In some embodiments, the above operations may be performed spontaneously when theuser 130 is detected. For example, thefirst user 130 may be viewing video content on thedisplay device 100 when the second user approaches the display device. The operations performed as above may occur after the second user is detected.

在其他实施方式中，可以在显示设备100开机(powering on)之后立即执行上述操作。In other embodiments, the above operations may be performed immediately after thedisplay device 100 is powered on.

在又一实施方式中，可以在已经接收关机(power off)指示之后执行所述操作。例如，如图13B所示，显示设备100可以在已被关闭后开机或者可以取消关闭操作，且用户界面1300可以被更新以示出最小量的信息，从而不会引起太多的注意力分散。例如，用户界面1300可以仅仅示出信息控件1305，以使用户130知晓例如即将开始的节目。可以提供控件1310，以允许用户130使显示设备100进入完全开机状态，以便于观看节目。In yet another embodiment, the operations may be performed after a power off indication has been received. For example, as shown in Figure 13B, thedisplay device 100 can be powered on after it has been shut down or the shut down operation can be cancelled, and theuser interface 1300 can be updated to show a minimal amount of information so as not to cause too much distraction. For example,user interface 1300 may only show information controls 1305 to letuser 130 be aware of, for example, upcoming programs.Controls 1310 may be provided to allow theuser 130 to bring thedisplay device 100 into a fully powered on state for viewing a program.

在其他实施方式中，可以确定与用户130相关联的一个或多个信息类型，并且用户界面1300可以被更新以示出属于所确定的信息类型的信息。例如，如图13C所示，可能已确定用户130对了解天气感兴趣。在这种情况下，显示设备100可以在最小功率状态下被开机，并且可以示出显示与天气相关的信息的信息控件1305。或者，信息控件1305可以被更新以显示与即将开始的电视剧集相关的信息，如图13D所示。在预定时间(例如，1分钟)之后，显示设备100可以被断电。In other implementations, one or more information types associated with theuser 130 may be determined, and theuser interface 1300 may be updated to show information pertaining to the determined information types. For example, as shown in Figure 13C, it may have been determined thatuser 130 is interested in knowing the weather. In this case, thedisplay device 100 may be powered on in a minimum power state and may show aninformation control 1305 displaying weather-related information. Alternatively, theinformation control 1305 may be updated to display information related to an upcoming TV episode, as shown in Figure 13D. After a predetermined time (eg, 1 minute), thedisplay apparatus 100 may be powered off.

图14示出了用于基于所检测的用户130的使用习惯(routine)调整各种智能家电的示例性操作。参考图15A-图15B，可以更好地理解图14的操作。FIG. 14 illustrates exemplary operations for adjusting various smart home appliances based on the detected usage routine of theuser 130 . 15A-15B, the operation of FIG. 14 may be better understood.

在步骤1400处，显示设备100可以接收与各种智能家电117的状态和显示设备100使用情况相关的数据。例如，可以接收与预先与显示设备100的使用情况相关的灯开关、计时器、帷帘控制器和其它智能家电117相关的数据。在这点上，显示设备100的通信电路可以连续地从智能家电117接收状态信息。支持数据库153可以存储智能家电117的状态信息以及显示设备100的使用情况信息。CPU 150可以将智能家电117的状态信息和显示设备100的使用情况信息相关联，以形成智能家电状态和显示设备使用之间的关系。所述关系可以指示用户130在显示设备100上观看视频内容时所遵循的使用习惯。Atstep 1400 , thedisplay device 100 may receive data related to the status of the varioussmart appliances 117 and the usage of thedisplay device 100 . For example, data related to light switches, timers, drapery controls, and othersmart appliances 117 that are previously associated with the usage of thedisplay device 100 may be received. In this regard, the communication circuitry of thedisplay device 100 may continuously receive status information from thesmart appliance 117 . Thesupport database 153 may store status information of thesmart home appliance 117 and usage information of thedisplay device 100 . TheCPU 150 may associate the status information of thesmart home appliance 117 with the usage information of thedisplay device 100 to form a relationship between the status of the smart home appliance and the usage of the display device. The relationship may indicate usage habits that theuser 130 follows when viewing video content on thedisplay device 100 .

状态信息可以定义智能家电117的激活状态。例如，智能灯是打开、关闭还是调暗到特定设置，例如50％。其它信息可包括智能帷帘是否闭合、部分闭合等。使用情况信息可以定义显示设备的使用时间、在显示设备上观看的节目类型、显示设备的特定用户列表、以及显示设备100的特定特性，诸如显示设备的音量、对比度和亮度等。The state information may define the activation state of thesmart appliance 117 . For example, whether the smart lights are on, off, or dimmed to a specific setting, such as 50%. Other information may include whether the smart drape is closed, partially closed, and the like. Usage information may define how long the display device has been used, the types of programs viewed on the display device, a list of specific users of the display device, and specific characteristics of thedisplay device 100, such as display device volume, contrast, and brightness.

在步骤1405处，可以确定显示设备的使用情况，在步骤1410处，可以基于所接收的数据确定一个或多个智能家电117的相应状态。例如，显示设备使用情况可以指示显示设备100被设置为电影频道、画面模式已被设置为影院模式以及在星期五晚上使用显示设备100。智能家电状态/显示设备使用情况的关联数据可以指示在这些情况下，显示设备100所在的房间的灯通常是关闭的，并且百叶窗是关闭的。Atstep 1405, the usage of the display device can be determined, and atstep 1410, the corresponding status of one or moresmart appliances 117 can be determined based on the received data. For example, display device usage may indicate thatdisplay device 100 is set to a movie channel, that the picture mode has been set to theater mode, and thatdisplay device 100 is being used on a Friday night. The smart appliance status/display device usage correlation data may indicate that under these conditions, the lights in the room in which thedisplay device 100 is located are typically off, and the shutters are closed.

在步骤1415处，可以根据在步骤1410处所确定的状态设置各种智能家电的状态。例如，CPU 150可以经由显示设备100的通信电路调整各种智能家电117。Atstep 1415, the states of various smart appliances may be set according to the state determined atstep 1410. For example, theCPU 150 may adjust varioussmart home appliances 117 via the communication circuit of thedisplay device 100 .

如图15A所示，用户界面1500可包括信息控件1505以通知用户130检测到使用习惯。例如，用户界面1500可以注意到(note)，显示设备100处于“影院模式”，并且当显示设备100处于该模式时，控制智能灯泡。如图15B所示，用户界面1500可以被更新以提供与所检测到的使用习惯相关的详细内容，例如为所述使用习惯分配的名称(例如，“电影时间8PM”)、进入模式“影院模式”的时间(例如，8:01PM)以及将智能家电设置到(例如，10％)的设置。As shown in FIG. 15A, theuser interface 1500 may include aninformation control 1505 to notify theuser 130 that a usage habit is detected. For example,user interface 1500 may note thatdisplay device 100 is in "cinema mode" and control smart light bulbs whiledisplay device 100 is in this mode. As shown in Figure 15B, theuser interface 1500 may be updated to provide details related to the detected usage habits, such as a name assigned to the usage pattern (eg, "Movie Time 8PM"), entry mode "Cinema Mode" ” (eg, 8:01PM) and the setting to set the smart appliance to (eg, 10%).

图16示出了可以形成上述系统、环境、设备等的一部分或实现它们的计算机系统1600。计算机系统1600可以包括一组指令1645，处理器1605可以执行该组指令1645以促使计算机系统1600执行上述操作中的任何操作。计算机系统1600可以作为独立设备运行，或者可以例如使用网络被连接至其它计算机系统或外围设备。FIG. 16 shows acomputer system 1600 that may form part of or implement the systems, environments, devices, etc. described above.Computer system 1600 may include a set ofinstructions 1645 thatprocessor 1605 may execute to causecomputer system 1600 to perform any of the operations described above.Computer system 1600 may operate as a stand-alone device, or may be connected to other computer systems or peripheral devices, eg, using a network.

在联网部署中，计算机系统1600可以作为服务器或服务器-客户端网络环境中的客户端计算机运行，或者作为对等(或分布式)环境中的对等计算机系统运行。计算机系统1600还可以被实现为或被结合至能够执行指令1645(顺序地或以其他方式)来促使设备执行一个或多个动作的各种设备中，诸如个人计算机或移动设备中。此外，所描述的系统中的每一系统可以包括子系统的集合，这些子系统单独地或联合地执行一组指令或多组指令以执行一个或多个计算机操作。In a networked deployment,computer system 1600 may operate as a client computer in a server or server-client network environment, or as a peer-to-peer computer system in a peer-to-peer (or distributed) environment.Computer system 1600 may also be implemented as or incorporated into various devices, such as personal computers or mobile devices, capable of executing instructions 1645 (sequentially or otherwise) to cause the device to perform one or more actions. Furthermore, each of the described systems may include a collection of subsystems that, individually or in combination, execute a set of instructions or sets of instructions to perform one or more computer operations.

计算机系统1600可包括与用于传送信息的总线1620通信连接的一个或多个存储器设备1610。另外，可操作以促使计算机系统执行上述操作的代码可存储在存储器1610中。存储器1610可以是随机存取存储器、只读存储器、可编程存储器、硬盘驱动器或任何其它类型的存储器或存储设备。Computer system 1600 may include one ormore memory devices 1610 in communication withbus 1620 for communicating information. Additionally, code operable to cause the computer system to perform the operations described above may be stored inmemory 1610 .Memory 1610 may be random access memory, read only memory, programmable memory, hard drive, or any other type of memory or storage device.

计算机系统1600可以包括显示器1630，例如液晶显示器(LCD)、阴极射线管(CRT)或适于显示信息的任何其他显示器。显示器1630可以充当用于用户观看由处理器1605产生的处理结果的界面。Computer system 1600 may include adisplay 1630, such as a liquid crystal display (LCD), cathode ray tube (CRT), or any other display suitable for displaying information.Display 1630 may serve as an interface for a user to view processing results produced byprocessor 1605.

另外，计算机系统1600可以包括被配置成允许用户与系统1600的组件交互的输入设备1625，例如键盘或鼠标或触摸屏。Additionally,computer system 1600 may include aninput device 1625, such as a keyboard or mouse or a touch screen, configured to allow a user to interact with components ofsystem 1600.

计算机系统1600还可以包括磁盘或光驱单元1615。驱动单元1615可以包括其中可以存储有指令1645的计算机可读介质1640。在由计算机系统1600执行期间，所述指令1645可以完全或至少部分地驻留在存储器1610内和/或处理器1605内。存储器1610和处理器1605还可以包括如上所述的计算机可读介质。Computer system 1600 may also include a disk oroptical drive unit 1615 .Drive unit 1615 may include computer readable medium 1640 in whichinstructions 1645 may be stored. Theinstructions 1645 may reside entirely or at least partially within thememory 1610 and/or within theprocessor 1605 during execution by thecomputer system 1600 .Memory 1610 andprocessor 1605 may also include computer-readable media as described above.

计算机系统1600可以包括通信接口1635，以支持经由网络1650的通信。网络1650可以包括有线网络、无线网络或其组合。通信接口1635可以经由任意数量的通信标准实现通信，诸如802.11、802.12、802.20、WiMAX、蜂窝电话标准或其他通信标准。Computer system 1600 may includecommunication interface 1635 to support communication vianetwork 1650 . Thenetwork 1650 may include a wired network, a wireless network, or a combination thereof.Communication interface 1635 may enable communication via any number of communication standards, such as 802.11, 802.12, 802.20, WiMAX, cellular telephony standards, or other communication standards.

因此，本文描述的方法和系统可以以硬件、软件或硬件和软件的组合来实现。所述方法和系统可以在至少一个计算机系统中以集中方式实现，或者以其中不同元件分布在互连的计算机系统上的分布式方式实现。可以采用适于执行本文所描述的方法的任何种类的计算机系统或其它装置。Accordingly, the methods and systems described herein may be implemented in hardware, software, or a combination of hardware and software. The methods and systems can be implemented in a centralized fashion in at least one computer system, or in a distributed fashion in which different elements are distributed over interconnected computer systems. Any kind of computer system or other apparatus suitable for carrying out the methods described herein may be employed.

本文描述的方法和系统还可以嵌入在计算机程序产品中，所述计算机程序产品包括能够实现本文描述的操作的所有特征，并且当被加载到计算机系统中时，能够执行这些操作。如本文所使用的计算机程序指的是一组机器可执行指令的机器可执行语言、代码或符号中的表达式，所述机器可执行指令旨在使设备直接执行特定功能，或者在以下a)和b)的一个或多个之后执行特定功能：a)将第一语言、代码或符号转换为另一语言、代码或符号以及b)第一语言、代码或符号的再现。The methods and systems described herein may also be embedded in a computer program product that includes all the features capable of carrying out the operations described herein and, when loaded into a computer system, is capable of performing these operations. A computer program as used herein refers to an expression in machine-executable language, code or notation of a set of machine-executable instructions intended to cause an apparatus to directly perform a specified function, or in a) below and one or more of b) then perform the specified function: a) conversion of the first language, code or symbol into another language, code or symbol and b) reproduction of the first language, code or symbol.

尽管已经参考某些实施方案描述了方法和系统，但是本领域技术人员将理解，在不脱离权利要求的范围的情况下，可以进行各种变化并且可以用等同物来替代。因此，本发明的方法和系统并不限于所公开的特定实施方案，而是所公开的方法和系统包括落入所附权利要求的范围内的所有实施方案。Although the methods and systems have been described with reference to certain embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the claims. Therefore, the methods and systems of the present invention are not limited to the particular embodiments disclosed, but the disclosed methods and systems include all embodiments falling within the scope of the appended claims.

Claims (17)

1, A display device, comprising:

user input circuitry for receiving user commands;

a display for outputting video content and a user interface, the video content including metadata;

a processor in communication with the user input circuitry and the display; and

a non-transitory computer readable medium in communication with the processor and storing instruction code that, when executed by the processor, causes the processor to:

receiving th scene command from the user input circuit, the th scene command to search for scenes in the video content that are of a scene type;

determining or more scenes in the video content that belong to the scene type from the metadata, and

updating the user interface to show or more scene images related to the or more scenes belonging to the scene type.

2. The display device of claim 1, wherein the instruction code causes the processor to:

determining or more candidate second scene commands related to the scene command from the scene command based on the metadata in the video content;

updating the user interface to show or more of the candidate second scene commands;

receiving the second scene command from the user input circuitry, the second scene command to show video content of a second scene type related to the th scene command and the second scene command;

determining or more scenes in the video content that belong to the second scene type from the metadata;

updating the user interface to show or more scene images related to the or more scenes belonging to the second scene type.

3. The display device of claim 1, wherein the instruction code causes the processor to:

updating the user interface to show a -only identifier on each of the one or more scene images;

receiving a third scene command from the user input circuitry specifying of the unique identifier;

video content from the scene image associated with the specified unique identifier is displayed.

4. The display device of claim 1, wherein the th scene command corresponds to a voice command, the instruction code causing the processor to:

implementing a natural language processor; and

determining, by the natural language processor, a meaning of the voice command.

5. The display device of claim 3, wherein the unique identifier comprises an Arabic number.

6. The display device of claim 3, wherein a of the -only identifiers are presented overlaid on a -th of the scene images.

7. The display device of claim 5, wherein the third scene command comprises a voice input of a user.

8. The display device of claim 5, wherein the third scene command comprises a user input via a remote control.

9, a method for controlling a display device, comprising:

receiving a user command through a user input circuit;

outputting, via a display, video content and a user interface, the video content including metadata;

receiving th scene command from the user input circuit, the th scene command to search for scenes in the video content that are of a scene type;

determining or more scenes in the video content that belong to the scene type from the metadata, and

updating the user interface to show or more scene images related to the or more scenes belonging to the scene type.

10. The method of claim 9, further comprising:

determining or more candidate second scene commands related to the scene command from the scene command based on the metadata in the video content;

updating the user interface to show or more of the candidate second scene commands;

receiving the second scene command from the user input circuitry, the second scene command to show video content of a second scene type related to the th scene command and the second scene command;

determining or more scenes in the video content that belong to the second scene type from the metadata;

updating the user interface to show or more scene images related to the or more scenes belonging to the second scene type.

11. The method of claim 9, further comprising:

updating the user interface to show a -only identifier on each of the one or more scene images;

receiving a third scene command from the user input circuitry specifying of the unique identifier;

video content from the scene image associated with the specified unique identifier is displayed.

12. The method of claim 9, the th scene command corresponding to a voice command, the method further comprising:

implementing a natural language processor; and

determining, by the natural language processor, a meaning of the voice command.

13. The method of claim 11, wherein the unique identifier comprises an arabic numeral.

14. The method of claim 11, wherein a of the unique identifiers is presented overlaid on a of the scene images.

15. The method of claim 13, wherein the third scene command comprises a voice input of a user.

16. The method of claim 13, wherein the third scene command comprises a user input via a remote control.

A non-transitory computer readable medium of the kind 17, , having stored thereon instruction code for controlling a display device, the instruction code executable by a computer to cause the computer to:

receiving th scene command from a user input circuit of the computer, the th scene command to search for scenes in video content that are of a scene type;

determining or more scenes in the video content that belong to the scene type from metadata of the video content, and

updating a user interface of the computer to show or more scene images related to the or more scenes belonging to the scene type.

Images