CN106663112A - Presenting information cards for events associated with entities - Google Patents

Abstract

Translated fromChinese

方法、系统、和装置包括在计算机可读存储介质上编码的计算机程序，其包括一种用于提供内容的方法。接收与用户对计算设备的使用相关联的快照。每个快照基于呈现给用户的内容。评估快照。针对每个相应的快照，识别相应快照所指示的相应实体集合。对相应实体集合的指示和指示相应快照被捕获的相应时间的相应时间戳被关联并存储。基于快照中的第一快照，确定用于向用户呈现一个或多个信息卡的第一时间。在第一时间，定位具有与第一时间相对应的时间戳的实体。基于所定位的实体来生成信息卡。所生成的信息卡被提供用于向用户呈现。

Methods, systems, and apparatus include a computer program encoded on a computer-readable storage medium, including a method for providing content. A snapshot associated with usage of the computing device by the user is received. Each snapshot is based on what is presented to the user. Evaluation snapshot. For each respective snapshot, a respective set of entities indicated by the respective snapshot is identified. An indication of the corresponding set of entities and a corresponding timestamp indicating the corresponding time at which the corresponding snapshot was captured are associated and stored. Based on a first of the snapshots, a first time for presenting the one or more information cards to the user is determined. At a first time, an entity having a timestamp corresponding to the first time is located. An information card is generated based on the located entity. The generated information card is provided for presentation to the user.

Description

Translated fromChinese

呈现与实体相关联的事件的信息卡An info card that presents events associated with the entity

背景技术Background technique

互联网提供对广泛多种资源的访问。例如，视频和/或音频文件，以及针对特定主题或特定新闻报道的网页，是可在互联网上访问的。对这些资源的访问带来了使其他内容(例如广告)与资源一起提供的机会。例如，网页能够包括能够呈现内容的槽位(slot)。这些槽位能够在网页中被限定或者被限定以用于例如连同搜索结果一起来利用网页呈现。这些示例中的内容能够处于各种格式，而消费(例如呈现)内容的设备能够同样在其类型和效能上不同。The Internet provides access to a wide variety of resources. For example, video and/or audio files, as well as web pages on specific topics or specific news stories, are accessible on the Internet. Access to these resources brings with it the opportunity to have other content, such as advertisements, served along with the resources. For example, a web page can include slots in which content can be presented. These slots can be defined in the web page or defined for use with web page presentation, for example, along with search results. The content in these examples can be in a variety of formats, and the devices consuming (eg, rendering) the content can likewise vary in their type and performance.

发明内容Contents of the invention

一般的，本说明书中所述的主题的一个创新方面能够以方法来实现，所述方法包括计算机实现的方法以用于提供内容。所述方法能够包括：由服务器设备接收与用户对计算设备的使用相关联的多个快照，所述多个快照中的每个快照都基于在所述计算设备上呈现给所述用户的内容。所述方法能够进一步包括，评估多个快照包括：针对每个相应快照，识别由该相应快照指示的相应实体集合，以及向存储器存储对该相应实体集合的指示和指示该相应快照被捕获的相应时间的相应时间戳，其中，所述相应实体集合和相应时间戳在所述存储器中被关联。所述方法能够进一步包括：基于所述多个快照中的第一快照，确定向所述用户呈现一个或多个信息卡的第一时间。所述方法能够进一步包括：在所述第一时间，在存储器中定位具有与所述第一时间相对应的时间戳的实体。所述方法能够进一步包括：基于所定位的实体中的一个或多个实体来生成信息卡。所述方法能够进一步包括：提供所生成的信息卡以用于向所述用户呈现。In general, an innovative aspect of the subject matter described in this specification can be implemented in a method that includes a computer-implemented method for providing content. The method can include receiving, by the server device, a plurality of snapshots associated with usage of the computing device by a user, each snapshot of the plurality of snapshots being based on content presented to the user on the computing device. The method can further include that evaluating the plurality of snapshots includes, for each respective snapshot, identifying a respective set of entities indicated by the respective snapshot, and storing to memory an indication of the respective set of entities and a corresponding set of entities indicating that the respective snapshot was captured. A corresponding timestamp of a time, wherein the corresponding set of entities and the corresponding timestamp are associated in the memory. The method can further include determining a first time to present one or more information cards to the user based on a first snapshot of the plurality of snapshots. The method can further comprise, at the first time, locating in memory an entity having a timestamp corresponding to the first time. The method can further include generating an information card based on one or more of the located entities. The method can further include providing the generated information card for presentation to the user.

附图说明Description of drawings

图1是用于递送内容的示例环境的框图。1 is a block diagram of an example environment for delivering content.

图2A示出用于基于与向用户呈现的内容的快照相关联的实体来呈现信息卡的示例系统。2A illustrates an example system for presenting information cards based on entities associated with a snapshot of content presented to a user.

图2B示出与电话号码实体相关联的示例信息卡。Figure 2B illustrates an example information card associated with a phone number entity.

图2C示出与位置实体相关联的示例信息卡。Figure 2C illustrates an example information card associated with a location entity.

图2D示出与主题实体相关联的示例信息卡。Figure 2D illustrates an example information card associated with a subject entity.

图3是用于基于从被呈现给用户的内容中提取的快照来提供信息卡的示例过程的流程图。3 is a flowchart of an example process for providing information cards based on snapshots extracted from content presented to a user.

图4是能够内用于实现本公开中所述的方法、系统和过程的示例计算机系统的框图。4 is a block diagram of an example computer system that can be used to implement the methods, systems, and processes described in this disclosure.

在各个附图中的类似附图标记和名称指示类似的元素。Like reference numerals and names in the various drawings indicate like elements.

具体实施方式detailed description

描述了用于提供基于呈现给用户的内容的一个或多个经评估的快照来确定的信息卡或其他形式的通知的系统、方法、和计算机程序产品。能够基于在一个或多个用户的相应用户设备上向其呈现的内容来持续地捕获和评估快照。内容可以被呈现给用户，例如在浏览器、应用(例如移动应用)、web站点、广告、社交网络页面、或者其他数字内容环境中呈现。每个快照能够包括以下中的一个或多个中的至少一部分：日历条目、地图、电子邮件消息、社交网络页面条目、web页面元素、图像、或一些其他内容。评估特定快照能够包括识别相关联的实体(例如人、地点(例如特定位置、地址、城市、州、国家、房间号、建筑物、或者其他特定地理位置)、事物(诸如电话号码)、主题、所排程的的事件(例如午餐约会、生日、会见)、或者其他可识别实体。与快照的接收相关联的时间戳还能够与快照和/或快照所基于的实体相关联地来被存储。基于例如与快照的接收相关联的时间戳，和/或基于使用快照所识别的一个或多个事件的时间，能够确定目标呈现时间。在与目标呈现时间相对应的时间，标识实体中的一个或多个的一个或多个信息卡能够被提供(例如用于向用户呈现)。例如，每个信息卡还能够指示用户能够用来理解对给定信息卡的显示的基本原理的场境(context)。对动作的至少一个调用也能够被包括在信息卡中，例如以允许用户执行与实体相关联的动作(诸如拨叫电话号码、获得导航路线指引(driving direction)、或者接收额外信息)。信息卡能够充当对种类的提示(例如，用于使用户想起概念和/或一些其他信息片(piece))，或者信息卡能够充当对即将到来的事件的提醒。Systems, methods, and computer program products are described for providing information cards or other forms of notifications determined based on one or more evaluated snapshots of content presented to a user. Snapshots can be continuously captured and evaluated based on content presented to one or more users on their respective user devices. Content may be presented to a user, such as in a browser, application (eg, mobile application), website, advertisement, social networking page, or other digital content environment. Each snapshot can include at least a portion of one or more of: a calendar entry, a map, an email message, a social networking page entry, a web page element, an image, or some other content. Evaluating a particular snapshot can include identifying associated entities (e.g., people, places (e.g., specific locations, addresses, cities, states, countries, room numbers, buildings, or other specific geographic locations), things (such as phone numbers), subjects, A scheduled event (eg, lunch appointment, birthday, meeting), or other identifiable entity. A timestamp associated with receipt of the snapshot can also be stored in association with the snapshot and/or the entity on which the snapshot is based. Based on, for example, a time stamp associated with the receipt of the snapshot, and/or based on the time of one or more events identified using the snapshot, a target presentation time can be determined. At a time corresponding to the target presentation time, one of the identified entities One or more information cards can be provided (e.g., for presentation to the user). For example, each information card can also indicate a context ( context). At least one call to an action can also be included in the information card, for example, to allow the user to perform an action associated with the entity (such as dialing a phone number, obtaining navigation route guidance (driving direction), or receiving additional information) An info card can serve as a reminder of a category (eg, to remind the user of a concept and/or some other piece of information), or an info card can serve as a reminder of an upcoming event.

对于这里讨论的系统收集和/或使用包括关于用户的个人信息的信息的情形，用户能够被提供有机会来启用/禁用或者控制可以收集和/或使用个人信息(例如，关于用户的社交网络、社交动作或活动、用户的偏好或用户的当前位置的信息)的程序或特征。此外，某些数据在其被存储或使用之前能够以一个或多个方式来处理，使得与用户相关联的个人可识别信息被删除。例如，用户的身份可以被匿名化，使得无法确定用户的个人可识别信息，或者可以在获得地理位置的情况下将用户的地理位置一般化(诸如到城市、邮编、或州层级)，使得用户的特定位置无法被确定。For situations where the systems discussed herein collect and/or use information including personal information about the user, the user can be provided the opportunity to enable/disable or control the collection and/or use of personal information (e.g., about the user's social networks, programs or features of social actions or activities, user preferences, or information about the user's current location). In addition, certain data can be processed in one or more ways before it is stored or used such that personally identifiable information associated with users is deleted. For example, a user's identity may be anonymized so that personally identifiable information about the user cannot be ascertained, or the user's geographic location may be generalized (such as to city, zip code, or state level) if obtained, such that the user The specific location of cannot be determined.

特定实施方式可以不实现或实现下面优点中的一个或多个。能够自动向用户呈现与事件或主题相关的信息卡，所述事件或主题与该用户所接收到的内容相关联。Particular implementations may achieve none or one or more of the following advantages. The user can be automatically presented with information cards related to events or topics associated with the content received by the user.

本说明书中所述的主题的一个或多个实施方式的细节在下面的附图和描述中阐述。根据说明书、附图和权利要求书，主题的其他特征、方面、和优点将变得显而易见。The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will be apparent from the description, drawings, and claims.

图1是用于递送内容的示例环境100的框图。示例环境100包括用于响应于对于内容的请求而选择和提供内容的内容管理系统110。示例环境100包括网络102，诸如局域网(LAN)、广域网(WAN)、互联网、或者其组合。网络102连接网站104、用户设备106、内容赞助者108(例如广告主)、发布者109、和内容管理系统110。示例环境100可以包括数以千计的网站104、用户设备106、内容赞助者108和发布者109。FIG. 1 is a block diagram of an example environment 100 for delivering content. The example environment 100 includes a content management system 110 for selecting and providing content in response to requests for content. The example environment 100 includes a network 102, such as a local area network (LAN), a wide area network (WAN), the Internet, or combinations thereof. Network 102 connects websites 104 , user devices 106 , content sponsors 108 (eg, advertisers), publishers 109 , and content management system 110 . The example environment 100 may include thousands of websites 104 , user devices 106 , content sponsors 108 and publishers 109 .

环境100能够包括多个数据存储库，其能够由内容管理系统110来本地存储、使用网络102在其他地方存储、根据来自相应数据源的需要而生成、或者这些的组合。实体131的数据存储库，例如，能够包括实体的列表，其能够被用于识别呈现给用户的内容的快照中的实体。例如，实体能够包括电话号码、位置(例如地址、城市、州、国家、房间号、建筑物、特定地理位置)、主题(例如与话题相关)、人物名称、所排程的事件(例如午餐约会、生日、会见)、电子邮件地址、组织名称、产品、电影、音乐、或者能够在知识图或其他信息表示中被表示的其他主题。Environment 100 can include multiple data repositories, which can be stored locally by content management system 110, stored elsewhere using network 102, generated on demand from respective data sources, or a combination of these. The data store of entities 131, for example, can include a list of entities that can be used to identify entities in snapshots of content presented to the user. For example, entities can include phone numbers, locations (e.g., address, city, state, country, room number, building, specific geographic location), topics (e.g., related to a topic), names of people, scheduled events (e.g., lunch date , birthday, meeting), email address, organization name, product, movie, music, or other topic that can be represented in a knowledge graph or other representation of information.

例如，实体131的数据存储库能够包括多元实体，被评估的每个快照各有一个实体。快照能够在被捕获之后被评估，并且被包括在快照中或在快照中引用的一个或多个排名最靠前或最显著的实体能够被存储为群组(例如，实体131的数据存储库中的实体)。For example, the data repository of entities 131 can include multiple entities, one entity for each snapshot being evaluated. A snapshot can be evaluated after being captured, and one or more top-ranked or most prominent entities included in or referenced in the snapshot can be stored as a group (e.g., in a data repository for entities 131 entity).

例如，时间戳132的数据存储库能够包括与相应快照被捕获的时间相关联的时间戳。时间戳能够与从相应快照中识别出的实体相关联。For example, the data store of timestamps 132 can include a timestamp associated with the time the corresponding snapshot was captured. Timestamps can be associated with identified entities from corresponding snapshots.

例如，事件133的数据存储库能够包括与已从相应快照中被识别出的事件相关联的信息。例如，事件的信息能够包括以下中的一个或多个：日期、开始时间、结束时间、持续时间、参与者名称、关联的位置、关联的电话号码和/或其他联系信息(例如电子邮件地址)、事件类型(例如会见、生日、午餐约会)、以及描述或场境(例如，从相应快照中获得)。For example, a data store of events 133 can include information associated with events that have been identified from corresponding snapshots. For example, information for an event can include one or more of: date, start time, end time, duration, attendee names, associated location, associated phone number, and/or other contact information (e.g., email address) , event type (eg, meeting, birthday, lunch date), and description or context (eg, obtained from the corresponding snapshot).

例如，目标呈现时间的数据存储库能够包括由内容管理系统110建立的、对相应信息卡的呈现的一个或多个时间。例如，针对午餐约会建立的目标呈现时间可以包括在午餐约会前一个小时的时间(例如，作为对离开或准备午餐约会的提醒)以及在午餐约会之前的白天或夜晚的所标定(designate)的时间以通知用户第二天的午餐约会。所讨论的数据存储库中的一些或全部能够在单个数据存储库中组合，诸如包括所识别实体、事件、时间戳和目标呈现时间的组合的数据存储库，全部都与单个快照相关联。For example, the data repository of target presentation times can include one or more times established by the content management system 110 for the presentation of corresponding information cards. For example, a target presentation time established for a lunch appointment may include a time one hour before the lunch appointment (e.g., as a reminder to leave or prepare for the lunch appointment) and a designated time of day or night before the lunch appointment to notify the user of the next day's lunch appointment. Some or all of the data repositories in question can be combined in a single data store, such as a data store comprising a combination of identified entities, events, timestamps and target presentation times, all associated with a single snapshot.

内容管理系统110能够包括多元引擎，其中的一些或全部能够被组合或分离，并且能够共处一处或者被分布(例如通过网络102连接)。例如，快照评估引擎121能够评估在设备上呈现给用户的内容的快照。例如，对于每个快照，快照评估引擎121能够识别被包括在快照中的实体和/或事件并且连同与相应快照被捕获的时间或呈现时间相关联的时间戳一起存储所识别的实体/事件。Content management system 110 can include multiple engines, some or all of which can be combined or separated, and can be co-located or distributed (eg, connected by network 102). For example, snapshot evaluation engine 121 can evaluate snapshots of content presented to a user on a device. For example, for each snapshot, the snapshot evaluation engine 121 can identify entities and/or events included in the snapshot and store the identified entities/events along with a timestamp associated with the time the corresponding snapshot was captured or presented.

例如，信息卡引擎122能够执行与以下相关联的功能：聚集在信息卡中使用的信息、生成信息卡、和确定呈现信息卡的时间。例如，在接收到的快照被评估之后，信息卡引擎122能够确定用于包括在信息卡中的内容和呈现一个或多个信息卡给用户的时间，包括确定呈现的目标时间。内容的选择和呈现的定时在下面更详细讨论。For example, information card engine 122 can perform functions associated with aggregating information for use in information cards, generating information cards, and determining when to present information cards. For example, after the received snapshots are evaluated, the information card engine 122 can determine content for inclusion in the information card and a time for presenting the one or more information cards to the user, including determining a target time for presentation. The selection of content and the timing of presentation are discussed in more detail below.

网站104包括与域名相关联且由一个或多个服务器托管的一个或多个资源105。示例网站是以超文本标记语言(HTML)来格式化的网页的集合，其能够包含文本、图像、多媒体内容、和诸如脚本的编程元素。每个网站104能够由内容发布者来维护，该内容发布者是控制、管理和/或拥有网站104的实体。Website 104 includes one or more resources 105 associated with a domain name and hosted by one or more servers. An example website is a collection of web pages formatted in Hypertext Markup Language (HTML), which can contain text, images, multimedia content, and programming elements such as scripts. Each website 104 can be maintained by a content publisher, which is the entity that controls, manages, and/or owns the website 104 .

资源105能够是能够通过网络102来提供的任何数据。资源105能够由与资源105相关联的资源地址来标识。资源包括HTML页面、文字处理文档、便携式文档格式(PDF)文档、图像、视频、和新闻馈给源，仅举数例。资源能够包括内容，诸如词、短语、图像、视频和声音，所述内容能够包括嵌入式信息(诸如元信息超链接)和/或嵌入式指令。Resource 105 can be any data that can be provided over network 102 . Resource 105 can be identified by a resource address associated with resource 105 . Resources include HTML pages, word processing documents, Portable Document Format (PDF) documents, images, videos, and news feeds, to name a few. Resources can include content, such as words, phrases, images, video, and sound, which can include embedded information (such as meta-information hyperlinks) and/or embedded instructions.

用户设备106是在用户控制下且能够通过网络102请求和接收资源的电子设备。示例用户设备106包括个人计算机(PC)、具有嵌入其中或耦合至其的一个或多个处理器的电视、机顶盒、游戏控制台、移动通信设备(例如智能电话)、平板计算机和能够通过网络102发送和接收数据的其他设备。用户设备106通常包括一个或多个用户应用，诸如web浏览器，以促进网络102上的数据的发送和接收。A user device 106 is an electronic device under the control of a user and capable of requesting and receiving resources over the network 102 . Example user devices 106 include personal computers (PCs), televisions with one or more processors embedded in or coupled to them, set-top boxes, game consoles, mobile communication devices (such as smartphones), tablet computers, and Other devices that send and receive data. User device 106 typically includes one or more user applications, such as a web browser, to facilitate sending and receiving data over network 102 .

用户设备106能够从网站104请求资源105。进而，表示资源105的数据能够被提供到用户设备106以用于由用户设备106呈现。表示资源105的数据还能够包括指定资源的一部分或用户显示的一部分的数据，诸如弹出窗口的呈现位置或能够呈现内容的第三方内容站点或网页的槽位。资源的这些所指定的部分或用户显示被称为槽位(例如广告位)。User device 106 can request resource 105 from website 104 . In turn, data representing resource 105 can be provided to user device 106 for presentation by user device 106 . The data representing the resource 105 can also include data specifying a portion of the resource or a portion of the user display, such as the rendering position of a pop-up window or the slot of a third-party content site or web page where the content can be rendered. These designated portions of resources, or user displays, are called slots (eg, advertising slots).

为了促进对这些资源的搜索，环境100能够包括搜索系统112，其通过爬取和索引由内容发布者在网站104上提供的资源来识别资源。关于资源的数据能够基于数据所对应的资源来被索引。被索引的且可选地被缓存的资源副本能够被存储在经索引缓存114中。To facilitate searching for these resources, the environment 100 can include a search system 112 that identifies resources by crawling and indexing resources provided by content publishers on the website 104 . Data about resources can be indexed based on the resource to which the data corresponds. Indexed and optionally cached resource copies can be stored in indexed cache 114 .

用户设备106能够通过网络102提交搜索查询116到搜索系统112。作为响应，例如，搜索系统112能够访问经索引缓存114以识别与搜索查询116相关的资源。搜索系统112识别处于搜索结果118形式的资源并且以搜索结果页面来向用户设备106返回搜索结果118。搜索结果118能够是由搜索系统112生成的数据，其标识响应于特定搜索查询而提供的资源，并且包括指向资源的链接。搜索结果页面还能够包括一个或多个槽位，在其中能够呈现其他内容项(例如广告)。User device 106 is capable of submitting a search query 116 to search system 112 over network 102 . In response, for example, search system 112 can access indexed cache 114 to identify resources relevant to search query 116 . The search system 112 identifies resources in the form of search results 118 and returns the search results 118 to the user device 106 in a page of search results. Search results 118 can be data generated by search system 112 that identifies resources provided in response to a particular search query and includes links to resources. The search results page can also include one or more slots in which other content items (eg, advertisements) can be presented.

当资源105、搜索结果118和/或其他内容(例如视频)被用户设备106请求时，内容管理系统110接收对于内容的请求。对于内容的请求能够包括针对所请求的资源或搜索结果页面来被限定的槽位的特性，并且能够提供给内容管理系统110。When resources 105, search results 118, and/or other content (eg, video) are requested by user device 106, content management system 110 receives a request for content. The request for content can include characteristics of the slots defined for the requested resource or search results page and can be provided to the content management system 110 .

例如，对针对其来限定槽位的资源、槽位的大小、和/或可用于在槽位中呈现的媒体类型的引用(例如URL)能够与给定请求相关联地被提供给内容管理系统110。类似地，与所请求的资源相关联的关键字(“资源关键字”)或针对其来请求搜索结果的搜索查询116也能够被提供给内容管理系统110以促进对与资源或搜索查询116相关的内容的识别。For example, references (e.g., URLs) to resources for which the slot is defined, the size of the slot, and/or the media types available for presentation in the slot can be provided to the content management system in association with a given request 110. Similarly, keywords associated with the requested resource ("resource keywords") or search query 116 for which search results are requested can also be provided to content management system 110 to facilitate search queries related to the resource or search query 116. identification of the content.

至少部分地基于被包括在请求中的数据，内容管理系统110能够选择有资格响应于请求而被提供的内容(“适格内容项”)。例如，适格内容项能够适格广告，其包括具有匹配广告位特性的特性并且被识别为与所指定的资源关键字或搜索查询116相关。此外，当没有执行搜索或没有关键字可用时(例如因为用户没有在浏览网页)，其他信息——诸如从一个或多个快照获得的信息——能够被用来对接收到的请求作出响应。在一些实施方式中，对适格内容项的选择能够进一步取决于用户信号，诸如群体特征信号、行为信号或者从用户档案导出的其他信号。Based at least in part on the data included in the request, content management system 110 can select content that is eligible to be provided in response to the request ("eligible content items"). For example, a qualified content item can be a qualified advertisement that includes characteristics that match the characteristics of the ad slot and is identified as being relevant to the specified resource keyword or search query 116 . Additionally, other information, such as information obtained from one or more snapshots, can be used to respond to received requests when no search is performed or no keywords are available (eg, because the user is not browsing the web). In some implementations, the selection of eligible content items can further depend on user signals, such as demographic signals, behavioral signals, or other signals derived from user profiles.

内容管理系统110能够至少部分地基于竞拍的结果(或者通过一些其他选择过程)来从待被提供用于在资源或搜索结果页面的槽位中呈现的适格内容项中选择。例如，对于适格内容项，内容管理系统110能够从内容赞助者108接收出价(offer)并且至少部分地基于接收到的出价来分配槽位(例如，基于竞拍成交的最高竞价者或者基于其他准则，诸如与满足开放预留和学习价值相关的那些准则)。出价表示内容赞助者愿意为其内容与资源或搜索结果页面一起呈现(或者对其内容的选择或与其内容的其他交互)而支付的数额。例如，出价能够指定内容赞助者愿意为内容项的每1000个闪现(impression)(即呈现)支付的数额，其被称为CPM竞价。替选地，出价能够指定内容赞助者愿意为对内容项的选择(即点击)或对内容项的选择之后的转化来支付的数额。例如，所选择的内容项能够基于出价来被单独确定，或者基于每个内容赞助者的出价乘以一个或多个因子来被确定，所述因子诸如从内容效能导出的质量分值、着陆页面分值、学习价值、和/或其他因子分值。Content management system 110 can select from among eligible content items to be offered for presentation in slots of resource or search results pages based at least in part on the results of the auction (or through some other selection process). For example, for eligible content items, content management system 110 can receive offers from content sponsors 108 and assign slots based at least in part on the received offers (e.g., based on the highest bidder in an auction deal or based on other criteria , such as those related to satisfying open reservations and learning values). The bid represents the amount that a content sponsor is willing to pay for its content to be presented with a resource or search results page (or for selection or other interaction with its content). For example, a bid can specify the amount a content sponsor is willing to pay for every 1000 impressions (ie, presentations) of a content item, known as a CPM bid. Alternatively, the bid can specify an amount that the content sponsor is willing to pay for a selection (ie, click) of a content item or a conversion following selection of a content item. For example, selected content items can be determined based on bids alone, or based on each content sponsor's bid multiplied by one or more factors, such as quality scores derived from content performance, landing page points, learning value, and/or other factor points.

当用户执行和与资源或搜索结果页面一起提供的内容项相关的特定交易或动作时，可以认为发生了转化。构成转化的要素能够因情况而不同，并且能够通过各种方式来确定。例如，转化可以发生在：当用户点击内容项(例如广告)，被引至网页，并且在离开该网页之前在该处消费了购买。转化还能够被内容提供商限定为任何可测量或可观察的用户动作，诸如下载白皮书(white paper)、导航到网站的至少给定深度、查看至少一定数目的网页、在web站点或网页上花费至少预定的时间量、在网站上注册、体验媒体、或者执行关于内容项(例如广告)的社交动作，诸如赞同(endorsing)、重新发布或共享内容项。也能够使用构成转化的其他动作。A conversion may be considered to occur when a user performs a particular transaction or action related to a content item provided with a resource or search results page. What constitutes a transformation can vary from case to case and can be determined in various ways. For example, a conversion may occur when a user clicks on a content item (eg, an ad), is directed to a web page, and consumes a purchase there before leaving the web page. A conversion can also be defined by the content provider as any measurable or observable user action, such as downloading a white paper, navigating to at least a given depth of a website, viewing at least a certain number of web pages, spending money on a web site or web page At least a predetermined amount of time, registering on a website, experiencing media, or performing a social action with respect to a content item (eg, an advertisement), such as endorsing, reposting, or sharing the content item. Other actions that constitute transformations can also be used.

图2A是用于基于与呈现给用户的内容的快照202相关联的实体来呈现信息卡201的系统200的框图。例如，快照202能够从在用户设备106a上呈现给用户206的内容204a、204b中随时间推移来被捕获。内容204a、204b能够是与用户设备相关联的显示区域中内容(例如，仅在活动窗口中的内容)中的的全部或一部分。内容204a、204b可以在以下中的一个或多个中呈现：浏览器、应用、web站点、广告、社交网络页面、或一些其他用户接口或应用。例如，内容204a、204b能够包括以下中的一个或多个：日历条目、地图、电子邮件消息、社交网络页面条目、web页面元素、图像、或者一些其他内容或元素。例如，内容204a、204b的快照202能够被评估以识别相关联的实体131，诸如电话号码、位置(例如地址、城市、州、国家、房间号、建筑物、特定地理位置)、或其他可识别实体。例如，与接收到的快照202相关联的时间戳132能够与所识别的实体131一起使用来标识与实体131相关联的信息卡201的目标呈现时间133。例如，在与目标呈现时间133相关联的时间，内容管理系统110能够提供信息卡201以用于向用户206呈现。2A is a block diagram of a system 200 for presenting an information card 201 based on entities associated with a snapshot 202 of content presented to a user. For example, snapshot 202 can be captured over time from content 204a, 204b presented to user 206 on user device 106a. The content 204a, 204b can be all or a portion of the content in the display area associated with the user device (eg, content only in the active window). The content 204a, 204b may be presented in one or more of: a browser, an application, a website, an advertisement, a social networking page, or some other user interface or application. For example, the content 204a, 204b can include one or more of: a calendar entry, a map, an email message, a social networking page entry, a web page element, an image, or some other content or element. For example, a snapshot 202 of content 204a, 204b can be evaluated to identify an associated entity 131, such as a phone number, location (e.g., address, city, state, country, room number, building, specific geographic location), or other identifiable entity. For example, the timestamp 132 associated with the received snapshot 202 can be used with the identified entity 131 to identify a target presentation time 133 for the information card 201 associated with the entity 131 . For example, at a time associated with target presentation time 133 , content management system 110 can provide information card 201 for presentation to user 206 .

在一些实施方式中，能够基于被包括在快照中的实体(例如标识人、地点和电话号码的日历条目)来识别一个或多个事件(例如午餐约会)。能够关于事件何时发生来确定(例如，在未来的)第一时间(例如，午餐约会会见时间)，并且事件能够连同第一时间一起被存储(例如在事件133的存储仓库中)。能够确定在事件之前的第二时间，诸如用户206需要被通知离开以便按时到达事件的时间。通常，第二时间能够是在事件要发生之前执行与该事件相关的动作的时间，诸如为周年纪念订购鲜花或者发送生日贺卡。确定呈现信息卡的时间(例如，与午餐约会相关联的信息卡)能够包括确定当前时间(例如现在时间)等于第二时间(例如午餐约会之前一个小时)。信息卡能够在第二时间为事件而呈现。在一些实施方式中，下面的示例阶段能够被用于提供信息卡。In some implementations, one or more events (eg, lunch appointments) can be identified based on entities included in the snapshot (eg, calendar entries identifying people, places, and phone numbers). A first time (eg, a lunch date meeting time) can be determined (eg, in the future) as to when the event occurred, and the event can be stored along with the first time (eg, in a storage repository for events 133). A second time before the event can be determined, such as a time at which user 206 needs to be notified to leave in order to arrive at the event on time. Typically, the second time can be a time when an action related to the event is performed before the event is about to occur, such as ordering flowers for an anniversary or sending a birthday card. Determining when to present an information card (eg, an information card associated with a lunch appointment) can include determining that a current time (eg, the present time) is equal to a second time (eg, one hour before the lunch appointment). The information card can be presented for the event at a second time. In some implementations, the following example stages can be used to provide an information card.

在阶段1，例如，内容管理系统110能够接收快照202，例如与用户206对用户设备106a的使用相关联的多个快照。例如，接收到的快照202能够包括在用户设备106a上呈现给用户206的内容204a、204b的快照。例如，快照202能够包括：从电子邮件消息(例如内容204a)或从图像(例如内容204b)和/或从呈现给用户206的其他内容拍摄的快照。In stage 1, for example, content management system 110 can receive snapshots 202, eg, a plurality of snapshots associated with user 206's use of user device 106a. For example, the received snapshot 202 can include a snapshot of the content 204a, 204b presented to the user 206 on the user device 106a. For example, snapshot 202 can include a snapshot taken from an email message (eg, content 204a ) or from an image (eg, content 204b ) and/or from other content presented to user 206 .

在阶段2，例如，快照评估引擎121能够评估接收到的快照202。例如，对于每个快照，快照评估引擎121能够识别被包括在快照202中的实体131。例如，对于从内容204a获得的快照202所识别的实体能够包括Bob、Carol、J’s餐馆、和Carol的手机号码。快照评估引擎121能够将所识别的实体(或者其子集，诸如最突出的实体)连同与相应快照被捕获的时间相关联的时间戳一起存储。在一些实施方式中，作为确定实体过程的一部分，能够对于任何所确定的实体是否彼此相关作出确定，诸如与公共的事件相关。相关性能够基于接近度(例如，实体处于互相紧密接近)或者快照中的一些其他关系。在一些实施方式中，时间戳能够被存储在时间戳132的数据存储库中，例如以用于稍后在生成和呈现与快照202相关的信息卡201时使用。在一些实施方式中，一个或多个实体能够与事件相关联。即，实体可以是人，并且事件可以涉及与人的会见(如被包括在正在被评估的快照中所示的电子邮件消息中的内容所指示)。当事件由快照评估引擎121识别时，能够在用户的日历中建立日历项，并且可选地，在与事件相关联的其他用户的日历中也建立日历项(例如包括不必是事件参加者的用户)。在一些实施方式中，被识别的事件能够包括用户不参加的事件，但是用户通过接收到信息卡仍旧可以从其获益(例如，在线销售的优惠券期满)。在下面更详细讨论事件。In stage 2, for example, the snapshot evaluation engine 121 can evaluate the received snapshot 202 . For example, for each snapshot, snapshot evaluation engine 121 can identify entities 131 included in snapshot 202 . For example, entities identified for snapshot 202 obtained from content 204a can include Bob, Carol, J's restaurant, and Carol's cell phone numbers. The snapshot evaluation engine 121 can store the identified entities (or a subset thereof, such as the most prominent entities) along with a timestamp associated with the time the corresponding snapshot was captured. In some implementations, as part of the determining entities process, a determination can be made as to whether any of the determined entities are related to each other, such as related to a common event. Dependencies can be based on proximity (eg, entities are in close proximity to each other) or some other relationship in the snapshot. In some implementations, the timestamp can be stored in the timestamp 132 data store, eg, for later use in generating and presenting the information card 201 associated with the snapshot 202 . In some implementations, one or more entities can be associated with an event. That is, the entity may be a person, and the event may involve a meeting with the person (as indicated by the content included in the email message shown in the snapshot being evaluated). When an event is identified by the snapshot evaluation engine 121, a calendar entry can be established in the user's calendar, and optionally also in the calendars of other users associated with the event (including, for example, users who are not necessarily attendees of the event) ). In some implementations, the identified events can include events that the user does not attend, but that the user can still benefit from by receiving an information card (eg, a coupon for an online sale expires). Events are discussed in more detail below.

例如，与内容204a相关联的快照202的评估能够确定：在用户206(例如Bob)和Carol之间存在午餐约会事件。从快照202中识别出的其他信息能够包括时间、位置信息、以及电话号码。在这个示例中，被识别的实体能够包括：Bob、Carol、餐厅(例如，J’s)、以及Carol的电话号码。作为快照评估的一部分，能够确定与快照和/或事件相关联的场境。例如，基于Bob、Carol、餐厅和Carol的电话号码的实体，能够确定“在日期X的中午与Carol在J’s进行午餐约会(lunch date at noon on date X with Carol at J's.)”的场境。例如，场境信息能够被确定、存储以及稍后访问，以向用户提供关于为什么呈现特定信息卡的信息。在一些实施方式中，其他信息能够被包括在场境中，诸如对应用的识别或者从其提取快照的其他源、评估信息的方式、或者与屏幕截图相关联的场境。在一些实施方式中，场境信息能够处于从其提取实体或事件的文本摘录(snippet)的形式。在一些实施方式中，例如，在场境信息被随后呈现时，场境所基于的摘录能够被格式化以高亮显示信息的相关片。For example, evaluation of snapshot 202 associated with content 204a can determine that there is a lunch date event between user 206 (eg, Bob) and Carol. Other information identified from snapshot 202 can include time, location information, and phone number. In this example, the identified entities can include: Bob, Carol, a restaurant (eg, J's), and Carol's phone number. As part of the snapshot evaluation, context associated with the snapshot and/or event can be determined. For example, based on the entities of Bob, Carol, the restaurant, and Carol's phone number, the context of "lunch date at noon on date X with Carol at J's." can be determined. For example, contextual information can be determined, stored, and later accessed to provide the user with information about why a particular information card was presented. In some implementations, other information can be included in the context, such as the identification of the application or other source from which the snapshot was taken, the manner in which the information was evaluated, or the context associated with the screenshot. In some implementations, contextual information can be in the form of textual snippets from which entities or events are extracted. In some implementations, for example, when the contextual information is subsequently presented, the excerpt on which the context is based can be formatted to highlight the relevant piece of information.

在阶段3，例如，在评估接收到的快照中的一个或多个之后，信息卡引擎122能够确定向用户呈现一个或多个信息卡的时间，包括确定目标时间。在一些实施方式中，目标时间能够被存储在目标呈现时间134的数据存储库中。例如，对于Bob与Carol的即将发生的午餐约会，信息卡引擎122能够确定对于Bob的提醒时间，其是所排程的的中午午餐约会之前一个小时。在一些实施方式中，能够确定呈现信息卡的多个时间，例如，其包括将在前一天晚上发送的提醒——次日Bob与Carol有午餐约会。在一些实施方式中，能够使用各种因素来确定目标时间，所述因素诸如交通模式、距离、位置和/或其他因素。例如，对于充当对用户的提示的信息卡，目标时间能够包括自概念最初被呈现给用户——例如以从其获得相应的快照的内容的形式来呈现——起的一个或多个时间。In stage 3, for example, after evaluating one or more of the received snapshots, the information card engine 122 can determine when to present one or more information cards to the user, including determining a target time. In some implementations, the target time can be stored in the target presentation time 134 data store. For example, for Bob's upcoming lunch appointment with Carol, the info card engine 122 can determine a reminder time for Bob that is one hour before the scheduled noon lunch appointment. In some implementations, multiple times can be determined to present the information card, for example, including a reminder to be sent the night before - Bob has a lunch date with Carol the next day. In some implementations, various factors can be used to determine the target time, such as traffic patterns, distance, location, and/or other factors. For example, for an information card serving as a reminder to the user, the target time can include one or more times since the concept was first presented to the user, eg, in the form of content from which a corresponding snapshot was taken.

在阶段4，例如，信息卡引擎122能够基于目标时间和与所存储的实体的相应实体相关联的时间戳的比较来从存储的实体中识别实体。例如，对于Carol和Bob来排程的午餐约会，信息卡引擎122能够识别要在与即将发生的午餐约会相关联的信息卡中使用的信息。例如，Carol的电话号码能够是能够为生成信息卡来识别的实体，例如以用于给Bob的提醒，所述提醒在午餐约会之前一个小时的目标时间被发送并且还包括Carol的手机号码。In stage 4, for example, the information card engine 122 can identify an entity from the stored entities based on a comparison of the target time and the time stamps associated with corresponding ones of the stored entities. For example, for a lunch appointment that Carol and Bob are scheduling, the information card engine 122 can identify information to use in the information card associated with the upcoming lunch appointment. For example, Carol's phone number can be an entity that can be identified for generating an information card, eg, for a reminder to Bob that is sent at a target time of one hour before the lunch appointment and also includes Carol's cell phone number.

在阶段5，例如，信息卡引擎122能够基于一个或多个所识别的实体131来生成信息卡201。例如，信息卡201能够包括与午餐约会相关联的信息和Carol的手机号码。在一些实施方式中，信息卡201能够被存储在例如内容管理系统110处以供在相同信息卡的多个后续呈现中使用。At stage 5, for example, information card engine 122 can generate information card 201 based on one or more identified entities 131 . For example, information card 201 can include information associated with a lunch appointment and Carol's cell phone number. In some implementations, the information card 201 can be stored, for example, at the content management system 110 for use in multiple subsequent presentations of the same information card.

在阶段6，例如，内容管理系统110能够提供信息卡201以用于向用户呈现。例如，信息卡201可以被提供给用户设备106a以用于在屏幕208c上呈现，其可以是与屏幕208a、208b相同或不同的屏幕，从所述屏幕208a、208b，快照202从用户206的多元用户会话210来获得。在一些实施方式中，屏幕208a、208b、208c能够是在用户设备106a中与用户206相关联的的多个用户设备上呈现的屏幕。例如，呈现信息卡的时间能够是自与该信息卡相关联的概念例如以从其获得相应快照的内容的形式来最初被呈现给用户起的时间。在这个示例中，能够提供信息卡以唤起用户的记忆。例如，呈现信息卡的时间还能够是和与信息卡相关联的事件(例如午餐约会)相关的时间。At stage 6, for example, content management system 110 can provide information card 201 for presentation to the user. For example, information card 201 may be provided to user device 106a for presentation on screen 208c, which may be the same or a different screen than screens 208a, 208b from which snapshot 202 is taken from user 206's multiple User session 210 to obtain. In some implementations, the screens 208a, 208b, 208c can be screens presented on multiple ones of the user devices 106a associated with the user 206 . For example, the time at which an information card is presented can be the time since the concept associated with the information card was initially presented to the user, eg, in the form of content from which the corresponding snapshot was taken. In this example, an information card can be provided to jog the user's memory. For example, the time at which the information card is presented can also be a time related to an event (eg, a lunch appointment) associated with the information card.

在一些实施方式中，一些信息卡201可能适用于多于一个的用户。例如，内容管理系统110能够将信息卡201提供给与事件相关联的所有方，诸如关于他们的即将发生的的午餐约会来向Bob和Carol二者提供。In some implementations, some information cards 201 may be suitable for more than one user. For example, content management system 110 can provide information card 201 to all parties associated with the event, such as to both Bob and Carol regarding their upcoming lunch appointment.

在一些实施方式中，当快照被评估以预测向用户提供信息卡的可能时，用户能够可选地接收通知(例如，顺着“你可能将基于X而接收信息卡……(You may be receivinginformation cards based on X...)”的用词的通知)。在一些实施方式中，用户能够具有选项来单独地按群组(或者按信息卡的类型)或者全局地改变何时和任何呈现信息卡。在一些实施方式中，用户能够被呈现控件以用于指定能够被用于信息卡的信息类型，诸如顺着“不要使用来自我的电子邮件的信息来生成信息卡(Don’t use information from myemail to generate information cards.)”的用词的复选框控制。在一些实施方式中，用户能够控制呈现信息卡的时间，例如日内时间或者特定快照的时间。在一些实施方式中，用户能够被提供针对任何特定信息卡的透明控件，例如以了解如何或为何准备和呈现信息卡。In some implementations, when a snapshot is evaluated to predict the likelihood of providing an information card to the user, the user can optionally receive a notification (e.g., along the lines of "You may be receiving information based on X... cards based on X...)" wording notice). In some implementations, the user can have the option to change when and at what information cards are presented either individually by group (or by type of information card) or globally. In some implementations, the user can be presented with controls for specifying the type of information that can be used for an information card, such as along the lines of "Don't use information from my email to generate an info card" to generate information cards.)" checkbox control. In some implementations, the user can control when the information card is presented, such as the time of day or the time of a particular snapshot. In some implementations, the user can be provided with transparent controls for any particular information card, eg, to understand how or why the information card was prepared and presented.

图2B示出与电话号码实体相关联的示例信息卡220a。例如，继续上面关于图2A所述的示例，信息卡220a能够在Bob和Carol的即将发生的午餐约会之前一个小时被呈现。例如，信息卡220a能够包括通知标题222a(例如“拨号器…(Dialer…)”)，其通知用户：该信息卡是与电话号码——例如Carol的手机号码相关联的类型。例如，场境224a能够识别与信息卡相关联的场境。在这个示例中，场境224a能够包括(或者根据以下被确定)快照202的一部分，包括Bob从Carol接收的电子邮件消息的摘录，其包含与即将发生的午餐约会相关联的信息(诸如位置、电话号码、日期226a)。例如，信息卡220a还能够包括，与信息卡一起被显示在Bob的智能电话上的诸如控件的行动召唤(call-to-action)228a，其用于拨打Carol的手机号码。其他行动召唤228a在本示例中是可能的，诸如用以显示到餐馆的地图的行动召唤。Figure 2B shows an example information card 220a associated with a phone number entity. For example, continuing the example described above with respect to FIG. 2A , information card 220a can be presented an hour before Bob and Carol's upcoming lunch date. For example, information card 220a can include a notification title 222a (eg, "Dialer...") that informs the user that the information card is of the type associated with a telephone number, eg, Carol's cell phone number. For example, context 224a can identify a context associated with an information card. In this example, context 224a can include (or be determined from below) a portion of snapshot 202, including an excerpt of an email message Bob received from Carol containing information associated with an upcoming lunch appointment (such as location, Phone number, date 226a). For example, the information card 220a can also include, along with the information card, a call-to-action 228a such as a control displayed on Bob's smartphone for dialing Carol's cell phone number. Other calls to action 228a are possible in this example, such as a call to action to display a map to a restaurant.

图2C示出与位置实体相关联的示例信息卡220b。例如，信息卡220b能够包括通知标题222b(例如“位置…(Location…)”)，其通知用户信息卡与位置——例如法国巴黎(Paris France)相关联。例如，信息卡220b能够根据与用户浏览与巴黎相关联的在线信息相关联的快照202来生成，诸如在线旅行或度假信息。例如，场境224b能够标识与信息卡相关联的场境。在这个示例中，场境224b能够包括(或者根据以下确定)快照202的一部分，包括可能被包括在快照中的地图或从快照中的信息中识别的地图。例如，信息卡220b还能够包括行动召唤228b，诸如控件，其在Bob的智能电话上显示，以用于获得到巴黎或巴黎内的导航路线指引。与信息卡220b的呈现相关联的时间能够基于当前时间和用户的当前位置(例如到达巴黎机场)来确定。FIG. 2C shows an example information card 220b associated with a location entity. For example, the information card 220b can include a notification title 222b (eg, "Location...") that informs the user that the information card is associated with a location—eg, Paris France. For example, information card 220b can be generated from snapshot 202 associated with a user browsing online information associated with Paris, such as online travel or vacation information. For example, context 224b can identify a context associated with an information card. In this example, context 224b can include (or be determined from below) a portion of snapshot 202, including a map that may be included in the snapshot or identified from information in the snapshot. For example, the information card 220b can also include a call to action 228b, such as a control displayed on Bob's smartphone for obtaining navigational directions to or within Paris. The time associated with the presentation of the information card 220b can be determined based on the current time and the user's current location (eg, arrival at Paris airport).

图2D示出与信息性实体相关联的示例信息卡220c。例如，信息卡220c能够包括通知标题222c(例如“应答…(Answer…)”)，其通知用户信息卡与主题——例如纽约证券交易所(NYSE)相关联。在这个示例中，NYSE还能够是位置。例如，“应答”类型的信息卡能够应用于信息性实体，例如来自摘录、传记、报价(quote)(例如在用户屏幕上显示的股票报价)、或者其他信息性内容。例如，信息卡220c能够根据与用户浏览与NYSE相关联的在线信息或来自其他源的信息相关联的快照而生成。例如，场境224c能够标识与信息卡相关联的场境。在这个示例中，场境224c能够包括(或者根据以下被确定)快照的一部分，其包括关于NYSE的文本的摘录，所述本文可能已经作为来自网站的内容而向用户呈现过。例如，信息卡220c还能够包括行动召唤228c，诸如控件，其在Bob的智能电话上显示以用于获得关于NYSE的更多信息。Figure 2D shows an example information card 220c associated with an informational entity. For example, information card 220c can include a notification title 222c (eg, "Answer...") that notifies the user that the information card is associated with a subject matter—eg, the New York Stock Exchange (NYSE). In this example, NYSE can also be a location. For example, an "answer" type of information card can be applied to an informational entity, such as from an excerpt, a biography, a quote (such as a stock quote displayed on a user's screen), or other informational content. For example, information card 220c can be generated from a snapshot associated with a user browsing online information associated with NYSE or information from other sources. For example, context 224c can identify a context associated with an information card. In this example, context 224c can include (or be determined from below) a portion of the snapshot that includes an excerpt of text on the NYSE that may have been presented to the user as content from the website. For example, information card 220c can also include a call to action 228c, such as a control displayed on Bob's smartphone for more information on the NYSE.

图3A是基于从呈现给用户的内容中提取的快照来提供信息卡的示例过程300的流程图。例如，并存(coincidence)能够包括向用户的感官内容项的同时、接近同时或者近期的呈现。在一些实施方式中，内容管理系统110能够使用由一个或多个处理器执行的指令来执行过程300的阶段。图1-2C被用于提供示例结构以用于执行过程300的步骤。FIG. 3A is a flowchart of an example process 300 for providing information cards based on snapshots extracted from content presented to a user. For example, coincidence can include simultaneous, near-simultaneous, or recent presentation of a sensory content item to a user. In some implementations, the content management system 110 can use instructions executed by one or more processors to perform the stages of the process 300 . 1-2C are used to provide example structures for performing the steps of process 300 .

由服务器设备接收与用户对计算设备的使用相关联的多个快照(302)。该多个快照中的每个快照基于在计算设备上呈现给用户的内容。例如，服务器设备——诸如内容管理系统110能够接收与对用户设备106a的使用相关联的快照202，包括呈现给用户206的内容204a、204b的快照202。A plurality of snapshots associated with usage of a computing device by a user are received by a server device (302). Each snapshot of the plurality of snapshots is based on content presented to the user on the computing device. For example, a server device such as content management system 110 can receive snapshots 202 associated with usage of user device 106a , including snapshots 202 of content 204a, 204b presented to user 206 .

在一些实施方式中，过程300能够进一步包括由设备获得多个快照。例如，用户设备106a能够拍摄快照202并且将它们提供给内容管理系统110。在一些实施方式中，快照202能够由内容管理系统110从内容管理系统110提供给用户设备106a的内容中获得。In some implementations, process 300 can further include taking, by the device, multiple snapshots. For example, user device 106a can take snapshots 202 and provide them to content management system 110 . In some implementations, snapshot 202 can be obtained by content management system 110 from content provided by content management system 110 to user device 106a.

在一些实施方式中，与用户对设备的使用相关联的快照能够包括：呈现给用户或者由用户所体验的音频。例如，快照202能够包括已经提供给用户设备106a的录音。在这个示例中，获得快照202还能够包括使用语言辨识或者其他辨识技术来获得所呈现的音频的文本翻译或识别(例如标题)。在一些实施方式中，获得快照202能够包括获得音频指纹(例如特定歌曲的音频指纹)以用于识别音频。In some implementations, snapshots associated with the user's use of the device can include audio presented to or experienced by the user. For example, snapshot 202 can include recordings that have been provided to user device 106a. In this example, obtaining snapshot 202 can also include using language recognition or other recognition techniques to obtain a textual translation or recognition (eg, title) of the presented audio. In some implementations, obtaining snapshot 202 can include obtaining an audio fingerprint (eg, an audio fingerprint of a particular song) for use in identifying audio.

在一些实施方式中，与用户对设备的使用相关联的快照能够包括与浏览器没有关联的内容。作为示例，快照202能够从诸如应用、web站点、社交网络站点、广告、和/或其他源的非浏览器源获得。In some implementations, snapshots associated with the user's use of the device can include content not associated with the browser. As an example, snapshot 202 can be obtained from non-browser sources such as applications, web sites, social networking sites, advertisements, and/or other sources.

在一些实施方式中，由设备获得多个快照能够周期性发生或者基于环境事件来发生。例如，快照202能够被周期性获得——诸如以N秒或者M分钟的间隔来获得，或者快照202能够在每当特定触发发生时获得——例如包括用户动作或其他触发。在一些实施方式中，环境事件能够由设备(例如用户设备106a)、由应用(例如，当用户开启应用或者执行触发动作时)、由与设备通信的服务(例如地图应用、日历、或者电子邮件)、由与设备相关联的操作系统触发，或者基于场境的改变、场景的改变、或者用户对设备的使用的改变。例如，新快照202能够发生在确定用户设备106a上的屏幕的阈值百分比已经改变的时候。In some implementations, taking multiple snapshots by the device can occur periodically or based on environmental events. For example, snapshot 202 can be obtained periodically, such as at intervals of N seconds or M minutes, or snapshot 202 can be obtained whenever a certain trigger occurs, including, for example, a user action or other trigger. In some implementations, ambient events can be generated by a device (such as user device 106a), by an application (such as when the user launches an application or performs a trigger action), or by a service that communicates with the device (such as a map application, calendar, or email). ), triggered by an operating system associated with the device, or based on a change in context, a change in scene, or a change in the use of the device by the user. For example, new snapshot 202 can occur when it is determined that a threshold percentage of the screen on user device 106a has changed.

在一些实施方式中，环境事件能够是在设备上执行的应用的场境的改变，其中，被用于检测场境改变的时间包括：大致当前的时间和先前的时间中的至少一个。例如，环境事件能够由以下来触发：用户206从应用或游戏的一个层级移动到另一层级、或者通过到达与应用或游戏相关联的里程碑。场境的改变(例如，层级的改变或到达里程碑)能够通过例如比较先前时间和当前时间的场境来确定。In some implementations, the context event can be a change in context of an application executing on the device, wherein the time used to detect the context change includes at least one of a substantially current time and a previous time. For example, an environmental event can be triggered by the user 206 moving from one level of the application or game to another, or by reaching a milestone associated with the application or game. A change in context (eg, a change in hierarchy or reaching a milestone) can be determined by, for example, comparing the context at a previous time with the current time.

多个快照被评估(304)。例如，快照评估引擎121能够评估接收到的快照202。例如，快照评估引擎121能够针对每个快照来识别被包括在快照202中的实体131。快照评估引擎121能够将识别出的实体连同与捕获相应快照的时间相关联的时间戳一起存储。Multiple snapshots are evaluated (304). For example, snapshot evaluation engine 121 can evaluate received snapshot 202 . For example, snapshot evaluation engine 121 can identify, for each snapshot, entities 131 included in snapshot 202 . The snapshot evaluation engine 121 can store the identified entities along with a timestamp associated with the time the corresponding snapshot was captured.

在一些实施方式中，接收与用户对设备的使用相关联的快照包括：接收表示被包括在相应快照中的内容的散列，以及评估接收到的快照包括：在评估中使用散列以代替原始内容。例如，代替评估快照202(或者除其之外)，快照评估引擎121能够评估与所提供内容相关联的散列信息。例如，信息能够包括，与内容相对应的文本(例如，“你的信用卡尾号*1437”)，或者与描述在内容中包含什么的内容相关联的元数据(例如“你的地址加邮编”)。In some implementations, receiving a snapshot associated with the user's use of the device includes receiving a hash representing content included in the corresponding snapshot, and evaluating the received snapshot includes using the hash in the evaluation in place of the original content. For example, instead of (or in addition to) evaluating snapshot 202, snapshot evaluation engine 121 can evaluate hash information associated with provided content. For example, the information can include, text corresponding to the content (e.g., "your credit card ending number *1437"), or metadata associated with the content describing what is included in the content (e.g., "your address plus zip code" ).

在一些实施方式中，评估接收到的快照能够进一步包括：基于被包括在快照中的实体来识别一个或多个事件，确定事件将发生的未来的第一时间，将事件连同第一时间一起来存储，确定事件之前的第二时间，并且确定呈现的时间能够包括：确定当前时间等于第二时间，并且呈现信息卡包括：在第二时间呈现事件的信息卡。例如，如上面结合图2A所述，评估快照202能够指示Bob和Carol之间的午餐约会事件的存在。午餐约会的时间/地点和Carol的手机号码也能够根据快照202来确定。内容管理系统110能够使用该信息来识别午餐约会并且在午餐约会将发生之前的预定时间生成一个或多个信息卡。In some implementations, evaluating the received snapshot can further include: identifying one or more events based on the entities included in the snapshot, determining a first time in the future when the event will occur, bringing the event together with the first time Storing, determining a second time before the event, and determining a time to present can include determining that the current time is equal to the second time, and presenting the information card includes presenting an information card for the event at the second time. For example, as described above in connection with FIG. 2A , evaluation snapshot 202 can indicate the existence of a lunch date event between Bob and Carol. The time/place of the lunch appointment and Carol's cell phone number can also be determined from the snapshot 202 . The content management system 110 can use this information to identify the lunch appointment and generate one or more information cards at a predetermined time before the lunch appointment will occur.

在一些实施方式中，识别被包括在快照中的实体能够进一步包括：在文本中识别对事件的自然语言描述。例如，快照评估引擎121能够识别内容204a中的文本，其描述事件(例如午餐约会)或指示与事件相关联的实体(例如Bob、Carol、J’s餐馆、和Carol的电话号码)。In some implementations, identifying entities included in the snapshot can further include identifying a natural language description of the event in the text. For example, snapshot evaluation engine 121 can recognize text in content 204a that describes an event (e.g., a lunch appointment) or indicates an entity associated with the event (e.g., Bob, Carol, J's restaurant, and Carol's phone number).

在一些实施方式中，事件能够是发生在未来的用户感兴趣的活动。作为示例，由快照评估引擎121识别的事件能够是Bob与Carol的午餐约会，其是Bob感兴趣的。In some implementations, an event can be an activity that occurs in the future that is of interest to the user. As an example, an event identified by snapshot evaluation engine 121 could be Bob's lunch date with Carol, which is of interest to Bob.

对于每个相应的快照，相应快照所指示的相应实体集被识别(306)。例如，快照评估引擎121能够识别被包括在从内容204a获得的快照202中的实体131。例如，针对快照202所识别的实体能够包括Bob、Carol、J’s餐馆、和Carol的手机号码。For each respective snapshot, a respective entity set indicated by the respective snapshot is identified (306). For example, snapshot evaluation engine 121 can identify entity 131 included in snapshot 202 obtained from content 204a. For example, entities identified for snapshot 202 can include Bob, Carol, J's restaurant, and Carol's cell phone numbers.

将对相应实体集的指示和指示捕获相应快照的相应时间的相应时间戳存储到存储器(308)。相应实体集和相应时间戳在存储器中相关联。作为示例，快照评估引擎121能够将最显著识别的实体连同与捕获相应快照的时间相关联的时间戳一起存储。例如，时间戳能够被存储在时间戳132的数据存储库中以供稍后用于生成和呈现与快照202及相关联的实体有关的信息卡201。An indication of the corresponding set of entities and a corresponding time stamp indicating the corresponding time at which the corresponding snapshot was captured is stored to memory (308). Corresponding entity sets and corresponding timestamps are associated in memory. As an example, the snapshot evaluation engine 121 can store the most prominently identified entities along with a timestamp associated with the time the corresponding snapshot was captured. For example, the timestamp can be stored in the timestamp 132 data store for later use in generating and presenting the information card 201 related to the snapshot 202 and associated entities.

基于多个快照中的第一快照，确定向用户呈现一个或多个信息卡的第一时间(310)。例如，信息卡引擎122能够确定向用户呈现一个或多个信息卡的时间，包括确定目标时间。例如，对于Bob与Carol的即将发生的午餐约会，信息卡引擎122能够确定对Bob的提醒时间，其是在所排程的的中午午餐约会之前一个小时。在一些实施方式中，能够确定呈现信息卡的多个时间，例如以包括Bob次日有与Carol的午餐约会的提醒，该提醒将在该约会之前的夜晚发送。对于已经被处理的非事件快照——诸如所获得的与NYSE相关联的快照202，目标时间能够和与快照202相关联的时间戳有关，诸如用以在稍后的时间向用户示出信息卡220c。在一些实施方式中，目标时间能够被计算为更接近事件的开始时间，或者能够基于将接收信息卡的用户的当前位置来被重新计算(例如，基于Bob的当前位置，Bob可能需要90分钟来驾车前往午餐约会)。Based on a first snapshot of the plurality of snapshots, a first time to present one or more information cards to a user is determined (310). For example, the information card engine 122 can determine when to present one or more information cards to the user, including determining a target time. For example, for Bob's upcoming lunch appointment with Carol, the info card engine 122 can determine a reminder time for Bob that is one hour before the scheduled noon lunch appointment. In some implementations, multiple times to present the information card can be determined, for example to include a reminder that Bob has a lunch appointment with Carol the next day, which will be sent the night before the appointment. For non-event snapshots that have already been processed, such as the snapshot 202 associated with the NYSE obtained, the target time can be related to the timestamp associated with the snapshot 202, such as to show the user an information card at a later time 220c. In some implementations, the target time can be calculated to be closer to the start time of the event, or can be recalculated based on the current location of the user who will receive the information card (e.g., based on Bob's current location, Bob may need 90 minutes to driving to a lunch date).

在一些实施方式中，目标时间能够是过去的时间，并且信息卡能够提供对在过去向用户显现的事件或实体的提醒。例如，信息卡220c能够不基于事件，而基于与NYSE相关的内容的过去的呈现。In some implementations, the target time can be in the past, and the information card can provide reminders of events or entities that were presented to the user in the past. For example, the information card 220c could be based not on events, but on past presentations of NYSE-related content.

在一些实施方式中，确定呈现一个或多个信息卡的时间能够包括确定过去的一个或多个预定时间，以及针对每个时间，确定用于向用户呈现的一个或多个信息卡。例如，信息卡引擎122能够确定呈现信息卡220c的多个时间，并且这些时间能够基于首次向用户呈现信息卡220c所基于的、与NYSE相关联的内容的时间。In some implementations, determining when to present the one or more information cards can include determining one or more predetermined times in the past, and for each time, determining one or more information cards for presentation to the user. For example, the information card engine 122 can determine a number of times at which the information card 220c is presented, and these times can be based on the time at which the content associated with the NYSE on which the information card 220c was based was first presented to the user.

在一些实施方式中，预定时间能够依赖于用户的当前场境而变化。例如，基于用户206的当前动作——例如在偶然在互联网冲浪或者应用中间，信息卡引擎122能够延迟或加速数据卡的生成(例如基于用户的当前位置)。在一些实施方式中，信息卡能够在用户请求时显现——诸如在打开显示和/或管理信息卡的应用或工具的时候，和/或通过请求所有或特定信息卡被呈现来显现。能够使用用于显现信息卡的其他信号。In some implementations, the predetermined time can vary depending on the user's current context. For example, the information card engine 122 can delay or speed up the generation of data cards (eg, based on the user's current location) based on the current actions of the user 206 - eg, while casually surfing the Internet or in the middle of an application. In some implementations, information cards can be presented upon user request, such as upon opening an application or tool that displays and/or manages information cards, and/or by requesting that all or specific information cards be presented. Other signals for revealing the information card can be used.

在第一时间，具有与第一时间相对应的时间戳的实体位于存储器中(312)。例如，基于目标时间和与所存储的实体中的相应实体相关联的时间戳的比较，信息卡引擎122能够从所存储的实体131中识别实体以用于生成信息卡。例如，对于为Carol和Bob所排程的午餐约会，信息卡引擎122能够识别与即将发生的午餐约会相关联的、待在信息卡中使用的信息。例如，Carol的电话号码能够是被识别以用于生成包括对Bob的提醒的信息卡的实体。信息卡能够在午餐约会之前一个小时的目标时间被发送并且能够包括Carol的手机号码。At a first time, an entity with a timestamp corresponding to the first time is located in memory (312). For example, based on a comparison of the target time and the time stamps associated with corresponding ones of the stored entities, the information card engine 122 can identify entities from the stored entities 131 for use in generating an information card. For example, for a lunch appointment scheduled for Carol and Bob, the information card engine 122 can identify information to be used in the information card associated with the upcoming lunch appointment. For example, Carol's phone number could be the entity identified for generating an information card including a reminder to Bob. The information card can be sent at a target time of one hour before the lunch appointment and can include Carol's cell phone number.

在一些实施方式中，识别实体能够进一步包括：辨识快照中的文本，以及对该文本进行解析以识别实体。例如，快照评估引擎121能够辨识：快照202包括文本。快照评估引擎121能够以各种方式来提取文本，诸如通过使用光学字符辨识(OCR)或者其他字符辨识技术来提取、通过从超文本标记语言(HTML)或被用于生成内容(例如，内容204a或204b)的其他代码中提取文本，或者通过其他技术来提取。在一些实施方式中，辨识快照中的文本能够包括：使用自然语言处理技术，例如其使用与文本中的词或短语相关联的语法，或者快照的源(例如，基于电子邮件格式，日历条目格式、或者其他格式)。在一些实施方式中，其他视觉辨识技术能够被应用于快照，例如对象识别、地标识别、和/或从图像中检测实体的其他方式。In some implementations, identifying the entity can further include: recognizing text in the snapshot, and parsing the text to identify the entity. For example, snapshot evaluation engine 121 can recognize that snapshot 202 includes text. Snapshot evaluation engine 121 can extract text in various ways, such as by using optical character recognition (OCR) or other character recognition techniques, by extracting from hypertext markup language (HTML), or by being used to generate content (e.g., content 204a or 204b), or by other techniques. In some implementations, recognizing the text in the snapshot can include using natural language processing techniques, such as those that use grammars associated with words or phrases in the text, or the source of the snapshot (e.g., based on email format, calendar entry format , or other formats). In some implementations, other visual recognition techniques can be applied to the snapshot, such as object recognition, landmark recognition, and/or other means of detecting entities from images.

信息卡基于经定位的实体中的一个或多个来生成(314)。例如，信息卡引擎122能够生成包括一个或多个所识别的实体131的信息卡201(例如，包括Carol的手机号码的信息卡)。An information card is generated based on one or more of the located entities (314). For example, the information card engine 122 can generate an information card 201 including one or more identified entities 131 (eg, an information card including Carol's cell phone number).

生成的信息卡被提供用于呈现给用户(316)。例如，一旦信息卡201被生成，信息卡201就可以被呈现多次，例如在用户设备106a的屏幕208c上呈现。The generated information card is provided for presentation to the user (316). For example, once information card 201 is generated, information card 201 may be presented multiple times, such as on screen 208c of user device 106a.

在一些实施方式中，存储所识别的实体能够包括：存储与所识别的实体相关联的场境信息，以及呈现信息卡能够进一步包括：将场境信息连同关于所识别实体的信息一起在信息卡上呈现。例如，当由快照评估引擎121评估快照202时，还能够针对场境信息来确定和存储信息，场境与包括实体的相应快照相关联(例如，识别电子邮件消息和即将发生的午餐约会)。例如，在信息卡201被提供用于呈现时，信息卡201能够包括场境224a(例如识别午餐约会电子邮件或相关联的信息)。其他示例场境在场境224b和224c中示出。In some implementations, storing the identified entity can include storing contextual information associated with the identified entity, and presenting the information card can further include: presenting the contextual information along with information about the identified entity in the information card presented above. For example, when snapshot 202 is evaluated by snapshot evaluation engine 121, information can also be determined and stored for context information associated with the corresponding snapshot including entities (eg, identifying email messages and upcoming lunch appointments). For example, when information card 201 is provided for presentation, information card 201 can include context 224a (eg, identifying a lunch appointment email or associated information). Other example contexts are shown in contexts 224b and 224c.

图4是可以被用来实现本文档中所述的系统和方法的、作为客户端或作为服务器或多个服务器的示例计算设备400、450的框图。计算设备400意图表示各种形式的数字计算机，诸如膝上型计算机、台式计算机、工作站、个人数字助理、服务器、刀片式服务器、大型主机、以及其他适当的计算机。计算设备400进一步意图表示任何其他的通常非移动的设备，诸如电视或具有嵌入其中或附接至其上的一个或多个处理器的其他电子设备。计算设备450意图表示各种形式的移动设备，诸如个人数字助理、蜂窝电话、智能电话、以及其他计算设备。这里所示的组件、它们的连接和关系、以及它们的功能仅意在作为示例，并且不意在限制在本文档中所描述和/或要求保护的技术的实施方式。4 is a block diagram of an example computing device 400, 450, either as a client or as a server or servers, that may be used to implement the systems and methods described in this document. Computing device 400 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Computing device 400 is further intended to represent any other generally non-mobile device, such as a television or other electronic device having one or more processors embedded in or attached thereto. Computing device 450 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, and other computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the techniques described and/or claimed in this document.

计算设备400包括处理器402、存储器404、存储设备406、连接到存储器404和高速扩展端口410的高速控制器408、以及连接到低速总线414和存储设备406的低速控制器412。组件402、404、406、408、410、和412中的每个都使用各种总线互连，并且可以在公共主板上或者视情况以其他方式安装。处理器402可以处理指令以用于在计算设备400内执行，所述指令包括存储在存储器404中或存储设备406上以在诸如耦合到高速控制器408的显示器416的外部输入/输出设备上显示GUI的图形信息的指令。在其他实施方式中，多个处理器和/或多个总线能够视情况与多个存储器和多个类型的存储器一起使用。而且，多个计算设备400可以被连接，其中每个设备提供必要操作的部分(例如，作为服务器组、刀片式服务器的群组、或者多处理器系统)。Computing device 400 includes processor 402 , memory 404 , storage device 406 , high-speed controller 408 connected to memory 404 and high-speed expansion port 410 , and low-speed controller 412 connected to low-speed bus 414 and storage device 406 . Each of components 402, 404, 406, 408, 410, and 412 are interconnected using various buses and may be mounted on a common motherboard or otherwise as appropriate. Processor 402 may process instructions for execution within computing device 400 including storage in memory 404 or on storage device 406 for display on an external input/output device such as display 416 coupled to high-speed controller 408 Instructions for graphical information of the GUI. In other implementations, multiple processors and/or multiple buses can be used, as appropriate, with multiple memories and types of memory. Also, multiple computing devices 400 may be connected, with each device providing portions of the necessary operations (eg, as a server bank, a group of blade servers, or a multi-processor system).

存储器404在计算设备400内存储信息。在一个实施方式中，存储器404是计算机可读介质。在一个实施方式中，存储器404是一个或多个易失性存储器单元。在另一实施方式中，存储器404是一个或多个非易失性存储器单元。Memory 404 stores information within computing device 400 . In one implementation, memory 404 is a computer readable medium. In one implementation, memory 404 is one or more volatile memory units. In another implementation, memory 404 is one or more non-volatile memory units.

存储设备406能够提供用于计算设备400的大容量存储。在一个实施方式中，存储设备406是计算机可读介质。在各种不同的实施方式中，存储设备406可以是软盘设备、硬盘设备、光盘设备、或者磁带设备、闪速存储器或其他类似的固态存储器设备、或者设备的阵列——包括存储区域网络或其他配置中的设备。在一个实施方式中，计算机程序产品有形地以信息载体来体现。计算机程序产品包含指令，其在被执行时实施一个或多个方法——诸如上述的那些方法。信息载体是计算机或机器可读介质，诸如存储器404、存储设备406、或者处理器402上的存储器。The storage device 406 is capable of providing mass storage for the computing device 400 . In one implementation, storage device 406 is a computer-readable medium. In various implementations, the storage device 406 may be a floppy disk device, a hard disk device, an optical disk device, or a magnetic tape device, a flash memory or other similar solid-state storage device, or an array of devices—including a storage area network or other Devices in configuration. In one embodiment, a computer program product is tangibly embodied in an information carrier. The computer program product comprises instructions which, when executed, implement one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as memory 404 , storage device 406 , or memory on processor 402 .

高速控制器408管理计算设备400的带宽密集型操作，而低速控制器412管理较低的带宽密集型操作。这样的职责分配只是示例。在一个实施方式中，高速控制器408耦合到存储器404、显示器416(例如，通过图形处理器或加速器)，以及耦合到高速扩展端口410，其可以接受各个扩展卡(未示出)。在该实施方式中，低速控制器412耦合到存储设备406和低速总线414。可以包括各种通信端口(例如，USB、以太网、无线以太网)的低速总线414(例如低速扩展端口)可以耦合到一个或多个输入/输出设备——诸如键盘、指示设备、扫描仪、或者例如通过网络适配器来耦合到诸如开关或路由器的联网设备。High-speed controller 408 manages bandwidth-intensive operations of computing device 400 , while low-speed controller 412 manages less bandwidth-intensive operations. Such assignment of responsibilities is just an example. In one embodiment, high-speed controller 408 is coupled to memory 404, display 416 (eg, through a graphics processor or accelerator), and to high-speed expansion ports 410, which may accept various expansion cards (not shown). In this embodiment, low-speed controller 412 is coupled to storage device 406 and low-speed bus 414 . Various communication ports (e.g., USB, Ethernet, Wireless Ethernet) a low-speed bus 414 (e.g., a low-speed expansion port) can be coupled to one or more input/output devices—such as a keyboard, pointing device, scanner, or, for example, through a network adapter to a device such as a switch or A router is a networking device.

计算设备400可以以多种不同形式来实现，如图所示。例如，其可以被实现为标准服务器420，或者在这样的服务器的群组中多次实现。其还可以被实现为机架式服务器系统424的一部分。此外，其可以以诸如膝上型计算机422的个人计算机来实现。替选地，来自计算设备400的组件可以与诸如计算设备450的移动设备中的其他组件(未示出)相组合。这样的设备中的每一个可以包含计算设备400、450中的一个或多个，并且整个系统可以由彼此通信的多个计算设备400、450构成。Computing device 400 may be implemented in many different forms, as shown. For example, it may be implemented as a standard server 420, or multiple times in a group of such servers. It can also be implemented as part of rack server system 424 . Furthermore, it can be implemented with a personal computer such as laptop computer 422 . Alternatively, components from computing device 400 may be combined with other components (not shown) in a mobile device, such as computing device 450 . Each of such devices may contain one or more of the computing devices 400, 450, and the overall system may consist of multiple computing devices 400, 450 in communication with each other.

除了其他组件以外，计算设备450包括处理器452、存储器464、诸如显示器454的输入/输出设备、通信接口466、以及收发器468。计算设备450还可以被提供有存储设备——诸如微型驱动器或者其他设备以提供额外的存储。组件450、452、464、454、466、和468中的每个使用各种总线互连，并且组件中的若干组件可以被安装在公共主板上或者视情况以其他方式安装。Computing device 450 includes, among other components, processor 452 , memory 464 , input/output devices such as display 454 , communication interface 466 , and transceiver 468 . Computing device 450 may also be provided with a storage device, such as a microdrive or other device, to provide additional storage. Each of the components 450, 452, 464, 454, 466, and 468 are interconnected using various buses, and several of the components may be mounted on a common motherboard or otherwise as appropriate.

处理器452能够处理指令以用于在计算设备450内执行，所述指令包括存储在存储器464中的指令。处理器还可以包括分离的模拟和数字处理器。例如，处理器可以提供计算设备450的其他组件的协调，诸如对用户接口、计算设备450运行的应用、以及计算设备450进行的无线通信的控制。Processor 452 is capable of processing instructions for execution within computing device 450 , including instructions stored in memory 464 . Processors may also include separate analog and digital processors. For example, a processor may provide coordination of other components of computing device 450 , such as control of a user interface, applications run by computing device 450 , and wireless communications by computing device 450 .

处理器452能够通过耦合到显示器454的控制接口458和显示接口456与用户通信。例如，显示器454可以是TFT LCD显示器或者OLED显示器，或者其他适当的显示技术。显示接口456可以包括适当电路以用于驱动显示器454向用户呈现图形和其他信息。控制接口458可以从用户接收命令并且将它们变换以用于提交给处理器452。此外，可以提供与处理器452通信的外部接口462，从而使得计算设备450能够与其他设备进行近区域通信。例如，外部接口462可以提供有线通信(例如经由对接程序(docking procedure))或者无线通信(例如经由或其他这样的技术)。The processor 452 is capable of communicating with a user through a control interface 458 coupled to a display 454 and a display interface 456 . For example, display 454 may be a TFT LCD display or OLED display, or other suitable display technology. Display interface 456 may include appropriate circuitry for driving display 454 to present graphics and other information to a user. Control interface 458 may receive commands from a user and transform them for submission to processor 452 . Additionally, an external interface 462 may be provided in communication with the processor 452, thereby enabling near-area communication between the computing device 450 and other devices. For example, external interface 462 may provide wired communication (eg, via a docking procedure) or wireless communication (eg, via or other such techniques).

存储器464在计算设备450内存储信息。在一个实施方式中，存储器464是计算机可读介质。在一个实施方式中，存储器464是一个或多个易失性存储器单元。在另一实施方式中，存储器464是一个或多个非易失性存储器单元。还能够提供扩展存储器474并通过扩展接口472将其连接到计算设备450，扩展接口472可以包括例如订户识别模块(SIM)卡接口。这样的扩展存储器474能够为计算设备450提供额外的存储空间，或者还可以存储计算设备450的应用或其他信息。具体地，扩展存储器474可以包括执行或补充上述过程的指令，并且可以还包括安全信息。因此，例如，扩展存储器474可以被提供作为计算设备450的安全模块，并且可以被编程有许可对计算设备450的安全使用的指令。此外，可以经由SIM卡提供连同额外信息一起的安全应用，诸如以不可非法侵入的方式将识别信息设置在SIM卡上。Memory 464 stores information within computing device 450 . In one embodiment, memory 464 is a computer readable medium. In one embodiment, memory 464 is one or more volatile memory units. In another implementation, memory 464 is one or more non-volatile memory units. Expansion memory 474 can also be provided and connected to computing device 450 through expansion interface 472, which may include, for example, a Subscriber Identity Module (SIM) card interface. Such expansion memory 474 can provide additional storage space for computing device 450 , or can also store applications or other information for computing device 450 . Specifically, expansion memory 474 may include instructions to perform or supplement the above-described processes, and may also include security information. Thus, for example, expansion memory 474 may be provided as a security module of computing device 450 and may be programmed with instructions that permit secure use of computing device 450 . Furthermore, security applications together with additional information can be provided via the SIM card, such as identification information being set on the SIM card in a hackable manner.

如下面所讨论的，存储器可以包括例如，闪速存储器和/或MRAM存储器。在一个实施方式中，计算机程序产品有形在信息载体中体现。计算机程序产品包含指令，其在被执行时实施一个或多个方法——诸如上述的那些方法。信息载体是计算机或机器可读介质，诸如存储器464、扩展存储器474、或者处理器452上的存储器。As discussed below, memory may include, for example, Flash memory and/or MRAM memory. In one embodiment, a computer program product is tangibly embodied in an information carrier. The computer program product comprises instructions which, when executed, implement one or more methods, such as those described above. The information carrier is a computer or machine readable medium, such as memory 464 , extended memory 474 , or memory on processor 452 .

在必要的情况下，计算设备450可以通过通信接口466无线地通信，通信接口466可以包括数字信号处理电路。通信接口466可以提供在各种模式或协议下的通信，除了其他以外，诸如GSM语言通话、SMS、EMS或MMS消息传递、CDMA、TDMA、PDC、WCDMA、CDMA2000或者GPRS。例如，这样的通信可以通过收发器468(例如，射频收发器)而发生。此外，短距离通信可以诸如使用WiFi、或者其他这样的收发器(未示出)而发生。此外，GPS接收器模块470可以向计算设备450提供额外无线数据，其视情况可以被运行在计算设备450上的应用使用。Where necessary, computing device 450 may communicate wirelessly through communication interface 466, which may include digital signal processing circuitry. The communication interface 466 may provide communication in various modes or protocols such as GSM voice calling, SMS, EMS or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000 or GPRS, among others. For example, such communication may occur through a transceiver 468 (eg, a radio frequency transceiver). In addition, short-range communication can be used such as WiFi, or other such transceivers (not shown). Additionally, GPS receiver module 470 may provide additional wireless data to computing device 450 , which may optionally be used by applications running on computing device 450 .

计算设备450还可以使用音频编解码器460来可听地通信，可以从用户接收说出的信息且将其变换为可使用的数字信息。同样，音频编解码器460可以为用户生成可听的声音，诸如通过例如，在计算设备450的送受话器中的扬声器来生成。这样的声音可以包括来自语音电话呼叫的声音，可以包括已记录的声音(例如，语音消息、音乐文件等)并且还可以包括在计算设备450上操作的应用所生成的声音。Computing device 450 may also communicate audibly using audio codec 460, which may receive spoken information from a user and transform it into usable digital information. Likewise, audio codec 460 may generate audible sound for the user, such as through a speaker in a headset of computing device 450 , for example. Such sounds may include sounds from voice telephone calls, may include recorded sounds (eg, voice messages, music files, etc.), and may also include sounds generated by applications operating on computing device 450 .

计算设备450可以用多个不同形式来实现，如图所示。例如，其可以被实现为蜂窝电话480。其还可以被实现为智能电话482、个人数字助理、或其他移动设备的一部分。Computing device 450 may be implemented in a number of different forms, as shown. It may be implemented as a cellular telephone 480, for example. It can also be implemented as part of a smartphone 482, personal digital assistant, or other mobile device.

这里所述的系统和技术的各个实施方式可以用数字电子电路、集成电路、专门设计的ASIC(专用集成电路)、计算机硬件、固件、软件、和/或其组合来实施。这些各个实施方式可以包括在可编程系统上可执行和/或可解释的一个或多个计算机程序中的实现，所述可编程系统包括至少一个可编程处理器、至少一个输入设备以及至少一个输出设备，所述可编程处理器可以是专用或通用的处理器，其被耦合以从存储系统接收数据和指令，以及传送数据和指令到存储系统。Various implementations of the systems and techniques described herein may be implemented in digital electronic circuitry, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include implementation in one or more computer programs executable and/or interpretable on a programmable system comprising at least one programmable processor, at least one input device, and at least one output device, the programmable processor, which may be a special-purpose or general-purpose processor, coupled to receive data and instructions from a storage system, and to transmit data and instructions to a storage system.

这些计算机程序(还被称为程序、软件、软件应用或代码)包括用于可编程处理器的机器指令，并且能够用高级过程和/或面向对象的编程语言、和/或汇编/机器语言来实现。能够使用其他编程范例，例如功能性编程、逻辑编程、或者其他编程。如本文所使用的，术语“机器可读介质”指代用于提供机器指令和/或数据给可编程处理器的任何计算机程序产品、装置、和/或设备(例如磁盘、光盘、存储器、可编程逻辑器件(PLD))，其包括接收机器指令以作为机器可读信号的机器可读介质。术语“机器可读信号”指代用于提供机器指令和/或数据给可编程处理器的任何信号。These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be written in a high-level procedural and/or object-oriented programming language, and/or assembly/machine language accomplish. Other programming paradigms can be used, such as functional programming, logic programming, or other programming. As used herein, the term "machine-readable medium" refers to any computer program product, apparatus, and/or device (such as a magnetic disk, optical disk, memory, programmable logic device (PLD)) that includes a machine-readable medium that receives machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

为了提供与用户的交互，这里所述的系统和技术能够在计算机上实现，该计算机具有用于向用户显示信息的显示设备(例如CRT(阴极射线管)或LCD(液晶显示器)监视器)以及用户能够通过其向计算机提供输入的键盘和指示设备(例如鼠标或轨迹球)。其他种类的设备也能够被用来提供与用户的交互；例如，提供给用户的反馈能够是任何形式的感官反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且来自用户的输入能够以任何形式接收，包括声学、语音、或触觉输入。To provide for interaction with the user, the systems and techniques described herein can be implemented on a computer having a display device (such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and A keyboard and pointing device (such as a mouse or trackball) through which a user can provide input to a computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be in the form of Receive in any form, including acoustic, speech, or tactile input.

这里所述的系统和技术能够在计算系统中实现，该计算系统包括后端组件(例如，作为数据服务器)、或包括中间件组件(例如，应用服务器)、或包括前端组件(例如，具有用户通过其能够与这里所述的系统和技术的实现交互的图形用户界面或web浏览器的客户端计算机)、或这样的后端、中间件、或前端组件的任何组合。系统的组件能够通过数字数据通信的任何形式或介质(例如通信网络)来互连。通信网络的示例包括局域网(“LAN”)、广域网(“WAN”)、和互联网。The systems and techniques described herein can be implemented in computing systems that include back-end components (e.g., as data servers), or include middleware components (e.g., application servers), or include front-end components (e.g., with user A client computer through which an implementation of the systems and technologies described herein can interact with a graphical user interface or a web browser), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), and the Internet.

计算系统能够包括客户端和服务器。客户端和服务器一般互相处于远程并且通常通过通信网络交互。客户端和服务器的关系借助于运行在相应计算机上且彼此具有客户端-服务器关系的计算机程序而产生。A computing system can include clients and servers. A client and server are generally remote from each other and interact, usually through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

尽管本说明书包含许多特定实施方式细节，但这些不应当被解释为对任何技术的范围或可以要求保护的范围的限制，而应当被解释为对特定技术的特定实施方式的特征的描述。本说明书中在单独实施例的场境中所描述的某些特征还能够在单个实施方式的组合中实现。相反，单个实施方式的场境中所述的各个特征也能够分别在多个实施方式中单独地或者以任何合适的子组合来实现。此外，尽管特征可以在上面被描述为以特定组合来动作并且甚至最初如此要求保护，但是在一些情况下来自所要求保护的组合的一个或多个特征能够从组合中被删去，并且所要求保护的组合可以涉及子组合或子组合的变型。While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any technology or of what may be claimed, but rather as descriptions of features of particular implementations of particular technologies. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, individual features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination. Furthermore, although features may have been described above as acting in particular combinations and even initially claimed as such, in some cases one or more features from a claimed combination can be omitted from the combination and the claimed A protected combination may involve a subcombination or a variation of a subcombination.

类似地，尽管附图中以特定次序描绘了操作，但是这不应当被理解为需要这样的操作以所示的特定次序或者以连续顺序来执行，或者所有图示的操作都被执行才能以达到所需结果。在某些状况下，多任务和并行处理可能是有利的。而且，上述的实施方式中的各个系统组件的分离不应当被理解为在所有实施方式中都需要这样的分离，并且应当理解，所述的程序组件和系统通常能够一起整合到单个软件产品中或者封装到多个软件产品中。因此，已经描述了本主题的特定实施方式。其他实施方式落入所附权利要求书的范围内。在一些情况下，权利要求书中所记载的动作能够以不同顺序来执行且仍旧达到所需结果。此外，附图中所描绘的过程不必要求所示的特定顺序或者连续顺序来达到所需结果。在某些实施方式中，多任务和并行处理可能是有利的。Similarly, while operations are depicted in the figures in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown, or in sequential order, or that all illustrated operations be performed, to achieve desired result. In certain situations, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together into a single software product or Packaged into multiple software products. Thus, certain embodiments of the present subject matter have been described. Other implementations are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.

Claims (20)

1. a kind of method, including：

Received and the associated plurality of snapshot of use of the user to computing device by server apparatus, it is every in the plurality of snapshotIndividual snapshot is all based on the content for being presented to the user on said computing device；

The plurality of snapshot is assessed, including for each corresponding snapshot：

Recognize the corresponding entity set indicated by the corresponding snapshot；And

To memory storage is to the instruction of the corresponding entity set and indicates the corresponding of the corresponding snapshot captured corresponding timeTimestamp, wherein, the corresponding entity set and corresponding timestamp are associated in which memory；

Based on the first snapshot in the plurality of snapshot, it is determined that to the user the first of one or more releases is presented whenBetween；

In the very first time, the entity with the timestamp corresponding with the very first time is positioned in memory；

Release is generated based on one or more in the entity for being positioned；And

Generated release is provided for presenting to the user.

2. the method for claim 1, wherein：

The received snapshot of assessment is further included：One or more events are recognized based on included entity in snapshot,Determine that the event, by the very first time occurring, in future, the event is stored together with the very first time, it is determined thatThe second time before the event, and

It is determined that the time for presenting includes：Determining that current time is equal to second time, and provide release includes：DescribedSecond time provided the release of the event.

3. method as claimed in claim 2, wherein, recognize that included entity in the snapshot is further included：In textThe natural language description to event is recognized in this.

4. method as claimed in claim 2, wherein, the event be by it is following occur, the work that the user is interestedIt is dynamic.

5. the method for claim 1, wherein：

Object time be in the past time, and

Described information card is provided for the prompting of the event or entity manifested to user in the past.

6. the method for claim 1, wherein storage includes：The context information that storage is associated with the entity for being recognized,And wherein, there is provided further include：By the context information together with the entity with regard to being recognized information in releaseUpper offer.

7. the method for claim 1, wherein recognize that included entity in the snapshot is further included：IdentificationText in the snapshot, and the text is parsed to recognize entity.

8. the method for claim 1, wherein determine that the time that one or more releases are presented includes：It is determined that in mistakeOne or more scheduled times gone, and for each time, it is determined that for one or more letters presented to the userBreath card.

9. method as claimed in claim 8, wherein, the scheduled time depends on the current context of user and changes.

10. the method for claim 1, further includes to obtain the plurality of snapshot by the equipment.

11. methods as claimed in claim 10, wherein, obtain snapshot and periodically occur or occurred based on environment event.

12. methods as claimed in claim 11, wherein, the environment event by the equipment, by application, by with the equipmentThe service of communication, by be associated with the equipment operating system triggering, or the environment event based on context change, fieldThe change for using of the change of scape or the user to the equipment.

13. methods as claimed in claim 12, wherein：

The environment event is the change of the context of the application for performing on said device, and

Being used for the time of the change of detection context includes：Substantially at least one of current time and previous time.

The method of claim 1, wherein 14. included with described using the snapshot being associated：It is presented to userOr audio frequency that user is experienced.

The method of claim 1, wherein 15. include not having with browser with described using the snapshot being associatedThe content of association.

The method of claim 1, wherein 16. receive the snapshot being associated with the use includes：Receive and representThe hash of the content being included in corresponding snapshot, and wherein, assessing received snapshot includes：Used in the assessmentThe hash rather than original contents.

The method of claim 1, wherein 17. provide the release includes providing from including calling, navigation or electronicsThe control of the associated user action selected in the group of mail.

A kind of 18. computer programs being included in non-transitory computer-readable medium including instruction, the instructionWhen executed so that one or more processors：

Received and the associated plurality of snapshot of use of the user to computing device by server apparatus, it is every in the plurality of snapshotIndividual snapshot is all based on the content for being presented to the user on said computing device；

Assessing the plurality of snapshot includes, for each corresponding snapshot：

Recognize the corresponding entity set indicated by the corresponding snapshot；And

To memory storage is to the instruction of the corresponding entity set and indicates the corresponding of the corresponding snapshot captured corresponding timeTimestamp, wherein, the corresponding entity set and corresponding timestamp are associated in memory；

Based on the first snapshot in the plurality of snapshot, it is determined that to the user the first of one or more releases is presented whenBetween；

In the very first time, the entity with the timestamp corresponding with the very first time is positioned in memory；

Release is generated based on one or more in the entity for being positioned；And

Generated release is provided for presenting to the user.

A kind of 19. systems, including：

One or more processors；And

Including one or more memory components of instruction, the instruction causes when executed one or more of processDevice：

Received and the associated plurality of snapshot of use of the user to computing device by server apparatus, it is every in the plurality of snapshotIndividual snapshot is all based on the content for being presented to the user on said computing device；

The plurality of snapshot is assessed, including for each corresponding snapshot：

The corresponding entity set that identification is indicated by corresponding snapshot；And

To memory storage to the instruction of the corresponding entity set and indicate the corresponding snapshot captured corresponding time it is corresponding whenBetween stab, wherein, the corresponding entity set and corresponding timestamp are associated in which memory；

Based on the first snapshot in the plurality of snapshot, it is determined that to the user the first of one or more releases is presented whenBetween；

In the very first time, the entity with the timestamp corresponding with the very first time is positioned in memory；

Release is generated based on one or more entities in the entity for being positioned；And

Generated release is provided for presenting to the user.

20. systems as claimed in claim 19, wherein, assess received snapshot and further include：Based on the institute in snapshotIncluding entity recognizing one or more events, determine the event by it is occurring, in following very first time, by the thingPart is stored together with the very first time, it is determined that the second time before the event, and wherein it is determined that present whenBetween include：Determining that current time is equal to second time, and provide release includes：There is provided described in second timeThe release of event.