CN107491283A - Apparatus, method and graphical user interface for dynamically adjusting presentation of audio output - Google Patents

Abstract

The invention provides an apparatus, method and graphical user interface for dynamically adjusting the presentation of audio output. An electronic device displays a user interface including a first interaction region and a second interaction region of an application on a display. While displaying the user interface, the device detects an input on the display caused by a contact on the touch-sensitive surface at a location corresponding to a user interface element in the first interaction region. In response to detecting an input caused by a contact, when the input satisfies an intensity-based activation criterion (e.g., the contact has a characteristic intensity above an intensity threshold), the device obscures a first interaction region of the application in addition to the user interface element without obscuring a second interaction region of the application. When the input satisfies a selection criterion (e.g., the contact has a characteristic intensity below an intensity threshold), the device performs a selection operation corresponding to the user interface element without obscuring the first interaction region of the application.

Description

Translated fromChinese

用于动态地调整音频输出的呈现的设备、方法和图形用户界面Apparatus, method and graphical user for dynamically adjusting presentation of audio outputinterface

技术领域technical field

本发明大体上涉及提供音频输出的电子设备，并且更具体地涉及动态地调整音频输出的呈现的设备、方法和图形用户界面。The present invention relates generally to electronic devices that provide audio output, and more particularly to devices, methods, and graphical user interfaces that dynamically adjust the presentation of audio output.

背景技术Background technique

一些电子设备利用视听接口作为提供关于用户与设备的交互的反馈的方式。Some electronic devices utilize audio-visual interfaces as a way of providing feedback about the user's interaction with the device.

发明内容Contents of the invention

然而，一些电子设备以有限的、低效的和令人沮丧的方式来提供视听反馈。例如，一些方法中断和停止提供用户当前正在收听的音频(例如，讲座)并突然切换到一些其他音频(例如，与短视频消息相关联的音频)。这些突然转变分散了用户的注意力(从而导致用户失去焦点并且必须重新播放他们正在收听的音频的某些部分)，迫使用户执行附加输入来恢复回到他们正在收听的音频，迫使用户禁用某些基于音频的效果，并且产生附加挫折感。通过要求用户执行附加输入和/或迫使用户重新播放某些部分，常规电子设备浪费能量。However, some electronic devices provide audiovisual feedback in a limited, inefficient and frustrating manner. For example, some methods interrupt and stop providing the audio the user is currently listening to (eg, a lecture) and abruptly switch to some other audio (eg, audio associated with a short video message). These sudden transitions distract the user (thus causing the user to lose focus and have to replay some portion of the audio they were listening to), force the user to perform additional input to resume back to the audio they were listening to, force the user to disable some Audio-based effects, and create additional frustration. Conventional electronic devices waste energy by requiring the user to perform additional input and/or forcing the user to replay certain portions.

一些电子设备利用视听接口作为提供关于用户与设备的交互的反馈的方式。然而，一些电子设备以有限的、低效的和令人沮丧的方式来提供视听反馈。例如，一些方法响应于用户与图形用户界面元素的交互来提供预先确定的音频反馈(例如，响应于用户在小键盘上针对电话应用程序键入数字来提供可听音调)。这种预先确定的音频反馈不基于用户交互而改变，从而迫使用户反复听到相同的预先确定的并且不变的音频反馈。因此，许多用户禁用某些基于音频的效果和/或删除已变得太烦人的某些应用程序。Some electronic devices utilize audio-visual interfaces as a way of providing feedback about the user's interaction with the device. However, some electronic devices provide audiovisual feedback in a limited, inefficient and frustrating manner. For example, some approaches provide predetermined audio feedback in response to user interaction with graphical user interface elements (eg, provide audible tones in response to a user typing numbers on a keypad for a phone application). This predetermined audio feedback does not change based on user interaction, thereby forcing the user to repeatedly hear the same predetermined and constant audio feedback. As a result, many users disable certain audio-based effects and/or delete certain applications that have become too annoying.

因此，电子设备需要具有更有效的方法和接口来提供视听反馈。此类方法和接口任选地补充或替代用于提供视听反馈的常规方法。此类方法和接口减少了来自用户的输入的数量、范围和/或性质，并且产生更有效的人机接口(例如，通过一起动态地模糊该音频输出，本文公开的实施方案允许用户有效地预览新的音频输出而不必突然停止收听当前音频输出)。此外，此类方法减少了处理触摸输入所消耗的处理电力，节省功率(从而增大电池充电之间的时间)，减少不必要/无关/重复的输入，并且潜在减少存储器使用。Therefore, there is a need for electronic devices to have more efficient methods and interfaces to provide audiovisual feedback. Such methods and interfaces optionally supplement or replace conventional methods for providing audiovisual feedback. Such methods and interfaces reduce the number, scope, and/or nature of input from the user, and result in a more efficient human-machine interface (e.g., by dynamically blurring the audio output together, embodiments disclosed herein allow the user to effectively preview new audio output without having to suddenly stop listening to the current audio output). Furthermore, such methods reduce the processing power consumed to process touch input, save power (thus increasing time between battery charges), reduce unnecessary/irrelevant/repetitive inputs, and potentially reduce memory usage.

根据一些实施方案，在与显示器和音频系统进行通信的电子设备处执行方法。该方法包括：向音频系统提供第一声音信息以呈现第一音频输出，该第一音频输出包括：音量和除音量之外的音频属性。该方法还包括在音频系统正在呈现第一音频输出时，接收与用于呈现第二音频输出的请求对应的输入。该方法还包括响应于接收到与用于呈现第二音频输出的请求对应的输入：向音频系统提供信息，以根据输入的量值来动态地调整第一音频输出的呈现，其中动态地调整第一音频输出的呈现包括随着输入的量值改变而动态地调整非音量音频属性；以及向音频系统提供第二声音信息，以与第一音频输出同时呈现第二音频输出。According to some implementations, the method is performed at an electronic device in communication with a display and an audio system. The method includes providing first sound information to an audio system to present a first audio output, the first audio output including volume and audio properties other than volume. The method also includes receiving input corresponding to a request to render a second audio output while the audio system is rendering the first audio output. The method also includes, in response to receiving an input corresponding to a request to render a second audio output: providing information to the audio system to dynamically adjust the rendering of the first audio output according to the magnitude of the input, wherein dynamically adjusting the second audio output Presenting an audio output includes dynamically adjusting non-volume audio attributes as the magnitude of the input changes; and providing second sound information to the audio system for presenting the second audio output concurrently with the first audio output.

根据一些实施方案，在与显示器和音频系统进行通信的电子设备处执行方法。该方法包括：当在显示器上显示包括一组一个或多个示能表示的用户界面时：在第一时间点检测指向该组一个或多个示能表示中的第一示能表示的第一输入。该方法还包括响应于检测到指向第一示能表示的第一输入，开始向音频系统提供第一声音信息，以呈现与第一示能表示对应的第一音频输出，其中第一音频输出具有第一音频配置文件。该方法还包括在第一时间点之后的第二时间点检测指向该组一个或多个示能表示中的第二示能表示的第二输入。该方法还包括响应于检测到指向第二示能表示的第二输入：根据确定满足音频更改标准：使得音频系统呈现与第一示能表示对应的经更改的第一音频输出，而不是继续利用第一音频配置文件来呈现第一音频输出，其中经更改的第一音频输出具有不同于第一音频配置文件的经更改的音频配置文件；以及向音频系统提供第二声音信息，以呈现与第二示能表示对应的第二音频输出，其中第二音频输出具有第二音频配置文件。该方法还包括根据确定不满足音频更改标准：使得音频系统继续呈现与第一示能表示对应并且具有第一音频配置文件的第一音频输出；以及向音频系统提供第三声音信息，以呈现与第二示能表示对应的第三音频输出，其中该第三音频输出具有第三音频配置文件。According to some implementations, the method is performed at an electronic device in communication with a display and an audio system. The method includes: when a user interface including a set of one or more affordances is displayed on the display: detecting a first affordance pointing to a first affordance in the set of one or more affordances at a first point in time. enter. The method also includes, in response to detecting a first input directed to a first affordance, beginning to provide first sound information to an audio system to present a first audio output corresponding to the first affordance, wherein the first audio output has First audio profile. The method also includes detecting a second input directed to a second affordance of the set of one or more affordances at a second point in time subsequent to the first point in time. The method also includes, in response to detecting a second input directed to a second affordance: upon determining that the audio modification criteria are met: causing the audio system to present a modified first audio output corresponding to the first affordance, rather than continuing to use the first audio profile to render the first audio output, wherein the altered first audio output has an altered audio profile different from the first audio profile; and providing second sound information to the audio system for rendering consistent with the first audio profile The two affordances represent a corresponding second audio output, wherein the second audio output has a second audio profile. The method also includes, based on determining that the audio modification criteria are not met: causing the audio system to continue presenting the first audio output corresponding to the first affordance and having the first audio profile; and providing third sound information to the audio system for presenting a The second affordance represents a corresponding third audio output, wherein the third audio output has a third audio profile.

根据一些实施方案，在具有显示器、触敏表面和用于检测与触敏表面的接触的强度的一个或多个传感器的电子设备上执行方法。该方法包括：在显示器上显示包括媒体项的表示的用户界面。该方法还包括在显示用户界面时在触敏表面上的与媒体项的表示对应的位置处检测通过接触导致的输入。该方法还包括响应于检测到通过接触导致的输入：根据确定该输入满足媒体提示标准，其中该媒体提示标准包括当该接触具有高于第一强度阈值的特征强度时满足的标准：开始播放媒体项的相应部分；以及当媒体项正在播放时，随着接触的特征强度改变而动态地改变媒体项的一组一个或多个音频属性。该方法还包括根据确定该输入不满足媒体提示标准，放弃开始播放媒体项的相应部分并且放弃随着接触的特征强度改变而动态地改变媒体项的该组一个或多个音频属性。According to some embodiments, the method is performed on an electronic device having a display, a touch-sensitive surface, and one or more sensors for detecting the intensity of contact with the touch-sensitive surface. The method includes displaying on a display a user interface including a representation of a media item. The method also includes detecting an input caused by a contact at a location on the touch-sensitive surface corresponding to the representation of the media item while the user interface is displayed. The method also includes, in response to detecting an input caused by a contact: based on determining that the input satisfies a media prompt criterion, wherein the media prompt criterion includes a criterion satisfied when the contact has a characteristic intensity above a first intensity threshold: starting to play the media corresponding portions of the item; and dynamically changing a set of one or more audio properties of the media item as the characteristic intensity of the contact changes while the media item is being played. The method also includes, based on determining that the input does not satisfy the media cue criteria, abstaining from starting to play the corresponding portion of the media item and abstaining from dynamically changing the set of one or more audio properties of the media item as the characteristic intensity of the contact changes.

根据一些实施方案，在具有显示器、触敏表面和用于检测与触敏表面的接触的强度的一个或多个传感器的电子设备上执行方法。该方法包括：在显示器上显示包括应用程序的第一交互区域和应用程序的第二交互区域的用户界面。该方法还包括在显示用户界面时在显示器上的在与第一交互区域中的第一用户界面元素对应的位置处检测通过触敏表面上的接触导致的第一输入。该方法还包括响应于检测到通过接触导致的第一输入：根据确定第一输入满足基于强度的激活标准，其中该基于强度的激活标准要求接触具有高于第一强度阈值的特征强度以便满足该基于强度的激活标准，除了第一用户界面元素之外，遮蔽应用程序的第一交互区域，而无需遮蔽应用程序的第二交互区域。该方法还包括根据确定第一输入满足第一选择标准，其中该第一选择标准不要求接触具有高于第一强度阈值的特征强度以便满足选择标准，执行与第一用户界面元素对应的第一选择操作，而无需遮蔽应用程序的第一交互区域。According to some embodiments, the method is performed on an electronic device having a display, a touch-sensitive surface, and one or more sensors for detecting the intensity of contact with the touch-sensitive surface. The method includes displaying on a display a user interface including a first interaction area of the application and a second interaction area of the application. The method also includes detecting, while displaying the user interface, a first input caused by contact on the touch-sensitive surface at a location on the display corresponding to the first user interface element in the first interaction area. The method also includes, in response to detecting the first input caused by the contact: determining that the first input satisfies an intensity-based activation criterion, wherein the intensity-based activation criterion requires the contact to have a characteristic intensity above a first intensity threshold in order to satisfy the Based on the intensity activation criteria, except for the first user interface element, a first interaction area of the application is shaded without needing to shade a second interaction area of the application. The method also includes performing a first selection criterion corresponding to the first user interface element upon determining that the first input satisfies a first selection criterion, wherein the first selection criterion does not require the contact to have a characteristic intensity above a first intensity threshold in order to satisfy the selection criterion. Select actions without obscuring the first interaction area of the application.

根据一些实施方案，与显示器和音频系统进行通信的电子设备包括一个或多个处理器、存储器和一个或多个程序；该一个或多个程序被存储在存储器中并被配置为由该一个或多个处理器执行，并且该一个或多个程序包括用于执行或使得执行本文描述的任何方法的操作的指令。根据一些实施方案，计算机可读存储介质在其中存储有指令，该指令当由与显示器和音频系统进行通信的电子设备执行时使得该设备执行或使得执行本文描述的任何方法的操作。根据一些实施方案，包括存储器和用于执行被存储在存储器中的一个或多个程序的一个或多个处理器的电子设备上的图形用户界面包括在本文描述的任何方法中显示的元素中的一个或多个元素，这些元素响应于输入而被更新，如本文描述的任何方法中所述的。根据一些实施方案，与显示器和音频系统进行通信的电子设备包括用于执行或使得执行本文描述的任何方法的操作的装置。根据一些实施方案，用于在与显示器和音频系统进行通信的电子设备中使用的信息处理设备包括用于执行或使得执行本文描述的任何方法的操作的装置。According to some embodiments, an electronic device in communication with a display and an audio system includes one or more processors, memory, and one or more programs; the one or more programs are stored in the memory and configured to be used by the one or more Multiple processors execute, and the one or more programs include instructions for performing, or causing to be performed, the operations of any of the methods described herein. According to some embodiments, a computer-readable storage medium has stored therein instructions that, when executed by an electronic device in communication with a display and an audio system, cause the device to perform or cause the operations of any of the methods described herein to be performed. According to some embodiments, a graphical user interface on an electronic device comprising a memory and one or more processors for executing one or more programs stored in the memory includes among the elements displayed in any of the methods described herein One or more elements that are updated in response to input, as described in any of the methods described herein. According to some embodiments, the electronic device in communication with the display and audio system includes means for performing, or causing to be performed, the operations of any of the methods described herein. According to some embodiments, an information processing device for use in an electronic device in communication with a display and an audio system includes means for performing, or causing to be performed, the operations of any of the methods described herein.

因此，与显示器和音频系统进行通信的电子设备被提供有更快、更有效的方法和接口以用于提供音频反馈并且模糊音频，从而提高此类设备的有效性、效率和用户满意度。此类方法和接口可补充或替代用于提供音频反馈并且模糊音频的常规方法。Accordingly, electronic devices that communicate with displays and audio systems are provided with faster, more efficient methods and interfaces for providing audio feedback and blurring audio, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces may supplement or replace conventional methods for providing audio feedback and blurring audio.

根据一些实施方案，电子设备包括显示器、触敏表面、任选地用于检测与触敏表面的接触的强度的一个或多个传感器、一个或多个处理器、存储器和一个或多个程序；该一个或多个程序被存储在存储器中并被配置为由该一个或多个处理器执行，并且该一个或多个程序包括用于执行或使得执行本文描述的任何方法的操作的指令。根据一些实施方案，计算机可读存储介质在其中存储有指令，该指令当由具有显示器、触敏表面和任选地用于检测与触敏表面的接触的强度的一个或多个传感器的电子设备执行时使得该设备执行或使得执行本文描述的任何方法的操作。根据一些实施方案，具有显示器、触敏表面、任选地用于检测与触敏表面的接触的强度的一个或多个传感器、存储器和用于执行被存储在存储器中的一个或多个程序的一个或多个处理器的电子设备上的图形用户界面包括在本文描述的任何方法中所显示的元素中的一个或多个元素，这些元素响应于输入而被更新，如本文描述的任何方法中所述的。根据一些实施方案，电子设备包括：显示器、触敏表面和任选地用于检测与触敏表面的接触的强度的一个或多个传感器；以及用于执行或使得执行本文描述的任何方法的操作的装置。根据一些实施方案，用于在具有显示器和触敏表面以及任选地用于检测与触敏表面的接触的强度的一个或多个传感器的电子设备中使用的信息处理设备包括用于执行或使得执行本文描述的任何方法的操作的装置。According to some embodiments, an electronic device includes a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, one or more processors, memory, and one or more programs; The one or more programs are stored in memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing or causing the operations of any of the methods described herein to be performed. According to some embodiments, a computer-readable storage medium has stored therein instructions that are activated by an electronic device having a display, a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface. When executed, causes the apparatus to perform or cause the operations of any method described herein to be performed. According to some embodiments, there is a display, a touch-sensitive surface, optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, a memory, and a program for executing one or more programs stored in the memory The graphical user interface on the electronic device of the one or more processors includes one or more of the elements displayed in any of the methods described herein that are updated in response to the input, as in any of the methods described herein as stated. According to some embodiments, an electronic device includes: a display, a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface; and operations for performing or causing to be performed any of the methods described herein s installation. According to some embodiments, an information processing device for use in an electronic device having a display and a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface, includes a device for performing or causing A means for performing the operations of any of the methods described herein.

因此，具有显示器、触敏表面和任选地用于检测与触敏表面的接触的强度的一个或多个传感器的电子设备被提供有更快、更有效的方法和接口以用于提供音频反馈并且模糊音频，从而提高此类设备的有效性、效率和用户满意度。此类方法和接口可补充或替代用于提供音频反馈并且模糊音频的常规方法。Thus, an electronic device having a display, a touch-sensitive surface, and optionally one or more sensors for detecting the intensity of contact with the touch-sensitive surface is provided with faster, more efficient methods and interfaces for providing audio feedback And blur audio, thereby increasing the effectiveness, efficiency and user satisfaction of such devices. Such methods and interfaces may supplement or replace conventional methods for providing audio feedback and blurring audio.

附图说明Description of drawings

为了更好地理解各种所描述的实施方案，应当结合以下附图参考下面的具体实施方式，在附图中类似的附图标号在所有图式中指代对应的部分。For a better understanding of the various described embodiments, reference should be made to the following detailed description when taken in conjunction with the following drawings, in which like reference numerals refer to corresponding parts throughout the several views.

图1A是示出根据一些实施方案的具有触敏显示器的便携式多功能设备的框图。1A is a block diagram illustrating a portable multifunction device with a touch-sensitive display, according to some embodiments.

图1B是示出根据一些实施方案的用于事件处理的示例性部件的框图。Figure IB is a block diagram illustrating exemplary components for event handling, according to some embodiments.

图2示出了根据一些实施方案的具有触摸屏的便携式多功能设备。Figure 2 illustrates a portable multifunction device with a touch screen, according to some embodiments.

图3是根据一些实施方案的具有显示器和触敏表面的示例性多功能设备的框图。3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface, according to some embodiments.

图4A示出了根据一些实施方案的便携式多功能设备上的应用程序菜单的示例性用户界面。4A illustrates an exemplary user interface of an application menu on a portable multifunction device, according to some embodiments.

图4B示出了根据一些实施方案的具有与显示器分开的触敏表面的多功能设备的示例性用户界面。4B illustrates an exemplary user interface of a multifunction device having a touch-sensitive surface separate from a display, according to some embodiments.

图4C至4E示出了根据一些实施方案的动态强度阈值的示例。4C-4E illustrate examples of dynamic intensity thresholds, according to some embodiments.

图5示出了根据一些实施方案的与显示器和触敏表面进行通信的示例性电子设备，其中对于这些电子设备的至少一个子集，显示器和/或触敏表面被集成到电子设备中。5 illustrates an exemplary electronic device in communication with a display and a touch-sensitive surface, where for at least a subset of these electronic devices, the display and/or touch-sensitive surface is integrated into the electronic device, according to some embodiments.

图6A至6Y示出了根据一些实施方案的用于动态地调整音频输出的呈现并且任选地动态地调整用户界面的视觉呈现的示例性用户界面。6A-6Y illustrate exemplary user interfaces for dynamically adjusting the presentation of audio output, and optionally dynamically adjusting the visual presentation of the user interface, according to some embodiments.

图7A至7G示出了根据一些实施方案的用于基于音频配置文件来提供音频输出的示例性用户界面。7A-7G illustrate example user interfaces for providing audio output based on audio profiles, according to some embodiments.

图8A至8B是示出根据一些实施方案的动态地调整音频输出的呈现的方法的流程图。8A-8B are flowcharts illustrating methods of dynamically adjusting the presentation of audio output, according to some embodiments.

图8C是示出根据一些实施方案的动态地调整音频输出的呈现的方法的流程图。8C is a flowchart illustrating a method of dynamically adjusting the presentation of audio output, according to some embodiments.

图8D至8F是示出根据一些实施方案的动态地调整音频输出的呈现的方法的流程图。8D through 8F are flowcharts illustrating methods of dynamically adjusting the presentation of audio output, according to some embodiments.

图8G至8H是示出根据一些实施方案的动态地调整用户界面的视觉呈现的方法的流程图。8G-8H are flowcharts illustrating methods of dynamically adjusting the visual presentation of a user interface, according to some embodiments.

图9A至9C是示出根据一些实施方案的基于音频配置文件来提供音频输出的方法的流程图。9A-9C are flowcharts illustrating methods of providing audio output based on audio profiles, according to some embodiments.

具体实施方式detailed description

所公开的实施方案提供一种用于动态地调整音频输出的呈现的方法和设备。更具体地，在一些实施方案中，本文提供的设备通过根据用户输入的量值(例如，触敏表面上的接触的强度和/或轻扫手势的长度)调整音频属性(例如，音量或非音量属性)来预览音频内容。在一些实施方案中，在设备正提供第一音频输出(例如，播放第一首歌曲)时，设备根据用户输入的量值而使第二音频输出(例如，第二首歌曲)“模糊淡入”。在一些实施方案中，使第二音频模糊淡入包括基于输入的量值来动态地调整第一音频内容的非音量属性(例如，立体声平衡、应用于第一音频内容的低通滤波器的截止频率)。The disclosed embodiments provide a method and apparatus for dynamically adjusting the presentation of audio output. More specifically, in some embodiments, devices provided herein operate by adjusting an audio property (e.g., volume or volume property) to preview the audio content. In some embodiments, while the device is providing a first audio output (e.g., playing a first song), the device "fuzzy fades in" a second audio output (e.g., a second song) based on the magnitude of the user input . In some embodiments, blurring the second audio includes dynamically adjusting non-volume attributes of the first audio content (e.g., stereo balance, cutoff frequency of a low-pass filter applied to the first audio content) based on the magnitude of the input. ).

考虑正在经由用户设备上的音乐应用程序播放歌曲的示例。同时，用户可打开正在显示视频消息的消息应用程序(想象视频尚未播放并且用户必须按压视频的图像以对其进行播放)。在一些实施方案中，用户可通过对视频消息进行的按压并保持手势来播放视频消息的“提示”。提示使歌曲模糊淡出(例如，通过随着按压并保持手势的强度增大而减小音量并减小低通滤波器的截止频率)并且使视频消息模糊淡入(通过随着按压并保持手势的强度增大而提高音量)。在一些实施方案中，高于特定接触强度，设备以全音量播放视频消息，并且在滤出歌曲的高频分量时以低音量播放歌曲，使得歌曲仅在背景中以轻柔的方式听起来像“轰轰声”。Consider the example of a song being played via a music application on a user device. At the same time, the user may open a messaging application that is displaying a video message (imagine the video has not played and the user has to press the image of the video to play it). In some implementations, the user can play a "reminder" of the video message through a press and hold gesture on the video message. The prompt fades the song out (e.g., by reducing the volume and reducing the cutoff frequency of the low-pass filter as the intensity of the press-and-hold gesture increases) and blurs the video message in (by increasing the intensity of the press-and-hold gesture) to increase the volume). In some embodiments, above a certain contact intensity, the device plays the video message at full volume, and plays the song at a low volume while filtering out the high frequency components of the song so that the song only sounds in the background in a soft manner like " Boom."

根据一些实施方案，还提供用于更改音频反馈的方法和设备。例如，当激活相应示能表示使得设备产生音频反馈时，快速连续地激活该相应示能表示两次使得设备更改音频反馈。According to some embodiments, methods and apparatus for altering audio feedback are also provided. For example, when activating the corresponding affordance causes the device to produce audio feedback, activating the corresponding affordance twice in quick succession causes the device to alter the audio feedback.

下面，图1A至1B、2和3提供对示例性设备的描述。In the following, Figures 1A-1B, 2 and 3 provide a description of exemplary devices.

图4A至4B和6A至6O示出了用于根据用户输入的量值来使音频输出“模糊淡入”(例如，当背景音频已经在播放时)的示例性用户界面。图8A至8B和图8C示出了根据用户输入的量值来使音频输出“模糊淡入”(例如，当背景音频已正在播放时)的两种方法。图6A至6O中的用户界面用于示出图8A至8C中的过程。4A-4B and 6A-60 illustrate example user interfaces for "blurring" audio output (eg, when background audio is already playing) according to the magnitude of user input. 8A-8B and 8C illustrate two methods of "blurring" audio output (eg, when background audio is already playing) according to the magnitude of user input. The user interfaces in Figures 6A to 6O are used to illustrate the process in Figures 8A to 8C.

图4A至4B和6P至6Y示出了用于根据用户输入的量值来动态地调整音频输出的属性(例如，在音频输出处提示)的示例性用户界面。图8D至8F示出了根据用户输入的量值来动态地调整音频输出的属性(例如，在音频输出处提示)的方法。图6P至6Y中的用户界面用于示出图8D至8F中的过程。4A-4B and 6P-6Y illustrate example user interfaces for dynamically adjusting properties of audio output (eg, prompting at audio output) according to the magnitude of user input. 8D through 8F illustrate methods of dynamically adjusting properties of audio output (eg, prompting at audio output) according to the magnitude of user input. The user interfaces in Figures 6P to 6Y are used to illustrate the process in Figures 8D to 8F.

此外，图4A至4B和6P至6Y示出了用于提供视觉反馈(例如，视觉模糊)的示例性用户界面。图9A至9C示出了根据一些实施方案的提供视觉反馈(例如，视觉模糊)的方法的流程图。图6P至6Y中的用户界面用于示出图9A至9C中的过程。Additionally, FIGS. 4A-4B and 6P-6Y illustrate example user interfaces for providing visual feedback (eg, blurred vision). 9A-9C illustrate a flowchart of a method of providing visual feedback (eg, blurred vision), according to some embodiments. The user interfaces in Figures 6P to 6Y are used to illustrate the process in Figures 9A to 9C.

示例性设备exemplary device

现在将详细参考实施方案，其示例在附图中示出。下面的详细描述中示出许多具体细节，以便提供对各种所描述的实施方案的彻底理解。但是，对本领域的普通技术人员将显而易见的是，各种所描述的实施方案可在没有这些具体细节的情况下被实施。在其他情况下，未详细描述熟知的方法、程序、部件、电路和网络，以免不必要地模糊该实施方案的各个方面。Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are shown in order to provide a thorough understanding of various described embodiments. It will be apparent, however, to one of ordinary skill in the art that various described embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

还将理解的是，虽然在一些情况下，术语“第一”、“第二”等在本文中用于描述各种元素，但是这些元素不应受到这些术语限制。这些术语只是用于将一个元素与另一元素区分开。例如，第一接触可被命名为第二接触，并且类似地，第二接触可被命名为第一接触，而不脱离各种所描述的实施方案的范围。第一接触和第二接触均为接触，但它们不是同一接触，除非上下文另外明确指示。It will also be understood that, although in some instances the terms "first", "second", etc. are used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact could be termed a second contact, and, similarly, a second contact could be termed a first contact, without departing from the scope of the various described embodiments. A first contact and a second contact are both contacts, but they are not the same contact unless the context clearly dictates otherwise.

在本文中对各种所述实施方案的描述中所使用的术语只是为了描述特定实施方案的目的，而并非旨在进行限制。如在对各种所述实施方案的描述和所附权利要求书中所使用的那样，单数形式“一个(“a”,“an”)和“该”旨在也包括复数形式，除非上下文另外明确地指示。还将理解的是，本文中所使用的术语“和/或”是指并且涵盖相关联地列出的项目中的一个或多个项目的任何和全部可能的组合。还将理解的是，术语“包括”、“包含”、“含有”和/或“具有”当在本说明书中使用时是指定存在所陈述的特征、整数、步骤、操作、元件和/或部件，但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元件、部件和/或其群组。The terminology used in describing the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of various described embodiments and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context dictates otherwise It will be clearly indicated. It will also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will also be understood Note that the terms "comprising", "comprising", "containing" and/or "having" when used in this specification designate the presence of stated features, integers, steps, operations, elements and/or parts, but do not It does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and/or groups thereof.

如本文中所使用，根据上下文，术语“如果”任选地被解释为意思是“当……时”(“when”或“upon”)或“响应于确定”或“响应于检测到”。类似地，根据上下文，短语“如果确定……”或“如果检测到[所陈述的条件或事件]”任选地被解释为是指“在确定……时”或“响应于确定……”或“在检测到[所陈述的条件或事件]时”或“响应于检测到[所陈述的条件或事件]”。As used herein, the term "if" is optionally interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that ..." or "if [the stated condition or event] is detected" are optionally construed to mean "when determining ..." or "in response to determining ..." depending on the context Or "upon detection of [stated condition or event]" or "in response to detection of [stated condition or event]".

本文描述了电子设备、此类设备的用户界面和使用此类设备的相关联的过程的实施方案。在一些实施方案中，该设备是还包含其他功能诸如PDA和/或音乐播放器功能的便携式通信设备，诸如移动电话。便携式多功能设备的示例性实施方案包括但不限于来自Apple Inc.(Cupertino,California)的设备、iPod设备和设备。任选地使用其他便携式电子设备，诸如具有触敏表面(例如，触摸屏显示器和/或触摸板)的膝上型电脑或平板电脑。还应当理解的是，在一些实施方案中，该设备并非便携式通信设备，而是具有触敏表面(例如，触摸屏显示器和/或触摸板)的台式计算机。Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described herein. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functionality, such as PDA and/or music player functionality. Exemplary embodiments of portable multifunction devices include, but are not limited to, the Apple Inc. (Cupertino, California) device, iPod equipment and equipment. Other portable electronic devices are optionally used, such as laptops or tablets with touch-sensitive surfaces (eg, touch screen displays and/or touchpads). It should also be appreciated that in some embodiments, the device is not a portable communication device, but rather a desktop computer with a touch-sensitive surface (eg, a touch-screen display and/or a touchpad).

在下面的讨论中，描述了一种包括显示器和触敏表面的电子设备。然而，应当理解，电子设备任选地包括一个或多个其他物理用户接口设备，诸如物理键盘、鼠标和/或操纵杆。In the following discussion, an electronic device is described that includes a display and a touch-sensitive surface. It should be understood, however, that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and/or joystick.

该设备通常支持多种应用程序，诸如以下应用程序中的一个或多个应用程序：笔记应用程序、绘图应用程序、呈现应用程序、文字处理应用程序、网站创建应用程序、盘编辑应用程序、电子表格应用程序、游戏应用程序、电话应用程序、视频会议应用程序、电子邮件应用程序、即时消息应用程序、健身支持应用程序、照片管理应用程序、数码相机应用程序、数字摄像机应用程序、网页浏览应用程序、数字音乐播放器应用程序和/或数字视频播放器应用程序。The device typically supports multiple applications, such as one or more of the following: note-taking applications, drawing applications, rendering applications, word processing applications, website creation applications, disk editing applications, electronic Table apps, gaming apps, phone apps, video conferencing apps, email apps, instant messaging apps, fitness support apps, photo management apps, digital camera apps, digital video camera apps, web browsing apps programs, digital music player applications, and/or digital video player applications.

在设备上执行的各种应用程序任选地使用至少一个通用物理用户接口设备，诸如触敏表面。触敏表面的一种或多种功能以及被显示在设备上的对应信息任选地随不同的应用程序而调整和/或变化，和/或在相应的应用程序内调整和/或变化。这样，设备的共用物理架构(诸如触敏表面)任选地利用对于用户而言直观且清楚的用户界面来支持各种应用程序。Various applications executing on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and/or varied from application to application and/or within a corresponding application. In this way, the common physical architecture of the devices, such as the touch-sensitive surface, optionally supports various applications with a user interface that is intuitive and clear to the user.

现在关注具有触敏显示器的便携式设备的实施方案。图1A是示出根据一些实施方案的具有触敏显示器系统112的便携式多功能设备100的框图。触敏显示器系统112有时为了方便被叫做“触摸屏”，并且有时被简称为触敏显示器。设备100包括存储器102(其任选地包括一个或多个计算机可读存储介质)、存储器控制器122、一个或多个处理单元(CPU)120、外围设备接口118、RF电路108、音频电路110、扬声器111、麦克风113、输入/输出(I/O)子系统106、其他输入或控制设备116、和外部端口124。设备100任选地包括一个或多个光学传感器164。设备100任选地包括用于检测设备100上的接触的强度的一个或多个强度传感器165(例如，触敏表面，诸如设备100的触敏显示器系统112)。设备100任选地包括用于在设备100上生成触觉输出(例如，在触敏表面诸如设备100的触敏显示器系统112或设备300的触摸板355上生成触觉输出)的一个或多个触觉输出发生器167。这些部件任选地经由一个或多个通信总线或信号线103进行通信。Attention is now directed to an embodiment of a portable device with a touch-sensitive display. 1A is a block diagram illustrating a portable multifunction device 100 with a touch-sensitive display system 112 according to some embodiments. Touch-sensitive display system 112 is sometimes referred to as a "touch screen" for convenience, and is sometimes referred to simply as a touch-sensitive display. Device 100 includes memory 102 (which optionally includes one or more computer-readable storage media), memory controller 122, one or more processing units (CPUs) 120, peripherals interface 118, RF circuitry 108, audio circuitry 110 , speaker 111 , microphone 113 , input/output (I/O) subsystem 106 , other input or control devices 116 , and external port 124 . Device 100 optionally includes one or more optical sensors 164 . Device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contacts on device 100 (eg, a touch-sensitive surface such as touch-sensitive display system 112 of device 100 ). Device 100 optionally includes one or more tactile outputs for generating tactile outputs on device 100 (e.g., on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or touchpad 355 of device 300). Generator 167. These components optionally communicate via one or more communication buses or signal lines 103 .

如本说明书和权利要求书中所使用的，术语“触觉输出”是指将由用户利用用户的触感检测到的设备相对于设备的先前位置的物理位移、设备的部件(例如，触敏表面)相对于设备的另一个部件(例如，外壳)的物理位移、或部件相对于设备的质心的位移。例如，在设备或设备的部件与用户的对触摸敏感的表面(例如，手指、手掌或用户手部的其他部分)接触的情况下，通过物理位移生成的触觉输出将由用户解读为触感，该触感对应于设备或设备的部件的物理特征的感知变化。例如，触敏表面(例如，触敏显示器或触控板)的移动任选地由用户解读为对物理致动器按钮的“按下点击”或“松开点击”。在一些情况下，用户将感觉到触感，诸如“按下点击”或“松开点击”，即使在通过用户的移动而物理地被按压(例如，被移位)的与触敏表面相关联的物理致动按钮没有移动时。在另一个示例中，触敏表面的移动任选地由用户解读或感测为触敏表面的“粗糙度”，即使在触敏表面的光滑度无变化时。虽然用户对触摸的此类解读将受到用户的个体化感官知觉的限制，但是存在触摸的许多感官知觉是大多数用户共有的。因此，当触觉输出被描述为对应于用户的特定感官知觉(例如，“按下点击”、“松开点击”、“粗糙度”)时，除非另外陈述，否则所生成的触觉输出对应于设备或其部件的物理位移，该物理位移将会生成典型(或普通)用户的感官知觉。As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a component of the device (e.g., a touch-sensitive surface) relative to Physical displacement of another part of the device (eg, housing), or displacement of a part relative to the center of mass of the device. For example, where a device or component of a device is in contact with a user's touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation that Corresponds to a perceived change in a physical characteristic of a device or part of a device. For example, movement of a touch-sensitive surface (eg, a touch-sensitive display or trackpad) is optionally interpreted by the user as a "click-down" or "click-release" of a physical actuator button. In some cases, the user will feel a tactile sensation, such as "press click" or "release click", even when the touch-sensitive surface associated with the touch-sensitive surface is physically pressed (e.g., displaced) by the user's movement. When the physical actuation button is not moving. In another example, movement of the touch-sensitive surface is optionally interpreted or sensed by the user as "roughness" of the touch-sensitive surface even when there is no change in the smoothness of the touch-sensitive surface. While such interpretations of touch by a user will be limited by the user's individualized sensory perception, there are many sensory perceptions of touch that are common to most users. Thus, when a haptic output is described as corresponding to a particular sensory perception of a user (e.g., "press click", "release click", "roughness"), unless otherwise stated, the generated haptic output corresponds to a device or the physical displacement of its parts that will generate the sensory perception of a typical (or average) user.

应当理解，设备100仅仅是便携式多功能设备的一个示例，并且设备100任选地具有比所示出的部件更多或更少的部件，任选地组合两个或更多个部件，或者任选地具有这些部件的不同配置或布置。图1A中所示的各种部件在硬件、软件、固件或它们的任何组合(包括一个或多个信号处理电路和/或专用集成电路)中实施。It should be understood that device 100 is merely one example of a portable multifunction device, and that device 100 optionally has more or fewer components than shown, optionally combining two or more components, or any Optionally have a different configuration or arrangement of these components. The various components shown in FIG. 1A are implemented in hardware, software, firmware, or any combination thereof, including one or more signal processing circuits and/or application specific integrated circuits.

存储器102任选地包括高速随机存取存储器，并且还任选地包括非易失性存储器，诸如一个或多个磁盘存储设备、闪存存储器设备、或其他非易失性固态存储器设备。设备100的其他部件诸如一个或多个CPU 120和外围设备接口118对存储器102的访问任选地由存储器控制器122来控制。Memory 102 optionally includes high-speed random access memory, and also optionally includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by other components of device 100 such as one or more CPUs 120 and peripherals interface 118 is optionally controlled by memory controller 122 .

外围设备接口118可用于将设备的输入外围设备和输出外围设备耦接到一个或多个CPU 120和存储器102。该一个或多个处理器120运行或实行被存储存在储器102中的各种软件程序和/或指令集，以执行设备100的各种功能并处理数据。Peripherals interface 118 may be used to couple the device's input and output peripherals to one or more CPUs 120 and memory 102 . The one or more processors 120 run or execute various software programs and/or sets of instructions stored in the memory 102 to perform various functions of the device 100 and process data.

在一些实施方案中，外围设备接口118、一个或多个CPU 120和存储器控制器122任选地在单个芯片诸如芯片104上实现。在一些其他实施方案中，它们任选地在独立芯片上实现。In some embodiments, peripherals interface 118 , one or more CPUs 120 , and memory controller 122 are optionally implemented on a single chip, such as chip 104 . In some other embodiments, they are optionally implemented on separate chips.

RF(射频)电路108接收和发送也被叫做电磁信号的RF信号。RF电路108将电信号转换为电磁信号/将电磁信号转换为电信号，并且经由电磁信号来与通信网络和其他通信设备进行通信。RF电路108任选地包括用于执行这些功能的熟知的电路，包括但不限于天线系统、RF收发器、一个或多个放大器、调谐器、一个或多个振荡器、数字信号处理器、编解码芯片组、用户身份模块(SIM)卡、存储器等等。RF电路108任选地通过无线通信来与网络和其他设备进行通信，这些网络诸如为互联网，也被称为万维网(WWW)，内联网和/或无线网络诸如蜂窝电话网络、无线局域网(LAN)和/或城域网(MAN)。无线通信任选地使用多种通信标准、协议和技术中的任一者，包括但不限于全球移动通信系统(GSM)、增强数据GSM环境(EDGE)、高速下行链路分组接入(HSDPA)、高速上行链路分组接入(HSUPA)、演进、纯数据(EV-DO)、HSPA、HSPA+、双小区HSPA(DC-HSPDA)、长期演进(LTE)、近场通信(NFC)、宽带码分多址(W-CDMA)、码分多址(CDMA)、时分多址(TDMA)、蓝牙、无线保真(Wi-Fi)(例如，IEEE802.11a、IEEE 802.11ac、IEEE 802.11ax、IEEE 802.11b、IEEE 802.11g和/或IEEE 802.11n)、互联网协议语音技术(VoIP)、Wi-MAX、电子邮件的协议(例如，互联网消息访问协议(IMAP)和/或邮局协议(POP))、即时消息(例如，可扩展消息传递和存在协议(XMPP)、用于即时消息和存在利用扩展的会话发起协议(SIMPLE)、即时消息和存在服务(IMPS))和/或短消息服务(SMS)，或者包括截止本文档的提交日期还未开发出的通信协议的任何其他适当的通信协议。RF (Radio Frequency) circuitry 108 receives and transmits RF signals, also called electromagnetic signals. The RF circuitry 108 converts/converts electrical signals to/from electromagnetic signals and communicates with communication networks and other communication devices via the electromagnetic signals. RF circuitry 108 optionally includes well-known circuitry for performing these functions, including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a Decodes chipsets, Subscriber Identity Module (SIM) cards, memory, and more. RF circuitry 108 communicates with networks and other devices, optionally by wireless communication, such as the Internet, also known as the World Wide Web (WWW), intranets, and/or wireless networks such as cellular telephone networks, wireless local area networks (LANs) and/or Metropolitan Area Network (MAN). Wireless communications optionally employ any of a variety of communications standards, protocols, and technologies, including but not limited to Global System for Mobile communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed Downlink Packet Access (HSDPA) , High Speed Uplink Packet Access (HSUPA), Evolution, Pure Data (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (for example, IEEE802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Message Access Protocol (IMAP) and/or Post Office Protocol (POP)), Instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Utilization Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)) and/or Short Message Service (SMS) , or any other appropriate communication protocol including a communication protocol that has not been developed as of the filing date of this document.

音频电路110、扬声器111和麦克风113提供用户与设备100之间的音频接口。音频电路110从外围设备接口118接收音频数据，将音频数据转换为电信号，并将电信号传输到扬声器111。扬声器111将电信号转换为人类可听的声波。音频电路110还接收由麦克风113从声波转换来的电信号。音频电路110将电信号转换为音频数据，并且将音频数据传输到外围设备接口118以用于处理。音频数据任选地由外围设备接口118检索自和/或传输至存储器102和/或RF电路108。在一些实施方案中，音频电路110还包括耳麦插孔(例如，图2中的212)。耳麦插孔提供音频电路110与可移除的音频输入/输出外围设备之间的接口，该外围设备诸如仅输出的耳机或者具有输出(例如，单耳耳机或双耳耳机)和输入(例如，麦克风)两者的耳麦。Audio circuitry 110 , speaker 111 and microphone 113 provide an audio interface between a user and device 100 . The audio circuit 110 receives audio data from the peripheral device interface 118 , converts the audio data into electrical signals, and transmits the electrical signals to the speaker 111 . The speaker 111 converts electrical signals into human-audible sound waves. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113 . Audio circuitry 110 converts the electrical signal to audio data and transmits the audio data to peripherals interface 118 for processing. Audio data is optionally retrieved from and/or transmitted to memory 102 and/or RF circuitry 108 by peripherals interface 118 . In some embodiments, audio circuitry 110 also includes a headset jack (eg, 212 in FIG. 2 ). The headset jack provides an interface between the audio circuitry 110 and a removable audio input/output peripheral such as an output-only headphone or an output (e.g., monaural or dual earphones) and input (e.g., microphone) for both headsets.

I/O子系统106将设备100上的输入/输出外围设备诸如触敏显示器系统112和其他输入或控制设备116与外围设备接口118耦接。I/O子系统106任选地包括显示控制器156、光学传感器控制器158、强度传感器控制器159、触觉反馈控制器161、和用于其他输入或控制设备的一个或多个输入控制器160。该一个或多个输入控制器160从其他输入或控制设备116接收电信号/将电信号发送到其他输入或控制设备116。其他输入或控制设备116任选地包括物理按钮(例如，下压按钮、摇臂按钮等)、拨号盘、滑块开关、操纵杆、点击轮等等。在一些另选实施方案中，一个或多个输入控制器160任选地耦接至以下各项中的任一者(或不耦接至以下各项中的任一者)：键盘、红外线端口、USB端口、触笔、和/或指针设备诸如鼠标。一个或多个按钮(例如，图2中的208)任选地包括用于扬声器111和/或麦克风113的音量控制的增大/减小按钮。一个或多个按钮任选地包括下压按钮(例如，图2中的206)。I/O subsystem 106 couples input/output peripherals on device 100 , such as touch-sensitive display system 112 and other input or control devices 116 , with peripherals interface 118 . I/O subsystem 106 optionally includes display controller 156, optical sensor controller 158, intensity sensor controller 159, tactile feedback controller 161, and one or more input controllers 160 for other input or control devices . The one or more input controllers 160 receive/send electrical signals from/to other input or control devices 116 . Other input or control devices 116 optionally include physical buttons (eg, push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, and the like. In some alternative embodiments, one or more input controllers 160 are optionally coupled to any (or none) of the following: keyboard, infrared port , a USB port, a stylus, and/or a pointing device such as a mouse. The one or more buttons (eg, 208 in FIG. 2 ) optionally include up/down buttons for volume control of speaker 111 and/or microphone 113 . The one or more buttons optionally include a push button (eg, 206 in FIG. 2 ).

触敏显示器系统112提供设备与用户之间的输入接口和输出接口。显示控制器156从触敏显示器系统112接收电信号和/或将电信号发送至触敏显示器系统112。触敏显示器系统112向用户显示视觉输出。视觉输出任选地包括图形、文本、图标、视频和它们的任何组合(统称为“图形”)。在一些实施方案中，一些视觉输出或全部的视觉输出对应于用户界面对象。如本文所使用，术语“示能表示”是指用户交互式图形用户界面对象(例如，被配置为对指向图形用户界面对象的输入作出响应的图形用户界面对象)。用户交互式图形用户界面对象的示例包括但不限于按钮、滑块、图标、可选菜单项、开关、超链接、或其他用户界面控件。Touch-sensitive display system 112 provides an input interface and an output interface between the device and a user. Display controller 156 receives electrical signals from and/or sends electrical signals to touch-sensitive display system 112 . Touch-sensitive display system 112 displays visual output to the user. Visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively "graphics"). In some implementations, some or all of the visual output corresponds to user interface objects. As used herein, the term "affordance" refers to a user-interactive graphical user interface object (eg, a graphical user interface object configured to respond to input directed to the graphical user interface object). Examples of user-interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls.

触敏显示器系统112具有基于触感和/或触觉接触来接受来自用户的输入的触敏表面、传感器或传感器组。触敏显示器系统112和显示控制器156(与存储器102中的任何相关联的模块和/或指令集一起)检测触敏显示器系统112上的接触(和该接触的任何移动或中断)，并且将所检测到的接触转换为与被显示在触敏显示器系统112上的用户界面对象(例如，一个或多个软按键、图标、网页或图像)的交互。在示例性实施方案中，触敏显示器系统112与用户之间的接触点对应于用户的手指或触笔。Touch-sensitive display system 112 has a touch-sensitive surface, sensor or set of sensors that accepts input from a user based on tactile sensation and/or tactile contact. Touch-sensitive display system 112 and display controller 156 (together with any associated modules and/or instruction sets in memory 102) detect contact (and any movement or interruption of that contact) on touch-sensitive display system 112, and will The detected contact translates into an interaction with a user interface object (eg, one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 112 . In an exemplary embodiment, the point of contact between touch-sensitive display system 112 and the user corresponds to a finger or stylus of the user.

触敏显示器系统112任选地使用LCD(液晶显示器)技术、LPD(发光聚合物显示器)技术、或LED(发光二极管)技术，但是在其他实施方案中使用其他显示技术。触敏显示器系统112和显示控制器156任选地使用现在已知的或以后开发的多种触摸感测技术中的任何技术以及其他接近传感器阵列或用于确定与触敏显示器系统112的一个或多个接触点的其他元件来检测接触及其任何移动或中断，该多种触摸感测技术包括但不限于电容性、电阻性、红外线和表面声波技术。在一个示例性实施方案中，使用投射式互电容感测技术，诸如从Apple Inc.(Cupertino,California)的iPod和中发现的技术。The touch sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, although in other embodiments other display technologies are used. Touch-sensitive display system 112 and display controller 156 optionally use any of a variety of touch-sensing technologies now known or later developed, as well as other proximity sensor arrays or one or more A variety of touch sensing technologies including, but not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies are used to detect contact and any movement or interruption thereof with other elements of a plurality of touch points. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as from Apple Inc. (Cupertino, California) iPod with technology found in .

触敏显示器系统112任选地具有超过100dpi的视频分辨率。在一些实施方案中，触摸屏视频分辨率超过400dpi(例如，500dpi、800dpi或更大)。用户任选地使用任何合适的物体或附加物诸如触笔、手指等等与触敏显示器系统112进行接触。在一些实施方案中，该用户界面被设计为与基于手指的接触和手势一起工作，由于手指在触摸屏上的接触区域较大，因此这可能不如基于触笔的输入那样精确。在一些实施方案中，设备将基于手指的粗略输入翻译为精确的指针/光标位置或命令，以用于执行用户所期望的动作。Touch-sensitive display system 112 optionally has a video resolution in excess of 100 dpi. In some embodiments, the touchscreen video resolution exceeds 400dpi (eg, 500dpi, 800dpi, or greater). A user optionally makes contact with touch-sensitive display system 112 using any suitable object or appendage, such as a stylus, finger, or the like. In some embodiments, the user interface is designed to work with finger-based contacts and gestures, which may not be as precise as stylus-based input due to the larger contact area of a finger on the touch screen. In some embodiments, the device translates rough finger-based input into precise pointer/cursor positions or commands for performing the user's desired action.

在一些实施方案中，除了触摸屏之外，设备100任选地包括用于激活或去激活特定功能的触摸板(未示出)。在一些实施方案中，触摸板是设备的触敏区域，该触敏区域与触摸屏不同，其不显示视觉输出。触摸板任选地是与触敏显示器系统112分开的触敏表面，或者是由触摸屏形成的触敏表面的延伸部分。In some embodiments, in addition to the touch screen, device 100 optionally includes a touchpad (not shown) for activating or deactivating certain functions. In some embodiments, a touchpad is a touch-sensitive area of a device that, unlike a touchscreen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from the touch-sensitive display system 112, or an extension of the touch-sensitive surface formed by the touchscreen.

设备100还包括用于为各种部件供电的电力系统162。电力系统162任选地包括电力管理系统、一个或多个电源(例如，电池、交流电(AC))、再充电系统、电力故障检测电路、功率转换器或逆变器、电力状态指示器(例如，发光二极管(LED))和与便携式设备中的电力生成、管理和分配相关联的任何其他部件。Device 100 also includes a power system 162 for powering various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, power failure detection circuitry, power converters or inverters, power status indicators (e.g., , Light Emitting Diodes (LEDs)) and any other components associated with power generation, management and distribution in portable devices.

设备100任选地还包括一个或多个光学传感器164。图1A示出与I/O子系统106中的光学传感器控制器158耦接的光学传感器。一个或多个光学传感器164任选地包括电荷耦合器件(CCD)或互补金属氧化物半导体(CMOS)光电晶体管。一个或多个光学传感器164从环境接收通过一个或多个透镜投射的光，并且将光转换为表示图像的数据。结合成像模块143(也叫做相机模块)，一个或多个光学传感器164任选地捕获静态图像和/或视频。在一些实施方案中，光学传感器位于设备100的与设备的前部上的触敏显示器系统112相背对的后部上，使得触摸屏能够用作用于静态图像和/或视频图像采集的取景器。在一些实施方案中，另一光学传感器位于设备的前部上，从而获取该用户的图像(例如用于自拍、用于在用户在触摸屏上观看其他视频会议参与者时进行视频会议等等)。Device 100 optionally also includes one or more optical sensors 164 . FIG. 1A shows an optical sensor coupled to optical sensor controller 158 in I/O subsystem 106 . One or more optical sensors 164 optionally include charge-coupled devices (CCD) or complementary metal-oxide-semiconductor (CMOS) phototransistors. One or more optical sensors 164 receive light from the environment projected through the one or more lenses and convert the light into data representing an image. In conjunction with imaging module 143 (also referred to as a camera module), one or more optical sensors 164 optionally capture still images and/or video. In some embodiments, an optical sensor is located on the rear of device 100 opposite touch-sensitive display system 112 on the front of the device, enabling the touchscreen to be used as a viewfinder for still image and/or video image capture. In some embodiments, another optical sensor is located on the front of the device to capture an image of the user (eg, for selfies, for videoconferencing while the user is viewing other videoconferencing participants on a touchscreen, etc.).

设备100任选地还包括一个或多个接触强度传感器165。图1A示出与I/O子系统106中的强度传感器控制器159耦接的接触强度传感器。一个或多个接触强度传感器165任选地包括一个或多个压阻应变计、电容式力传感器、电力传感器、压电力传感器、光学力传感器、电容式触敏表面或其他强度传感器(例如，用于测量触敏表面上的接触的力(或压力)的传感器)。一个或多个接触强度传感器165从环境接收接触强度信息(例如，压力信息或压力信息的代用物)。在一些实施方案中，至少一个接触强度传感器与触敏表面(例如，触敏显示器系统112)并置排列或邻近。在一些实施方案中，至少一个接触强度传感器位于设备100的与位于设备100的前部上的触摸屏显示器系统112相背对的后部上。Device 100 optionally also includes one or more contact intensity sensors 165 . FIG. 1A shows a contact intensity sensor coupled to intensity sensor controller 159 in I/O subsystem 106 . The one or more contact intensity sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., with A sensor for measuring the force (or pressure) of a contact on a touch-sensitive surface). One or more contact intensity sensors 165 receive contact intensity information (eg, pressure information or a surrogate for pressure information) from the environment. In some implementations, at least one contact intensity sensor is collocated or adjacent to a touch-sensitive surface (eg, touch-sensitive display system 112 ). In some embodiments, at least one contact intensity sensor is located on the rear of the device 100 opposite the touch screen display system 112 located on the front of the device 100 .

设备100任选地还包括一个或多个接近传感器166。图1A示出与外围设备接口118耦接的接近传感器166。作为另外一种选择，接近传感器166与I/O子系统106中的输入控制器160耦接。在一些实施方案中，当多功能设备被放置在用户耳朵附近时(例如，用户正在打电话时)，接近传感器关闭并禁用触敏显示器系统112。Device 100 optionally also includes one or more proximity sensors 166 . FIG. 1A shows proximity sensor 166 coupled to peripherals interface 118 . Alternatively, proximity sensor 166 is coupled to input controller 160 in I/O subsystem 106 . In some implementations, the proximity sensor turns off and disables the touch-sensitive display system 112 when the multifunction device is placed near the user's ear (eg, while the user is on the phone).

设备100任选地还包括一个或多个触觉输出发生器167。图1A示出与I/O子系统106中的触觉反馈控制器161耦接的触觉输出发生器。一个或多个触觉输出发生器167任选地包括一个或多个电声设备，诸如扬声器或其他音频部件，和/或将能量转换成线性运动的机电设备，诸如马达、螺线管、电活性聚合器、压电致动器、静电致动器或其他触觉输出生成部件(例如，将电信号转换成设备上的触觉输出的部件)。该一个或多个触觉输出发生器167从触觉反馈模块133接收触觉反馈生成指令，并且在设备100上生成能够由设备100的用户感觉到的触觉输出。在一些实施方案中，至少一个触觉输出发生器与触敏表面(例如，触敏显示器系统112)并置排列或邻近，并且任选地通过竖直地(例如，向设备100的表面内/外)或侧向地(例如，在与设备100的表面相同的平面中向后和向前)移动触敏表面来生成触觉输出。在一些实施方案中，至少一个触觉输出发生器传感器位于设备100的与位于设备100的前部上的触敏显示器系统112相背对的后部上。Device 100 optionally also includes one or more haptic output generators 167 . FIG. 1A shows a haptic output generator coupled with haptic feedback controller 161 in I/O subsystem 106 . One or more tactile output generators 167 optionally include one or more electro-acoustic devices, such as speakers or other audio components, and/or electro-mechanical devices that convert energy into linear motion, such as motors, solenoids, electro-active Aggregators, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (eg, components that convert electrical signals into haptic outputs on the device). The one or more haptic output generators 167 receive haptic feedback generation instructions from the haptic feedback module 133 and generate haptic outputs on the device 100 that can be felt by a user of the device 100 . In some embodiments, at least one tactile output generator is juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112 ), and is optionally positioned vertically (e.g., in/outward of the surface of device 100 ) or laterally (eg, backward and forward in the same plane as the surface of device 100) to generate tactile output. In some implementations, at least one tactile output generator sensor is located on the rear of device 100 opposite touch-sensitive display system 112 located on the front of device 100 .

设备100任选地还包括一个或多个加速度计168。图1A示出与外围设备接口118耦接的加速度计168。作为另外一种选择，加速度计168任选地与I/O子系统106中的输入控制器160耦接。在一些实施方案中，基于对从一个或多个加速度计接收的数据的分析在触摸屏显示器上以纵向视图或横向视图来显示信息。设备100任选地除了一个或多个加速度计168之外还包括磁力仪(未示出)和GPS(或GLONASS或其他全球导航系统)接收器(未示出)，以用于获取关于设备100的位置和取向(例如，纵向或横向)的信息。Device 100 optionally also includes one or more accelerometers 168 . FIG. 1A shows accelerometer 168 coupled to peripherals interface 118 . Alternatively, accelerometer 168 is optionally coupled to input controller 160 in I/O subsystem 106 . In some embodiments, information is displayed on the touch screen display in either portrait view or landscape view based on analysis of data received from one or more accelerometers. Device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) in addition to one or more accelerometers 168 for obtaining information about device 100 position and orientation (eg, portrait or landscape) information.

在一些实施方案中，被存储在存储器102中的软件部件包括操作系统126、通信模块(或指令集)128、接触/运动模块(或指令集)130、图形模块(或指令集)132、触觉反馈模块(或指令集)133、文本输入模块(或指令集)134、全球定位系统(GPS)模块(或指令集)135、应用程序(或指令集)136、以及特定于音频的模块(包括音频预览模块163-1、音频修改模块163-2和音频更改模块163-3)。此外，在一些实施方案中，存储器102存储设备/全局内部状态157，如图1A和图3中所示。设备/全局内部状态157包括以下各项中的一者或多者：活动应用程序状态，该活动应用程序状态用于指示哪些应用程序(如果有的话)当前是活动的；显示状态，该显示状态用于指示什么应用程序、视图或其他信息占据触敏显示器系统112的各个区域；传感器状态，该传感器状态包括从设备的各个传感器和其他输入或控制设备116获取的信息；和关于设备位置和/或姿态的位置和/或定位信息。In some embodiments, the software components stored in memory 102 include operating system 126, communication module (or instruction set) 128, contact/motion module (or instruction set) 130, graphics module (or instruction set) 132, haptic Feedback module (or instruction set) 133, text input module (or instruction set) 134, Global Positioning System (GPS) module (or instruction set) 135, application program (or instruction set) 136, and audio-specific modules (including audio preview module 163-1, audio modification module 163-2 and audio modification module 163-3). Additionally, in some embodiments, memory 102 stores device/global internal state 157 , as shown in FIGS. 1A and 3 . Device/global internal state 157 includes one or more of the following: active application state, which is used to indicate which applications (if any) are currently active; display state, which displays States are used to indicate what applications, views, or other information occupy various areas of the touch-sensitive display system 112; sensor states, which include information obtained from the device's various sensors and other input or control devices 116; and information about the device's location and and/or position and/or positioning information of gestures.

操作系统126(例如，iOS、Darwin、RTXC、LINUX、UNIX、OSX、WINDOWS、或嵌入式操作系统诸如VxWorks)包括用于控制和管理一般系统任务(例如，存储器管理、存储设备控制、电力管理等)的各种软件部件和/或驱动器，并且有利于各种硬件和软件部件之间的通信。Operating system 126 (e.g., iOS, Darwin, RTXC, LINUX, UNIX, OSX, WINDOWS, or an embedded operating system such as VxWorks) includes tools for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) ) of various software components and/or drivers, and facilitates communication between various hardware and software components.

通信模块128促进通过一个或多个外部端口124来与其他设备进行通信，并且还包括用于处理由RF电路108和/或外部端口124所接收的数据的各种软件部件。外部端口124(例如，通用串行总线(USB)、火线等)适于直接耦接到其他设备或间接地经由网络(例如，互联网、无线LAN等)耦接。在一些实施方案中，外部端口是与Apple Inc.(Cupertino,California)的一些设备、iPod设备、和iPod设备中所使用的30针连接器相同或类似和/或兼容的多针(例如，30针)连接器。在一些实施方案中，外部端口是与AppleInc.(Cupertino,California)的一些设备、iPod设备、和iPod设备中所使用的Lightning连接器相同或类似和/或兼容的Lightning连接器。Communications module 128 facilitates communicating with other devices through one or more external ports 124 and also includes various software components for processing data received by RF circuitry 108 and/or external ports 124 . External port 124 (eg, Universal Serial Bus (USB), FireWire, etc.) is suitable for coupling to other devices directly or indirectly via a network (eg, Internet, wireless LAN, etc.). In some embodiments, the external port is a product of Apple Inc. (Cupertino, California) device, iPod device, a multi-pin (eg, 30-pin) connector identical or similar and/or compatible to the 30-pin connector used in the iPod device. In some embodiments, the external ports are some of Apple Inc. (Cupertino, California) device, iPod device, and a Lightning connector identical or similar and/or compatible with the Lightning connector used in the iPod device.

接触/运动模块130任选地检测与触敏显示器系统112(结合显示控制器156)和其他触敏设备(例如，触摸板或物理点击轮)的接触。接触/运动模块130包括各种软件部件以用于执行与接触(例如，通过手指或通过触笔)的检测相关的各种操作，诸如确定是否已发生接触(例如，检测手指按下事件)、确定接触的强度(例如，接触的力或压力，或者接触的力或压力的替代物)、确定是否存在接触的移动并跟踪跨触敏表面的移动(例如，检测一个或多个手指拖动事件)、以及确定接触是否已停止(例如，检测手指抬起事件或者接触断开)。接触/运动模块130从触敏表面接收接触数据。确定接触点的移动任选地包括确定接触点的速率(量值)、速度(量值和方向)和/或加速度(量值和/或方向的变化)，该接触点的移动由一系列接触数据来表示。这些操作任选地被应用于单点接触(例如，单指接触或触笔接触)或者多点同时接触(例如，“多点触摸”/多指接触)。在一些实施方案中，接触/运动模块130和显示控制器156检测触摸板上的接触。Contact/motion module 130 optionally detects contact with touch-sensitive display system 112 (in conjunction with display controller 156 ) and other touch-sensitive devices (eg, a touchpad or physical click wheel). The contact/motion module 130 includes various software components for performing various operations related to the detection of contact (e.g., by a finger or by a stylus), such as determining whether a contact has occurred (e.g., detecting a finger press event), Determine the strength of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determine whether there is movement of the contact, and track movement across the touch-sensitive surface (e.g., detect one or more finger drag events ), and determining whether contact has ceased (eg, detecting a finger lift event or contact breaking). The contact/motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the velocity (magnitude), velocity (magnitude and direction) and/or acceleration (change in magnitude and/or direction) of the contact point, the movement of the contact point being determined by a series of contacts data to represent. These operations are optionally applied to single-point contacts (eg, single-finger or stylus contacts) or multiple simultaneous contacts (eg, "multi-touch"/multi-finger contacts). In some implementations, contact/motion module 130 and display controller 156 detect contact on a touchpad.

接触/运动模块130任选地检测用户的手势输入。触敏表面上的不同手势具有不同的接触图案(例如，所检测到的接触的不同运动、定时和/或强度)。因此，任选地通过检测特定接触图案来检测手势。例如，检测单指轻击手势包括检测手指按下事件，然后在与手指按下事件相同的位置(或基本上相同的位置)处(例如，在图标位置处)检测手指抬起(抬离)事件。在另一个示例中，在触敏表面上检测到手指轻扫手势包括检测到手指按下事件，然后检测到一个或多个手指拖动事件，并且随后检测到手指抬起(抬离)事件。类似地，任选地通过检测触笔的特定接触图案针对触笔来检测轻击、轻扫、拖动和其他手势。The contact/motion module 130 optionally detects gesture input by the user. Different gestures on the touch-sensitive surface have different contact patterns (eg, different motion, timing, and/or intensity of detected contacts). Thus, gestures are optionally detected by detecting specific contact patterns. For example, detecting a single-finger tap gesture includes detecting a finger-down event and then detecting a finger-up (lift-off) at the same location (or substantially the same location) as the finger-down event (e.g., at the icon location) event. In another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger down event, then detecting one or more finger drag events, and then detecting a finger up (lift off) event. Similarly, taps, swipes, drags, and other gestures are detected for the stylus, optionally by detecting specific contact patterns of the stylus.

在一些实施方案中，检测手指轻击手势依赖于检测到手指按下事件与手指抬起事件之间的时间长度，但是独立于检测到手指按下事件与手指抬起事件之间的手指接触的强度。在一些实施方案中，根据确定手指按下事件与手指抬起事件之间的时间长度小于预先确定的值(例如，小于0.1、0.2、0.3、0.4或0.5秒)来检测轻击手势，而不管在轻击期间手指接触的强度是否满足给定强度阈值(大于标称接触检测强度阈值)，例如轻压强度阈值或深压强度阈值。因此，手指轻击手势可满足不要求接触的特征强度满足给定强度阈值以便满足特定输入标准的特定输入标准。为了清楚起见，轻击手势中的手指接触通常需要满足标称接触检测强度阈值以便检测到手指按下事件，低于该标称接触检测强度阈值不能检测到接触。类似分析应用于检测通过触笔或其他接触而进行的轻击手势。在设备能够检测到悬停在触敏表面上方的手指或触笔接触的情况下，标称接触检测强度阈值任选地不对应于手指或触笔与触敏表面之间的物理接触。In some embodiments, detecting a finger tap gesture is dependent on the length of time between the detection of a finger down event and a finger up event, but is independent of the amount of time between the detection of a finger down event and a finger up event. strength. In some embodiments, a tap gesture is detected based on determining that the length of time between a finger-down event and a finger-up event is less than a predetermined value (eg, less than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of Whether the intensity of the finger contact during a tap meets a given intensity threshold (greater than the nominal contact detection intensity threshold), such as a light-press intensity threshold or a deep-press intensity threshold. Thus, a finger tap gesture may satisfy certain input criteria that do not require that the characteristic intensity of the contact meet a given intensity threshold in order to satisfy the particular input criteria. For clarity, a finger contact in a tap gesture generally needs to meet a nominal contact detection intensity threshold below which no contact can be detected in order to detect a finger down event. Similar analysis is applied to detect tap gestures via a stylus or other contact. Where the device is capable of detecting contact by a finger or stylus hovering over the touch-sensitive surface, the nominal contact detection intensity threshold optionally does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.

相同概念以类似方式应用于其他类型的手势。例如，任选地基于满足与被包括在手势中的接触的强度无关或者不要求执行手势的一个或多个接触达到强度阈值以便被识别的标准来检测轻扫手势、捏合手势、取消捏合手势和/或长按手势。例如，基于一个或多个接触的移动量来检测轻扫手势；基于两个或更多个接触朝向彼此的移动来检测捏合手势；基于两个或更多个接触远离彼此的移动来检测取消捏合手势；并且基于触敏表面上的具有小于阈值移动量的接触的持续时间来检测长按手势。因此，特定手势识别标准不要求一个或多个接触的强度满足相应强度阈值以便满足特定手势识别标准的陈述意味着特定手势识别标准能够在手势中的一个或多个接触没有达到相应强度阈值的情况下得到满足，并且还能够在手势中的接触中的一个或多个接触达到或超过相应强度阈值的情况下得到满足。在一些实施方案中，基于确定在预定义的时间段内检测到手指按下和手指抬起事件来检测轻击手势，而不考虑在预定义的时间段期间接触是高于还是低于相应强度阈值，并且基于确定接触移动大于预定义的量值来检测轻扫手势，即使在接触移动结束时接触也高于相应强度阈值。即使在对手势的检测受到执行手势的接触的强度的影响(例如，设备在接触的强度高于强度阈值时较快地检测到长按，或者在接触的强度较高时延迟对轻击输入的检测)的具体实施中，对那些手势的检测不要求接触达到特定强度阈值，只要能够在接触没有达到特定强度阈值的情况下满足用于识别手势的标准(例如，即使识别手势所花的时间量改变)。The same concepts apply to other types of gestures in a similar manner. For example, swipe gestures, pinch gestures, un-pinch gestures, and /or long press gesture. For example, detect a swipe gesture based on the amount of movement of one or more contacts; detect a pinch gesture based on the movement of two or more contacts toward each other; detect a cancel pinch based on the movement of two or more contacts away from each other gesture; and detecting a long press gesture based on a duration of the contact on the touch-sensitive surface having less than a threshold amount of movement. Thus, a statement that a particular gesture recognition standard does not require the intensity of one or more contacts to meet a corresponding intensity threshold in order to satisfy a particular gesture recognition standard means that a particular gesture recognition standard can be used in situations where one or more contacts in a gesture do not meet a corresponding intensity threshold and can also be satisfied if one or more of the contacts in the gesture meets or exceeds a corresponding intensity threshold. In some embodiments, a tap gesture is detected based on determining that finger-down and finger-up events are detected within a predefined time period, regardless of whether the contact was higher or lower than the corresponding intensity during the predefined time period. threshold, and the swipe gesture is detected based on determining that the contact movement is greater than a predefined magnitude, even if the contact is above a corresponding intensity threshold at the end of the contact movement. Even when the detection of a gesture is affected by the strength of the contact performing the gesture (for example, the device detects a long press faster when the strength of the contact is above a strength threshold, or delays the response to a tap input when the strength of the contact is higher) detection), the detection of those gestures does not require the contact to reach a particular intensity threshold, as long as the criteria for recognizing the gesture can be met without the contact reaching the particular intensity threshold (e.g., even if the amount of time it takes to recognize the gesture Change).

在某些情况下，接触强度阈值、持续时间阈值和移动阈值被组合成多种不同的组合，以便创建用于区分指向相同输入元素或区域的两个或更多个不同手势的启发法，使得启用与相同输入元素的多个不同交互以提供一组更丰富的用户交互和响应。一组特定手势识别标准不要求一个或多个接触的强度满足相应强度阈值以便满足特定手势识别标准的陈述并不排除其他强度依赖手势识别标准的同时评估以识别具有在手势包括具有高于相应强度阈值的强度的接触时满足的标准的其他手势。例如，在某些情况下，用于第一手势的第一手势识别标准(其不要求一个或多个接触的强度满足相应强度阈值以便满足第一手势识别标准)与用于第二手势的第二手势识别标准(其依赖于一个或多个接触达到相应强度阈值)竞争。在此类竞争中，如果首先满足用于第二手势的第二手势识别标准，则手势任选地不被识别为满足用于第一手势的第一手势识别标准。例如，如果接触在接触移动预定义的移动量之前达到相应强度阈值，则检测到深压手势而不是轻扫手势。相反，如果接触在接触达到相应强度阈值之前移动预定义的移动量，则检测到轻扫手势而不是深压手势。即使在此类情况下，用于第一手势的第一手势识别标准仍然不要求一个或多个接触的强度满足相应强度阈值以便满足第一手势识别标准，因为如果接触保持低于相应强度阈值直到手势结束(例如，具有不增大到高于相应强度阈值的强度的接触的轻扫手势)，则手势将已被第一手势识别标准识别为轻扫手势。因此，不要求一个或多个接触的强度满足相应强度阈值以便满足特定手势识别标准的特定手势识别标准将(A)在某些情况下忽略接触相对于强度阈值的强度(例如，对于轻击手势)和/或(B)在某种情况下仍然依赖于接触相对于强度阈值的强度，在某种意义上如果在特定手势识别标准识别到与输入对应的手势(例如，对于正与深压手势进行竞争以进行识别的长按手势)之前一组竞争性强度相依手势识别标准(例如，对于深压手势)将输入识别为对应于强度相依手势，则该特定手势识别标准(例如，对于长按手势)将失败。In some cases, contact intensity thresholds, duration thresholds, and movement thresholds are combined into various combinations in order to create heuristics for distinguishing between two or more different gestures directed at the same input element or area such that Enables multiple different interactions with the same input element to provide a richer set of user interactions and responses. A statement that a particular set of gesture recognition criteria does not require that the intensity of one or more contacts meet a corresponding intensity threshold in order to satisfy a particular gesture recognition criterion does not preclude the simultaneous evaluation of other intensity-dependent gesture recognition criteria to identify people with gestures that include The threshold strength of the contact when other gestures meet the criteria. For example, in some cases, a first gesture recognition criterion for a first gesture (which does not require the intensity of one or more contacts to meet a corresponding intensity threshold in order to satisfy the first gesture recognition criterion) is different from that for a second gesture. A second gesture recognition criterion (which relies on one or more contacts reaching respective intensity thresholds) competes. In such competitions, a gesture is optionally not recognized as satisfying the first gesture recognition criteria for the first gesture if the second gesture recognition criteria for the second gesture is first met. For example, a deep press gesture is detected instead of a swipe gesture if the contact reaches the corresponding intensity threshold before the contact moves a predefined amount of movement. Conversely, if the contact moves a predefined amount of movement before the contact reaches the corresponding intensity threshold, a swipe gesture is detected instead of a deep press gesture. Even in such cases, the first gesture recognition criterion for the first gesture still does not require the intensity of one or more contacts to meet the corresponding intensity threshold in order to satisfy the first gesture recognition criterion, because if the contact remains below the corresponding intensity threshold until Once the gesture ends (eg, a swipe gesture with a contact that does not increase in intensity above the corresponding intensity threshold), the gesture will have been recognized as a swipe gesture by the first gesture recognition criterion. Accordingly, certain gesture recognition criteria that do not require the intensity of one or more contacts to meet a corresponding intensity threshold in order to satisfy certain gesture recognition criteria will (A) ignore the intensity of the contact relative to the intensity threshold in some cases (e.g., for a tap gesture ) and/or (B) in some cases still depend on the intensity of the contact relative to the intensity threshold, in the sense that if a gesture corresponding to the input is recognized under a particular gesture recognition criterion (e.g. for positive and deep pressure gestures Long press gestures competing for recognition) before a set of competing intensity-dependent gesture recognition criteria (e.g., for a deep press gesture) recognizes the input as corresponding to an intensity-dependent gesture, then that particular gesture recognition criterion (e.g., for a long press gesture) will fail.

图形模块132包括用于在触敏显示器系统112或其他显示器上呈现和显示图形的各种已知的软件部件，包括用于改变所显示的图形的视觉冲击(例如，亮度、透明度、饱和度、对比度或其他视觉属性)的部件。如本文所用，术语“图形”包括可被显示给用户的任何对象，包括但不限于文本、网页、图标(诸如包括软键的用户界面对象)、数字图像、视频、动画等。Graphics module 132 includes various known software components for rendering and displaying graphics on touch-sensitive display system 112 or other displays, including for changing the visual impact (e.g., brightness, transparency, saturation, contrast or other visual properties). As used herein, the term "graphics" includes any object that may be displayed to a user, including but not limited to text, web pages, icons (such as user interface objects including soft keys), digital images, video, animations, and the like.

在一些实施方案中，图形模块132存储待使用的用于表示图形的数据。每个图形任选地被分配有对应的代码。图形模块132从应用程序等接收指定待显示的图形的一个或多个代码，在必要的情况下还接收坐标数据和其他图形属性数据，并且然后生成屏幕图像数据以输出到显示控制器156。In some implementations, graphics module 132 stores data for representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes specifying graphics to be displayed, coordinate data and other graphics attribute data if necessary, from an application program or the like, and then generates screen image data to output to the display controller 156 .

触觉反馈模块133包括用于生成指令的各种软件部件，该指令由一个或多个触觉输出发生器167使用，以便响应于用户与设备100的交互而在设备100上的一个或多个位置处产生触觉输出。Haptic feedback module 133 includes various software components for generating instructions that are used by one or more haptic output generators 167 to respond to user interactions with device 100 at one or more locations on device 100. Produces tactile output.

任选地为图形模块132的部件的文本输入模块134提供用于在各种应用程序(例如，联系人137、电子邮件140、IM 141、浏览器147和需要文本输入的任何其他应用程序)中输入文本的软键。Text input module 134, optionally part of graphics module 132, provides for use in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application that requires text input) Soft key for entering text.

GPS模块135确定设备的位置，并且提供该信息以在各种应用程序中使用(例如，提供给电话138以用于基于位置的拨号、提供给相机143作为照片/视频元数据、以及提供给提供基于位置的服务的应用程序，诸如天气桌面小程序、本地黄页桌面小程序、和地图/导航桌面小程序)。The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to the phone 138 for location-based dialing, to the camera 143 as photo/video metadata, and to the location-based services, such as the Weather widget, the Local Yellow Pages widget, and the Maps/Navigation widget).

应用136任选地包括以下模块(或指令集)或者其子集或超集：Applications 136 optionally include the following modules (or sets of instructions) or subsets or supersets thereof:

·联系人模块137(有时称为通讯录或联系人列表)；· Contacts module 137 (sometimes referred to as an address book or contact list);

·电话模块138；· Telephone module 138;

·视频会议模块139；· Video conferencing module 139;

·电子邮件客户端模块140；• Email client module 140;

·即时消息(IM)模块141；• Instant Messaging (IM) module 141;

·新闻模块142；· News module 142;

·用于静止图像和/或视频图像的相机模块143；· Camera module 143 for still images and/or video images;

·图像管理模块144；· image management module 144;

·浏览器模块147；· Browser module 147;

·日历模块148；· Calendar module 148;

·桌面小程序模块149，其任选地包括以下各项中的一者或多者：天气桌面小程序149-1、股市桌面小程序149-2、计算器桌面小程序149-3、闹钟桌面小程序149-4、词典桌面小程序149-5和由用户获取的其他桌面小程序，以及用户创建的桌面小程序149-6；Widget module 149, which optionally includes one or more of the following: Weather widget 149-1, Stock Market widget 149-2, Calculator widget 149-3, Alarm Clock widget Applet 149-4, Dictionary applet 149-5, and other applets acquired by users, as well as user-created applets 149-6;

·用于制作用户创建的桌面小程序149-6的桌面小程序创建器模块150；an applet creator module 150 for making user-created applets 149-6;

·搜索模块151；· Search module 151;

·视频和音乐播放器模块152，其任选地由视频播放器模块和音乐播放器模块构成；· Video and music player module 152, which optionally consists of a video player module and a music player module;

·记事本模块153；Notepad module 153;

·地图模块154；和/或• Map module 154; and/or

·在线视频模块155。· Online video module 155 .

任选地被存储在存储器102中的其他应用程序136的示例包括其他文字处理应用程序、其他图像编辑应用程序、绘图应用程序、呈现应用程序、支持JAVA的应用程序、加密、数字版权管理、语音识别和语音复制。Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption, digital rights management, speech Recognition and speech reproduction.

结合触敏显示器系统112、显示控制器156、接触模块130、图形模块132以及文本输入模块134，联系人模块137包括用于管理通讯录或联系人列表(例如，被存储在存储器102或存储器370中的联系人模块137的应用程序内部状态192中)的可执行指令，包括：将一个或多个姓名添加到通讯录；从通讯录中删除一个或多个姓名；使一个或多个电话号码、一个或多个电子邮件地址、一个或多个物理地址或其他信息与姓名相关联；使图像与姓名相关联；对姓名进行分类和排序；提供电话号码和/或电子邮件地址以发起和/或促进通过电话138、视频会议139、电子邮件140或IM141进行的通信；等等。In conjunction with touch-sensitive display system 112, display controller 156, touch module 130, graphics module 132, and text input module 134, contacts module 137 includes functions for managing an address book or contact list (e.g., stored in memory 102 or memory 370). Executable instructions in the application internal state 192 of the contacts module 137 in ), including: adding one or more names to the address book; deleting one or more names from the address book; making one or more phone numbers , one or more email addresses, one or more physical addresses, or other information associated with a name; associate an image with a name; categorize and sort names; provide a phone number and/or email address to initiate and/or or facilitate communications via telephone 138, video conferencing 139, email 140, or IM 141;

结合RF电路108、音频电路110、扬声器111、麦克风113、触敏显示器系统112、显示控制器156、接触模块130、图形模块132、和文本输入模块134，电话模块138包括用于输入与电话号码对应的字符序列、访问通讯录137中的一个或多个电话号码、修改已输入的电话号码、拨打相应的电话号码、进行会话以及当会话完成时断开或挂断的可执行指令。如上所述，无线通信任选地使用多种通信标准、协议和技术中的任一者。In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, touch module 130, graphics module 132, and text input module 134, phone module 138 includes a Executable instructions for corresponding character sequences, accessing one or more phone numbers in the address book 137, modifying the input phone number, dialing the corresponding phone number, conducting a conversation, and disconnecting or hanging up when the conversation is completed. As noted above, wireless communications optionally employ any of a variety of communications standards, protocols, and techniques.

结合RF电路108、音频电路110、扬声器111、麦克风113、触敏显示器系统112、显示控制器156、一个或多个光学传感器164、光学传感器控制器158、接触模块130、图形模块132、文本输入模块134、联系人列表137和电话模块138，视频会议模块139包括用于根据用户指令发起、进行和终止用户与一个或多个其他参与方之间的视频会议的可执行指令。In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, text input Module 134, contact list 137, and telephony module 138, videoconferencing module 139 includes executable instructions for initiating, conducting, and terminating a videoconference between the user and one or more other participants in accordance with user instructions.

结合RF电路108、触敏显示器系统112、显示控制器156、接触模块130、图形模块132和文本输入模块134，电子邮件客户端模块140包括用于响应于用户指令来创建、发送、接收和管理电子邮件的可执行指令。结合图像管理模块144，电子邮件客户端模块140使得非常容易创建和发送具有由相机模块143拍摄的静态图像或视频图像的电子邮件。In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes functions for creating, sending, receiving, and managing Executable instructions for email. In conjunction with the image management module 144 , the email client module 140 makes it very easy to create and send emails with still images or video images captured by the camera module 143 .

结合RF电路108、触敏显示器系统112、显示控制器156、接触模块130、图形模块132和文本输入模块134，即时消息模块141包括用于输入与即时消息对应的字符序列、修改先前输入的字符、发送相应即时消息(例如，使用针对基于电话的即时消息的短消息服务(SMS)或多媒体消息服务(MMS)协议或者使用针对基于互联网的即时消息的XMPP、SIMPLE、Apple推送通知服务(APNs)或IMPS)、接收即时消息以及查看所接收的即时消息的可执行指令。在一些实施方案中，所传输和/或接收的即时消息任选地包括图形、照片、音频文件、视频文件和/或在MMS和/或增强消息服务(EMS)中所支持的其他附件。如本文所用，“即时消息”是指基于电话的消息(例如，使用SMS或MMS发送的消息)和基于互联网的消息(例如，使用XMPP、SIMPLE、APNs或IMPS发送的消息)两者。In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, instant message module 141 includes functions for entering character sequences corresponding to instant messages, modifying previously entered characters , send a corresponding instant message (for example, using the Short Message Service (SMS) or Multimedia Message Service (MMS) protocol for phone-based instant messaging or using XMPP, SIMPLE, Apple Push Notification Service (APNs) for Internet-based instant messaging or IMPS), receive instant messages, and view executable instructions for received instant messages. In some embodiments, transmitted and/or received instant messages optionally include graphics, photos, audio files, video files, and/or other attachments supported in MMS and/or Enhanced Message Service (EMS). As used herein, "instant messaging" refers to both phone-based messages (eg, messages sent using SMS or MMS) and Internet-based messages (eg, messages sent using XMPP, SIMPLE, APNs, or IMPS).

结合触敏显示器系统112、显示控制器156、接触/运动模块130和图形模块132，新闻模块142包括用于显示特定于用户的新闻文章(例如，基于特定于用户的偏好从多种出版物来源收集的文章)并且允许用户与特定于用户的新闻文章进行交互(或与特定于用户的新闻文章内所包含的部分内容进行交互)的可执行指令。In conjunction with touch-sensitive display system 112, display controller 156, contact/motion module 130, and graphics module 132, news module 142 includes functions for displaying user-specific news articles (e.g., from various publication sources based on user-specific preferences). Articles collected) and executable instructions that allow a user to interact with a user-specific news article (or interact with portions of content contained within a user-specific news article).

结合触敏显示器系统112、显示控制器156、一个或多个光学传感器164、光学传感器控制器158、接触模块130、图形模块132和图像管理模块144，相机模块143包括用于捕获静态图像或视频(包括视频流)并且将它们存储到存储器102中、修改静态图像或视频的特征、和/或从存储器102删除静态图像或视频的可执行指令。In conjunction with touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes a (including video streams) and store them into memory 102 , modify the characteristics of still images or videos, and/or delete still images or videos from memory 102 .

结合触敏显示器系统112、显示控制器156、接触模块130、图形模块132、文本输入模块134、和相机模块143，图像管理模块144包括用于排列、修改(例如，编辑)、或以其他方式操控、加标签、删除、呈现(例如，在数字幻灯片或相册中)、以及存储静态图像和/或视频图像的可执行指令。In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes functions for arranging, modifying (e.g., editing), or otherwise Executable instructions to manipulate, tag, delete, present (eg, in a digital slideshow or photo album), and store still images and/or video images.

结合RF电路108、触敏显示器系统112、显示系统控制器156、接触模块130、图形模块132和文本输入模块134，浏览器模块147包括根据用户指令来浏览互联网(包括搜索、链接到、接收和显示网页或其部分以及链接到网页的附件和其他文件)的可执行指令。In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, browser module 147 includes functions for browsing the Internet according to user commands (including searching, linking to, receiving and executable instructions that display web pages or portions thereof and attachments and other files linked to web pages).

结合RF电路108、触敏显示器系统112、显示系统控制器156、接触模块130、图形模块132、文本输入模块134、电子邮件客户端模块140和浏览器模块147，日历模块148包括用于根据用户指令来创建、显示、修改和存储日历以及与日历相关联的数据(例如，日历条目、待办事项等)的可执行指令。In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, touch module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions to create, display, modify, and store calendars and data associated with calendars (eg, calendar entries, to-dos, etc.).

结合RF电路108、触敏显示器系统112、显示系统控制器156、接触模块130、图形模块132、文本输入模块134和浏览器模块147，桌面小程序模块149是任选地由用户下载和使用的微型应用程序(例如，天气桌面小程序149-1、股市桌面小程序149-2、计算器桌面小程序149-3、闹钟桌面小程序149-4和词典桌面小程序149-5)或由用户创建的微型应用程序(例如，用户创建的桌面小程序149-6)。在一些实施方案中，桌面小程序包括HTML(超文本标记语言)文件、CSS(层叠样式表)文件和JavaScript文件。在一些实施方案中，桌面小程序包括XML(可扩展标记语言)文件和JavaScript文件(例如，Yahoo！桌面小程序)。In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, touch module 130, graphics module 132, text input module 134, and browser module 147, widget module 149 is optionally downloaded and used by the user Mini-applications (e.g., Weather widget 149-1, Stock Market widget 149-2, Calculator widget 149-3, Alarm Clock widget 149-4, and Dictionary widget 149-5) or by the user Created mini-applications (eg, user-created applet 149-6). In some embodiments, the applet includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, an applet includes an XML (Extensible Markup Language) file and a JavaScript file (eg, Yahoo! Applet).

结合RF电路108、触敏显示器系统112、显示系统控制器156、接触模块130、图形模块132、文本输入模块134、和浏览器模块147，桌面小程序创建器模块150包括用于创建桌面小程序(例如，将网页的用户指定部分转到桌面小程序中)的可执行指令。In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, widget creator module 150 includes widgets for creating widgets. (for example, transfer a user-specified portion of a web page to a desktop applet).

结合触敏显示器系统112、显示系统控制器156、接触模块130、图形模块132和文本输入模块134，搜索模块151包括用于根据用户指令来搜索存储器102中的与一个或多个搜索条件(例如，一个或多个用户指定的搜索词)匹配的文本、音乐、声音、图像、视频和/或其他文件的可执行指令。In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, search module 151 includes functions for searching memory 102 with one or more search criteria (e.g., , one or more user-specified search terms) matches executable instructions for text, music, sound, images, video, and/or other files.

结合触敏显示器系统112、显示系统控制器156、接触模块130、图形模块132、音频电路110、扬声器111、RF电路108和浏览器模块147，视频和音乐播放器模块152包括允许用户下载并重放以一种或多种文件格式诸如MP3或AAC文件存储的记录音乐和其他声音文件的可执行指令，以及用于显示、呈现或以其他方式重放视频(例如，在触敏显示器系统112上或在无线地或经由外部端口124连接的外部显示器上)的可执行指令。在一些实施方案中，设备100任选地包括MP3播放器诸如iPod(苹果公司的商标)的功能。In conjunction with touch-sensitive display system 112, display system controller 156, touch module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes a Executable instructions for recording music and other sound files stored in one or more file formats, such as MP3 or AAC files, and for displaying, presenting, or otherwise replaying video (e.g., on touch-sensitive display system 112 or executable instructions on an external display connected wirelessly or via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (trademark of Apple Inc.).

结合触敏显示器系统112、显示控制器156、接触模块130、图形模块132和文本输入模块134，记事本模块153包括用于根据用户指令来创建和管理记事本、待办事项等的可执行指令。In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, notepad module 153 includes executable instructions for creating and managing notepads, to-do lists, etc. in accordance with user instructions .

结合RF电路108、触敏显示器系统112、显示系统控制器156、接触模块130、图形模块132、文本输入模块134、GPS模块135和浏览器模块147，地图模块154包括用于根据用户指令来接收、显示、修改和存储地图以及与地图相关联的数据(例如，驾驶方向；关于特定位置处或附近的商店和其他兴趣点的数据；以及其他基于位置的数据)的可执行指令。In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 includes a , executable instructions that display, modify, and store maps and data associated with maps (eg, driving directions; data about stores and other points of interest at or near particular locations; and other location-based data).

结合触敏显示器系统112、显示系统控制器156、接触模块130、图形模块132、音频电路110、扬声器111、RF电路108、文本输入模块134、电子邮件客户端模块140和浏览器模块147，在线视频模块155包括可执行指令，这些可执行指令允许用户访问、浏览、接收(例如，通过流媒体和/或下载)、重放(例如，在触摸屏112上或在无线地或经由外部端口124连接的外部显示器上)、发送具有特定在线视频的链接的电子邮件，以及以其他方式管理呈一种或多种文件格式诸如H.264的在线视频。在一些实施方案中，使用即时消息模块141而不是电子邮件客户端模块140来发送至特定在线视频的链接。In conjunction with touch-sensitive display system 112, display system controller 156, touch module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, online Video module 155 includes executable instructions that allow a user to access, browse, receive (e.g., via streaming and/or download), replay (e.g., external display), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, the instant messaging module 141 is used instead of the email client module 140 to send a link to a particular online video.

如图1A所描绘，便携式多功能设备100还包括用于向音频系统提供声音信息的音频输出提供模块163，使得音频系统可呈现音频输出(例如，如图5所示，音频系统可被包括在便携式多功能设备100中或与便携式多功能设备100分开)。音频输出提供模块163任选地包括以下模块(或指令集)或者其子集或超集：As depicted in FIG. 1A , portable multifunction device 100 also includes an audio output providing module 163 for providing sound information to an audio system so that the audio system can present audio output (e.g., as shown in FIG. 5 , the audio system can be included in portable multifunction device 100 or separate from portable multifunction device 100). The audio output providing module 163 optionally includes the following modules (or instruction sets) or a subset or a superset thereof:

用于存储关于对应于或响应于例如与便携式多功能设备100的用户交互而产生的音频特性的信息的音频配置文件402(例如，音频包络特征、音高特性、左右(L-R)平衡特性、混响曲线、频率滤波特性)；Audio profile 402 for storing information about audio characteristics (e.g., audio envelope characteristics, pitch characteristics, left-right (L-R) balance characteristics, reverberation curve, frequency filter characteristics);

音频预览模块163-1，其包括用于提供信息以播放音频内容(例如，特定歌曲)的预览并且任选地响应于对呈现第一音频输出的请求而调整不同内容(例如，不同歌曲)的呈现(例如，使得这两首歌曲模糊在一起)的可执行指令；an audio preview module 163-1, which includes components for providing information to play a preview of audio content (e.g., a particular song) and optionally adjusting different content (e.g., a different song) in response to a request to present the first audio output Executable instructions to render (e.g., blur these two songs together);

音频修改模块163-2，其包括用于响应于检测到指向用户界面中的示能表示(例如，相同示能表示或不同示能表示)的第二用户输入而更改第一音频输出(例如，由指向用户界面中的示能表示的第一用户输入产生的音频输出输入)的可执行指令。在一些实施方案中，音频修改模块163-1修改相应的音频配置文件402；以及an audio modification module 163-2 comprising means for modifying the first audio output (e.g., Executable instructions for an audio output resulting from a first user input directed to an affordance in the user interface (in). In some embodiments, the audio modification module 163-1 modifies the corresponding audio configuration file 402; and

音频更改标准模块163-3，其包括用于基于音频更改标准来确定音频修改模块163-2是否应该修改第一音频输出的可执行指令。The audio modification criteria module 163-3, which includes executable instructions for determining whether the audio modification module 163-2 should modify the first audio output based on the audio modification criteria.

上述所识别的每个模块和应用程序对应于用于执行上述一种或多种功能以及在本申请中所描述的方法(例如，本文中所描述的计算机实现的方法和其他信息处理方法)的一组可执行指令。这些模块(即，指令集)不必被实现为分开的软件程序、过程或模块，并且因此这些模块的各种子组任选地在各种实施方案中被组合或以其他方式重新布置。在一些实施方案中，存储器102任选地存储上述模块和数据结构的子集。此外，存储器102任选地存储上面未描述的附加模块和数据结构。Each of the modules and applications identified above corresponds to a program for performing one or more functions described above and the methods described in this application (for example, the computer-implemented methods and other information processing methods described herein) A set of executable instructions. These modules (ie, sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the modules and data structures described above. Furthermore, memory 102 optionally stores additional modules and data structures not described above.

在一些实施方案中，设备100是这样一种设备，即该设备上的一组预定义的功能的操作唯一地通过触摸屏和/或触摸板来执行。通过使用触摸屏和/或触摸板作为用于设备100的操作的主要输入控制设备，任选地减少设备100上的物理输入控制设备(诸如下压按钮、拨号盘等等)的数量。In some embodiments, device 100 is a device on which operation of a predefined set of functions is performed exclusively through a touchscreen and/or a touchpad. By using a touchscreen and/or touchpad as the primary input control device for operation of device 100, the number of physical input control devices (such as push buttons, dials, etc.) on device 100 is optionally reduced.

唯一地通过触摸屏和/或触摸板执行的该预定义的的一组功能任选地包括在用户界面之间的导航。在一些实施方案中，触摸板在被用户触摸时将设备100从设备100上所显示的任何用户界面导航到主菜单、home菜单或根菜单。在此类实施方案中，使用触摸板来实现“菜单按钮”。在一些其他实施方案中，菜单按钮是物理下压按钮或者其他物理输入控制设备而不是触摸板。The predefined set of functions performed exclusively through the touchscreen and/or touchpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 from any user interface displayed on the device 100 to the main menu, home menu, or root menu. In such embodiments, the "menu button" is implemented using a touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device rather than a touchpad.

图1B是示出根据一些实施方案的用于事件处理的示例性部件的框图。在一些实施方案中，存储器102(图1A中)或370(图3)包括事件分类器170(例如，在操作系统126中)以及相应的应用程序136-1(例如，前述应用程序136、137至155、380至390中的任一个应用程序)。Figure IB is a block diagram illustrating exemplary components for event handling, according to some embodiments. In some embodiments, memory 102 (in FIG. 1A ) or 370 (FIG. 3 ) includes event sorter 170 (e.g., in operating system 126) and corresponding application 136-1 (e.g., the aforementioned applications 136, 137 to any one of 155, 380 to 390 applications).

事件分类器170接收事件信息并确定要将事件信息递送到的应用程序136-1和应用程序136-1的应用程序视图191。事件分类器170包括事件监视器171和事件分配器模块174。在一些实施方案中，应用程序136-1包括应用程序内部状态192，该应用程序内部状态指示当应用程序是活动的或正在执行时在触敏显示器系统112上显示的一个或多个当前应用程序视图。在一些实施方案中，设备/全局内部状态157被事件分类器170用于确定哪个(哪些)应用程序当前是活动的，并且应用程序内部状态192被事件分类器170用于确定要将事件信息递送到的应用程序视图191。The event sorter 170 receives the event information and determines the application 136-1 and the application view 191 of the application 136-1 to which to deliver the event information. Event sorter 170 includes event monitor 171 and event dispatcher module 174 . In some embodiments, application 136-1 includes application internal state 192 that indicates one or more current application programs displayed on touch-sensitive display system 112 while the application program is active or executing. view. In some embodiments, device/global internal state 157 is used by event sorter 170 to determine which application(s) are currently active, and application internal state 192 is used by event sorter 170 to determine which event information to deliver Go to the application view 191.

在一些实施方案中，应用程序内部状态192包括附加信息，诸如以下各项中的一者或多者：当应用程序136-1恢复执行时将被使用的恢复信息、指示正被应用程序136-1显示的信息或准备好用于被应用程序136-1显示的信息的用户界面状态信息、用于使得用户能够返回到应用程序136-1的前一状态或视图的状态队列、以及用户采取的先前动作的重复/撤销队列。In some embodiments, application internal state 192 includes additional information, such as one or more of: resume information to be used when application 136-1 resumes execution, 1 Information displayed or user interface state information ready for display by application 136-1, a state queue for enabling the user to return to a previous state or view of application 136-1, and actions taken by the user A repeat/undo queue of previous actions.

事件监视器171从外围设备接口118接收事件信息。事件信息包括关于子事件(例如，作为多点触摸手势的一部分的触敏显示器系统112上的用户触摸)的信息。外围设备接口118传输其从I/O子系统106或传感器诸如接近传感器166、一个或多个加速度计168和/或麦克风113(通过音频电路110)接收的信息。外围设备接口118从I/O子系统106所接收的信息包括来自触敏显示器系统112或触敏表面的信息。Event monitor 171 receives event information from peripherals interface 118 . The event information includes information about sub-events (eg, a user touch on touch-sensitive display system 112 as part of a multi-touch gesture). Peripherals interface 118 communicates information it receives from I/O subsystem 106 or sensors such as proximity sensor 166 , one or more accelerometers 168 , and/or microphone 113 (via audio circuitry 110 ). Information that peripherals interface 118 receives from I/O subsystem 106 includes information from touch-sensitive display system 112 or a touch-sensitive surface.

在一些实施方案中，事件监视器171以预先确定的间隔将请求发送至外围设备接口118。作为响应，外围设备接口118传输事件信息。在其他实施方案中，外围设备接口118仅当存在显著事件(例如，接收到高于预先确定的噪声阈值的输入和/或超过预先确定的持续时间)时传输事件信息。In some embodiments, event monitor 171 sends requests to peripherals interface 118 at predetermined intervals. In response, peripherals interface 118 transmits event information. In other embodiments, the peripherals interface 118 transmits event information only when there is a significant event (eg, receiving an input above a predetermined noise threshold and/or exceeding a predetermined duration).

在一些实施方案中，事件分类器170还包括命中视图确定模块172和/或活动事件识别器确定模块173。In some embodiments, the event sorter 170 also includes a hit view determination module 172 and/or an active event recognizer determination module 173 .

当触敏显示器系统112显示多于一个视图时，命中视图确定模块172提供用于确定子事件已在一个或多个视图内的什么地方发生的软件过程。视图由用户可在显示器上看到的控件和其他元件构成。When touch-sensitive display system 112 displays more than one view, hit view determination module 172 provides a software process for determining where a sub-event has occurred within one or more views. Views consist of controls and other elements that the user sees on the display.

与应用程序相关联的用户界面的另一方面是一组视图，在本文中有时也称为应用程序视图或用户界面窗口，在其中显示信息并且发生基于触摸的手势。在其中检测到触摸的(相应应用程序的)应用程序视图任选地对应于应用程序的程序化或视图分级结构内的程序化水平。例如，在其中检测到触摸的最低水平视图任选地被称为命中视图，并且被认为是正确输入的事件集任选地至少部分地基于初始触摸的命中视图来确定，该初始触摸开始基于触摸的手势。Another aspect of the user interface associated with an application is the set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which the touch is detected optionally corresponds to the application's programmatic or programmatic level within the view hierarchy. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events considered to be a correct input is optionally determined based at least in part on the hit view of the initial touch that began based on the touch gesture.

命中视图确定模块172接收与基于接触的手势的子事件相关的信息。当应用程序具有在分级结构中组织的多个视图时，命中视图确定模块172将命中视图识别为应对子事件进行处理的分级结构中的最低视图。在大多数情况下，命中视图是发起子事件(即形成事件或潜在事件的子事件序列中的第一子事件)在其中发生的最低水平视图。一旦命中视图被命中视图确定模块所识别，命中视图通常接收与其被识别为命中视图所针对的同一触摸或输入源相关的所有子事件。Hit view determination module 172 receives information related to sub-events of contact-based gestures. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy for which sub-events should be processed. In most cases, the hit view is the lowest level view in which an initiating sub-event (ie, the first sub-event in a sequence of sub-events forming an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

活动事件识别器确定模块173确定视图分级结构内的哪个或哪些视图应接收特定子事件序列。在一些实施方案中，活动事件识别器确定模块173确定仅命中视图应接收特定子事件序列。在其他实施方案中，活动事件识别器确定模块173确定包括子事件的物理位置的所有视图是活跃参与的视图，并且因此确定所有活跃参与的视图应接收特定子事件序列。在其他实施方案中，即使触摸子事件完全被局限到与一个特定视图相关联的区域，分级结构中的较高视图将仍然保持为活跃参与的视图。The active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some implementations, the active event recognizer determination module 173 determines that only hit views should receive a particular sequence of sub-events. In other embodiments, the active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if touch sub-events are completely confined to the area associated with one particular view, views higher in the hierarchy will still remain actively participating views.

事件分配器模块174将事件信息分配到事件识别器(例如，事件识别器180)。在包括活动事件识别器确定模块173的实施方案中，事件分配器模块174将事件信息递送到由活动事件识别器确定模块173确定的事件识别器。在一些实施方案中，事件分配器模块174在事件队列中存储事件信息，该事件信息由相应事件接收器模块182进行检索。Event dispatcher module 174 distributes event information to event recognizers (eg, event recognizer 180 ). In embodiments that include active event recognizer determination module 173 , event dispatcher module 174 delivers event information to the event recognizers determined by active event recognizer determination module 173 . In some embodiments, the event dispatcher module 174 stores event information in an event queue, which is retrieved by the corresponding event receiver module 182 .

在一些实施方案中，操作系统126包括事件分类器170。作为另外一种选择，应用程序136-1包括事件分类器170。在又一个实施方案中，事件分类器170是独立模块，或者是被存储在存储器102中的另一个模块诸如接触/运动模块130的一部分。In some embodiments, the operating system 126 includes an event sorter 170 . Alternatively, application 136 - 1 includes event sorter 170 . In yet another embodiment, event sorter 170 is a stand-alone module, or part of another module such as contact/motion module 130 stored in memory 102 .

在一些实施方案中，应用程序136-1包括多个事件处理程序190和一个或多个应用程序视图191，其中每个应用程序视图包括用于处理发生在应用程序的用户界面的相应视图内的触摸事件的指令。应用136-1的每个应用视图191包括一个或多个事件识别器180。通常，相应应用视图191包括多个事件识别器180。在其他实施方案中，事件识别器180中的一个或多个事件识别器是独立模块的一部分，该独立模块诸如用户界面工具包(未示出)或应用程序136-1从中继承方法和其他属性的更高水平的对象。在一些实施方案中，相应事件处理程序190包括以下一者或多者：数据更新器176、对象更新器177、GUI更新器178和/或从事件分类器170接收的事件数据179。事件处理程序190任选地利用或调用数据更新器176、对象更新器177或GUI更新器178来更新应用程序内部状态192。作为另外一种选择，应用程序视图191中的一个或多个应用程序视图包括一个或多个相应事件处理程序190。另外，在一些实施方案中，数据更新器176、对象更新器177和GUI更新器178中的一者或多者被包括在相应应用程序视图191中。In some embodiments, the application 136-1 includes a plurality of event handlers 190 and one or more application views 191, wherein each application view includes functions for processing events that occur within a corresponding view of the application's user interface. Instructions for touch events. Each application view 191 of application 136 - 1 includes one or more event recognizers 180 . Typically, a corresponding application view 191 includes a plurality of event recognizers 180 . In other embodiments, one or more of event recognizers 180 are part of a separate module, such as a user interface toolkit (not shown) or from which application 136-1 inherits methods and other properties. higher level objects. In some embodiments, corresponding event handler 190 includes one or more of: data updater 176 , object updater 177 , GUI updater 178 , and/or event data 179 received from event sorter 170 . Event handler 190 optionally utilizes or invokes data updater 176 , object updater 177 or GUI updater 178 to update application internal state 192 . Alternatively, one or more of the application views 191 includes one or more corresponding event handlers 190 . Additionally, in some embodiments, one or more of data updater 176 , object updater 177 , and GUI updater 178 are included in respective application views 191 .

相应的事件识别器180从事件分类器170接收事件信息(例如，事件数据179)，并且从事件信息识别事件。事件识别器180包括事件接收器182和事件比较器184。在一些实施方案中，事件识别器180还包括至少以下一个子集：元数据183和事件传递指令188(其任选地包括子事件传递指令)。A corresponding event recognizer 180 receives event information (eg, event data 179 ) from event classifier 170 and recognizes an event from the event information. Event recognizer 180 includes event receiver 182 and event comparator 184 . In some embodiments, event recognizer 180 also includes at least a subset of: metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

事件接收器182从事件分类器170接收事件信息。事件信息包括关于子事件例如触摸或触摸移动的信息。根据子事件，事件信息还包括附加信息，诸如子事件的位置。当子事件涉及触摸的运动时，事件信息任选地还包括子事件的速率和方向。在一些实施方案中，事件包括设备从一个取向旋转到另一取向(例如，从纵向取向旋转到横向取向，或反之亦然)，并且事件信息包括关于设备的当前取向(也被称为设备姿态)的对应信息。Event receiver 182 receives event information from event sorter 170 . The event information includes information on sub-events such as touch or touch movement. Depending on the sub-event, the event information also includes additional information such as the location of the sub-event. When the sub-event involves the motion of a touch, the event information optionally also includes the velocity and direction of the sub-event. In some embodiments, an event includes a rotation of the device from one orientation to another (eg, from a portrait orientation to a landscape orientation, or vice versa), and the event information includes information about the current orientation of the device (also referred to as the device pose ) corresponding information.

事件比较器184将事件信息与预定义的的事件或子事件定义进行比较，并且基于该比较，确定一个事件或子事件，或者确定或更新事件或子事件的状态。在一些实施方案中，事件比较器184包括事件定义186。事件定义186包含事件(例如事件1(187-1)、事件2(187-2)以及其他事件)的定义(例如，预定义的的子事件序列)。在一些实施方案中，事件187中的子事件包括例如触摸开始、触摸结束、触摸移动、触摸取消和多点触摸。在一个示例中，事件1(187-1)的定义是在所显示的对象上的双击。例如，双击包括被显示对象上的预先确定时长的第一次触摸(触摸开始)、预先确定时长的第一次抬离(触摸结束)、被显示对象上的预先确定的时长的第二次触摸(触摸开始)、以及预先确定时长的第二次抬离(触摸结束)。在另一个示例中，事件2(187-2)的定义是显示对象上的拖动。例如，拖动包括被显示对象上的预先确定时长的触摸(或接触)、触摸在触敏显示器系统112上的移动、以及触摸的抬起(触摸结束)。在一些实施方案中，事件还包括用于一个或多个相关联的事件处理程序190的信息。The event comparator 184 compares event information to predefined event or sub-event definitions, and based on the comparison, determines an event or sub-event, or determines or updates the status of an event or sub-event. In some implementations, event comparator 184 includes event definitions 186 . Event definitions 186 contain definitions (eg, predefined sequences of sub-events) of events (eg, event 1 ( 187 - 1 ), event 2 ( 187 - 2 ), and other events. In some embodiments, sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 ( 187 - 1 ) is defined as a double click on a displayed object. For example, a double tap includes a first touch (touch start) for a predetermined duration on the displayed object, a first lift-off (touch end) for a predetermined duration, and a second touch for a predetermined duration on the displayed object (touch to start), and a second lift-off (touch to end) of a predetermined duration. In another example, event 2 ( 187 - 2 ) is defined as a drag on the display object. For example, dragging includes a predetermined duration of touch (or contact) on the displayed object, movement of the touch on the touch-sensitive display system 112, and lift-off of the touch (touch end). In some embodiments, an event also includes information for one or more associated event handlers 190 .

在一些实施方案中，事件定义187包括用于相应用户界面对象的事件的定义。在一些实施方案中，事件比较器184执行命中测试以确定哪个用户界面对象与子事件相关联。例如，在触敏显示器系统112上显示三个用户界面对象的应用程序视图中，当在触敏显示器系统112上检测到触摸时，事件比较器184执行命中测试以确定这三个用户界面对象中的哪个用户界面对象与该触摸(子事件)相关联。如果每个所显示的对象与相应的事件处理程序190相关联，则事件比较器使用该命中测试的结果来确定哪个事件处理程序190应当被激活。例如，事件比较器184选择与子事件和触发该命中测试的对象相关联的事件处理程序。In some embodiments, event definitions 187 include definitions of events for corresponding user interface objects. In some implementations, event comparator 184 performs hit testing to determine which user interface object is associated with the sub-event. For example, in an application view displaying three user interface objects on the touch-sensitive display system 112, when a touch is detected on the touch-sensitive display system 112, the event comparator 184 performs a hit test to determine whether one of the three user interface objects is touched. Which UI object of the touch is associated with that touch (subevent). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

在一些实施方案中，相应事件187的定义还包括延迟动作，该延迟动作延迟事件信息的递送，直到已确定子事件序列是否确实对应于或不对应于事件识别器的事件类型。In some embodiments, the definition of the corresponding event 187 also includes a delay action that delays delivery of the event information until it has been determined whether the sequence of sub-events does or does not correspond to the event recognizer's event type.

当相应事件识别器180确定子事件序列不与事件定义186中的任何事件匹配时，该相应事件识别器180进入事件不可能、事件失败或事件结束状态，在此之后忽略基于触摸的手势的后续子事件。在这种情况下，对于命中视图保持活动的其他事件识别器(如果有的话)继续跟踪和处理持续的基于触摸的手势的子事件。When the corresponding event recognizer 180 determines that the sequence of sub-events does not match any event in the event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent touch-based gestures are ignored. sub event. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

在一些实施方案中，相应事件识别器180包括具有指示事件递送系统应当如何执行对活跃参与的事件识别器的子事件递送的可配置属性、标记和/或列表的元数据183。在一些实施方案中，元数据183包括指示事件识别器彼此如何交互或如何能够交互的可配置属性、标志和/或列表。在一些实施方案中，元数据183包括指示子事件是否递送到视图或程序化分级结构中的不同层级的可配置属性、标志和/或列表。In some embodiments, a respective event recognizer 180 includes metadata 183 with configurable properties, flags, and/or lists indicating how the event delivery system should perform delivery of sub-events to actively participating event recognizers. In some embodiments, metadata 183 includes configurable properties, flags, and/or lists that indicate how event recognizers interact or are capable of interacting with each other. In some embodiments, metadata 183 includes configurable properties, flags, and/or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

在一些实施方案中，当识别事件的一个或多个特定子事件时，相应事件识别器180激活与事件相关联的事件处理程序190。在一些实施方案中，相应事件识别器180将与事件相关联的事件信息递送到事件处理程序190。激活事件处理程序190不同于将子事件发送(和延期发送)到相应命中视图。在一些实施方案中，事件识别器180抛出与所识别的事件相关联的标记，并且与该标记相关联的事件处理程序190获取该标记并执行预定义的过程。In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates an event handler 190 associated with the event. In some embodiments, the respective event recognizer 180 delivers event information associated with the event to the event handler 190 . Activating an event handler 190 is distinct from sending (and deferred sending) sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a flag associated with the recognized event, and the event handler 190 associated with the flag takes the flag and performs a predefined process.

在一些实施方案中，事件递送指令188包括递送关于子事件的事件信息而无需激活事件处理程序的子事件递送指令。相反，子事件递送指令将事件信息递送到与子事件序列相关联的事件处理程序或者递送到活跃参与的视图。与子事件序列或与活跃参与的视图相关联的事件处理程序接收事件信息并执行预先确定的过程。In some embodiments, event delivery instructions 188 include subevent delivery instructions that deliver event information about a subevent without activating an event handler. Instead, subevent delivery instructions deliver event information to event handlers associated with subevent sequences or to actively participating views. Event handlers associated with sub-event sequences or with actively participating views receive event information and perform predetermined procedures.

在一些实施方案中，数据更新器176创建并更新在应用程序136-1中使用的数据。例如，数据更新器176对联系人模块137中所使用的电话号码进行更新，或者对视频播放器模块145中所使用的视频文件进行存储。在一些实施方案中，对象更新器177创建并更新在应用程序136-1中使用的对象。例如，对象更新器177创建新的用户界面对象或更新用户界面对象的位置。GUI更新器178更新GUI。例如，GUI更新器178准备显示信息并将其发送到图形模块132以供在触敏显示器上显示。In some embodiments, the data updater 176 creates and updates data used in the application 136-1. For example, the data updater 176 updates the phone number used in the contact module 137 , or stores the video file used in the video player module 145 . In some embodiments, the object updater 177 creates and updates objects used in the application 136-1. For example, the object updater 177 creates new user interface objects or updates the location of user interface objects. The GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends it to graphics module 132 for display on a touch-sensitive display.

在一些实施方案中，一个或多个事件处理程序190包括数据更新器176、对象更新器177和GUI更新器178或者具有对它们的访问权限。在一些实施方案中，数据更新器176、对象更新器177和GUI更新器178被包括在相应应用程序136-1或应用程序视图191的单个模块中。在其他实施方案中，它们被包括在两个或更多个软件模块中。In some embodiments, one or more event handlers 190 include or have access to data updater 176 , object updater 177 , and GUI updater 178 . In some embodiments, the data updater 176, the object updater 177, and the GUI updater 178 are included in a single module of the respective application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

应当理解，关于触敏显示器上的用户触摸的事件处理的上述讨论还应用于利用输入设备来操作多功能设备100的其他形式的用户输入，并不是所有用户输入都是在触摸屏上发起的。例如，任选地利用鼠标移动和鼠标按钮按压，任选地结合单次或多次键盘按压或保持；触摸板上的接触移动，诸如轻击、拖动、滚动等；触笔输入；设备的移动；口头指令；检测到的眼睛运动；生物识别输入；和/或其任何组合作为与定义要识别的事件的子事件对应的输入。It should be understood that the above discussion regarding event handling of user touches on a touch sensitive display also applies to other forms of user input utilizing input devices to operate the multifunction device 100, not all user input being initiated on a touch screen. For example, optionally utilizing mouse movements and mouse button presses, optionally combined with single or multiple keyboard presses or holds; contact movements on a touchpad, such as tapping, dragging, scrolling, etc.; stylus input; Movement; verbal commands; detected eye movements; biometric input; and/or any combination thereof as input corresponding to sub-events defining the event to be recognized.

图2示出了根据一些实施方案的具有触摸屏(例如，图1A的触敏显示器系统112)的便携式多功能设备100。触摸屏任选地在用户界面(UI)200内显示一个或多个图形。在本实施方案中以及在下文中描述的其他实施方案中，用户能够通过例如利用一个或多个手指202(在附图中没有按比例绘制)或者利用一个或多个触笔203(在附图中没有按比例绘制)在图形上作出手势来选择这些图形中的一个或多个图形。在一些实施方案中，当用户中断与一个或多个图形的接触时，将发生对一个或多个图形的选择。在一些实施方案中，手势任选地包括一次或多次轻击、一次或多次轻扫(从左向右、从右向左、向上和/或向下)和/或已与设备100发生接触的手指的滚动(从右向左、从左向右、向上和/或向下)。在一些具体实施中或在一些情况下，不经意地与图形接触不会选择图形。例如，当与选择对应的手势是轻击时，在应用程序图标上方扫动的轻扫手势任选地不会选择对应应用程序。FIG. 2 illustrates a portable multifunction device 100 with a touch screen (eg, touch-sensitive display system 112 of FIG. 1A ), according to some embodiments. The touch screen optionally displays one or more graphics within user interface (UI) 200 . In this embodiment, as well as other embodiments described hereinafter, the user can, for example, use one or more fingers 202 (not drawn to scale in the drawings) or use one or more stylus 203 (in the drawings). not drawn to scale) gestures over the figures to select one or more of the figures. In some embodiments, selection of one or more graphics will occur when the user breaks contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (left to right, right to left, up and/or down) and/or has occurred with device 100 Scrolling (right to left, left to right, up and/or down) of the finger in contact. In some implementations or in some cases, inadvertent contact with a graphic does not select the graphic. For example, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

设备100任选地还包括一个或多个物理按钮，诸如“home”按钮或菜单按钮204。如前所述，菜单按钮204任选地用于导航到任选地在设备100上被执行的一组应用程序中的任何应用程序136。作为另外一种选择，在一些实施方案中，菜单按钮被实现为被显示在触摸屏显示器112上的GUI中的软按键。Device 100 optionally also includes one or more physical buttons, such as a “home” button or menu button 204 . As previously mentioned, menu button 204 is optionally used to navigate to any application 136 of a set of applications optionally executing on device 100 . Alternatively, in some embodiments, the menu buttons are implemented as softkeys in a GUI displayed on the touchscreen display 112 .

在一些实施方案中，设备100包括触摸屏显示器、菜单按钮204、用于使设备通电/断电和用于锁定设备的下压按钮206、一个或多个音量调节按钮208、用户身份模块(SIM)卡槽210、耳麦插孔212和对接/充电外部端口124。下压按钮206任选地用于通过按下按钮并使按钮在下压状态中保持预定义的时间间隔来使设备通电/断电；通过按下按钮并在经过预定义的时间间隔之前释放按钮来锁定设备；和/或对设备进行解锁或发起解锁过程。在一些实施方案中，设备100还通过麦克风113来接受用于激活或去激活某些功能的语音输入。设备100还任选地包括用于检测触敏显示器系统112上的接触的强度的一个或多个接触强度传感器165、和/或用于为设备100的用户生成触觉输出的一个或多个触觉输出发生器167。In some embodiments, the device 100 includes a touch screen display, a menu button 204, a push button 206 for powering on/off the device and for locking the device, one or more volume adjustment buttons 208, a Subscriber Identity Module (SIM) Card slot 210 , headset jack 212 and docking/charging external port 124 . The push button 206 is optionally used to power on/off the device by pressing the button and holding the button in the depressed state for a predefined time interval; by pressing the button and releasing the button before the predefined time interval elapses lock the device; and/or unlock the device or initiate an unlocking process. In some embodiments, device 100 also accepts voice input through microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of a contact on touch-sensitive display system 112, and/or one or more tactile output sensors for generating a tactile output for a user of device 100. Generator 167.

图3是根据一些实施方案的具有显示器和触敏表面的多功能设备的示例的框图。设备300不必是便携式的。在一些实施方案中，设备300是膝上型电脑、台式计算机、平板电脑、多媒体播放器设备、导航设备、教育设备(诸如儿童学习玩具)、游戏系统或控制设备(例如，家用控制器或工业用控制器)。设备300通常包括一个或多个处理单元(CPU)310、一个或多个网络或其他通信接口360、存储器370和用于使这些部件互连的一个或多个通信总线320。通信总线320任选地包括使系统部件互连并且控制系统部件之间的通信的电路(有时叫做芯片组)。设备300包括具有显示器340的输入/输出(I/O)接口330，该显示器通常是触摸屏显示器。I/O接口330还任选地包括键盘和/或鼠标(或其他指向设备)350和触摸板355、用于在设备300上生成触觉输出的触觉输出发生器357(例如，类似于上文参考图1A所述的一个或多个触觉输出发生器167)、传感器359(例如，光学传感器、加速度传感器、接近传感器、触敏传感器和/或类似于上文参考图1A所述的一个或多个接触强度传感器165的接触强度传感器)。存储器370包括高速随机存取存储器诸如DRAM、SRAM、DDR RAM或其他随机存取固态存储器设备，并且任选地包括非易失性存储器诸如一个或多个磁盘存储设备、光盘存储设备、闪存存储器设备或其他非易失性固态存储设备。存储器370任选地包括远离一个或多个CPU 310定位的一个或多个存储设备。在一些实施方案中，存储器370存储与被存储便在携式多功能设备100(图1A)的存储器102中的程序、模块和数据结构类似的程序、模块、和数据结构，或它们的子组。此外，存储器370任选地存储在便携式多功能设备100的存储器102中不存在的附加程序、模块、和数据结构。例如，设备300的存储器370任选地存储绘图模块380、呈现模块382、文字处理模块384、网站创建模块386、盘编辑模块388、和/或电子表格模块390，而便携式多功能设备100(图1A)的存储器102任选地不存储这些模块。3 is a block diagram of an example of a multifunction device with a display and a touch-sensitive surface, according to some implementations. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., a home controller or industrial with the controller). Device 300 generally includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. Communication bus 320 optionally includes circuitry (sometimes called a chipset) that interconnects and controls communications between system components. Device 300 includes an input/output (I/O) interface 330 having a display 340, which is typically a touch screen display. I/O interface 330 also optionally includes keyboard and/or mouse (or other pointing device) 350 and touchpad 355, tactile output generator 357 for generating tactile output on device 300 (e.g., similar to One or more tactile output generators 167 described in FIG. 1A ), sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch-sensitive sensors, and/or one or more contact intensity sensor 165). Memory 370 includes high speed random access memory such as DRAM, SRAM, DDR RAM, or other random access solid state memory devices, and optionally includes nonvolatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from one or more CPUs 310 . In some embodiments, memory 370 stores programs, modules, and data structures similar to those stored in memory 102 of portable multifunction device 100 (FIG. 1A), or a subset thereof. . In addition, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100 . For example, memory 370 of device 300 optionally stores drawing module 380, rendering module 382, word processing module 384, website creation module 386, disk editing module 388, and/or spreadsheet module 390, while portable multifunction device 100 (FIG. The memory 102 of 1A) optionally does not store these modules.

图3中的上述所识别的元件中的每个元件任选地被存储在一个或多个先前提到的存储器设备中。上述所识别的模块中的每个模块对应于用于执行上述功能的指令集。上述所识别的模块或程序(即，指令集)不必被实现为单独的软件程序、过程或模块，并且因此这些模块的各种子组任选地在各种实施方案中被组合或以其他方式重新布置。在一些实施方案中，存储器370任选地存储上述模块和数据结构的子集。此外，存储器370任选地存储上面未描述的附加模块和数据结构。Each of the above-identified elements in FIG. 3 is optionally stored in one or more of the previously mentioned memory devices. Each of the above-identified modules corresponds to a set of instructions for performing a function described above. The modules or programs (i.e., sets of instructions) identified above need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules are optionally combined or otherwise rearrange. In some embodiments, memory 370 optionally stores a subset of the modules and data structures described above. Furthermore, memory 370 optionally stores additional modules and data structures not described above.

现在将注意力转到任选地在便携式多功能设备100上实现的用户界面(“UI”)的实施方案。Attention is now turned to an implementation of a user interface (“UI”) optionally implemented on portable multifunction device 100 .

图4A示出了根据一些实施方案的便携式多功能设备100上的应用程序菜单的示例性用户界面。类似的用户界面任选地在设备300上实现。在一些实施方案中，用户界面400包括以下元素或者其子集或超集：FIG. 4A illustrates an exemplary user interface of an application menu on portable multifunction device 100, according to some embodiments. A similar user interface is optionally implemented on device 300 . In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:

·用于一个或多个无线通信诸如蜂窝信号和Wi-Fi信号的一个或多个信号强度指示符402；• One or more signal strength indicators 402 for one or more wireless communications, such as cellular and Wi-Fi signals;

·时间404；time 404;

·蓝牙指示符405；· Bluetooth indicator 405;

·电池状态指示符406；· Battery status indicator 406;

·具有针对常用应用程序的图标的托盘408，这些图标诸如：• Tray 408 with icons for frequently used applications such as:

○电话模块138的被标记为“电话”的图标416，该图标416任选地包括未接来电或语音留言的数量的指示符414；o an icon 416 labeled "Phone" for the phone module 138, which optionally includes an indicator 414 of the number of missed calls or voice messages;

○电子邮件客户端模块140的被标记为“邮件”的图标418，该图标418任选地包括未读电子邮件的数量的指示符410；o an icon 418 labeled "Mail" for the email client module 140, which optionally includes an indicator 410 of the number of unread emails;

○浏览器模块147的被标记为“浏览器”的图标420；以及o the icon 420 of the browser module 147 labeled "Browser"; and

○视频和音乐播放器模块152(也称为iPod(Apple Inc.的商标)模块152)的被标记为“iPod”的图标422；以及o the icon 422 labeled "iPod" for the video and music player module 152 (also referred to as the iPod (trademark of Apple Inc.) module 152); and

·其他应用程序的图标，诸如：· Icons for other applications, such as:

○IM模块141的被标记为“消息”的图标424；o Icon 424 of the IM module 141 labeled "Messages";

○日历模块148的被标记为“日历”的图标426；o Icon 426 of calendar module 148 labeled "Calendar";

○图像管理模块144的被标记为“照片”的图标428；o Icon 428 of Image Management Module 144 labeled "Photos";

○相机模块143的被标记为“相机”的图标430；o Camera module 143's icon 430 labeled "Camera";

○在线视频模块155的被标记为“在线视频”的图标432；o The icon 432 of the online video module 155 labeled "Online Video";

○股市桌面小程序149-2的被标记为“股市”的图标434；○ The icon 434 of the stock market applet 149-2 labeled "stock market";

○地图模块154的被标记为“地图”的图标436；o the icon 436 of the map module 154 labeled "Map";

○天气桌面小程序149-1的被标记为“天气”的图标438；o Icon 438 labeled "Weather" for Weather widget 149-1;

○闹钟桌面小程序149-4的被标记为“时钟”的图标440；○ The icon 440 of the alarm clock applet 149-4 labeled "clock";

○新闻模块142的被标记为“新闻”的图标442；o Icon 442 of News module 142 labeled "News";

○记事本模块153的被标记为“记事本”的图标444；和o Notepad module 153's icon 444 labeled "Notepad"; and

○设置应用程序或模块的图标446，该图标446提供对设备100及其各种应用程序136的设置的访问。o Settings application or module icon 446 that provides access to settings for the device 100 and its various applications 136 .

应当指出的是，图4A中示出的图标标签仅仅是示例。例如，在一些实施方案中，视频和音乐播放器模块152的图标422被标记为“音乐”或“音乐播放器”。其他标签任选地用于各种应用程序图标。在一些实施方案中，相应应用程序图标的标签包括与该相应应用程序图标对应的应用程序的名称。在一些实施方案中，特定应用程序图标的标签不同于与该特定应用程序图标对应的应用程序的名称。It should be noted that the icon labels shown in FIG. 4A are examples only. For example, in some embodiments, the icon 422 of the video and music player module 152 is labeled "Music" or "Music Player." Other tags are optionally used for various application icons. In some implementations, the label of the corresponding application icon includes the name of the application corresponding to the corresponding application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

图4B示出了具有与显示器450分开的触敏表面451(例如，图3中的平板或触摸板355)的设备(例如，图3中的设备300)上的示例性用户界面。设备300还任选地包括用于检测触敏表面451上的接触的强度的一个或多个接触的强度传感器(例如，传感器359中的一个或多个传感器)，和/或用于为设备300的用户生成触觉输出的一个或多个触觉输出发生器359。尽管将参考触摸屏显示器112(其中组合了触敏表面和显示器)上的输入给出随后许多实施方案，但是在一些实施方案中，设备检测与显示器分开的触敏表面上的输入，如图4B中所示。在一些实施方案中，触敏表面(例如，图4B中的451)具有与显示器(例如，450)上的主轴(例如，图4B中的453)对应的主轴(例如，图4B中的452)。根据这些实施方案，设备检测在对应于显示器上的相应位置的位置(例如，在图4B中，460对应于468并且462对应于470)处与触敏表面451的接触(例如，图4B中的460和462)。这样，在触敏表面(例如，图4B中的451)与多功能设备的显示器(例如，图4B中的450)分开时，由设备在触敏表面上检测到的用户输入(例如，接触460和462及其移动)被该设备用于操纵显示器上的用户界面。应当理解，类似方法任选地用于本文所述的其他用户界面。4B illustrates an example user interface on a device (eg, device 300 in FIG. 3 ) having a touch-sensitive surface 451 (eg, tablet or touchpad 355 in FIG. 3 ) separate from display 450 . Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 359) for detecting the intensity of a contact on touch-sensitive surface 451, and/or for providing One or more haptic output generators 359 that generate haptic output for the user. Although many of the embodiments that follow will be given with reference to input on a touchscreen display 112 (where the touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, as in FIG. 4B . shown. In some embodiments, the touch-sensitive surface (eg, 451 in FIG. 4B ) has a major axis (eg, 452 in FIG. 4B ) that corresponds to a major axis (eg, 453 in FIG. 4B ) on the display (eg, 450 ). . According to these embodiments, the device detects contact with touch-sensitive surface 451 (e.g., in FIG. 460 and 462). In this way, when the touch-sensitive surface (eg, 451 in FIG. 4B ) is separated from the display (eg, 450 in FIG. 4B ) of the multifunction device, user input (eg, contact 460 ) detected by the device on the touch-sensitive surface and 462 and their movement) are used by the device to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

另外，虽然主要是参考手指输入(例如，手指接触、手指轻击手势、手指轻扫手势等)来给出下面的实施方案，但是应当理解，在一些实施方案中，这些手指输入中的一个或多个手指输入由来自另一输入设备的输入(例如，基于鼠标的输入或触笔输入)替代。例如，轻扫手势任选地由鼠标点击(例如，代替接触)，之后是光标沿着轻扫的路径的移动(例如，代替接触的移动)来替代。作为另一个示例，轻击手势任选地由在光标位于轻击手势的位置上方时的鼠标点击(例如，代替对接触的检测，之后是停止检测接触)来替代。类似地，当同时检测到多个用户输入时，应当理解，多个计算机鼠标任选地被同时使用，或鼠标和手指接触任选地被同时使用。Additionally, while the following embodiments are primarily presented with reference to finger inputs (e.g., finger contacts, finger tap gestures, finger swipe gestures, etc.), it should be understood that in some embodiments, one or more of these finger inputs Multiple finger input is replaced by input from another input device (eg, mouse-based or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (eg, instead of a contact) followed by movement of the cursor along the path of the swipe (eg, instead of movement of the contact). As another example, a tap gesture is optionally replaced by a mouse click while the cursor is over the location of the tap gesture (eg, instead of detection of the contact followed by ceasing to detect the contact). Similarly, when multiple user inputs are detected simultaneously, it is understood that multiple computer mice are optionally used simultaneously, or mouse and finger contacts are optionally used simultaneously.

如本文所使用，术语“焦点选择器”是指示用户正与其进行交互的用户界面的当前部分的输入元素。在包括光标或其他位置标记的一些具体实施中，光标充当“焦点选择器”，使得当光标在特定用户界面元素(例如，按钮、窗口、滑块或其他用户界面元素)上方时在触敏表面(例如，图3中的触摸板355或图4B中的触敏表面451)上检测到输入(例如，按压输入)的情况下，根据检测到的输入来调整该特定用户界面元素。在包括实现与触摸屏显示器上的用户界面元素的直接交互的触摸屏显示器(例如，图1A中的触敏显示器系统112或图4A中的触摸屏)的一些具体实施中，在触摸屏上所检测到的接触充当“焦点选择器”，使得当在触摸屏显示器上在特定用户界面元素(例如，按钮、窗口、滑块或其他用户界面元素)的位置处检测到输入(例如，通过接触的按压输入)时，根据所检测到的输入来调整该特定用户界面元素。在一些具体实施中，焦点从用户界面的一个区域移动到用户界面的另一个区域，而无需光标的对应移动或触摸屏显示器上的接触的移动(例如，通过使用制表键或箭头键将焦点从一个按钮移动到另一个按钮)；在这些具体实施中，焦点选择器根据用户界面的不同区域之间的焦点移动而移动。不考虑焦点选择器所采取的具体形式，焦点选择器通常是由用户控制以便传送与用户界面的用户打算的交互(例如，通过向设备指示用户界面的用户打算与其进行交互的元素)的用户界面元素(或触摸屏显示器上的接触)。例如，在触敏表面(例如，触摸板或触摸屏)上检测到按压输入时，焦点选择器(例如，光标、接触或选择框)在相应按钮上方的位置将指示用户期望激活相应按钮(而不是设备显示器上示出的其他用户界面元素)。As used herein, the term "focus selector" is an input element that indicates the current portion of a user interface with which a user is interacting. In some implementations that include a cursor or other position marker, the cursor acts as a "focus selector" such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the cursor is on the touch-sensitive surface. Where an input (eg, a press input) is detected on (eg, touchpad 355 in FIG. 3 or touch-sensitive surface 451 in FIG. 4B ), that particular user interface element is adjusted in accordance with the detected input. In some implementations that include a touchscreen display that enables direct interaction with user interface elements on the touchscreen display (e.g., touch-sensitive display system 112 in FIG. 1A or the touchscreen in FIG. 4A ), a contact detected on the touchscreen Acts as a "focus selector" such that when an input (e.g., a press input by a contact) is detected on the touchscreen display at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element), The particular user interface element is adjusted according to the detected input. In some implementations, focus is moved from one area of the user interface to another without corresponding movement of a cursor or movement of contact on a touchscreen display (e.g., by using the tab or arrow keys to move focus from button to button); in these implementations, the focus selector moves according to focus movement between different areas of the user interface. Regardless of the particular form that a focus selector takes, a focus selector is typically a user interface that is controlled by the user in order to convey a user-intended interaction with the user interface (e.g., by indicating to the device the element of the user interface that the user intends to interact with) element (or contact on a touchscreen display). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of a focus selector (e.g., a cursor, contact, or selection box) over the corresponding button will indicate that the user desires to activate the corresponding button (rather than other user interface elements shown on the device display).

如在本说明书和权利要求书中所使用的，术语触敏表面上的接触的“强度”是指触敏表面上的接触(例如，手指接触或触笔接触)的力或压力(每单位面积的力)，或是指触敏表面上的接触的力或压力的替代物(代用物)。接触的强度具有值范围，该值范围包括至少四个不同的值并且更典型地包括上百个不同的值(例如，至少256个)。接触的强度任选地使用各种方法和各种传感器或传感器的组合来确定(或测量)。例如，在触敏表面下方或相邻于触敏表面的一个或多个力传感器任选地用于测量触敏表面上的不同点处的力。在一些具体实施中，来自多个力传感器的力测量值被合并(例如，加权平均或者加和)，以确定估计的接触的力。类似地，触笔的压敏顶端任选地用于确定触笔在触敏表面上的压力。作为另外一种选择，在触敏表面上检测到的接触区域的尺寸和/或其变化、接触附近的触敏表面的电容和/或其变化以及/或者接触附近的触敏表面的电阻和/或其变化任选地被用作触敏表面上的接触的力或压力的替代物。在一些具体实施中，接触力或压力的替代测量值直接用于确定是否已超过强度阈值(例如，强度阈值是以与替代测量值对应的单位来描述的)。在一些具体实施中，将接触力或压力的替代测量值转换为预估力或压力，并且使用预估力或压力来确定是否已超过强度阈值(例如，强度阈值是以压力单位测量的压力阈值)。使用接触的强度作为用户输入的属性，从而允许用户访问用户在用于(例如，在触敏显示器上)显示示能表示和/或接收用户输入(例如，经由触敏显示器、触敏表面或物理控件/机械控件诸如旋钮或按钮)的实地面积有限的尺寸更小的设备上本来不能容易地访问的附加设备功能。As used in this specification and claims, the term "strength" of a contact on a touch-sensitive surface refers to the force or pressure (per unit area force), or a proxy (surrogate) referring to the force or pressure of a contact on a touch-sensitive surface. The intensity of the contact has a range of values that includes at least four different values and more typically hundreds of different values (eg, at least 256). The intensity of contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors beneath or adjacent to the touch-sensitive surface are optionally used to measure force at various points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (eg, weighted averaged or summed) to determine an estimated force of the contact. Similarly, the pressure sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch sensitive surface. Alternatively, the size of the detected touch area on the touch-sensitive surface and/or its change, the capacitance of the touch-sensitive surface in the vicinity of the touch and/or its change and/or the resistance of the touch-sensitive surface in the vicinity of the touch and/or Or variations thereof are optionally used as a surrogate for force or pressure of contact on a touch-sensitive surface. In some implementations, the surrogate measurement of contact force or pressure is used directly to determine whether an intensity threshold has been exceeded (eg, the intensity threshold is described in units corresponding to the surrogate measurement). In some implementations, a surrogate measure of contact force or pressure is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., an intensity threshold is a pressure threshold measured in pressure units ). Using the intensity of contact as an attribute of user input, allowing the user to access the user's ability to display affordances (e.g., on a touch-sensitive display) and/or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical Controls/Mechanical Controls (such as knobs or buttons) have limited real estate that would otherwise not allow easy access to additional device functionality on smaller sized devices.

在一些实施方案中，接触/运动模块130使用一组一个或多个强度阈值来确定用户是否已执行操作(例如，以便确定用户是否已在图标上进行“点击”)。在一些实施方案中，根据软件参数来确定强度阈值的至少一个子集(例如，强度阈值不是由特定物理致动器的激活阈值来确定的，并且可在不改变设备100的物理硬件的情况下被调节)。例如，可将触控板或触摸屏显示器的鼠标“点击”阈值设置为大范围的预定义的阈值中的任何一个预定义的阈值，而无需改变触控板或触摸屏显示硬件。另外，在一些实施方式中，设备的用户被提供有软件设置，以用于调整该组强度阈值中的一个或多个(例如，通过调整单独的强度阈值和/或通过利用系统级点击“强度”参数来同时调整多个强度阈值)。In some implementations, the contact/motion module 130 uses a set of one or more intensity thresholds to determine whether the user has performed an action (eg, to determine whether the user has "clicked" on the icon). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of specific physical actuators and can be determined without changing the physical hardware of device 100). is regulated). For example, the mouse "click" threshold of a touchpad or touchscreen display can be set to any one of a wide range of predefined thresholds without requiring changes to the touchpad or touchscreen display hardware. Additionally, in some embodiments, the user of the device is provided with a software setting for adjusting one or more of the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and/or by utilizing a system-level click on the "Intensity " parameter to adjust multiple intensity thresholds simultaneously).

如在说明书和权利要求书中所使用的，接触的术语“特征强度”是指基于一个或多个接触的强度的接触特征。在一些实施方案中，特征强度基于多个强度样本。特性强度任选地基于相对于预定义的事件(例如，在检测到接触之后，在检测到接触抬起之前，在检测到接触开始移动之前或之后，在检测到接触结束之前，在检测到接触的强度增大之前或之后和/或在检测到接触的强度减小之前或之后)而言在预先确定的时间段(例如，0.05秒、0.1秒、0.2秒、0.5秒、1秒、2秒、5秒、10秒)期间采集的预定义数量的强度样本或一组强度样本。接触的特性强度任选地基于以下各项中的一者或多者：接触强度的最大值、接触强度的均值、接触的强度平均值、接触强度的前10％处的值、接触强度的半最大值、接触强度的90％最大值等。在一些实施方案中，在确定特性强度时使用接触的持续时间(例如，在特性强度是接触的强度在时间上的平均值时)。在一些实施方案中，将特性强度与一组一个或多个强度阈值进行比较，以确定用户是否已执行操作。例如，该组一个或多个强度阈值可包括第一强度阈值和第二强度阈值。在本示例中，特征强度未超过第一阈值的接触导致第一操作，特征强度超过第一强度阈值但未超过第二强度阈值的接触导致第二操作，并且特征强度超过第二阈值的接触导致第三操作。在一些实施方案中，使用特征强度和一个或多个强度阈值之间的比较来确定是否要执行一个或多个操作(例如，是否执行相应选项或放弃执行相应操作)，而不是用于确定执行第一操作还是第二操作。As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of a contact based on the intensity of one or more contacts. In some embodiments, the characteristic intensity is based on a plurality of intensity samples. The feature strength is optionally based on relative to a predefined event (e.g. after a contact is detected, before a contact lift is detected, before or after a contact is detected to start moving, before a contact is detected to end, after a contact is detected for a predetermined period of time (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds , 5 seconds, 10 seconds) a predefined number of intensity samples or a set of intensity samples. The characteristic intensity of the contact is optionally based on one or more of the following: maximum value of the contact intensity, mean of the contact intensity, average of the intensity of the contact, values at the top 10% of the contact intensity, half of the contact intensity maximum, 90% maximum of contact strength, etc. In some embodiments, the duration of contact is used in determining the characteristic intensity (eg, when the characteristic intensity is the average of the intensity of the contact over time). In some embodiments, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether the user has performed an action. For example, the set of one or more intensity thresholds may include a first intensity threshold and a second intensity threshold. In this example, contacts whose characteristic intensity does not exceed the first threshold result in a first action, contacts whose characteristic intensity exceeds the first intensity threshold but not the second intensity threshold result in a second action, and contacts whose characteristic intensity exceeds the second threshold result in a second action. third operation. In some embodiments, a comparison between the intensity of a feature and one or more intensity thresholds is used to determine whether to perform one or more actions (e.g., whether to perform a corresponding option or to abstain from performing a corresponding operation), rather than to determine whether to perform The first operation or the second operation.

在一些实施方案中，识别手势的一部分以用于确定特征强度。例如，触敏表面可接收连续的轻扫接触，该连续的轻扫接触从起始位置过渡并到达结束位置(例如拖动手势)，在该结束位置处，接触的强度增大。在该示例中，接触在结束位置处的特征强度可仅基于连续轻扫接触的一部分，而不是整个轻扫接触(例如，仅结束位置处的轻扫接触部分)。在一些实施方案中，可在确定接触的特征强度之前向轻扫手势的强度应用平滑化算法。例如，该平滑化算法任选地包括以下各项中的一者或多者：不加权轻扫平均平滑化算法、三角平滑化算法、中值滤波器平滑化算法和/或指数平滑化算法。在一些情况下，为了确定特征强度的目的，这些平滑化算法消除了轻扫接触的强度中的窄尖峰或下降。In some implementations, a portion of the gesture is recognized for use in determining feature strength. For example, a touch-sensitive surface may receive successive swipe contacts that transition from a starting location to an ending location (eg, a drag gesture) where the intensity of the contact increases. In this example, the characteristic intensity of the contact at the end location may be based on only a portion of the continuous swipe contact, rather than the entire swipe contact (eg, only the portion of the swipe contact at the end location). In some implementations, a smoothing algorithm may be applied to the strength of the swipe gesture prior to determining the characteristic strength of the contact. For example, the smoothing algorithm optionally includes one or more of the following: an unweighted swipe average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and/or an exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow spikes or dips in the intensity of swipe contacts for the purpose of determining feature intensities.

这里描述的用户界面图(例如，图6A-6Y和图7A-7G)可选地包括示出了触敏表面上的接触相对于一个或多个强度阈值(例如，接触检测强度阈值IT₀、轻按压强度阈值IT_L、深按压强度阈值IT_D(例如，至少初始高于I_L))和/或一个或多个其他强度阈值(例如，低于I_H的强度阈值I_L)的当前强度的各种强度图。该强度图通常不是所显示的用户界面的一部分，但是被提供以帮助解释该图。在一些实施方案中，轻按压强度阈值对应于设备将执行通常与点击物理鼠标的按钮或触控板相关联的操作的强度。在一些实施方案中，深按压强度阈值对应于这样的强度：在该强度下设备将执行与通常与点击物理鼠标或触控板的按钮相关联的操作不同的操作。在一些实施方案中，当检测到特征强度低于轻按压强度阈值(例如，并且高于低于其不再检测到接触的标称接触检测强度阈值IT₀)的接触时，该设备将根据触敏表面上的接触移动来移动焦点选择器，而无需执行与轻按压强度阈值或深按压强度阈值相关联的操作。通常，除非另有说明，否则这些强度阈值在不同组的用户界面图之间是一致的。The user interface diagrams described herein (e.g., FIGS. 6A-6Y and FIGS. 7A-7G ) optionally include graphs showing the relationship of a contact on a touch-sensitive surface with respect to one or more intensity thresholds (e.g., contact detection intensity threshold IT₀ , The current intensity of a light press intensity threshold IT_L , a deep press intensity threshold IT_D (e.g., at least initially above_IL )), and/or one or more other intensity thresholds (e.g., an intensity threshold_IL below_IH ) various intensity maps. This intensity graph is usually not part of the displayed user interface, but is provided to help interpret the graph. In some implementations, the light press intensity threshold corresponds to an intensity at which the device will perform an operation typically associated with clicking a button of a physical mouse or a trackpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device will perform operations other than those normally associated with clicking a button of a physical mouse or trackpad. In some embodiments, when a contact is detected with a characteristic intensity below a light press intensity threshold (e.g., and above a nominal contact detection intensity threshold IT₀ below which it no longer detects contacts), the device will Contact movement on the sensitive surface moves the focus selector without performing an action associated with a light press intensity threshold or a deep press intensity threshold. In general, these intensity thresholds are consistent across different sets of UI diagrams unless otherwise noted.

在一些实施方案中，设备对设备所检测到的输入的响应取决于基于输入期间的接触强度的标准。例如，对于一些“轻按压”输入，在输入期间超过第一强度阈值的接触的强度触发第一响应。在一些实施方案中，设备对由设备检测的输入的响应取决于包括输入期间的接触强度和基于时间的标准两者的标准。例如，对于一些“深按压”输入，只有当满足第一强度阈值与满足第二强度阈值之间已过去延迟时间时，在输入期间超过第二强度阈值并大于轻按压的第一强度阈值的接触的强度才触发第二响应。该延迟时间的持续时间通常小于200ms(例如，40ms、100ms或120ms，这取决于第二强度阈值的量值，其中延迟时间随着第二强度阈值增大而增大)。这个延迟时间帮助避免意外的深按压输入。作为另一个示例，对于一些“深按压”输入，在满足第一强度阈值之后会出现敏感度减小的时间段。在该敏感度减小的时间段期间，第二强度阈值增大。第二强度阈值的这种暂时增大也有助于避免意外的深按压输入。对于其他深按压输入，对深按压输入检测的响应不取决于基于时间的标准。In some embodiments, the response of the device to the input detected by the device is based on criteria based on the intensity of contact during the input. For example, for some "light press" inputs, the intensity of the contact exceeding a first intensity threshold during the input triggers the first response. In some embodiments, the response of the device to an input detected by the device is dependent on criteria including both intensity of contact during the input and time-based criteria. For example, for some "deep press" inputs, contacts that exceed the second intensity threshold and are greater than the first intensity threshold for light presses during the input only if the delay time has elapsed between meeting the first intensity threshold and satisfying the second intensity threshold Only when the intensity of the signal triggers the second response. The duration of this delay time is typically less than 200ms (eg, 40ms, 100ms or 120ms, depending on the magnitude of the second intensity threshold, where the delay time increases as the second intensity threshold increases). This delay time helps avoid accidental deep press inputs. As another example, for some "deep press" inputs, a period of reduced sensitivity may occur after the first intensity threshold is met. During this period of reduced sensitivity, the second intensity threshold increases. This temporary increase in the second intensity threshold also helps avoid accidental deep press inputs. As with other deep-press inputs, the response to deep-press input detection does not depend on time-based criteria.

在一些实施方案中，输入强度阈值和/或对应输出中的一者或多者基于一个或多个因素而变化，诸如用户设置、接触运动、输入定时、应用程序运行、施加强度的速率、并发输入的数量、用户历史、环境因素(例如，环境噪声)、焦点选择器位置等。在美国专利申请序列号14/399,606和14/624,296中描述了示例性因素，该专利申请的全部内容以引用方式并入本文。In some embodiments, one or more of the input intensity thresholds and/or corresponding outputs vary based on one or more factors, such as user settings, contact motion, input timing, application execution, rate at which intensity is applied, concurrent Number of inputs, user history, environmental factors (eg, ambient noise), focus selector position, etc. Exemplary factors are described in US Patent Application Serial Nos. 14/399,606 and 14/624,296, the entire contents of which are incorporated herein by reference.

例如，图4C示出了部分地基于随时间的触摸输入476的强度而随时间改变的动态强度阈值480。动态强度阈值480是以下两个分量的和：第一分量474，其在自从初始检测到触摸输入476的预定义的延迟时间p1之后随时间衰减；以及第二分量478，其随时间追踪触摸输入476的强度。第一分量474的初始高强度阈值减少“深按压”响应的意外触发，同时如果触摸输入476提供足够的强度，则仍然允许立即的“深按压”响应。第二分量478通过触摸输入中的逐渐强度起伏来减少“深按压”响应的无意触发。在一些实施方案中，当触摸输入476满足动态强度阈值480(例如，在图4C中的点481处)时，触发“深按压”响应。For example, FIG. 4C illustrates a dynamic intensity threshold 480 that changes over time based in part on the intensity of touch input 476 over time. The dynamic intensity threshold 480 is the sum of two components: a first component 474, which decays over time after a predefined delay time p1 since the touch input 476 was initially detected; and a second component 478, which tracks the touch input over time. 476 strength. The initial high intensity threshold of the first component 474 reduces accidental triggering of a "deep press" response while still allowing an immediate "deep press" response if the touch input 476 provides sufficient intensity. The second component 478 reduces inadvertent triggering of a "deep press" response through gradual intensity fluctuations in the touch input. In some implementations, a "deep press" response is triggered when touch input 476 satisfies dynamic intensity threshold 480 (eg, at point 481 in FIG. 4C ).

图4D示出了另一个动态强度阈值486(例如，强度阈值I_D)。图4D还示出了两个其他强度阈值：第一强度阈值I_H和第二强度阈值I_H。在图4D中，虽然触摸输入484在时间p2之前满足第一强度阈值I_H和第二强度阈值I_L，但直到在时间482已过去延迟时间p2才提供响应。同样在图4D中，动态强度阈值486随时间衰减，其中在从时间482经过去预定义的延迟时间p1之后，该衰减在时间488开始(当触发与第二强度阈值I_L相关联的响应时)。这种类型的动态强度阈值减少了：紧接着触发与较低强度阈值相关联的响应、或在触发与较低强度阈值相关联的响应的同时，意外触发与动态强度阈值I_D相关联的响应，该较低强度阈值诸如第一强度阈值I_H或第二强度阈值I_H。FIG. 4D shows another dynamic intensity threshold₄₈₆ (eg, intensity threshold ID ). Figure 4D also shows two other intensity thresholds: a first intensity threshold_IH and a second intensity threshold_IH . In FIG. 4D , although touch input 484 satisfies first and second intensity thresholds I_H and I_L prior to time p2 , a response is not provided until delay time p2 has elapsed at time 482 . Also in FIG. 4D , dynamic intensity threshold 486 decays over time, where the decay begins at time 488 after a predefined delay time p1 has elapsed from time 482 (when a response associated with the second intensity threshold_IL is triggered ). This type of dynamic intensity threshold reduces the chance of accidentally triggering a response associated with a dynamic intensity threshold ID immediately after, or at the same time as,_triggering a response associated with a lower intensity threshold , the lower intensity threshold such as the first intensity threshold I_H or the second intensity threshold I_H .

图4E示出了又一个动态强度阈值492(例如，强度阈值I_D)。在图4E中，在自从初始检测到触摸输入490已过去延迟时间p2之后，触发与强度阈值I_L相关联的响应。同时，在自从初始检测到触摸输入490已过去预定义的延迟时间p1之后，动态强度阈值492衰减。因此，即使当触摸输入490的强度低于另一个强度阈值(例如，强度阈值I_L)时，在触发与强度阈值I_L相关联的响应之后触摸输入490的强度减小，随后触摸输入490的强度增大，而无需释放触摸输入490，可触发与强度阈值I_D相关联的响应(例如，在时间494)。Figure 4E shows yet another dynamic intensity threshold₄₉₂ (eg, intensity threshold ID). In FIG. 4E , the response associated with the intensity threshold_IL is triggered after a delay time p2 has elapsed since the initial detection of the touch input 490 . At the same time, the dynamic intensity threshold 492 decays after a predefined delay time p1 has elapsed since the touch input 490 was initially detected. Thus, even when the intensity of touch input 490 is below another intensity threshold (e.g., intensity threshold_IL ), the intensity of touch input 490 decreases after triggering a response associated with intensity threshold_IL , and subsequent touch input 490 An increase in intensity, without releasing the touch input 490, may trigger a response associated with the intensity threshold ID (eg, at time₄₉₄ ).

接触的特征强度从低于轻按压强度阈值IT_L的强度增大到介于轻按压强度阈值IT_L与深按压强度阈值IT_D之间的强度有时被称为“轻按压”输入。接触的特征强度从低于深按压强度阈值IT_D的强度到高于深按压强度阈值IT_D的强度的增大有时被称为“深按压”输入。接触的特征强度从低于接触检测强度阈值IT₀的强度增大到介于接触检测强度阈值IT₀与轻按压强度阈值IT_L之间的强度有时被称为检测到触摸表面上的接触。接触的特征强度从高于接触检测强度阈值IT₀的强度减小到低于接触检测强度阈值IT₀的强度有时被称为检测到接触从触摸表面抬起。在一些实施方案中，IT₀为零。在一些实施方案中，IT₀大于零。在一些图示中，阴影圆或椭圆用于表示触敏表面上的接触的强度。在一些图示中，没有阴影的圆或椭圆用于表示触敏表面上的相应接触而无需指定相应接触的强度。An increase in the characteristic intensity of a contact from an intensity below the light press intensity threshold IT_L to an intensity between the light press intensity threshold IT_L and the deep press intensity threshold IT_D is sometimes referred to as a "light press" input. An increase in the characteristic intensity of a contact from an intensity below the deep press intensity threshold IT_D to an intensity above the deep press intensity threshold IT_D is sometimes referred to as a "deep press" input. An increase in the characteristic intensity of a contact from an intensity below the contact detection intensity threshold IT₀ to an intensity between the contact detection intensity threshold IT₀ and the light press intensity threshold IT_L is sometimes referred to as a detected contact on the touch surface. A decrease in the characteristic intensity of a contact from an intensity above the contact detection intensity threshold IT₀ to an intensity below the contact detection intensity threshold IT₀ is sometimes referred to as a detected lift of the contact from the touch surface. In some embodiments, IT₀ is zero. In some embodiments, IT₀ is greater than zero. In some illustrations, a shaded circle or ellipse is used to represent the intensity of a contact on the touch-sensitive surface. In some illustrations, unshaded circles or ellipses are used to represent corresponding contacts on the touch-sensitive surface without specifying the intensity of the corresponding contacts.

在本文中所述的一些实施方案中，响应于检测到包括相应按压输入的手势或响应于检测到利用相应接触(或多个接触)执行的相应按压输入来执行一个或多个操作，其中至少部分地基于检测到该接触(或多个接触)的强度增大到高于按压输入强度阈值而检测到相应按压输入。在一些实施方案中，响应于检测到相应接触的强度增大到高于按压输入强度阈值来执行相应操作(例如，在相应按压输入的“向下冲程”上执行相应操作)。在一些实施方案中，按压输入包括相应接触的强度增大到高于按压输入强度阈值以及该接触的强度随后减小到低于按压输入强度阈值，并且响应于检测到相应接触的强度随后减小到低于按压输入阈值来执行相应操作(例如，在相应按压输入的“向上冲程”上执行相应操作)。In some embodiments described herein, one or more operations are performed in response to detecting a gesture comprising a corresponding press input or in response to detecting a corresponding press input performed with a corresponding contact (or contacts), wherein at least A corresponding press input is detected based in part on detecting an increase in intensity of the contact (or contacts) above a press input intensity threshold. In some implementations, the corresponding operation is performed in response to detecting an increase in intensity of the corresponding contact above a press input intensity threshold (eg, performing the corresponding operation on a "down stroke" of the corresponding press input). In some embodiments, the press input includes an increase in the intensity of the corresponding contact above a press input intensity threshold and a subsequent decrease in the intensity of the contact below the press input intensity threshold, and the subsequent decrease in intensity in response to detecting the corresponding contact to below the press input threshold to perform the corresponding action (eg, perform the corresponding action on the "up stroke" of the corresponding press input).

在一些实施方案中，设备采用强度滞后以避免有时称为“抖动”的意外输入，其中设备限定或选择与按压输入强度阈值具有预定义的关系的滞后强度阈值(例如，滞后强度阈值比按压输入强度阈值低X个强度单位，或滞后强度阈值是按压输入强度阈值的75％、90％或某个合理比例)。因此，在一些实施方案中，按压输入包括相应接触的强度增大到高于按压输入强度阈值以及该接触的强度随后减小到低于对应于按压输入强度阈值的滞后强度阈值，并且响应于检测到相应接触的强度随后减小到低于滞后强度阈值来执行相应操作(例如，在相应按压输入的“向上冲程”上执行相应操作)。类似地，在一些实施方案中，仅在设备检测到接触的强度从等于或低于滞后强度阈值的强度增大到等于或高于按压输入强度阈值的强度并且任选地接触的强度随后减小到等于或低于滞后强度的强度时才检测到按压输入，并且响应于检测到按压输入(例如，根据环境，接触的强度增大或接触的强度减小)来执行相应操作。In some embodiments, the device employs an intensity hysteresis to avoid accidental inputs sometimes referred to as "jitter," wherein the device defines or selects a hysteresis intensity threshold that has a predefined relationship to the press input intensity threshold (e.g., the hysteresis intensity threshold is greater than the press input intensity threshold). The intensity threshold is X intensity units lower, or the hysteresis intensity threshold is 75%, 90%, or some reasonable ratio of the press input intensity threshold). Thus, in some embodiments, a press input includes an increase in the intensity of a corresponding contact above a press input intensity threshold and a subsequent decrease in the intensity of the contact below a hysteresis intensity threshold corresponding to the press input intensity threshold, and in response to detecting The intensity to the corresponding contact then decreases below the hysteresis intensity threshold to perform the corresponding operation (eg, perform the corresponding operation on an "up stroke" of the corresponding press input). Similarly, in some embodiments, only if the device detects an increase in the intensity of the contact from an intensity at or below the hysteresis intensity threshold to an intensity at or above the press input intensity threshold and optionally a subsequent decrease in the intensity of the contact The press input is detected until the intensity is equal to or lower than the hysteresis intensity, and a corresponding operation is performed in response to the detection of the press input (for example, the intensity of the contact is increased or the intensity of the contact is decreased according to circumstances).

为了容易解释，任选地，响应于检测到以下情况而触发对响应于与按压输入强度阈值相关联的按压输入或响应于包括按压输入的手势而执行的操作的描述：接触的强度增大到高于按压输入强度阈值、接触的强度从低于滞后强度阈值的强度增大到高于按压输入强度阈值的强度、接触的强度减小到低于按压输入强度阈值、或接触的强度减小到低于与按压输入强度阈值对应的滞后强度阈值。另外，在将操作描述为响应于检测到接触的强度减小到低于按压输入强度阈值而执行的示例中，任选地响应于检测到接触的强度减小到低于对应于并且小于按压输入强度阈值的滞后强度阈值来执行操作。如上所述，在一些实施方案中，对这些响应的触发也取决于满足基于时间的标准(例如，在满足第一强度阈值与满足第二强度阈值之间已过去延迟时间)。For ease of explanation, optionally, a description of an operation performed in response to a press input associated with a press input intensity threshold, or in response to a gesture including a press input, is triggered in response to detecting that the intensity of the contact increases to Above the press input intensity threshold, the intensity of the contact increases from an intensity below the hysteresis intensity threshold to an intensity above the press input intensity threshold, the intensity of the contact decreases below the press input intensity threshold, or the intensity of the contact decreases to Below the hysteresis strength threshold corresponding to the press input strength threshold. Additionally, in examples where operations are described as being performed in response to a detected intensity of a contact being decreased below a press input intensity threshold, optionally in response to a detected intensity of a contact being decreased below a value corresponding to and less than a press input Hysteresis for intensity thresholds to perform actions on. As noted above, in some embodiments the triggering of these responses is also dependent on meeting time-based criteria (eg, a delay time has elapsed between meeting a first intensity threshold and meeting a second intensity threshold).

图5示出了与显示器450和触敏表面451进行通信的示例性电子设备。对于电子设备的至少一个子集，根据一些实施方案，显示器450和/或触敏表面451被集成到电子设备中。虽然参考与电子设备(例如，图1A-1B中的便携式多功能设备100或图3中的设备300)进行通信的触敏表面451和显示器450来描述以下更详细描述的示例，但应当理解，根据一些实施方案，触敏表面和/或显示器与电子设备集成，而在其他实施方案中，触敏表面和显示器中的一个或多个与电子设备分开。另外，在一些实施方案中，电子设备具有集成显示器和/或集成触敏表面，并且与和电子设备分开的一个或多个附加显示器和/或触敏表面进行通信。FIG. 5 illustrates an exemplary electronic device in communication with a display 450 and a touch-sensitive surface 451 . For at least a subset of electronic devices, according to some embodiments, display 450 and/or touch-sensitive surface 451 are integrated into the electronic device. While the examples described in more detail below are described with reference to a touch-sensitive surface 451 and display 450 in communication with an electronic device (e.g., portable multifunction device 100 in FIGS. 1A-1B or device 300 in FIG. 3 ), it should be understood that According to some embodiments, the touch-sensitive surface and/or the display are integrated with the electronic device, while in other embodiments, one or more of the touch-sensitive surface and the display are separate from the electronic device. Additionally, in some embodiments, the electronic device has an integrated display and/or integrated touch-sensitive surface and communicates with one or more additional displays and/or touch-sensitive surfaces that are separate from the electronic device.

在一些实施方案中，对具有用户界面导航逻辑部件483的单个电子设备(例如，以下参考图5描述的计算设备A)执行以下参考图6A-6Y和7A-7G描述的所有操作。然而，应当理解，通常将多个不同的电子设备链接在一起，以便执行以下参考图6A-6Y和7A-7G描述的操作(例如，具有用户界面导航逻辑部件483的电子设备与具有显示器450的独立电子设备和/或具有触敏表面451的独立电子设备进行通信)。在这些实施方案中的任何一个实施方案中，以下参考图6A-6Y和7A-7G描述的电子设备是包含用户界面导航逻辑部件483的电子设备(或多个设备)。另外，应当理解，在各种实施方案中，用户界面导航逻辑部件483可在多个不同的模块或电子设备之间进行划分；然而，为了本文的描述的目的，用户界面导航逻辑部件483将主要指驻留在单个电子设备中，以避免不必要地模糊实施方案的其他方面。In some embodiments, all of the operations described below with reference to FIGS. 6A-6Y and 7A-7G are performed on a single electronic device with user interface navigation logic 483 (eg, computing device A described below with reference to FIG. 5 ). It should be understood, however, that typically a plurality of different electronic devices are linked together in order to perform the operations described below with reference to FIGS. Standalone electronic device and/or stand-alone electronic device with touch-sensitive surface 451 for communication). In any of these embodiments, the electronic device described below with reference to FIGS. 6A-6Y and 7A-7G is an electronic device (or devices) that includes user interface navigation logic 483 . Additionally, it should be understood that in various embodiments, user interface navigation logic 483 may be divided between a number of different modules or electronic devices; however, for purposes of the description herein, user interface navigation logic 483 will primarily Refers to residing in a single electronic device to avoid unnecessarily obscuring other aspects of the implementation.

在一些实施方案中，用户界面导航逻辑部件483包括一个或多个模块(例如，一个或多个事件处理器190，其包括一个或多个对象更新器177和一个或多个GUI更新器178，如以上参考图1C更详细地描述的)，该一个或多个模块接收解释输入并且响应于这些解释输入来生成用于根据解释输入来更新图形用户界面的指令，该指令随后用于更新显示器上的图形用户界面。在一些实施方案中，解释输入是已被检测(例如，通过图1A-1B和3中的接触运动130)、识别(例如，通过图1C中的事件识别器180)、和/或优先化(例如，通过图1C中的事件分类器170)的输入。在一些实施方案中，解释输入由电子设备处的模块生成(例如，电子设备接收原始接触输入数据以便从原始接触输入数据识别手势)。在一些实施方案中，一些或所有的解释输入由电子设备作为解释输入而接收(例如，包括触敏表面451的电子设备处理原始接触输入数据以便从原始接触输入数据识别手势，并且将指示手势的信息发送到包括用户界面导航逻辑部件483的电子设备)。In some embodiments, user interface navigation logic 483 includes one or more modules (e.g., one or more event handlers 190, which include one or more object updaters 177 and one or more GUI updaters 178, As described in more detail above with reference to FIG. 1C ), the one or more modules receive interpretive inputs and, in response to these interpretive inputs, generate instructions for updating the graphical user interface based on the interpretive inputs, which are then used to update the graphical user interface on the display. graphical user interface. In some embodiments, interpretation input is detected (e.g., by contact motion 130 in FIGS. 1A-1B and 3 ), recognized (e.g., by event recognizer 180 in FIG. 1C ), and/or prioritized ( For example, through the input of the event classifier 170 in Figure 1C). In some embodiments, the interpretation input is generated by a module at the electronic device (eg, the electronic device receives raw contact input data to recognize a gesture from the raw contact input data). In some embodiments, some or all of the interpreted input is received by the electronic device as interpreted input (e.g., an electronic device including touch-sensitive surface 451 processes raw contact input data to recognize gestures from the raw contact input data and The information is sent to the electronic device including the user interface navigation logic 483).

在一些实施方案中，显示器450和触敏表面451两者与包含用户界面导航逻辑部件483的电子设备(例如，图5中的计算设备A)集成。例如，电子设备可以是具有集成显示器(例如，图3中的340)和触摸板(例如，图3中的355)的台式计算机或膝上型计算机。作为另一个示例，电子设备可以是具有触摸屏(例如，图2中的112)的便携式多功能设备100(例如，智能电话、PDA、平板计算机等)。In some embodiments, both display 450 and touch-sensitive surface 451 are integrated with an electronic device (eg, computing device A in FIG. 5 ) that includes user interface navigation logic 483 . For example, the electronic device may be a desktop or laptop computer with an integrated display (eg, 340 in FIG. 3 ) and touchpad (eg, 355 in FIG. 3 ). As another example, the electronic device may be a portable multifunction device 100 (eg, smartphone, PDA, tablet computer, etc.) having a touch screen (eg, 112 in FIG. 2 ).

在一些实施方案中，触敏表面451与电子设备集成，而显示器450未与包含用户界面导航逻辑部件483的电子设备(例如，图5中的计算设备B)集成。例如，电子设备可以是具有(通过有线连接或无线连接)连接到单独显示器(例如，计算机、监视器、电视机等)的集成触摸板(例如，图3中的355)的设备300(例如，台式计算机或膝上型计算机)。作为另一示例，电子设备可以是具有(通过有线连接或无线连接)连接到单独显示器(例如，计算机监视器、电视机等)的触摸屏(例如，图2中的112)的便携式多功能设备100(例如，智能电话、PDA、平板计算机等)。In some embodiments, touch-sensitive surface 451 is integrated with the electronic device, while display 450 is not integrated with the electronic device (eg, computing device B in FIG. 5 ) that includes user interface navigation logic 483 . For example, the electronic device may be device 300 (eg, 355 in FIG. 3 ) having an integrated touchpad (eg, 355 in FIG. desktop or laptop computer). As another example, the electronic device may be a portable multifunction device 100 having a touch screen (eg, 112 in FIG. 2 ) connected (via a wired or wireless connection) to a separate display (eg, computer monitor, television, etc. (eg, smart phone, PDA, tablet computer, etc.).

在一些实施方案中，显示器450与电子设备集成，而触敏表面451未与包含用户界面导航逻辑部件483的电子设备(例如，图5中的计算设备C)集成。例如，电子设备可以是具有(通过有线连接或无线连接)连接到单独触敏表面(例如，远程触摸板、便携式多功能设备等)的集成显示器(例如，图3中的340)的设备300(例如，台式计算机、膝上型计算机、具有集成机顶盒的电视机)。作为另一个示例，电子设备可以是具有(通过有线连接或无线连接)连接到单独触摸敏感表面(例如，远程触摸板、具有用作远程触摸板的触摸屏的另一个便携式多功能设备等)的触摸屏(例如，图2中的112)的便携式多功能设备100(例如，智能电话、PDA、平板计算机等)。In some embodiments, display 450 is integrated with the electronic device, while touch-sensitive surface 451 is not integrated with the electronic device (eg, computing device C in FIG. 5 ) that includes user interface navigation logic 483 . For example, the electronic device may be device 300 (e.g., 340 in FIG. 3 ) having an integrated display (e.g., 340 in FIG. For example, desktop computers, laptop computers, televisions with integrated set-top boxes). As another example, the electronic device may be a touchscreen with a touchscreen connected (either through a wired or wireless connection) to a separate touch-sensitive surface (e.g., a remote touchpad, another portable multifunction device with a touchscreen that acts as a remote touchpad, etc.) (eg, 112 in FIG. 2 ) of the portable multifunction device 100 (eg, smart phone, PDA, tablet computer, etc.).

在一些实施方案中，显示器450和触敏表面451两者不与包含用户界面导航逻辑部件483的电子设备(例如，图5中的计算设备D)集成。例如，电子设备可以是(通过有线连接或无线连接)连接到单独触敏表面(例如，远程触摸板、便携式多功能设备等)和单独显示器(例如，计算机监视器、电视机等)的独立电子设备300(例如，台式计算机、膝上型计算机、控制台、机顶盒等)。作为另一个示例，电子设备可以是具有(通过有线连接或无线连接)连接到单独触摸敏感表面(例如，远程触摸板、具有用作远程触摸板的触摸屏的另一个便携式多功能设备等)的触摸屏(例如，图2中的112)的便携式多功能设备100(例如，智能电话、PDA、平板计算机等)。In some embodiments, neither display 450 nor touch-sensitive surface 451 are integrated with an electronic device (eg, computing device D in FIG. 5 ) that includes user interface navigation logic 483 . For example, an electronic device may be a stand-alone electronic device connected (via a wired or wireless connection) to a separate touch-sensitive surface (e.g., remote touchpad, portable multifunction device, etc.) and a separate display (e.g., computer monitor, television, etc.). Device 300 (eg, desktop computer, laptop computer, console, set-top box, etc.). As another example, the electronic device may be a touchscreen with a touchscreen connected (either through a wired or wireless connection) to a separate touch-sensitive surface (e.g., a remote touchpad, another portable multifunction device with a touchscreen that acts as a remote touchpad, etc.) (eg, 112 in FIG. 2 ) of the portable multifunction device 100 (eg, smart phone, PDA, tablet computer, etc.).

在一些实施方案中，计算设备具有集成音频系统。在一些实施方案中，计算设备与和该计算设备分开的音频系统进行通信。在一些实施方案中，音频系统(例如，集成在电视机单元中的音频系统)与单独显示器450集成。在一些实施方案中，音频系统(例如，立体声系统)是与计算设备和显示器450分开的独立系统。In some embodiments, the computing device has an integrated audio system. In some embodiments, the computing device communicates with an audio system that is separate from the computing device. In some implementations, an audio system (eg, an audio system integrated in a television unit) is integrated with the separate display 450 . In some implementations, the audio system (eg, a stereo system) is a separate system from the computing device and display 450 .

用户界面和相关联的过程User Interface and Associated Processes

现在注意可使用与显示器和触敏表面进行通信和/或包括其的电子设备来实现的用户界面(“UI”)和相关联的过程的实施方案，该电子设备诸如计算设备(例如，图5中的计算设备A-D中的一个计算设备)。在一些实施方案中，该计算设备包括用于检测与触敏表面的接触的强度的一个或多个传感器。在一些实施方案中，该计算设备包括显示器。在一些实施方案中，该计算设备包括音频系统。在一些实施方案中，该计算设备既不包括显示器也不包括音频系统。在一些实施方案中，显示器包括音频系统(例如，显示器和音频系统是电视机的部件)。在一些实施方案中，音频系统和显示器的某些部件是分开的(例如，显示器是电视机的部件，并且音频系统包括与电视机分开的条形音箱)。在一些实施方案中，计算设备与单独遥控器通信，该计算设备通过该遥控器接收用户输入(例如，遥控器包括用户通过其与计算设备进行交互的触敏表面或触摸屏)。在一些实施方案中，遥控器包括用于检测遥控器运动(例如，用户拾起遥控器)的运动传感器(例如，加速度计和/或陀螺仪)。Attention is now directed to embodiments of a user interface (“UI”) and associated processes that may be implemented using an electronic device that communicates with and/or includes a display and a touch-sensitive surface, such as a computing device (e.g., FIG. 5 one of computing devices A-D in ). In some implementations, the computing device includes one or more sensors for detecting the intensity of contact with the touch-sensitive surface. In some embodiments, the computing device includes a display. In some embodiments, the computing device includes an audio system. In some embodiments, the computing device includes neither a display nor an audio system. In some embodiments, the display includes an audio system (eg, the display and audio system are components of a television). In some embodiments, certain components of the audio system and display are separate (eg, the display is part of a television and the audio system includes a sound bar separate from the television). In some embodiments, the computing device communicates with a separate remote control through which the computing device receives user input (eg, the remote control includes a touch-sensitive surface or touch screen through which the user interacts with the computing device). In some implementations, the remote includes motion sensors (eg, accelerometers and/or gyroscopes) for detecting motion of the remote (eg, a user picks up the remote).

尽管将参考与显示器450分开的触敏表面451上的输入来给出以下的一些示例，但是在一些实施方案中，如图4A所示，设备检测触摸屏显示器上的输入(其中触敏表面与显示器组合)。为了便于说明，将参考对具有触敏显示系统112的设备执行的操作来讨论一些实施方案。在此类实施方案中，焦点选择器可选地是：相应的手指或触笔触点、与手指或触笔触点对应的代表点(例如，相应触点的形心或与相应触点相关联的点)，或在触敏显示系统112上检测到的两个或更多个触点的形心。然而，可选地，在显示显示器450上的图中所示的用户界面时响应于在触敏表面451上检测到接触而对具有显示器450和单独触敏表面451以及焦点选择器的设备执行类似的操作。Although some of the following examples will be given with reference to input on touch-sensitive surface 451 that is separate from display 450, in some embodiments, as shown in FIG. combination). For ease of illustration, some implementations will be discussed with reference to operations performed on a device having a touch-sensitive display system 112 . In such embodiments, the focus selector is optionally: the corresponding finger or stylus contact, a representative point corresponding to the finger or stylus contact (e.g., the centroid of the corresponding contact or the point), or the centroid of two or more touch points detected on touch-sensitive display system 112. Alternatively, however, a device having a display 450 and a separate touch-sensitive surface 451 and a focus selector performs a similar operation in response to detecting a contact on the touch-sensitive surface 451 while displaying the user interface shown in the figure on the display 450. operation.

图6A-6Y示出了根据一些实施方案的用于动态地调整音频输出的呈现的示例性用户界面。更具体地，图6A-6Y示出了基于用户输入的量值来调整音频内容的音量和/或非音量特性的用户界面。例如，当在设备上播放歌曲时，用户可执行用于请求提供其他音频内容的手势。例如，可在音乐应用程序中播放歌曲，并且用户可在相同音乐应用程序或不同应用程序中请求另一个音频内容(例如，另一个音频内容可以是在即时消息应用程序中由另一个用户发送的视频的音频部分)。如以下更详细地解释的，音频内容的模糊包括动态地调整第一音频内容(例如，用户输入之前在设备上播放的歌曲)的非音量特性(例如，立体声平衡)。6A-6Y illustrate example user interfaces for dynamically adjusting the presentation of audio output, according to some embodiments. More specifically, FIGS. 6A-6Y illustrate user interfaces for adjusting volume and/or non-volume characteristics of audio content based on the magnitude of user input. For example, while a song is playing on the device, the user may perform a gesture to request additional audio content. For example, a song can be played in a music application, and the user can request another audio content in the same music application or a different application (for example, another audio content can be sent by another user in an instant messaging application audio portion of the video). As explained in more detail below, blurring of the audio content includes dynamically adjusting non-volume characteristics (eg, stereo balance) of a first audio content (eg, a song playing on the device prior to user input).

图6A-6H示出了以下示例：设备(例如，为了简单起见而称为设备100)在音乐应用程序中呈现第一首歌曲或专辑时，根据轻扫手势的长度而在音乐应用程序中的第二首歌曲或专辑中模糊。6A-6H illustrate the following example: when a device (e.g., referred to as device 100 for simplicity) presents the first song or album in the music application, according to the length of the swipe gesture, the Blurred in the second song or album.

图6A示出了在设备100的触摸屏112上显示的音乐应用程序用户界面602(为了视觉清晰起见，设备100的除了触摸屏112之外的剩余部分未被显示)。音乐应用程序用户界面602包括指示设备100当前正在播放贝多芬的月光奏鸣曲的标题604。即，设备100正在向音频系统提供声音信息606-1，使得音频系统可呈现贝多芬的月光奏鸣曲(或来自贝多芬的月光奏鸣曲的音轨/歌曲)。作为简写，短语“设备100呈现音频内容”等用于表示设备100向音频系统提供声音信息，使得音频系统可呈现与音频内容对应的音频输出。如参考图5所解释的，音频系统可集成到设备100中或与设备100分开。FIG. 6A shows a music application user interface 602 displayed on touch screen 112 of device 100 (for visual clarity, the remainder of device 100 other than touch screen 112 is not shown). Music application user interface 602 includes title 604 indicating that device 100 is currently playing Beethoven's Moonlight Sonata. That is, device 100 is providing sound information 606-1 to the audio system so that the audio system can present Beethoven's Moonlight Sonata (or a track/song from Beethoven's Moonlight Sonata). As shorthand, the phrase "device 100 presents audio content" and the like is used to indicate that device 100 provides sound information to an audio system such that the audio system can present audio output corresponding to the audio content. As explained with reference to FIG. 5 , the audio system may be integrated into the device 100 or separated from the device 100 .

如音频图608示意性所示的，贝多芬的月光奏鸣曲由声音信息606-1表示。声音信息606-1包括音量(由声音信息606-1在音频图表608的垂直轴线上的位置表示)以及除音量之外的音频特性，在该示例中该音频特性为左右立体声平衡(由声音信息606-1在音频图608的水平轴线上的位置表示的“L-R平衡”)。在一些实施方案中，非音量音频特性是滤波特性(例如，低通滤波器的截止频率和/或衰减因子)。As schematically shown in audio graph 608, Beethoven's Moonlight Sonata is represented by sound information 606-1. Sound information 606-1 includes volume (represented by the position of sound information 606-1 on the vertical axis of audio graph 608) and an audio characteristic other than volume, which in this example is left-right stereo balance (represented by sound information 606-1 on the vertical axis of audio graph 608). 606-1 on the horizontal axis of the audio graph 608 represents the "L-R balance"). In some implementations, the non-volume audio characteristic is a filter characteristic (eg, cutoff frequency and/or attenuation factor of a low-pass filter).

音乐应用程序用户界面602还包括音频内容表示610(例如，表示贝多芬的月光奏鸣曲的音频内容表示610-1；表示莫扎特的安魂曲的音频内容表示610-2；以及表示勃拉姆斯的女低音狂想曲的音频内容表示610-3)。音频内容表示610是各自占据触摸屏112上的用户界面602的相应区域的图形对象(也称为“图形用户界面对象”)。在各种情况下，音频内容表示610表示：歌曲、专辑、铃声、视频内容对象(例如，视频内容在文本即时消息窗口中出现的位置)、在文本即时消息窗口中出现的音频文件对象、或包括音频成分的任何其他类型的媒体内容。在此示例中，音频内容表示610是专辑的表示并且包括所显示的专辑封面。The music application user interface 602 also includes audio content representations 610 (e.g., an audio content representation 610-1 representing Beethoven's Moonlight Sonata; an audio content representation 610-2 representing Mozart's Requiem; and an audio content representation representing Brahms' The audio content of Alto Rhapsody indicates 610-3). Audio content representations 610 are graphical objects (also referred to as “graphical user interface objects”) that each occupy a corresponding area of user interface 602 on touch screen 112 . In each case, the audio content representation 610 represents: a song, an album, a ringtone, a video content object (e.g., where the video content appears in a text instant message window), an audio file object that appears in a text instant message window, or Any other type of media content that includes an audio component. In this example, audio content representation 610 is a representation of an album and includes displayed album art.

在图6A-6H所示的示例的开始，L-R平衡是均衡的，并且除了声音信息606-1之外，设备100不提供任何其他声音信息。At the beginning of the example shown in FIGS. 6A-6H , the L-R balance is equalized, and device 100 does not provide any other sound information except sound information 606-1.

如图6B所示，当音频系统正在呈现贝多芬的月光奏鸣曲时，设备100接收对应于呈现莫扎特的安魂曲的请求的输入612(例如，轻扫手势)。在此示例中，因为通过与莫扎特的安魂曲对应的音频内容表示610-2来接收输入612，所以输入612是用于呈现莫扎特的安魂曲的请求。As shown in FIG. 6B , while the audio system is presenting Beethoven's Moonlight Sonata, device 100 receives input 612 (eg, a swipe gesture) corresponding to a request to present Mozart's Requiem. In this example, because input 612 is received through audio content representation 610-2 corresponding to Mozart's Requiem, input 612 is a request to present Mozart's Requiem.

在一些实施方案中，响应于输入612的初始部分(例如，轻扫手势的轻微移动)，设备100暗示(例如，听觉地和/或视觉上)其将开始模糊音频。在一些实施方案中，如本文所使用的，术语“模糊音频”是指改变音频特性以便改变模糊音频在其播放时的突出性，以使得用户能够更好地区分与模糊音频同时播放的其他音频。当音频“模糊”时，音频中的“模糊渐入”对应于增大模糊音频的突出性(例如，通过增大低通滤波器的截止频率和/或使音频朝向中心声道偏移)，而“模糊渐出”对应于减少模糊音频的突出性(例如，通过减少低通滤波器的截止频率和/或将音频从中心声道移开)。在一些实施方案中，模糊渐入的音频特性包括音量。在一些实施方案中，模糊渐入的音频特性包括一个或多个非音量音频特性(例如，低通滤波器的截止频率或左/右平衡)。例如，设备100提供听觉提示，即通过用输入612的初始部分的量值(例如，轻扫手势的轻微移动)动态地调整声音信息606-1的L-R平衡，音频将开始模糊。在此示例中，声音信息606-1的L-R平衡向左轻微偏移。作为听觉提示的另一个示例，音量信息606-1的音量减小。在一些实施方案中，设备100还通过增大音频内容表示610-2(其在图6B中由围绕音频内容表示610-2的粗体边框示意性地示出)的视觉突出性来提供视觉提示。在一些实施方案中，增大视觉突出性包括视觉上模糊除音频内容表示610-2之外的其他音频内容表示(以及可选地同样视觉上模糊用户界面的其余部分)。在一些实施方案中，如下所述，在莫扎特的安魂曲听觉上“模糊渐入”之前提供这些提示。In some implementations, in response to an initial portion of input 612 (eg, a slight movement of a swipe gesture), device 100 suggests (eg, audibly and/or visually) that it will begin to blur audio. In some embodiments, as used herein, the term "blurred audio" refers to changing the characteristics of the audio so as to change the prominence of the blurred audio when it is played, so that the user can better distinguish other audio that is played at the same time as the blurred audio . When the audio is "fuzzy", "fuzzy fading" in the audio corresponds to increasing the prominence of the fuzzy audio (for example, by increasing the cutoff frequency of the low-pass filter and/or shifting the audio towards the center channel), Whereas "fuzzy fade-out" corresponds to reducing the prominence of fuzzy audio (for example, by reducing the cutoff frequency of the low-pass filter and/or moving the audio away from the center channel). In some implementations, the audio characteristic of the fade-in includes volume. In some implementations, the audio characteristics of the fade-in include one or more non-volume audio characteristics (eg, cutoff frequency or left/right balance of a low-pass filter). For example, device 100 provides an auditory cue that by dynamically adjusting the L-R balance of sound information 606-1 with the magnitude of the initial portion of input 612 (eg, a slight movement of a swipe gesture), the audio will begin to blur. In this example, the L-R balance of sound message 606-1 is shifted slightly to the left. As another example of an auditory cue, the volume of volume information 606-1 decreases. In some embodiments, device 100 also provides a visual cue by increasing the visual prominence of audio content representation 610-2 (which is schematically illustrated in FIG. 6B by a bold border surrounding audio content representation 610-2). . In some embodiments, increasing visual prominence includes visually obscuring audio content representations other than audio content representation 610-2 (and optionally visually obscuring the rest of the user interface as well). In some embodiments, these cues are provided prior to the auditory "fuzzy fade-in" of Mozart's Requiem, as described below.

在一些实施方案中，视觉效果(例如，模糊半径)与动态地调整的非音量音频特性结合地改变(例如，随着轻扫延长，用户界面602(除音频内容表示610-2之外)通过偏移声音信息606-1的L-R平衡以锁步方式模糊)。In some implementations, the visual effect (e.g., blur radius) changes in conjunction with dynamically adjusted non-volume audio characteristics (e.g., as the swipe is extended, user interface 602 (in addition to audio content representation 610-2) passes The L-R balance of the offset sound message 606-1 is blurred in lock-step).

如图6C-6D所示，设备100向音频系统提供声音信息606-2，以便与贝多芬的月光奏鸣曲同时呈现莫扎特的安魂曲。在一些实施方案中，响应于输入612的量值超过预先确定的阈值(例如，该轻扫移动超过小预先确定的量)，完成第二音频内容的“模糊渐入”。在一些实施方案中，设备100静态地呈现莫扎特的安魂曲；即，一旦莫扎特的安魂曲开始播放，其便以固定的音量和固定的L-R平衡(例如，均衡的L-R平衡)来进行。可替代地，如图所示，也根据输入612的量值来动态地调整莫扎特的安魂曲的呈现。例如，当用户将输入612进一步左移动时，莫扎特的安魂曲的L-R平衡更向左偏移(如图6C-6D中的音频图608中的声音信息606-2的向左偏移所示的)。继续根据轻扫手势来动态地调整贝多芬的月光奏鸣曲的呈现。例如，当用户将输入612进一步左移动时，贝多芬的月光奏鸣曲的L-R平衡更向左偏移(如图6C-6D中的音频图608中的声音信息606-1的向左偏移所示的)。这给予用户以下感觉：贝多芬的月光奏鸣曲正在从中心舞台向左偏移，并且莫扎特的安魂曲正在从右侧偏移到中心舞台上，从而占据月光奏鸣曲的位置。As shown in Figures 6C-6D, the device 100 provides sound information 606-2 to the audio system to present Mozart's Requiem concurrently with Beethoven's Moonlight Sonata. In some implementations, the "fuzzy fade-in" of the second audio content is accomplished in response to the magnitude of the input 612 exceeding a predetermined threshold (eg, the swipe movement exceeds a small predetermined amount). In some embodiments, device 100 presents Mozart's Requiem statically; that is, once Mozart's Requiem begins to play, it does so at a fixed volume and with a fixed L-R balance (e.g., a balanced L-R balance). Alternatively, as shown, the presentation of Mozart's Requiem is also dynamically adjusted according to the magnitude of the input 612 . For example, when the user moves input 612 further to the left, the L-R balance of Mozart's Requiem is shifted more to the left (as shown by the left shift of sound information 606-2 in audio graph 608 in FIGS. 6C-6D of). Continue to dynamically adjust the presentation of Beethoven's Moonlight Sonata based on swipe gestures. For example, when the user moves input 612 further to the left, the L-R balance of Beethoven's Moonlight Sonata is shifted more to the left (as shown by the left shift of sound information 606-1 in audio graph 608 in FIGS. 6C-6D of). This gives the user the feeling that Beethoven's Moonlight Sonata is being shifted from center stage to the left, and Mozart's Requiem is being shifted from the right onto center stage, taking the position of the Moonlight Sonata.

在一些实施方案中，设备100还根据轻扫手势的量值来动态地调整贝多芬的月光奏鸣曲的音量(例如，当输入612进一步向左移动时，设备100减小音量，如图6C-6D中的音频图608中声音信息606-1的向下偏移所表示的)。在一些实施方案中，设备100还根据轻扫手势的大小来动态地调整莫扎特的安魂曲的音量(例如，当输入612进一步向左移动时，设备100增大莫扎特的安魂曲的音量，如图6C-6D中的音频图608中声音信息606-2的向上偏移所表示的)。在一些实施方案中，贝多芬的月光奏鸣曲和莫扎特的安魂曲的音量大体上与显示器的预定义的区域(例如，整个显示器或显示器的中心区域)内分别显示的表示610-1和610-2的分数成比例。因此，当用户将音频内容表示610-2拉到显示器的中心上时，莫扎特的安魂曲变得在听觉上更加突出以及在听觉上更加居中。In some embodiments, device 100 also dynamically adjusts the volume of Beethoven's Moonlight Sonata based on the magnitude of the swipe gesture (e.g., when input 612 is moved further to the left, device 100 decreases the volume, as in FIGS. 6C-6D Indicated by the downward shift of sound message 606-1 in audio graph 608). In some embodiments, device 100 also dynamically adjusts the volume of Mozart's Requiem based on the size of the swipe gesture (e.g., when input 612 is moved further to the left, device 100 increases the volume of Mozart's Requiem, As indicated by the upward shift of sound information 606-2 in audio graph 608 in Figures 6C-6D). In some embodiments, the volumes of Beethoven's Moonlight Sonata and Mozart's Requiem are substantially the same as the volumes of representations 610-1 and 610- A fraction of 2 is proportional. Thus, when the user pulls the audio content representation 610-2 over the center of the display, Mozart's Requiem becomes more aurally prominent and more aurally centered.

图6E示出了设备100检测用户输入612的结束(例如，用户输入612已被释放)。因此，图6E中不再示出用户输入612。FIG. 6E shows device 100 detecting the end of user input 612 (eg, user input 612 has been released). Accordingly, user input 612 is no longer shown in FIG. 6E.

在一些实施方案中，响应于检测到用户输入612的结束，设备100基于确定输入612的量值超过(或已超过)预先确定的阈值来确定是否继续呈现第一音频内容(例如，并且停止呈现第二音频内容)，或反之亦然。In some embodiments, in response to detecting the end of user input 612, device 100 determines whether to continue presenting the first audio content (e.g., and stop presenting) based on determining that the magnitude of input 612 exceeds (or has exceeded) a predetermined threshold. second audio content), or vice versa.

图6F示出了在此示例中，因为输入612已拖动音频内容表示610-2，使得它比音频内容表示610-1更多地被显示，所以输入612的释放使得音频内容表示610-2移动到屏幕中间的位置，其中设备100仅显示莫扎特的安魂曲。因此，响应于检测到输入612的结束，设备100停止呈现第一音频内容(例如，月光奏鸣曲)并且继续呈现第二音频内容(莫扎特的安魂曲)。在一些实施方案中，设备100完成对莫扎特的安魂曲的动态调整(例如，在释放输入612之后，在0.5秒的时间段内逐渐改变L-R平衡和音量，使得以预设置的音量来均衡地呈现莫扎特的安魂曲)。这通过图6E-6F中的音频图608中的声音信息606-2的向上和向左偏移来示意性地示出。在一些实施方案中，响应于检测到输入612的结束，视觉效果(例如，模糊)也被反转。Figure 6F shows that in this example, since input 612 has dragged audio content representation 610-2 so that it is displayed more than audio content representation 610-1, release of input 612 causes audio content representation 610-2 Move to a location in the middle of the screen where device 100 displays only Mozart's Requiem. Thus, in response to detecting the end of input 612, device 100 stops presenting the first audio content (eg, Moonlight Sonata) and continues presenting the second audio content (Mozart's Requiem). In some embodiments, device 100 completes dynamic adjustments to Mozart's Requiem (e.g., after releasing input 612, gradually changes L-R balance and volume over a period of 0.5 seconds such that the volume is balanced at a preset volume Presenting Mozart's Requiem). This is schematically shown by the upward and leftward shifting of sound information 606-2 in audio graph 608 in Figures 6E-6F. In some implementations, in response to detecting the end of input 612, the visual effect (eg, blurring) is also reversed.

图6G-6H示出了如果输入612没有拖动音频内容表示610-2以使其比音频内容表示610-1更多地被显示则在此示例中将会发生什么。因此，图6G示出了在紧接在释放比输入612更短但类似于输入612的输入之后的用户界面602的状态。作为响应，如图6H所示，设备100停止呈现(例如，致使音频系统停止呈现)莫扎特的安魂曲，并且重新开始仅呈现贝多芬的月光奏鸣曲。然后呈现贝多芬的月光奏鸣曲，而无需对其进行动态地调整(例如，在0.5秒内反转L-R平衡和音量的改变，以使得均衡地并且以预设置的音量来呈现贝多芬的月光奏鸣曲)。这通过图6G-6H中的音频图608中的声音信息606-1的向上和向左偏移来示意性地示出。在一些实施方案中，响应于检测到输入的结束，视觉效果(例如，模糊)也被反转。6G-6H illustrate what would happen in this example if input 612 did not drag audio content representation 610-2 to be displayed more than audio content representation 610-1. Accordingly, FIG. 6G shows the state of user interface 602 immediately after releasing an input that is shorter than, but similar to, input 612 . In response, as shown in FIG. 6H , device 100 stops rendering (eg, causes the audio system to stop rendering) Mozart's Requiem and resumes rendering only Beethoven's Moonlight Sonata, as shown in FIG. 6H . Then render Beethoven's Moonlight Sonata without dynamically adjusting it (e.g. invert L-R balance and volume change in 0.5 seconds so that Beethoven's Moonlight Sonata is rendered evenly and at preset volume) . This is schematically shown by the upward and leftward shifting of sound information 606-1 in audio graph 608 in Figures 6G-6H. In some implementations, the visual effect (eg, blurring) is also reversed in response to detecting the end of the input.

在一些实施方案中，当输入612的量值满足大于第一预先确定的阈值的第二预先确定的阈值时，设备100在视觉上和听觉上将第二音频输出“弹出”到适当的位置(例如，存在第一阈值，其中如果释放输入，则第二音频输出移动到适当位置，并且存在第二阈值，其中甚至在释放输入之前，第二音频输出弹出到适当的位置)。In some embodiments, when the magnitude of input 612 satisfies a second predetermined threshold greater than the first predetermined threshold, device 100 visually and audibly "pops" the second audio output into place ( For example, there is a first threshold where the second audio output moves into place if the input is released, and a second threshold where the second audio output pops into place even before the input is released).

在一些实施方案中，作为图6E-6H所示的示例的替代方案，响应于检测到输入612的结束，设备100继续呈现调整的第一音频输出并且继续呈现第二音频输出。In some embodiments, as an alternative to the example shown in FIGS. 6E-6H , in response to detecting the end of input 612 , device 100 continues to present the adjusted first audio output and continues to present the second audio output.

图6I-6O示出了以下示例：设备(例如，为了简单起见而称为设备100)在音乐应用程序中呈现歌曲或专辑时，根据按压输入的强度而在即时消息应用程序(例如，与音乐应用程序不同的应用程序)中的发送到用户的视频的音频部分中模糊。图6I-60所示的示例的各个方面类似于图6A-6H所示的示例。在此不重复这些细节。图6I-60所示的示例与图6A-6H所示的示例之间的差异在于：在图6I-60中，输入是按压输入，其量值是按压输入的强度；并且另外，在图6I-60中，设备100模糊来自两个不同应用(例如，音乐应用程序和即时消息应用程序)的音频。6I-6O show the following example: when a device (e.g., referred to as device 100 for simplicity) presents a song or an album in a music application, the instant messaging application (e.g., associated with music) The audio portion of the video sent to the user in different applications) is blurry. Aspects of the example shown in Figures 6I-60 are similar to the example shown in Figures 6A-6H. These details are not repeated here. The difference between the example shown in FIGS. 6I-60 and the example shown in FIGS. 6A-6H is that in FIG. 6I-60, the input is a press input whose magnitude is the intensity of the press input; - 60, the device 100 blurs audio from two different applications (eg, a music application and an instant messaging application).

此外，图6I-60示出了通过动态地调整低通滤波器截止频率而不是L-R平衡来模糊音频。为此，图6I-60包括音频图613。音频图613的垂直轴线表示音量，并且水平轴线表示低通滤波器截止频率。Furthermore, Figures 6I-60 illustrate blurring audio by dynamically adjusting the low-pass filter cutoff frequency instead of L-R balance. To this end, FIG. 6I-60 includes an audio graph 613 . The vertical axis of the audio graph 613 represents volume, and the horizontal axis represents the low-pass filter cutoff frequency.

图6I示出了用于在触摸屏112上显示的即时消息应用程序的用户界面614。用户界面614显示消息616，该消息616中的一些消息由设备100的用户(例如，消息616-1和616-2)接收，并且该消息616中的一些消息由设备100的用户发送(例如，消息616-3)。用户界面614包括指示会话中的参与者的头像618(例如，其头像为618-1的“Alex”、以及其头像为618-2的“Tina”)。例如，Tina向Alex发送为视频的表示的消息616-1。FIG. 61 shows a user interface 614 for an instant messaging application displayed on the touch screen 112 . User interface 614 displays messages 616, some of which were received by a user of device 100 (e.g., messages 616-1 and 616-2) and some of which were sent by a user of device 100 (e.g., Message 616-3). The user interface 614 includes avatars 618 indicating the participants in the session (eg, "Alex" whose avatar is 618-1, and "Tina" whose avatar is 618-2). For example, Tina sends Alex a message 616-1 that is a representation of a video.

在图6I中，音频图613包括声音信息606-3的表示，从而意味着设备正在呈现音频内容。在这个示例中，声音信息606-3对应于勃拉姆斯的女低音狂想曲，因此首先设备100通过与即时消息应用程序分开或不同的音乐应用程序来呈现勃拉姆斯的女低音狂想曲。以特定音量以及通过设置为高值的低通滤波器截止频率(下文中称为“截止频率”)来提供勃拉姆斯的女低音狂想曲。在一些实施方案中，开始时的截止频率高于人们可听的最大频率(例如，约20kHz)，使得当不执行音频模糊时，低通滤波器对音频输出没有可听见的影响。在一些实施方案中，在开始时(即，当不执行音频模糊时)不应用低通滤波器。相反，当音频模糊开始时，打开低通滤波器。为了简单起见，图6I示出了通过开始时设置为高值的截止频率来呈现的声音信息606-3。In FIG. 6I , audio graph 613 includes a representation of sound information 606-3, meaning that the device is presenting audio content. In this example, sound information 606-3 corresponds to Brahms' Rhapsody for Alto, so first device 100 presents Brahms' Rhapsody for Alto through a separate or different music application than the instant messaging application. Brahms' Alto Rhapsody is provided at a certain volume and with a low-pass filter cutoff frequency (hereinafter referred to as "cutoff frequency") set to a high value. In some embodiments, the cutoff frequency is initially higher than the maximum human audible frequency (eg, about 20 kHz), so that the low pass filter has no audible effect on the audio output when no audio blurring is performed. In some implementations, no low-pass filter is applied initially (ie, when no audio blurring is performed). Instead, turn on the low-pass filter when audio fuzz begins. For simplicity, FIG. 61 shows the sound information 606-3 presented with the cutoff frequency initially set to a high value.

如图6J-6K所示，当设备100呈现女低音狂想曲时，设备100接收与用于呈现视频消息616-1(包括视频消息616-1的音频)的请求对应的输入620(例如，输入是在触摸屏112的与视频消息616-1对应的区域上的输入)。6J-6K, when device 100 is presenting Alto Rhapsody, device 100 receives input 620 corresponding to a request to present video message 616-1 (including the audio of video message 616-1) (e.g., the input is input on the area of the touch screen 112 corresponding to the video message 616-1).

在此示例中，输入620是触摸屏112上的按压输入(例如，按压并保持输入)。触摸屏112具有用于检测接触的强度的一个或多个传感器。输入620的强度被表示在强度图622上。在图6J中，输入620的强度高于提示阈值(例如，IT_H)。因此，设备100开始将视频消息616-1的音频部分在听觉上模糊到勃拉姆斯的女低音狂想曲中。即，当输入620的强度高于提示阈值(例如，IT_H)但没有高于窥视阈值(例如，IT_L)时，设备100在输入620的强度增大时减小女低音狂想曲的音量并且减小截止频率。这些效果通过音频图613中的声音信息606-3的向下和向左偏移示意性地示出。减小低通滤波器的截止频率的效果是滤除女低音狂想曲的越来越多的高频分量从而只剩下其低音。In this example, input 620 is a press input (eg, a press and hold input) on touchscreen 112 . The touch screen 112 has one or more sensors for detecting the intensity of a contact. The intensity of the input 620 is represented on an intensity graph 622 . In FIG. 6J, the intensity of the input 620 is above the cue threshold (eg, IT_H ). Accordingly, device 100 begins to aurally blur the audio portion of video message 616-1 into Brahms' Alto Rhapsody. That is, when the intensity of input 620 is above a cue threshold (e.g., IT_H ) but not above a peek threshold (e.g., IT_L ), device 100 decreases the volume of Rhapsody Alto and small cutoff frequency. These effects are schematically illustrated by the downward and leftward shifting of the sound information 606 - 3 in the audio graph 613 . The effect of reducing the cutoff frequency of the low pass filter is to filter out more and more of the high frequency content of the Alto Rhapsody so that only its bass is left.

同时，随着接触620的强度增大(如与视频消息616-1对应的声音信息606-4的向上箭头所表示的)，视频消息616-1的音量增大(例如，成比例地)。声音信息606-4实际上未被低通滤波(例如，低通滤波器通过高于人类可听范围的截止频率对其进行滤波，或者其根本未被低通滤波)。Concurrently, the volume of video message 616-1 increases (eg, proportionally) as the intensity of contact 620 increases (as indicated by the upward arrow of sound information 606-4 corresponding to video message 616-1). The sound information 606-4 is not actually low-pass filtered (eg, a low-pass filter filters it with a cutoff frequency above the human audible range, or it is not low-pass filtered at all).

在一些实施方案中，只要输入620的强度已高于提示阈值(例如，IT_H)但尚未超过窥视阈值(例如，IT_L)，则视频消息616-1的视觉突出性随着输入620的强度而动态改变(例如，视频的视觉突出性和对应音频的听觉突出性随着强度的增大而增大，并且随着强度的减小而减小)。例如，如图6J-6K所示的，增大输入620的强度增大了视频消息616-1的尺寸，使视频消息616-1居中，并且通过与输入620的强度成比例的模糊半径使得用户界面614中除视频消息616-1之外的任何消息模糊(在这些图中的增大的模糊半径由除视频消息616-1之外覆盖所有用户界面614的图案的减小的透明度示意性地表示)。In some embodiments, the visual prominence of video message 616-1 increases with the intensity of input 620 as long as the intensity of input 620 is above a cue threshold (e.g., IT_H ) but not yet above a peek threshold (e.g., IT_L ). Instead, it changes dynamically (eg, the visual prominence of the video and the auditory prominence of the corresponding audio increase as the intensity increases and decrease as the intensity decreases). For example, as shown in FIGS. 6J-6K , increasing the intensity of input 620 increases the size of video message 616-1, centers video message 616-1, and makes the user blur with a blur radius that is proportional to the intensity of input 620. Any messages in interface 614 other than video message 616-1 are blurred (increased blur radius in these figures is schematically illustrated by the decreased transparency of the pattern covering all user interface 614 except video message 616-1 express).

通过比较图6J和图6K可看出IT_H和IT_L之间的增大的强度的影响。在一些实施方案中，用户可通过增大和减少输入620的强度来反复地净化女低音狂想曲和视频消息616-1的音量(以及女低音狂想曲的截止频率)。例如，如果图6J和6K被反转，则声音信息606-3的音量和截止频率将增大，并且声音信息606-4的音量将减小。The effect of increased intensity between IT_H and IT_L can be seen by comparing Figure 6J and Figure 6K. In some implementations, the user can repeatedly clear the volume of the Rhapsody Alto and video message 616 - 1 (and the cutoff frequency of Rhapsody Alto) by increasing and decreasing the intensity of the input 620 . For example, if FIGS. 6J and 6K were inverted, the volume and cutoff frequency of sound information 606-3 would increase, and the volume of sound information 606-4 would decrease.

在此示例中，当设备100呈现视频消息616-1的音频输出时，设备100还呈现视频消息616-1的视频输出。因此，图6J-60示出了视频进展(例如，视频消息616-1是鸟围绕框架飞翔的简单视频)。In this example, when device 100 presents the audio output of video message 616-1, device 100 also presents the video output of video message 616-1. Thus, Figure 6J-60 shows the video progression (eg, video message 616-1 is a simple video of a bird flying around a frame).

如图6L-6M所示，当输入620的强度达到“窥视”阈值时，设备100通过预设置的截止频率和第一预设置的音量(例如，低音量)来呈现低音狂想曲，并且通过第二预设置的音量(例如，比第一预设置的音量更高的音量)来呈现视频消息616-1，而与输入620的强度无关(例如，只要输入620的强度保持低于“弹出”阈值，在此示例中该“弹出”阈值为IT_D)。例如，设备100检测到输入620的强度增大高于IT_L(图6L)，随后输入620的强度减少低于IT_L(图6M)，但是在音频图613中将声音信息606-3和声音信息606-4维持在其“窥视”位置处(例如，即使接触的强度减少到低于IT_L，视频的视觉突出性和对应音频的听觉突出性也被维持在至少6K所示的级别)。效果是主要地呈现来自视频消息616-1的音频，其中仅女低音狂想曲的安静低音背景保持可听。As shown in FIGS. 6L-6M , when the intensity of the input 620 reaches the "peep" threshold, the device 100 presents bass rhapsody through a preset cutoff frequency and a first preset volume (eg, low volume), and through a second preset volume. Preset volume (e.g., a volume higher than the first preset volume) to present video message 616-1, regardless of the intensity of input 620 (e.g., as long as the intensity of input 620 remains below the "pop" threshold, In this example the "eject" threshold is IT_D ). For example, the device 100 detects that the intensity of the input 620 increases above IT_L (FIG. 6L), and then the intensity of the input 620 decreases below the IT_L (FIG. 6M), but in the audio map 613 the sound information 606-3 and the sound Information 606-4 is maintained at its "peek" position (eg, the visual prominence of the video and the auditory prominence of the corresponding audio are maintained at a level of at least 6K even if the intensity of the contact is reduced below IT_L ). The effect is to render predominantly the audio from video message 616-1, where only the quiet bass background of the Alto Rhapsody remains audible.

此外，在输入620的强度达到“窥视”阈值之后，视觉改变被锁定。在图6L中，视频消息616-1居中定位，其中用户界面614的剩余部分严重模糊。当输入620的强度减小到低于IT_L时，在图6M中保持这种模糊以及视频消息616-1的中心位置。Furthermore, after the intensity of input 620 reaches the "peek" threshold, the visual change is locked. In FIG. 6L, video message 616-1 is centered, with the remainder of user interface 614 severely blurred. This blurring and the center position of video message 616-1 is maintained in FIG. 6M when the intensity of input 620 is reduced below IT_L.

然而，在一些实施方案中，输入620在图6J-6M中的任何一点处的释放将导致设备100停止呈现视频消息616-1以及恢复完全呈现女低音狂想曲(例如，以第二预设置的音量并且无滤波)，并且可选地返回到图6I中显示的用户界面的状态。However, in some embodiments, release of input 620 at any point in FIGS. 6J-6M will cause device 100 to stop presenting video message 616-1 and resume full presentation of Alto Rhapsody (e.g., at a second preset volume and no filtering), and optionally return to the state of the user interface shown in Figure 6I.

图6N-60示出了在一些实施方案中，当输入620的强度高于弹出阈值(例如，IT_D)时，设备100将视频消息616-1“弹出”到适当的位置。即使输入620的强度随后下降到低于IT_D或者输入端620完全结束，一旦输入620的强度高于深按压阈值IT_D，视频消息616-1的音量便保持在第二预设置的音量并且保持未被滤波(图60)。在一些实施方案中，设备100将视频消息616-1在视觉上弹出到适当的位置(例如，视频消息616-1扩展以便占据整个屏幕或大部分屏幕)。此外，设备100停止呈现第一音频输出(例如，女低音狂想曲)。6N-60 illustrates that, in some embodiments, device 100 "pops" video message 616-1 into place when the intensity of input 620 is above a pop threshold (eg, IT_D ). Even if the intensity of input 620 subsequently drops below IT_D or input 620 ends completely, once the intensity of input 620 is above the deep press threshold IT_D , the volume of video message 616-1 remains at the second preset volume and remains Unfiltered (Figure 60). In some embodiments, device 100 visually pops video message 616-1 into place (eg, video message 616-1 expands to take up the entire screen or a majority of the screen). Additionally, device 100 stops presenting the first audio output (eg, Alto Rhapsody).

图6P-6Y示出了以下示例：根据一些实施方案，由于接触的特征强度改变，设备(例如，为了简单起见而称为设备100)通过动态地改变媒体项的一个或多个音频特性的集合来预览来自媒体项的音频。例如，至少对于一定范围的接触强度，当设备100接收到歌曲的表示上的接触时，设备100将以与接触的强度成比例的音量来播放歌曲。在一些实施方案中，当接触达到第一强度阈值时，即使接触的强度随后下降，设备100也将通过把音量维持在预设置的级别来“窥视”预览媒体项。在一些实施方案中，当接触达到第二强度阈值时，设备100将执行另一操作(例如，除了预览媒体项之外)，诸如执行选择操作。6P-6Y illustrate the following example: According to some embodiments, due to the change in the characteristic strength of the contact, the device (for example, referred to as the device 100 for simplicity) dynamically changes the set of one or more audio characteristics of the media item. to preview the audio from the media item. For example, at least for a range of contact intensities, when device 100 receives a contact on a representation of a song, device 100 will play the song at a volume proportional to the intensity of the contact. In some embodiments, when a contact reaches a first intensity threshold, device 100 will "peek" at a preview media item by maintaining the volume at a pre-set level, even if the intensity of the contact subsequently decreases. In some embodiments, when the contact reaches the second intensity threshold, device 100 will perform another operation (eg, in addition to previewing the media item), such as performing a selection operation.

图6P-6Y还示出了以下示例：响应于输入，设备100隐藏(例如，模糊)整个用户界面(除所选择的用户界面对象之外)或仅隐藏用户界面的一部分。在一些实施方案中，当接触满足基于强度的激活标准时，执行遮蔽用户界面(例如，整个用户界面或仅用户界面的一部分)。例如，强度上升到高于提示阈值IT_H的按压并保持输入导致遮蔽，而轻击手势导致执行不同的操作(例如，对用户界面对象的选择操作)。6P-6Y also illustrate examples in which, in response to an input, device 100 hides (eg, blurs) the entire user interface (except for the selected user interface object) or only a portion of the user interface. In some implementations, obscuring the user interface (eg, the entire user interface or only a portion of the user interface) is performed when the contact meets intensity-based activation criteria. For example, a press-and-hold input whose intensity rises above the hint threshold IT_H results in shading, while a tap gesture results in a different operation being performed (eg, a selection operation on a user interface object).

图6P示出了用于即时消息应用程序的用户界面640。用户界面640包括交互区域642-1。交互区域642-1是包括会话参与者(例如，Alex和Tina)之间的多个消息644(其中的消息644-1、消息644-2和消息644-3是代表性的)的会话区域。用户可通过滚动通过消息644(例如，会话区域中的位置由滚动条646示出)、或通过与各个消息644交互(例如，在消息644-3上轻击以便播放视频，或在消息644-3上按压(例如并保持)以便“窥视”视频，如参考图6I-60所述的)来与交互区域642-1进行交互。交互区域642-1还包括多个示能表示(例如，图标648-1、图标648-2、图标648-3和图标648-4，其各自能够实现特定的设备功能)和架650(例如，用户已输入但尚未发送给其他会话参与者的内容的展现区域)。Figure 6P shows a user interface 640 for an instant messaging application. User interface 640 includes interaction area 642-1. Interaction area 642-1 is a conversation area that includes a plurality of messages 644 (of which message 644-1, message 644-2, and message 644-3 are representative) between conversation participants (eg, Alex and Tina). The user can scroll through messages 644 (e.g., the position in the conversation area is shown by scroll bar 646), or by interacting with individual messages 644 (e.g., tap on message 644-3 to play a video, or tap on message 644-3 to 3 (eg and hold) to "peek" the video, as described with reference to Figures 6I-60) to interact with the interaction area 642-1. Interaction area 642-1 also includes a plurality of affordances (e.g., icon 648-1, icon 648-2, icon 648-3, and icon 648-4, each of which enables a particular device function) and shelf 650 (e.g., display area for content that the user has entered but not yet sent to other session participants).

图6P示出了图标648-3上的输入652(例如，轻击手势)。如图6Q所示，轻击手势652使得用户界面640显示交互区域642-2(例如，与交互区域642-1不同的独立交互区域)。在一些实施方案中，交互区域642-1是用于主应用程序(例如，即时消息应用程序)的用户界面，并且交互区域642-2被配置成显示来自在主应用程序内操作的不同微型应用程序的内容。用户可在微型应用程序之间轻扫，从而改变在交互区域642-2中显示的微型应用程序。交互区域642-2还包括提出了将在交互区域642-2中显示的微型应用程序的列表的示能表示659。指示符654指示当前在交互区域642-2中显示哪个微型应用程序(例如，向左轻扫将导致在区域642-2中显示不同的微型应用程序，并且指示符654中的实心点将其向右移动)。FIG. 6P shows input 652 (eg, a tap gesture) on icon 648-3. As shown in FIG. 6Q , tap gesture 652 causes user interface 640 to display interaction area 642 - 2 (eg, a separate interaction area from interaction area 642 - 1 ). In some embodiments, interactive area 642-1 is a user interface for the main application (e.g., an instant messaging application), and interactive area 642-2 is configured to display information from different mini-applications operating within the main application. the content of the program. A user may swipe between mini-apps, thereby changing the mini-app displayed in interaction area 642-2. Interactive area 642-2 also includes affordance 659 presenting a list of mini-apps to be displayed in interactive area 642-2. Indicator 654 indicates which mini-app is currently displayed in interaction area 642-2 (e.g., swiping left will cause a different mini-app to be displayed in area 642-2, and a solid dot in indicator 654 will move it toward to move right).

在此示例中，微型应用程序是用于选择在会话参与者之间的会话中将共享的媒体的媒体选择微型应用程序656。(例如，显示在设备100上播放的最近30首歌曲的可滚动表单)。媒体选择微型应用程序包括媒体项658的表示(例如，媒体项658-1至658-4的表示)。在一些实施方案中，媒体项是歌曲、专辑、视频等。表示658-3包括有关媒体项(例如，艺术家、歌曲名称)和专辑封面图片的显示信息。在一些实施方案中，相应表示658上的轻击手势(未示出)通过将其放置在架650中来选择相应表示658(例如，作为音频消息，使得其被分段以用于发送到会话)。在一些实施方案中，相应表示658上的轻击手势(未示出)通过播放媒体项(例如，在设备100上本地播放)来选择相应的表示658。In this example, the mini-app is a media selection mini-app 656 for selecting media to be shared in a session between session participants. (eg, displaying a scrollable list of the last 30 songs played on device 100). The media selection mini-app includes representations of media items 658 (eg, representations of media items 658-1 through 658-4). In some implementations, the media item is a song, album, video, or the like. Representation 658-3 includes display information about the media item (eg, artist, song title) and album cover art. In some embodiments, a tap gesture (not shown) on the corresponding representation 658 selects the corresponding representation 658 by placing it in the shelf 650 (e.g., as an audio message so that it is segmented for sending to the session ). In some implementations, a tap gesture (not shown) on a respective representation 658 selects the respective representation 658 by playing the media item (eg, locally on device 100).

图6R示出了包括表示658-2上的接触(例如，表示658-2上的按压并保持手势)的输入660。如强度图668所示的，通过具有高于提示阈值IT_H的接触强度，按下并保持手势满足媒体提示标准(例如，基于强度的激活标准)。因此，设备100开始播放媒体项的与表示658-2对应的一部分(例如，开始部分或被选择为包含媒体项的可识别部分的代表部分)。在音频图662中通过音频输出664来表示媒体项的播放。音频图662示出了媒体项的音量和时间关系。FIG. 6R shows input 660 including a contact on representation 658-2 (eg, a press and hold gesture on representation 658-2). As shown in the intensity graph 668, the press and hold gesture satisfies the media prompting criteria (eg, intensity-based activation criteria) by having a contact intensity above the prompting threshold IT_H. Accordingly, device 100 begins playing a portion of the media item corresponding to representation 658-2 (eg, the beginning portion or a representative portion selected to contain an identifiable portion of the media item). Playing of the media item is represented in audio graph 662 by audio output 664 . Audio graph 662 shows the volume and time relationship of media items.

如图6R-6S所示，当媒体项正在播放时，设备100随着接触的特征强度改变而动态地改变媒体项的音量(例如，媒体项的音量随着接触的特征强度增大而增大，并且随着接触的特征强度减少而减少)。例如，从图6R到图6S的接触的强度增大(例如，从正好高于IT_H到正低于IT_L)伴随着对应的音量增大(图6R-6V中的强度图668中的箭头示出了来自前图的强度的改变；类似地，图6R-6U中的音频图662中的实线示出了该图的与强度图668中的箭头对应的改变，而虚线表示前图中的改变)。如果强度被反转(例如，如果图6S在图6R之前发生)，则媒体项的音量将减小。As shown in Figures 6R-6S, when the media item is playing, the device 100 dynamically changes the volume of the media item as the characteristic intensity of the contact changes (e.g., the volume of the media item increases as the characteristic intensity of the contact increases , and decreases as the characteristic strength of the contact decreases). For example, the increase in intensity of the contact from FIG. 6R to FIG. 6S (e.g., from just above IT_H to just below IT_L ) is accompanied by a corresponding increase in volume (arrows in intensity graph 668 in FIGS. 6R-6V The change in intensity from the previous graph is shown; similarly, the solid line in the audio graph 662 in FIGS. changes). If the intensity is reversed (eg, if Figure 6S occurs before Figure 6R), the volume of the media item will decrease.

在一些实施方案中，当在媒体项开始播放之前已产生不同的音频输出时，如参考图6A-60、方法800、方法840和图8A-8C所描述的，设备100将两个音频输出模糊在一起。In some embodiments, when different audio outputs have been produced before the media item begins playing, as described with reference to FIGS. 6A-60, method 800, method 840, and FIGS. together.

此外，图6R-6S示出了当媒体项的音量改变时，设备100动态地改变交互区域642-2的外观。例如，当接触的特征强度改变时，设备100动态隐藏(例如，模糊)交互区域642-2(例如，以具有音频的改变的锁步方式，使得随着接触的特征强度增大，模糊半径也增大)。因此，从图6R到图6S的接触的强度增大伴随着模糊度的对应增大(当设备从图6R移动到图6S时，这些图中的增大的模糊度由覆盖交互区域642-2中的其他内容的图案的减小的透明度示意性地表示)。在此示例中，设备100动态隐藏交互区域642-2而无需隐藏交互区域642-1(例如，以便指示该提示涉及用户界面的微型应用程序部分并且不涉及整个用户界面)。Additionally, Figures 6R-6S illustrate that device 100 dynamically changes the appearance of interaction area 642-2 when the volume of a media item changes. For example, as the characteristic strength of the contact changes, device 100 dynamically hides (e.g., blurs) interaction region 642-2 (e.g., in a changing lock-step fashion with audio such that as the characteristic strength of the contact increases, the blur radius also increases. increase). Thus, an increase in the intensity of the contact from FIG. 6R to FIG. 6S is accompanied by a corresponding increase in blurriness (as the device moves from FIG. 6R to FIG. The reduced transparency of the pattern of other content in is schematically indicated). In this example, device 100 dynamically hides interaction area 642-2 without hiding interaction area 642-1 (eg, to indicate that the hint relates to a mini-app portion of the user interface and not to the entire user interface).

此外，未被隐藏的表示658-2随着接触强度的增大而在尺寸上增大并且移动到交互区域642-2的中心。在此示例中，只要接触强度未达到“窥视”强度阈值(例如，IT_L)，用户便可通过在“提示”范围的强度内(例如，在IT_H与IT_L之间)改变接触的强度而反复地净化视觉改变和听觉改变。Furthermore, the unconcealed representation 658-2 increases in size and moves to the center of the interaction area 642-2 as the contact intensity increases. In this example, as long as the contact intensity does not reach the "peek" intensity threshold (e.g., IT_L ), the user can change the intensity of the contact by varying the intensity within the "tip" range (e.g., between IT_H and IT_L ). And repeatedly purify visual changes and auditory changes.

图6T-6U示出了接触达到窥视阈值的结果(例如，设备100检测到高于IT_L的接触的特征强度增大的结果)。在此示例中，一旦接触的强度达到窥视阈值，音频输出664的音量便保持固定(例如，锁定到预设置的级别，诸如用户将设备100设置成的任何音量)。因此，IT_L与IT_D之间的接触的强度增大不导致音量的增大。类似地，如图6U所示，随后检测到低于IT_L的接触的特征强度的减小对体积没有影响(例如，只要连续维持接触；在某些实施方案中，提升逐渐或立即停止媒体项的呈现并且将视觉变化反转到交互区域642-2)。6T-6U illustrate the results of a contact reaching the peek threshold (eg, the result of device 100 detecting an increase in the characteristic strength of a contact above IT_L ). In this example, once the intensity of the contact reaches the peek threshold, the volume of the audio output 664 remains fixed (eg, locked to a pre-set level, such as whatever volume the user has set the device 100 to). Therefore, an increase in the intensity of the contact between IT_L and IT_D does not result in an increase in volume. Similarly, as shown in Figure 6U, subsequent reductions in the characteristic intensity of contacts detected below the IT_L have no effect on volume (e.g., as long as the contact is continuously maintained; in some embodiments, lifting gradually or immediately stops the media item and reverse the visual change to the interactive area 642-2).

在一些实施方案中，一旦接触达到IT_L，设备100便显示指示661，即接触的特征强度增大将使得设备相对于媒体项来执行选择操作(例如，话语“3D TOUCH TO SHARE”)。低于IT_L的接触的强度的随后减少不修改互动区域642-2的其余部分的遮蔽、表示658-2的尺寸、或“3D TOUCH TO SHARE”指示。In some embodiments, once the contact reaches IT_L , the device 100 displays an indication 661 that an increase in the characteristic strength of the contact will cause the device to perform a selection operation with respect to the media item (eg, the utterance "3D TOUCH TO SHARE"). Subsequent reductions in the intensity of touches below IT_L do not modify the shading of the remainder of the interaction area 642-2, the size of the representation 658-2, or the "3D TOUCH TO SHARE" indication.

图6V示出了设备100检测到高于“弹出”阈值(例如，IT_D)的接触的特征强度增大。因此，设备100停止播放媒体项(因此，图6V-6X中没有音频图662)并且针对媒体项来执行选择操作(例如，设备100将媒体项放置在架650中)。FIG. 6V shows that the device 100 detects an increase in the characteristic intensity of a contact above a "pop-up" threshold (eg, IT_D ). Accordingly, device 100 stops playing the media item (hence, there is no audio map 662 in FIGS. 6V-6X ) and performs a selection operation for the media item (eg, device 100 places the media item in shelf 650 ).

在一些实施方案中，如图所示，在检测到超过IT_D的接触的强度之后立即执行选择操作。输入660然后变为无效的。换句话说，尽管在图6V中输入660被示出在原始表示658-2上，但输入660对原始表示658-2没有影响。用户可通过释放输入660并且输入新的用户输入来重新选择原始表示658-2。In some embodiments, as shown, the selection operation is performed immediately after the intensity of the contact exceeding IT_D is detected. Entering 660 then becomes invalid. In other words, although input 660 is shown on original representation 658-2 in FIG. 6V, input 660 has no effect on original representation 658-2. The user may reselect original representation 658-2 by releasing input 660 and entering a new user input.

在图6V中，媒体项现在由音频消息644-4表示，其对应于如图6Q-6U所示的表示658-2。在一些实施方案中，音频消息644-4看起来与图6R所示的表示658-2相同。此外，设备100停止遮蔽交互区域642-2(例如，反转模糊)。In FIG. 6V, the media item is now represented by audio message 644-4, which corresponds to representation 658-2 as shown in FIGS. 6Q-6U. In some implementations, the audio message 644-4 looks the same as the representation 658-2 shown in Figure 6R. In addition, device 100 ceases to obscure interaction region 642-2 (eg, reverse blur).

将内容放置在架650中也导致设备100显示“发送”按钮670。图6W示出了选择发送按钮670的用户输入672(例如，轻击手势)，其将音频消息644-4发送到会话参与者并且导致将音频消息644-4添加到会话区域，如图6X所示。Placing content in shelf 650 also causes device 100 to display a “Send” button 670 . FIG. 6W shows user input 672 (e.g., a tap gesture) selecting the send button 670, which sends the audio message 644-4 to the session participants and causes the audio message 644-4 to be added to the session area, as shown in FIG. 6X Show.

图6Y示出了包括在音频消息644-4上的接触(例如，音频消息644-4上的按压并保持手势)的输入674。因为输入674接触中的接触高于IT_H，所以除了音频消息644-4之外，设备100隐藏交互区域642-1和交互区域642-2。在一些实施方案中，响应于在高于IT_H的接触下的按压并保持输入，设备100还以类似于先前描述的方式的方式(例如，包括动态地改变音频)来预览音频消息644-4。相比之下，响应于轻击手势(例如，以低于IT_H的强度)，设备100相对于音频消息644-4来执行选择操作(例如，播放整个音频消息644-4)，而无需遮蔽应用程序的第二交互区域并且无需遮蔽应用程序的第一交互区域。FIG. 6Y shows input 674 that includes contact on audio message 644-4 (eg, a press and hold gesture on audio message 644-4). Because the contact in the input 674 contact is higher than IT_H , the device 100 hides the interaction area 642-1 and the interaction area 642-2 except for the audio message 644-4. In some embodiments, device 100 also previews audio message 644-4 in a manner similar to that previously described (e.g., including dynamically changing the audio) in response to a press and hold input at a contact above IT_H . In contrast, in response to a tap gesture (e.g., at an intensity lower than IT_H ), device 100 performs a selection operation (e.g., plays the entire audio message 644-4) relative to audio message 644-4 without covering The second interactive area of the application program does not need to obscure the first interactive area of the application program.

以下参考图8A-8H来提供有关图6A-6Y(以及其中所示的用户界面)的附加细节。Additional details regarding FIGS. 6A-6Y (and the user interfaces shown therein) are provided below with reference to FIGS. 8A-8H .

图7A至7G示出了根据一些实施方案的用于基于音频配置文件来提供音频输出的示例性用户界面。更具体地，图7A-7G示出了以下示例：示能表示(例如，触摸屏112上的按钮)的激活使得设备输出具有音频配置文件(例如，管理音调、混响、和/或音频输出随时间的衰减的音频配置文件)的声音。当激活第二示能表示时，设备确定是否满足音频更改标准，并且如果满足，则修改与第一示能表示对应的音频配置文件。例如，当激活第一示能表示不久之后激活第二示能表示时，该设备使得第一示能表示的声音更快地衰减。7A-7G illustrate example user interfaces for providing audio output based on audio profiles, according to some embodiments. More specifically, FIGS. 7A-7G illustrate examples where activation of an affordance (e.g., a button on the touch screen 112) causes the device output to have an audio profile (e.g., manage pitch, reverb, and/or audio output with time decay of the audio profile) of the sound. When the second affordance is activated, the device determines whether the audio modification criteria are met, and if so, modifies the audio profile corresponding to the first affordance. For example, the device causes the sound of the first affordance to decay more quickly when the first affordance is activated shortly after the second affordance is activated.

这些图中的用户界面用于说明以下描述的过程，包括图9A-9C中的过程。The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 9A-9C.

为了方便起见，参照设备100来描述图7A-7G。作为简写，短语“设备100呈现与示能表示对应的音频”等用于意味着设备100向音频系统提供(或发起提供)声音信息，以使得音频系统可呈现与示能表示对应的音频输出。如参考图5所解释的，音频系统可集成到设备100中或与设备100分开。7A-7G are described with reference to device 100 for convenience. As shorthand, the phrase "device 100 presents audio corresponding to the affordance" and the like is used to mean that device 100 provides (or initiates providing) sound information to the audio system so that the audio system can present audio output corresponding to the affordance. As explained with reference to FIG. 5 , the audio system may be integrated into the device 100 or separated from the device 100 .

更具体地，图7A-7G示出了以下示例：当在第一示能表示的激活的阈值时间T_thres内激活第二示能表示时，满足音频更改标准。More specifically, FIGS. 7A-7G illustrate an example in which an audio modification criterion is met when a second affordance is activated within a threshold time_Tthres of activation of a first affordance.

图7A示出了用于电话应用程序(例如，电话模块138)的用户界面702。用户界面显示小键盘704，用户可通过该小键盘704来将字符例如输入到拨号号码栏706中。为此，小键盘704包括与数字对应的一组示能表示708(例如，与“5”对应的示能表示708-1和与“2”对应的示能表示708-2)。FIG. 7A shows a user interface 702 for a telephony application (eg, telephony module 138). The user interface displays a keypad 704 by which a user may enter characters, for example, into a dial number field 706 . To this end, keypad 704 includes a set of affordances 708 that correspond to numbers (eg, affordance 708-1 corresponding to "5" and affordance 708-2 corresponding to "2").

如图7B所示，设备100在时间T₁检测到用于激活示能表示708-1的第一输入710，并且在拨号号码栏706中放置“5”。图7B还示出了提供随时间的音频输出表示(例如，音频配置文件)的音频图712。检测到第一输入710时的时间T₁在音频图表712的水平轴线上被标记。音频图形712的垂直轴线指示在音频图形712上示出的音频输出的音量。第一音频配置文件714-1是第一音频输出的与示能表示708-1对应并且响应于示能表示708-1而被呈现的表示。在图7D的音频图712中完全显示第一音频输出的第一音频配置文件714-1。第一音频输出在时间T₁之后急剧上升，然后最终衰减到零音量。此外，音频图712指示了以c升音音调来产生第一音频输出。As shown in FIG. 7B , device 100 detects first input 710 for activating affordance 708 - 1 at time T₁ and places a "5" in dialed number field 706 . FIG. 7B also shows an audio graph 712 that provides a representation of audio output (eg, an audio profile) over time. Time T₁ when the first input 710 is detected is marked on the horizontal axis of the audio graph 712 . The vertical axis of the audio graph 712 indicates the volume of the audio output shown on the audio graph 712 . The first audio profile 714-1 is a representation of the first audio output that corresponds to the affordance 708-1 and is presented in response to the affordance 708-1. The first audio profile 714-1 for the first audio output is fully displayed in the audio graph 712 of FIG. 7D. The_first audio output rises sharply after time T1 and then eventually decays to zero volume. Additionally, the audio graph 712 indicates that the first audio output is produced with a c sharp pitch.

图7C示出了设备100在第二时间T₂(例如，T1之后的时间)检测到指向示能表示708-2的第二输入716(例如，检测到dui的“2”按钮的激活，其将“2”放置到拨号号码栏706中)。在此示例中，当设备100检测到对示能表示708-2的激活时，设备100产生由第二音频配置文件714-2表示的第二声音输出。在一些实施方案中，第二音频配置文件714-2与第一音频配置文件714-1相同(例如，小键盘声音的默认音频配置文件)。7C shows device 100 detecting a second input 716 directed at affordance 708-2 at a_second time T2 (e.g., a time after T1) (e.g., detecting activation of the "2" button of dui, which Place "2" in the dial number field 706). In this example, when device 100 detects activation of affordance 708-2, device 100 produces a second sound output represented by second audio profile 714-2. In some implementations, the second audio profile 714-2 is the same as the first audio profile 714-1 (eg, the default audio profile for keypad sounds).

因为检测到第二输入716的第二时间T₂在阈值时间T_thres之后，所以设备100继续根据第一音频配置文件714-1来呈现第一音频输出。相比之下，图7E-7G类似于图7B-7D，除了检测到第二输入716的第二时间T₂在阈值时间T_thres之前之外。在该情况下，满足音频更改标准，因此设备100不继续呈现具有第一音频配置文件714-1的第一音频输出，而是呈现具有经更改的音频配置文件714-3的经更改的第一音频输出(经更改的第一音频输出遵循图7F-7G中标记为714-3的实线；被标记为714-1的虚线用于示出如果未满足音频更改标准则未经更改的音频配置文件应当是什么)。Because the second time T2 at which the_second input 716 is detected is after the threshold time_Tthres , the device 100 continues to present the first audio output according to the first audio profile 714-1. In contrast, FIGS. 7E-7G are similar to FIGS. 7B-7D except that the second time T₂ at which the second input 716 is detected is before the threshold time T_thres . In this case, the audio modification criteria are met, so instead of continuing to present the first audio output with the first audio profile 714-1, the device 100 presents the modified first audio output with the modified audio profile 714-3. Audio output (modified first audio output follows the solid line labeled 714-3 in Figures 7F-7G; the dashed line labeled 714-1 is used to show the unmodified audio configuration if the audio modification criteria are not met what the file should be).

在一些实施方案中，当设备100呈现第一音频输出时，设备100呈现与第一音频输出对应的视觉效果。例如，如图7C-7D所示，视觉效果包括从示能表示708-1向外(例如，远离)延伸的一个或多个图形718(例如，环、波纹)。类似地，当设备100呈现第二音频输出时，设备100呈现远离示能表示708-2向外延伸的图形720(例如，环、波纹)。In some implementations, when device 100 presents the first audio output, device 100 presents a visual effect corresponding to the first audio output. For example, as shown in FIGS. 7C-7D , the visual effect includes one or more graphics 718 (eg, rings, ripples) extending outward (eg, away) from affordance 708-1. Similarly, when device 100 presents the second audio output, device 100 presents graphic 720 (eg, ring, ripple) extending outwardly away from affordance 708-2.

以下参考图9A-9C来提供有关图7A-7G(以及其中所示的用户界面)的附加细节。Additional details regarding Figures 7A-7G (and the user interfaces shown therein) are provided below with reference to Figures 9A-9C.

图8A-8B是描绘根据一些实施方案的动态地调整音频输出的呈现的方法800的流程图。图6A-60用于说明图8A-8B的方法和/或过程。尽管将参考触感显示器(其中触敏表面和显示器被组合)上的输入来给出以下的一些示例，但是在一些实施方案中，设备检测到与显示器450分开的触敏表面451上的输入，如图4B所示。8A-8B are flowcharts depicting a method 800 of dynamically adjusting the presentation of audio output, according to some implementations. 6A-60 are used to illustrate the method and/or process of FIGS. 8A-8B. Although some of the following examples will be given with reference to inputs on a touch-sensitive display (where the touch-sensitive surface and display are combined), in some implementations, the device detects an input on a touch-sensitive surface 451 that is separate from the display 450, such as Figure 4B.

在一些实施方案中，方法800由电子设备(例如，图1A的便携式多功能设备100)和/或电子设备(例如，I/O子系统106、操作系统126等)的一个或多个部件执行。在一些实施方案中，方法800由指令管理，该指令被存储在非暂态计算机可读存储介质中并且由设备的一个或多个处理器(诸如设备100(图1A)的一个或多个处理器122)执行。为了便于说明，以下描述由设备100执行的方法800。在一些实施方案中，参考图1A，方法800的操作至少部分地由音频预览模块163-1、音频修改模块163-2和触敏显示器(例如，触摸屏112)执行或对其进行使用。方法800中的一些操作任选地被组合，和/或一些操作的顺序任选地被改变。In some embodiments, method 800 is performed by an electronic device (e.g., portable multifunction device 100 of FIG. 1A ) and/or one or more components of an electronic device (e.g., I/O subsystem 106, operating system 126, etc.) . In some embodiments, method 800 is governed by instructions stored in a non-transitory computer-readable storage medium and processed by one or more processors of a device, such as one or more processors of device 100 (FIG. 1A ). device 122) executes. For ease of illustration, the method 800 performed by the device 100 is described below. In some embodiments, referring to FIG. 1A , the operations of method 800 are performed at least in part by or using audio preview module 163 - 1 , audio modification module 163 - 2 , and a touch-sensitive display (eg, touch screen 112 ). Some operations in method 800 are optionally combined, and/or the order of some operations is optionally changed.

如下所述，方法800(和相关联的界面)减少了来自用户的输入的数量、范围和/或性质，并且产生更有效的人机界面，从而为用户提供易于使用且直观的方式来与用户界面进行交互。对于电池供电的电子设备，方法800通过提供易于理解和信息丰富的音频输出来实现有效、无缝和快速的交互，其节省电力并且增大电池充电之间的时间(例如，通过减少汲取电池电量的广泛和低效的用户交互的需求)。As described below, the method 800 (and associated interface) reduces the number, scope, and/or nature of input from the user and produces a more efficient human-machine interface, thereby providing the user with an easy-to-use and intuitive way to communicate with the user. Interface to interact. For battery-powered electronic devices, the method 800 enables effective, seamless, and fast interaction by providing easy-to-understand and informative audio output that conserves power and increases the time between battery charges (e.g., by reducing the drain on battery power). extensive and inefficient user interaction requirements).

在一些实施方案中，当设备向与该设备进行通信的音频系统提供(802)第一声音信息以便呈现包括音量和除音量之外的音频特性(“非音量音频特性”)时，方法800开始。例如，在图6A-6H中，第一音频输出是贝多芬的月光奏鸣曲。在各种情况下，第一音频输出是在文本即时消息窗口中出现的歌曲、专辑、铃声、视频内容的音频部分、或来自音频文件对象的音频。In some embodiments, the method 800 begins when a device provides (802) first sound information to an audio system in communication with the device to present audio characteristics including volume and audio characteristics other than volume ("non-volume audio characteristics") . For example, in Figures 6A-6H, the first audio output is Beethoven's Moonlight Sonata. In various cases, the first audio output is a song, album, ringtone, audio portion of video content, or audio from an audio file object that appears in the text instant message window.

在一些实施方案中，非音量音频特性是(804)第一音频输出的混响时间、第一音频输出的低通滤波器截止(例如，截止频率)、或第一音频输出的立体声平衡(也被称为左右平衡、“L-R”平衡)。In some embodiments, the non-volume audio characteristic is (804) the reverberation time of the first audio output, the low-pass filter cutoff (e.g., cutoff frequency) of the first audio output, or the stereo balance of the first audio output (also Known as left-right balance, "L-R" balance).

当音频系统呈现第一音频输出时，设备接收(806)与用于呈现第二音频输出的请求对应的输入。在一些实施方案中，当焦点选择器位于图形对象之上时，在与电子设备进行通信的显示器上接收(808)与第二音频输出对应(或以其他方式相关联)的输入。While the audio system is rendering the first audio output, the device receives (806) an input corresponding to a request to render the second audio output. In some implementations, an input corresponding to (or otherwise associated with) the second audio output is received (808) on the display in communication with the electronic device when the focus selector is positioned over the graphical object.

例如，输入是触敏表面(例如，与电子设备集成的触敏显示器、或与电子设备进行信号通信的遥控器上的触敏表面)上的按压输入。在与第二音频输出相关联的图形用户界面对象(例如，文本消息窗口中的视频内容对象，图6I-60)上接收按压输入。For example, the input is a press input on a touch-sensitive surface (eg, a touch-sensitive display integrated with the electronic device, or a touch-sensitive surface on a remote control in signal communication with the electronic device). A press input is received on a graphical user interface object associated with the second audio output (eg, a video content object in a text message window, Figures 6I-60).

作为另一个示例，输入对应于在与第二音频输出相关联的图形用户界面对象(例如，专辑封面的原图，图6A-6H)上开始的触敏表面上的轻扫手势。As another example, the input corresponds to a swipe gesture on the touch-sensitive surface beginning on a graphical user interface object (eg, an artwork of an album cover, FIGS. 6A-6H ) associated with the second audio output.

在一些实施方案中，当图形用户界面对象表示歌曲、专辑或视频时，第二音频输出是歌曲、专辑或视频的预览(例如，歌曲、专辑或视频的一部分，其已被预选为代表或可识别歌曲、专辑或视频)。在一些实施方案中，歌曲、专辑或视频的一部分具有预设置的长度(例如，30秒)。In some embodiments, when the GUI object represents a song, album, or video, the second audio output is a preview of the song, album, or video (e.g., a portion of the song, album, or video that has been pre-selected to represent or can be identify a song, album or video). In some embodiments, a portion of a song, album, or video has a preset length (eg, 30 seconds).

例如，输入对应于触敏表面上的轻扫手势，该轻扫手势在与第二音频输出相关联的图形用户界面对象(例如，图形用户界面对象包括用于专辑封面的原图，图6A-6H)上方(或其上)开始。上述另一个示例是：其中在聚焦选择器位于与第二音频输出相关联的图形用户界面对象(例如，图形用户界面对象包括文本消息窗口中的视频内容对象，图6I-60)上方时，接收按压输入。图形用户界面对象的其他示例包括铃声的文本表示、在文本即时消息窗口中出现的音频文件对象等。For example, the input corresponds to a swipe gesture on the touch-sensitive surface on a GUI object associated with the second audio output (e.g., the GUI object includes an artwork for an album cover, Figure 6A- 6H) above (or above) start. Another example of the above is where when the focus selector is over a GUI object associated with the second audio output (e.g., the GUI object includes a video content object in a text message window, Figure 6I-60), receiving Press enter. Other examples of graphical user interface objects include textual representations of ringtones, audio file objects appearing in text instant message windows, and the like.

响应于接收到与用于呈现第二音频输出的请求对应的输入，设备向音频系统提供(810)信息，以便根据输入的量值来动态地调整第一音频输出的呈现。在一些实施方案中，输入的量值是(812)输入中的接触的特征强度、输入中的接触的时间长度、或输入中的接触行进的距离(例如，轻扫手势的长度)。In response to receiving an input corresponding to a request to render the second audio output, the device provides (810) information to the audio system to dynamically adjust the rendering of the first audio output according to the magnitude of the input. In some implementations, the magnitude of the input is (812) a characteristic strength of the contact in the input, a length of time for the contact in the input, or a distance traveled by the contact in the input (eg, the length of a swipe gesture).

随着输入的量值改变，设备动态地调整(814)非音量音频特性。随着输入的量值改变而调整非音量音频特性允许用户有效地预览或收听第二音频输出而无需中断第一音频输出，并且通过单个用户输入来给予用户对音频输出的附加控制。这增强了设备的可操作性，并且使得用户设备接口更有效(例如，通过减少用户在操作设备/与设备交互时必须进行的用户输入的数量)，这另外通过使得用户能够更快和更有效地使用设备来减少电力使用并且改善设备的电池寿命。As the magnitude of the input changes, the device dynamically adjusts (814) the non-volume audio characteristics. Adjusting non-volume audio characteristics as the magnitude of the input changes allows the user to effectively preview or listen to the second audio output without interrupting the first audio output, and gives the user additional control over the audio output through a single user input. This enhances the operability of the device and makes the user device interface more efficient (e.g., by reducing the amount of user input the user must make when operating/interacting with the device), which additionally makes the user interface faster and more efficient Use your device more efficiently to reduce power usage and improve your device's battery life.

在一些实施方案中，该设备偏移第一音频输出的立体声平衡度，同时偏移第二音频输出的立体声平衡度(816)(例如，如参考图6A-6O所述)。例如，该输入对应于触敏表面上的轻扫手势，该轻扫手势在第一方向上移动(例如，从左向右轻扫)远离图形用户界面对象的与第二音频输出相关联的初始位置(例如，该轻扫手势将专辑封面的原图朝向显示器的中心拖动，图6A-6H)。在该示例中，偏移立体声平衡度包括偏移第一音频输出和第二音频输出的呈现，使得它们跟踪轻扫输入沿第一方向的移动(例如，第一音频输出的立体声平衡度向右偏移，而第二音频输出从音频系统的左边开始淡入)。在其他实施方案中，基于输入的接触的强度的变化或基于输入的接触时间而偏移第一音频输出和第二音频输出的立体声平衡度。In some implementations, the device shifts the degree of stereo balance of the first audio output while simultaneously shifting the degree of stereo balance of the second audio output (816) (eg, as described with reference to FIGS. 6A-60 ). For example, the input corresponds to a swipe gesture on the touch-sensitive surface that moves in a first direction (eg, swipes from left to right) away from the initial position of the graphical user interface object associated with the second audio output. position (eg, the swipe gesture drags the artwork of the album cover towards the center of the display, FIGS. 6A-6H ). In this example, shifting the stereo balance includes shifting the presentation of the first audio output and the second audio output so that they track movement of the swipe input in a first direction (e.g., the stereo balance of the first audio output to the right offset, while the second audio output fades in from the left of the audio system). In other embodiments, the stereo balance of the first audio output and the second audio output is shifted based on a change in intensity of the input contact or based on a time of the input contact.

作为另一个实施方案，在一些实施方案中，非音量音频属性为第一音频输出的低通滤波器截止。该设备根据输入的量值的增大而使低通滤波器截止向下偏移。因此，随着输入的量值增大，第一音频内容减小到低音背景(例如，“轰轰轰”)，使得第二音频内容能够以使第二音频内容清晰可听的方式与第一音频内容同时产生(例如，因为背景音频的突出性已由于应用低通滤波器而减小)。As another implementation, in some implementations, the non-volume audio attribute is a low pass filter cutoff of the first audio output. This device shifts the low pass filter cutoff downwards as the magnitude of the input increases. Thus, as the magnitude of the input increases, the first audio content decreases to a bass background (e.g., "boom boom boom") so that the second audio content can be compared to the first audio content in a manner that makes the second audio content clearly audible. Audio content is produced simultaneously (eg, because the prominence of background audio has been reduced by applying a low-pass filter).

在一些实施方案中，该设备随着输入的量值的变化而调整第一音频输出的音量(818)(例如，音量和非音频属性两者随着输入的量值的变化而变化)。在一些实施方案中，第一音频输出的音量随着输入的量值的增大而减小。以这种方式调整第一音频输出的音量还增大了第二音频输出的听觉突出，而无需中断第一音频输出。这使得两个音频输出的模糊更有效，因而增强了该设备的可操作性并且使得用户设备界面更有效。In some implementations, the device adjusts the volume of the first audio output (818) as the magnitude of the input changes (eg, both volume and non-audio properties change as the magnitude of the input changes). In some implementations, the volume of the first audio output decreases as the magnitude of the input increases. Adjusting the volume of the first audio output in this manner also increases the aural prominence of the second audio output without interrupting the first audio output. This enables more efficient blurring of the two audio outputs, thus enhancing the operability of the device and making the user device interface more efficient.

在一些实施方案中，该设备在第二音频输出的呈现之前，随着输入的量值的变化而动态地调整非音量音频属性，直到输入的量值满足第一预先确定的阈值(820)(例如，输入中的接触的特征强度保持低于预览阈值但高于提示阈值，由此该设备提供该设备即将预览第二音频输出的听觉提示和/或视觉提示，如参考图6B所述)。一旦输入的量值满足第一预先确定的阈值(例如，超过轻微轻扫移动，图6C)，第二音频输出即开始播放，并且在一些实施方案中，第一音频输出随着输入的量值的变化而进一步调整。在第二音频输出的呈现为用户提供音频模糊即将发生的提示之前，随着输入的量值的变化而动态地调整非音量音频属性，该提示可用于例如在用户偶然发现方法800中所描述的设备输入功能时警告用户。In some embodiments, prior to presentation of the second audio output, the device dynamically adjusts non-volume audio properties as the magnitude of the input changes until the magnitude of the input satisfies a first predetermined threshold (820) ( For example, the characteristic intensity of the contact in the input remains below the preview threshold but above the cue threshold, whereby the device provides an audible and/or visual cue that the device is about to preview the second audio output, as described with reference to FIG. 6B ). Once the magnitude of the input meets a first predetermined threshold (e.g., beyond a slight swipe movement, FIG. 6C ), the second audio output starts playing, and in some embodiments, the first audio output follows the magnitude of the input. further adjustments for changes. Dynamically adjust non-volume audio attributes as the magnitude of the input changes before the presentation of the second audio output provides the user with a cue that audio blurring is about to occur, which can be used, for example, as described in user stumble method 800 Warn the user when the device enters a function.

响应于接收到与用于呈现第二音频输出的请求对应的输入，该设备为音频系统提供第二声音信息，以与第一音频输出同时呈现第二音频输出(822)。在一些实施方案中，该设备提供信息以根据输入的量值而动态地调整第二音频输出的呈现(824)。在一些实施方案中，在根据输入的量值而动态地调整第一音频输出的呈现的同时执行对第二音频输出的呈现。这样，第一音频输出和第二音频输出同时动态地模糊(例如，第一音频输出逐渐淡出，而第二音频输出逐渐淡入)。因此，根据输入的量值而动态地调整第二音频输出的呈现提供从第一音频输出的突出性到第二音频输出的突出性的无缝过渡，这使得音频的模糊对于偶然发现方法800中所描述的设备输入功能的用户来说更令人愉悦并且不那么引发恐慌。这继而增大了用户保持在其设备上实现这种功能的可能性，并且因此通过增大用户与设备的交互的效率而节省了电池功率。In response to receiving input corresponding to a request to present a second audio output, the device provides second sound information to the audio system to present the second audio output concurrently with the first audio output (822). In some implementations, the device provides information to dynamically adjust the presentation of the second audio output according to the magnitude of the input (824). In some implementations, the rendering of the second audio output is performed while dynamically adjusting the rendering of the first audio output according to the magnitude of the input. In this way, the first audio output and the second audio output are dynamically blurred simultaneously (eg, the first audio output gradually fades out while the second audio output gradually fades in). Thus, dynamically adjusting the presentation of the second audio output according to the magnitude of the input provides a seamless transition from the prominence of the first audio output to the prominence of the second audio output, which makes audio ambiguity less important in the serendipity method 800. The described device input functions are more pleasing and less panic-inducing to the user. This in turn increases the likelihood that the user will remain on their device for such functionality, and thus saves battery power by increasing the efficiency of the user's interaction with the device.

在一些实施方案中，响应于接收到与用于呈现第二音频输出的请求对应的输入，该设备向显示器提供数据以显示结合动态地调整非音量音频属性而变化的视觉效应(826)。例如，该设备向显示器提供数据以在视觉上模糊用户界面中的除了与第二音频输出对应的图形用户界面对象之外的任何事物，其中模糊半径与输入的量值成比例。显示结合动态地调整非音量音频属性而变化的视觉效应提供与变化的音频对应的直观的视觉提示，这有助于用户与他们的设备进行交互。In some implementations, in response to receiving the input corresponding to the request to present the second audio output, the device provides data to the display to display the visual effect that changes in conjunction with dynamically adjusting the non-volume audio property (826). For example, the device provides data to the display to visually blur anything in the user interface other than a graphical user interface object corresponding to the second audio output, wherein the blur radius is proportional to the magnitude of the input. Displaying visual effects that change in conjunction with dynamically adjusting non-volume audio properties provides intuitive visual cues that correspond to changing audio, which helps users interact with their devices.

在一些实施方案中，该设备检测到输入的量值满足大于第一预先确定的阈值的第二预先确定的阈值并且作为响应，使得音频系统停止呈现第一音频输出(并且继续呈现第二音频输出)(828)。在一些实施方案中，第二预先确定的阈值(“弹出”阈值)与第一预先确定的阈值相比对应于该输入的更高的接触的强度(例如，弹出阈值在图6I-6O中为IT_D)、该输入行进的更大的距离或该输入与触敏表面保持接触的更长的时间量。因此，音频系统停止呈现第一音频输出为用户提供一种方式以使用用户通过其预览第一音频输出的相同输入而切换到第二音频输出的“完全呈现”。这也减少了实现特定功能所需的用户交互的数量。In some embodiments, the device detects that the magnitude of the input satisfies a second predetermined threshold greater than the first predetermined threshold and in response causes the audio system to stop presenting the first audio output (and continue presenting the second audio output )(828). In some embodiments, the second predetermined threshold (the "pop-up" threshold) corresponds to a higher intensity of contact for the input than the first predetermined threshold (e.g., the pop-up threshold in FIGS. 6I-6O is IT_D ), the greater distance the input travels, or the greater amount of time the input remains in contact with the touch-sensitive surface. Thus, the audio system ceasing to render the first audio output provides a way for the user to switch to "full rendering" of the second audio output using the same input through which the user previewed the first audio output. This also reduces the amount of user interaction required to achieve a particular function.

在一些实施方案中，该设备检测到输入的结束(830)(例如，如图6E-6H所示)。在一些另选的实施方案中，替代检测到输入的结束，该设备检测到输入的量值的变化已停止(例如，接触的强度回到其原始强度或输入回到其原始位置)。在一些另选的实施方案中，替代检测到输入的结束，该设备检测到输入的超过预先确定的阈值(例如，弹出阈值)的量值的增大。在一些另选的实施方案中，替代检测到输入的结束，该设备检测到输入的量值的预先确定的变化。例如，输入的量值下降到低于预先确定的阈值(例如，输入的量值下降到低于弹出阈值，而输入与触敏表面保持接触)。在一些实施方案中，取代检测到输入的结束，该设备检测到输入的变化已停止预先确定的时间段。In some embodiments, the device detects the end of the input (830) (eg, as shown in Figures 6E-6H). In some alternative embodiments, instead of detecting the end of the input, the device detects that the change in magnitude of the input has ceased (eg, the intensity of the contact returns to its original intensity or the input returns to its original position). In some alternative implementations, instead of detecting the end of the input, the device detects an increase in the magnitude of the input beyond a predetermined threshold (eg, a pop-up threshold). In some alternative embodiments, instead of detecting the end of the input, the device detects a predetermined change in magnitude of the input. For example, the magnitude of the input falls below a predetermined threshold (eg, the magnitude of the input falls below a pop-up threshold while the input remains in contact with the touch-sensitive surface). In some embodiments, instead of detecting the end of the input, the device detects that the change in the input has ceased for a predetermined period of time.

在任何情况下，响应于适当的条件，该设备使得音频系统进行以下操作中的一个操作：In any case, in response to the appropriate conditions, the device causes the audio system to do one of the following:

·停止呈现第二音频输出并且呈现第一音频输出，而无需对第一音频输出进行动态调整(832)(例如，撤销对第一音频输出的动态调整)。在一些实施方案中，在播放第二音频输出时，调整而不是暂停第一音频输出的播放，并且第一音频输出在删除调整的情况下，在第二音频输出停止播放之后继续播放。这为用户提供了一种直观的方式返回到仅收听第一音频输出，从而减少了用户输入的数量。例如，在图6H中，该设备停止呈现第二音频输出并且在无动态调整的情况下呈现第一音频输出。• Stop presenting the second audio output and present the first audio output without dynamic adjustment to the first audio output (832) (eg, undo the dynamic adjustment to the first audio output). In some embodiments, while the second audio output is playing, playback of the first audio output is adjusted rather than paused, and the first audio output continues playing after the second audio output stops playing, with the adjustment removed. This provides an intuitive way for the user to return to listening to only the first audio output, thereby reducing the amount of user input. For example, in FIG. 6H, the device stops presenting the second audio output and presents the first audio output without dynamic adjustment.

·停止呈现第一音频输出并且继续呈现第二音频输出(834)。这为用户提供了一种直观的方式切换成仅收听第二音频输出，从而减少了用户输入的数量。例如，在图6H中，该设备停止呈现第一音频输出并且在无动态调整的情况下呈现第二音频输出。• Stop presenting the first audio output and continue presenting the second audio output (834). This provides an intuitive way for the user to switch to only listening to the second audio output, reducing the amount of user input. For example, in Figure 6H, the device stops presenting the first audio output and presents the second audio output without dynamic adjustment.

·停止呈现经调整的第一音频输出并且继续呈现第二音频输出(836)。这为用户提供了一种直观的方式来继续模糊音频；或者• Stop presenting the adjusted first audio output and continue presenting the second audio output (836). This provides an intuitive way for the user to continue blurring the audio; or

·根据确定量值满足预先确定的阈值(例如，在输入期间超过弹出阈值)，停止呈现第一音频输出并且继续呈现第二音频输出；并且根据确定量值不满足该阈值(例如，该输入在输入期间保持低于弹出阈值)，停止呈现第二音频输出并且在无动态调整的情况下呈现第一音频输出(例如，恢复对第一音频输出的动态调整)(838)。这样，该系统自动地确定用户实际上想要继续收听第一音频输出还是想要切换到第二音频输出，从而为用户提供一种直观的方式来实现任一种结果并且减少用户输入的数量。例如，在图6E-6H中，该设备根据确定用户输入的量值满足还是不满足阈值来确定是否停止呈现第二音频输出并且在无动态调整的情况下呈现第一音频输出。Stop presenting the first audio output and continue presenting the second audio output based on the determined magnitude meeting a predetermined threshold (e.g., the pop-up threshold was exceeded during the input); and based on the determined magnitude not meeting the threshold (e.g., the input was remains below the pop-up threshold during input), ceases presenting the second audio output and presents the first audio output without dynamic adjustment (eg, resumes dynamic adjustment to the first audio output) (838). In this way, the system automatically determines whether the user actually wants to continue listening to the first audio output or to switch to the second audio output, thereby providing an intuitive way for the user to achieve either outcome and reducing the amount of user input. For example, in FIGS. 6E-6H , the device determines whether to stop presenting the second audio output and present the first audio output without dynamic adjustment based on determining whether the magnitude of the user input meets or fails to meet the threshold.

应当理解，对图8A-8H中已进行描述的操作的具体次序仅是一个示例，并非旨在表明所述次序是可执行这些操作的唯一次序。本领域的普通技术人员会想到多种方式来对本文所述的操作进行重新排序。在一些实施方案中，将方法800的一个或多个操作与本文所述的其他方法(例如，方法840、854、875和/或900)的一个或多个操作组合或由它们补充或替代。It should be understood that the specific order in which the operations have been described in Figures 8A-8H is an example only, and is not intended to suggest that the described order is the only order in which these operations may be performed. There are many ways to reorder the operations described herein will occur to those of ordinary skill in the art. In some embodiments, one or more operations of method 800 are combined with, supplemented with, or replaced by one or more operations of other methods described herein (eg, methods 840, 854, 875, and/or 900).

图8C是示出根据一些实施方案的动态地调整音频输出的呈现的方法840的流程图。在一些实施方案中，将方法840的一个或多个操作与本文所述的其他方法(例如，方法800)的一个或多个操作组合或由它们补充或替代。此外，参考方法840所述的很多操作与以上参考方法800所述的操作共享相同的益处(例如，音频的直观性模糊、用户必须形成的用户输入的数量的减少)。因此，方法840还(例如，通过帮助用户在操作该设备/与该设备进行交互时提供正确的输入和减少用户错误)增强设备的可操作性并使得用户设备界面更有效，这另外还通过使用户能够更快速并且更有效地使用该设备而减少了功率使用并且改进了该设备的电池寿命。FIG. 8C is a flowchart illustrating a method 840 of dynamically adjusting the presentation of audio output, according to some embodiments. In some embodiments, one or more operations of method 840 are combined with, supplemented to, or replaced by one or more operations of other methods described herein (eg, method 800 ). Furthermore, many of the operations described with reference to method 840 share the same benefits as those described above with reference to method 800 (eg, intuitive obfuscation of audio, reduction in the amount of user input a user must form). Accordingly, the method 840 also enhances the operability of the device and makes the user device interface more efficient (e.g., by helping the user provide correct input and reducing user error when operating/interacting with the device), which in addition The user is able to use the device more quickly and efficiently with reduced power usage and improved battery life of the device.

在一些实施方案中，方法840在具有存储器和一个或多个处理器的电子设备处执行，并且该电子设备与显示器和音频系统进行通信。In some embodiments, the method 840 is performed at an electronic device having memory and one or more processors, and the electronic device is in communication with a display and an audio system.

在一些实施方案中，在该设备向显示器提供数据以呈现包括媒体对象的用户界面时，方法840开始，该媒体对象表示至少一个媒体内容项(842)。In some embodiments, the method 840 begins when the device provides data to a display to present a user interface comprising a media object representing at least one item of media content (842).

该设备向音频系统提供第一声音信息以呈现不与媒体对象对应的第一音频输出(844)。The device provides first sound information to an audio system to present a first audio output that does not correspond to a media object (844).

在向音频系统提供第一声音信息时，该设备检测到输入的指向媒体对象的第一部分(846)。Upon providing the first sound information to the audio system, the device detects a first portion of the input directed to the media object (846).

响应于检测到输入的指向媒体对象的第一部分，该设备：(i)开始向音频系统提供第二声音信息以呈现与媒体对象对应的第二音频输出；以及(ii)继续向音频系统提供第一声音信息以呈现不与媒体对象对应的第一音频输出(848)。In response to detecting the first part of the input pointing to the media object, the device: (i) begins to provide the second sound information to the audio system to present a second audio output corresponding to the media object; and (ii) continues to provide the second sound information to the audio system. A sound message is output to present the first audio that does not correspond to the media object (848).

在向音频系统提供第一声音信息和第二声音信息时，该设备检测到输入的指向媒体对象的第二部分，其中检测到输入的第二部分包括检测到输入的参数的变化(850)(例如，当焦点选择器在媒体对象上方时检测到与触敏表面的接触的强度的变化或检测到焦点选择器的位置或移动的变化)。While providing the first sound information and the second sound information to the audio system, the device detects a second portion of the input directed to the media object, wherein detecting the second portion of the input includes detecting a change in a parameter of the input (850)( For example, detecting a change in the intensity of contact with the touch-sensitive surface when the focus selector is over a media object or detecting a change in the position or movement of the focus selector).

响应于检测到输入的参数的变化，该设备：In response to detecting a change in an input parameter, the device:

(i)向显示器提供(852)数据以(例如，通过使用户界面背景在视觉上模糊和/或放大媒体对象)根据输入的参数的变化来动态地更改所呈现的用户界面；(i) providing (852) data to the display to dynamically alter the presented user interface according to changes in the input parameters (e.g., by visually blurring the user interface background and/or enlarging media objects);

(ii)向音频系统提供信息以(例如，通过减小音量、增大混响时间、减小低通滤波器截止频率、和/或将音频的L-R平衡度向右或向左移动)根据输入的参数的变化来动态地更改不与媒体对象对应的第一音频输出；以及(ii) Provide information to the audio system to (for example, by reducing the volume, increasing the reverberation time, reducing the low-pass filter cutoff frequency, and/or shifting the L-R balance of the audio to the right or left) based on the input to dynamically change the first audio output not corresponding to the media object; and

(iii)向音频系统提供信息以(例如，通过减小音量、增大混响时间、减小低通滤波器截止频率、和/或将音频的L-R平衡度向右或向左移动)根据输入的参数的变化来动态地更改与媒体对象对饮的第二音频输出(852)。(iii) provide information to the audio system to (for example, by reducing the volume, increasing the reverberation time, reducing the low-pass filter cutoff frequency, and/or shifting the L-R balance of the audio to the right or left) based on the input Changes in the parameters of the media object dynamically alter the second audio output for drinking (852).

应当理解，对图8C中已经进行描述的操作的具体次序仅仅是一个示例，并非旨在表明所述次序是可执行这些操作的唯一次序。本领域的普通技术人员会想到多种方式来对本文所述的操作进行重新排序。在一些实施方案中，将方法840的一个或多个操作与本文所述其他方法(例如，方法800、854、875和/或900)的一个或多个操作组合或由它们补充或替代。It should be understood that the specific order in which the operations have been described in FIG. 8C is an example only, and is not intended to indicate that the described order is the only order in which these operations may be performed. There are many ways to reorder the operations described herein will occur to those of ordinary skill in the art. In some embodiments, one or more operations of method 840 are combined with, supplemented with, or replaced by one or more operations of other methods described herein (eg, methods 800, 854, 875, and/or 900).

图8D-8F是示出根据一些实施方案的动态地调整音频输出的呈现的方法854的流程图。图6P-6Y用于示出图8D-8F的方法和/或过程。尽管将参考触敏显示器(其中组合了触敏表面和显示器)上的输入给出随后的实施方案中的一些实施方案，但在一些实施方案中，设备检测与显示器450分开的触敏表面451上的输入，如图4B所示。8D-8F are flowcharts illustrating a method 854 of dynamically adjusting the presentation of audio output, according to some embodiments. 6P-6Y are used to illustrate the method and/or process of FIGS. 8D-8F. Although some of the embodiments that follow will be given with reference to input on a touch-sensitive display (where the touch-sensitive surface and display are combined), in some embodiments the device detects input on a touch-sensitive surface 451 that is separate from the display 450. input, as shown in Figure 4B.

在一些实施方案中，方法854由电子设备(例如，便携式多功能设备100，图1A)和/或该电子设备的一个或多个部件(例如，I/O子系统106、操作系统126等)执行。在一些实施方案中，方法854通过被存储在非暂态计算机可读存储介质中并且由设备的一个或多个处理器诸如设备100的一个或多个处理器122(图1A)执行的指令来支配。为了容易解释，下文描述了由设备100执行的方法854。在一些实施方案中，参考图1A，方法854的操作至少部分地由音频预览模块163-1、音频修改模块163-2和触敏显示器(例如，触摸屏112)执行或至少部分地对其进行使用。方法854中的一些操作任选地被组合，和/或一些操作的顺序任选地被改变。In some embodiments, method 854 is performed by an electronic device (e.g., portable multifunction device 100, FIG. 1A ) and/or one or more components of the electronic device (e.g., I/O subsystem 106, operating system 126, etc.) implement. In some embodiments, method 854 is implemented by instructions stored in a non-transitory computer-readable storage medium and executed by one or more processors of a device, such as one or more processors 122 of device 100 (FIG. 1A ). dominate. For ease of explanation, the method 854 performed by the device 100 is described below. In some embodiments, referring to FIG. 1A , the operations of method 854 are performed at least in part by or at least in part used by audio preview module 163-1, audio modification module 163-2, and a touch-sensitive display (e.g., touch screen 112). . Some operations in method 854 are optionally combined, and/or the order of some operations is optionally changed.

如下所述，方法854(和相关联的界面)减少了来自用户的输入的数量、范围和/或性质，并且产生更有效的人机界面，从而为用户提供易于使用且直观的方式来与用户界面进行交互。对于电池操作的电子设备，方法854允许通过提供节省功率并且增大电池两次充电之间的时间间隔的容易理解且信息性的音频输出(例如，通过减少对耗尽电池功率的密集且低效的用户交互的需求)来实现有效、无缝且快速的交互。As described below, the method 854 (and associated interface) reduces the number, scope, and/or nature of input from the user and produces a more efficient human-machine interface, thereby providing the user with an easy-to-use and intuitive way to communicate with the user. Interface to interact. For battery-operated electronic devices, method 854 allows for easy-to-understand and informative audio output that conserves power and increases the time between battery charges (e.g., by reducing intensive and inefficient response to draining battery power). user interaction requirements) to achieve effective, seamless and fast interaction.

在一些实施方案中，该设备在显示器上显示包括媒体项的表示的用户界面(855)时，方法854开始。例如，图6Q示出了包括媒体项658的表示的用户界面640。In some embodiments, the method 854 begins when the device displays a user interface on the display (855) that includes a representation of the media item. For example, FIG. 6Q shows user interface 640 including a representation of media item 658 .

在显示用户界面时，该设备在触敏表面上的与媒体项的表示对应的位置处检测到通过接触(例如，手指接触或触笔接触)导致的输入(856)(例如，在触敏显示器上的媒体项的表示上检测到接触)。例如，图6R示出了用户输入660的开始，该用户输入是在表示658-2上检测到的按压并保持手势。While displaying the user interface, the device detects (856) an input caused by contact (e.g., finger contact or stylus contact) at a location on the touch-sensitive surface corresponding to a representation of the media item (e.g., on a touch-sensitive display Touch detected on the representation of the media item on). For example, FIG. 6R shows the beginning of user input 660, which is a detected press and hold gesture on representation 658-2.

响应于检测到通过接触导致的输入：根据确定该输入满足媒体提示标准，其中该媒体提示标准包括当该接触具有高于第一强度阈值(例如，IT_H，图6R)的特征强度时满足的标准：该设备开始播放媒体项的相应部分(例如，被选择成包含媒体项的可识别部分的开始部分或代表部分)(例如，播放它们的音频和/或视频)(857)。例如，在图6R中，设备100开始播放Brahms’s Alto Rhapsody。而且，在播放媒体项时，该设备随着接触的特征强度改变而动态地改变一组一个或多个音频属性(例如，随着接触的特征强度通过多个值改变，该媒体项的音频属性通过多个值改变)。在一些实施方案中，媒体项的音量随着接触的特征强度的增大而增大并且随着接触的特征强度的减小而减小。例如，随着输入660的强度从图6R到图6S增大，Brahms’s Alto Rhapsody的音量增大。Responsive to detecting an input caused by a contact: upon determining that the input satisfies a media cueing criterion, wherein the media cueing criterion includes satisfying when the contact has a characteristic intensity above a first intensity threshold (e.g., IT_H , FIG. 6R ). Criterion: The device begins playing the corresponding portion of the media item (e.g., the beginning or representative portion selected to contain an identifiable portion of the media item) (e.g., playing their audio and/or video) (857). For example, in Figure 6R, device 100 starts playing Brahms' Alto Rhapsody. Also, as the media item is played, the device dynamically changes a set of one or more audio properties as the characteristic strength of the contact changes (e.g., the audio property of the media item changes as the characteristic strength of the contact changes through multiple values). changed by multiple values). In some implementations, the volume of the media item increases as the characteristic intensity of the contact increases and decreases as the characteristic intensity of the contact decreases. For example, as the intensity of input 660 increases from FIG. 6R to FIG. 6S , the volume of Brahms' Alto Rhapsody increases.

在一些实施方案中，所播放的媒体项基于输入中的接触的位置而确定(例如，根据确定在与第一媒体项对应的位置处检测到接触，以上步骤针对第一媒体项而执行，并且根据确定在与第二媒体项对应的位置处检测到接触，以上步骤针对第二媒体项而不是针对第一媒体项而执行)。例如，如果在表示658-1而不是表示658-2上检测到图6R中的输入660，则设备100将会开始播放与表示658-1对应的媒体项。In some embodiments, the media item played is determined based on the location of the contact in the input (e.g., based on determining that a contact was detected at a location corresponding to the first media item, the above steps are performed for the first media item, and Based on a determination that a contact is detected at a location corresponding to the second media item, the above steps are performed with respect to the second media item rather than with respect to the first media item). For example, if input 660 in FIG. 6R was detected on representation 658-1 instead of representation 658-2, device 100 would begin playing the media item corresponding to representation 658-1.

根据确定该输入不满足媒体提示标准，该设备放弃开始播放媒体项的相应部分，并且放弃随着接触的特征强度改变而动态地改变媒体项的这组一个或多个音频属性。在一些实施方案中，该设备相对于媒体项执行另选的操作。例如，响应于媒体项的表示上的轻击手势，该设备相对于媒体项执行选择操作。Based on determining that the input does not satisfy the media cueing criteria, the device abstains from starting to play the corresponding portion of the media item, and abstains from dynamically changing the set of one or more audio properties of the media item as the characteristic intensity of the contact changes. In some embodiments, the device performs an alternative operation with respect to the media item. For example, in response to a tap gesture on a representation of a media item, the device performs a selection operation with respect to the media item.

在一些实施方案中，媒体项的音频属性包括音量、低通滤波器的截止频率和/或均衡器设置(858)(例如，图6P-6Y示出了其中设备随着接触的特征强度改变而动态地改变媒体项的音量的一个示例)。In some embodiments, the audio properties of the media item include volume, cutoff frequency of the low-pass filter, and/or equalizer settings (858) (e.g., FIGS. An example of dynamically changing the volume of a media item).

在一些实施方案中，当媒体项的这组一个或多个音频属性动态地改变时，该设备动态地改变用户界面的视觉外观(859)。在一些实施方案中，动态地改变用户界面的视觉外观包括随着接触的特征强度的增大而增大媒体项的表示的尺寸(860)(例如，随着接触的特征强度的增大而增大媒体项的表示的尺寸以及随着接触的特征强度的减小而减小媒体项的表示的尺寸)。例如，随着输入660的强度从图6R到图6S增大，媒体项的音量和表示658-2的尺寸均增大。在一些实施方案中，这是在用户界面中初始显示的同一表示(如图6R-6T所示，表示658-2由其原始尺寸和位置“生长”，朝向用户界面的第二区域的中心部分移动，同时也增大了尺寸)。在其他实施方案中，当初始显示的表示与用户界面的第二区域保持在同一位置时，该表示与初始表示不同，并且媒体项的第二不同表示随着接触的特征强度的增大而增大尺寸并且朝向用户界面的第二区域的中心部分移动(例如，该不同表示显示在初始表示的顶部)。In some embodiments, when the set of one or more audio properties of the media item dynamically changes, the device dynamically changes the visual appearance of the user interface (859). In some embodiments, dynamically changing the visual appearance of the user interface includes increasing the size of the representation of the media item (860) as the characteristic strength of the contact increases (e.g., increasing the size of the media item as the characteristic strength of the contact increases). The size of the representation of the media item is large and the size of the representation of the media item is reduced as the characteristic intensity of the contact decreases). For example, as the intensity of input 660 increases from FIG. 6R to FIG. 6S, both the volume of the media item and the size of representation 658-2 increase. In some embodiments, this is the same representation that was initially displayed in the user interface (as shown in FIGS. moved, and also increased in size). In other embodiments, when the initially displayed representation remains in the same location as the second area of the user interface, the representation is different from the initial representation, and the second different representation of the media item increases as the characteristic intensity of the contact increases. Large in size and moving towards a central portion of the second area of the user interface (eg, the different representation is displayed on top of the initial representation).

在一些实施方案中，动态地改变用户界面的视觉外观包括随着接触的特征强度改变而动态地改变用户界面的邻近媒体项的表示的一部分的模糊量(861)。例如，随着输入660的强度从图6R到图6S增大，交互区域642-2模糊，其模糊半径与接触660的强度成比例(这些附图中，增大的模糊半径通过覆盖交互区域642-2的图案的减小的透明度而示意性地表示)。In some embodiments, dynamically changing the visual appearance of the user interface includes dynamically changing an amount of blurring of a portion of the user interface's representation of adjacent media items as the characteristic intensity of the contact changes (861). For example, as the intensity of input 660 increases from FIG. 6R to FIG. 6S , interaction region 642-2 blurs with a blur radius proportional to the intensity of contact 660 (in these figures, the increasing blur radius by covering interaction region 642 -2 is schematically represented by the reduced transparency of the pattern).

在一些实施方案中，在开始播放媒体项的相应部分之后：该设备检测到接触的特征强度的增大，之后检测到接触的特征强度的减小(862)(例如，图6T示出了接触660的强度的增大；图6U示出了接触660的强度的减小)。在一些实施方案中，下面描述了多个条件操作，这些条件操作根据确定接触的强度特征是否满足媒体预览标准而进行调节。这些操作响应于检测到接触的特征强度的增大，之后检测到接触的特征强度的减小而执行。在一些实施方案中，在检测到接触的特征强度的增大，之后检测到接触的特征强度的减小时，执行下面描述的条件操作。In some embodiments, after starting to play the corresponding portion of the media item: the device detects an increase in the characteristic intensity of the contact, followed by a decrease in the characteristic intensity of the contact (862) (e.g., FIG. 660; FIG. 6U shows a decrease in the intensity of contact 660). In some embodiments, a number of conditional operations are described below that are conditioned upon determining whether the intensity characteristics of the contact meet the media preview criteria. These operations are performed in response to detecting an increase in the characteristic intensity of the contact followed by a detected decrease in the characteristic intensity of the contact. In some implementations, the conditional operations described below are performed when an increase in the characteristic intensity of a contact is detected followed by a decrease in the characteristic intensity of the contact.

为此，在媒体项正在播放时，在检测到接触的特征强度的减小之前，根据确定接触的特征强度满足媒体预览标准，该媒体预览标准包括当接触的特征强度增大到高于比第一强度阈值(例如，IT_H)更高的第二强度阈值(例如，IT_L，图6T-6U)时满足的标准，该设备随着接触的特征强度的增大而以第一方式来改变媒体项的音频属性(例如，增大媒体项的音频音量)并且随着接触的特征强度的减小而使音频属性保持在第一状态(例如，继续以设定音频音量来回放媒体项)。在一些实施方案中，随着媒体项的音频属性改变，背景音频的音频属性也发生改变(例如，使背景音频随着强度的减小而淡入以及使背景音频随着强度的增大而淡出)。使音频属性随着接触的特征强度的减小而保持在第一状态(例如，继续以设定音频音量回放媒体项)允许用户进入“窥视”模式，其中媒体项的音频属性是固定的，这对于希望收听媒体项的一部分的用户是方便的，而无需持续修改音频属性。由于音频属性仅在接触强度增大到高于第二强度阈值之后才是固定的，因此用户可灵活地选择他或她想要的操作模式。这减少了实现期望设备功能所需的用户输入的数量，因而(例如，通过帮助用户在操作该设备/与该设备交互时提供正确的输入并且减少用户错误)而增强了设备的可操作性并使得用户设备界面更有效，这另外还通过使用户能够更快速且更有效地使用该设备而减少了功率使用并且改善了该设备的电池寿命。To this end, while the media item is playing, before a decrease in the characteristic strength of the contact is detected, the media preview criteria are determined based on a determination that the characteristic strength of the contact satisfies the media preview criteria, which includes when the characteristic strength of the contact increases above the first threshold. Criteria to be satisfied when one intensity threshold (e.g., IT_H ) is higher than a second intensity threshold (e.g.,_ITL , FIGS. 6T-6U ), the device changes in the first way as the characteristic intensity of the contact increases. The audio property of the media item (eg, increasing the audio volume of the media item) and maintaining the audio property in the first state (eg, continuing to play back the media item at the set audio volume) as the characteristic intensity of the contact decreases. In some embodiments, as the audio properties of the media item change, the audio properties of the background audio also change (e.g., fading the background audio in as the intensity decreases and fading the background audio out as the intensity increases) . Keeping the audio property in the first state (e.g., continuing to play back the media item at a set audio volume) as the characteristic intensity of the contact decreases allows the user to enter a "peek" mode, where the audio property of the media item is fixed, which It is convenient for a user who wishes to listen to a portion of a media item without continuously modifying the audio properties. Since the audio properties are only fixed after the contact intensity increases above the second intensity threshold, the user has the flexibility to select his or her desired mode of operation. This reduces the amount of user input required to achieve the desired device functionality, thereby enhancing the operability of the device (for example, by helping the user provide correct input and reducing user error when operating/interacting with the device) and Making the user device interface more efficient additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and efficiently.

例如，由于接触660的强度在图6T中高于IT_L，因此媒体项的音量被“锁定”到预览音量(例如，全音量)，使得接触660的强度随后如图6U所示下降到低于IT_L不会导致音量的变化。根据确定接触的特征强度不满足媒体预览标准(即，接触的特征强度不增大到高于第二强度阈值)，当播放媒体项时(以及当接触保持在触敏表面上时)，该设备随着接触的特征强度的增大而以第一方式来改变媒体项的音频属性(例如，增大媒体项的音频音量)并且随着接触的特征强度的减小而以第二方式来改变媒体项的音频属性(例如，反过来增大音频音量)。例如，如果图6S-6R所示的序列反向，则媒体项的音量随着强度的减小而减小。因此，在图6S-6R中，用户可通过增大和减小接触660的强度而上下滚动媒体项的音量，只要接触660的强度不超过IT_L。在一些实施方案中，媒体项的相应部分随着接触的特征强度改变而连续地播放，其中这组一个或多个音频参数随着媒体项的播放而改变。For example, since the intensity of contact 660 is above IT_L in FIG. 6T , the volume of the media item is “locked” to the preview volume (e.g., full volume), so that the intensity of contact 660 then drops below IT as shown in FIG. 6U_L does not cause a change in volume. Upon determining that the characteristic intensity of the contact does not meet the media preview criteria (i.e., the characteristic intensity of the contact does not increase above the second intensity threshold), the device, while playing the media item (and while the contact remains on the touch-sensitive surface), Change the audio property of the media item in a first way (for example, increase the audio volume of the media item) as the characteristic strength of the contact increases and change the media in a second way as the characteristic strength of the contact decreases. The item's audio properties (eg, increase audio volume in turn). For example, if the sequence shown in Figures 6S-6R is reversed, the volume of the media item decreases as the intensity decreases. Thus, in FIGS. 6S-6R , the user can scroll up and down the volume of the media item by increasing and decreasing the intensity of contact 660 as long as the intensity of contact 660 does not exceed IT_L . In some embodiments, corresponding portions of the media item are played continuously as the characteristic intensity of the contact changes, wherein the set of one or more audio parameters changes as the media item is played.

在一些实施方案中，响应于检测到该输入满足媒体预览标准，该设备在显示器上显示(863)接触的特征强度的增大将使得设备相对于媒体项执行选择操作的指示(例如，“更用力地按压以进行选择”或“进行3D触摸以共享”)。In some embodiments, in response to detecting that the input satisfies the media preview criteria, the device displays (863) on the display an indication that an increase in the characteristic strength of the contact will cause the device to perform a selection operation relative to the media item (e.g., "Hold harder press firmly to select" or "3D touch to share").

在一些实施方案中，在开始播放媒体项的相应部分后，该设备在接触保持在触敏表面上时，检测到接触的特征强度的增大(864)。响应于检测到接触的特征强度的增大：根据确定接触的特征强度满足媒体选择标准，该媒体选择标准包括当接触的特征强度大于比第一强度阈值更高(并且也比第二强度阈值更高，操作862)的选择强度阈值时满足的标准，该设备停止显示媒体项的相应部分并且针对媒体项来执行选择操作(例如，将媒体项放置在消息组成区域中或与即时消息会话中的另一用户共享媒体项)。例如，在图6V-6W中，由于接触660的强度超过IT_D，因此选择表示658-2，使得对应的音频消息644-4被放置在架650中。根据确定接触的特征强度不满足媒体选择标准，该设备继续播放媒体项的相应部分，而无需执行选择操作。In some embodiments, after beginning to play the corresponding portion of the media item, the device detects an increase in the characteristic intensity of the contact while the contact remains on the touch-sensitive surface (864). Responsive to detecting an increase in the characteristic intensity of the contact: based on determining that the characteristic intensity of the contact satisfies media selection criteria comprising when the characteristic intensity of the contact is greater than a first intensity threshold (and also greater than a second intensity threshold high, operation 862), the device stops displaying the corresponding portion of the media item and performs a selection operation for the media item (for example, placing the media item in a message composition area or with an instant messaging session). Another user shares the media item). For example, in FIGS. 6V-6W , since the intensity of contact 660 exceeds IT_D , representation 658 - 2 is selected, causing a corresponding audio message 644 - 4 to be placed in shelf 650 . Upon determining that the characteristic intensity of the contact does not satisfy the media selection criteria, the device continues to play the corresponding portion of the media item without performing a selection operation.

在一些实施方案中，当播放媒体项时，该设备检测到输入的结束(865)(例如，检测到致使媒体项开始播放的接触的抬离)。响应于检测到输入的结束，该设备停止播放媒体项。在一些实施方案中，响应于检测到输入的结束而停止播放媒体项包括使邻近媒体项的表示的内容的视觉模糊反转并且改变音频项目的这组一个或多个属性以使播放的媒体项逐渐淡出(866)。例如，如果输入660在图6R-6U的任一个图中已终止，则设备100将会停止播放媒体项并且使交互区域642-2的模糊反转。In some embodiments, while the media item is playing, the device detects the end of the input (865) (eg, detecting lift-off of a contact that causes the media item to begin playing). In response to detecting the end of the input, the device stops playing the media item. In some embodiments, stopping the playback of the media item in response to detecting the end of the input includes reversing the visual blurring of the content of the representation of the adjacent media item and changing the set of one or more properties of the audio item so that the played media item Fade out gradually (866). For example, if input 660 had terminated in any of Figures 6R-6U, device 100 would stop playing the media item and reverse the blurring of interaction area 642-2.

在一些实施方案中，在响应于检测到输入的结束而停止播放媒体项后，该设备在与媒体项的表示(例如，轻击手势)对应的位置处检测到通过触敏表面上的第二接触导致的选择输入，其中该第二接触不具有达到第一强度阈值的特征强度(867)。响应于检测到选择输入，该设备针对媒体项来执行选择操作(例如，将媒体项放置在消息组成区域诸如架650(图6P)中或与即时消息会话中的另一用户共享媒体项)。In some embodiments, after stopping playback of the media item in response to detecting the end of the input, the device detects, at a location corresponding to a representation of the media item (e.g., a tap gesture), passing a second tap gesture on the touch-sensitive surface. A selection input resulting from a contact wherein the second contact does not have a characteristic intensity that reaches the first intensity threshold (867). In response to detecting a selection input, the device performs a selection operation on the media item (eg, placing the media item in a message composition area such as shelf 650 (FIG. 6P) or sharing the media item with another user in the instant messaging session).

操作868-870描述了音频(即，在设备上已播放的具有背景媒体的媒体项)的模糊。方法800和840详细描述了音频模糊。因此，操作868-870可共享这些方法中所描述的特征中的任一个特征。Operations 868-870 describe the blurring of audio (ie, media items that have played on the device with background media). Methods 800 and 840 detail audio blurring. Accordingly, operations 868-870 may share any of the features described in these methods.

在一些实施方案中，当在设备上播放背景媒体时，检测到输入(868)(例如，在操作856中检测到的输入)。根据确定该输入满足媒体提示标准：当播放媒体项时，该设备随着接触的特征强度改变而动态地改变背景媒体项的一组一个或多个音频属性(869)。因此，在一些实施方案中，该设备使背景媒体上方的媒体项“淡入”，这提供了参考方法800/840所述的全部随附的益处。In some embodiments, an input is detected (868) (eg, the input detected in operation 856) while the background media is playing on the device. Based on determining that the input satisfies the media cueing criteria: the device dynamically changes a set of one or more audio attributes of the background media item as the media item is played as the characteristic intensity of the contact changes (869). Thus, in some embodiments, the device "fades in" the media item above the background media, which provides all of the attendant benefits described with reference to methods 800/840.

根据确定该输入不满足媒体提示标准，该设备继续播放背景媒体，而无需改变背景媒体的这组一个或多个音频属性。在一些实施方案中，当播放媒体项时，该设备检测到输入的结束(870)。响应于检测到输入的结束，该设备恢复播放背景媒体项，其中这组一个或多个音频属性在检测到输入之前返回到它们的值。Based on determining that the input does not satisfy the media cueing criteria, the device continues to play the background media without changing the set of one or more audio properties of the background media. In some embodiments, when the media item is played, the device detects the end of the input (870). In response to detecting the end of the input, the device resumes playing the background media item with the set of one or more audio properties returning to their values before the input was detected.

应当理解，对图8D-8F中已进行描述的操作的具体次序仅是一个示例，并非旨在表明所述次序是可执行这些操作的唯一次序。本领域的普通技术人员会想到多种方式来对本文所述的操作进行重新排序。在一些实施方案中，将方法854的一个或多个操作与本文所述其他方法(例如，方法800、840、875和/或900)的一个或多个操作组合或由它们补充或替代。It should be understood that the specific order in which the operations have been described in Figures 8D-8F is an example only, and is not intended to imply that the described order is the only order in which these operations may be performed. There are many ways to reorder the operations described herein will occur to those of ordinary skill in the art. In some embodiments, one or more operations of method 854 are combined with, supplemented to, or replaced by one or more operations of other methods described herein (eg, methods 800, 840, 875, and/or 900).

图8G-8H是根据一些实施方案示出遮蔽图形用户界面的多个部分的方法875的流程图。图6P-6Y用于示出图8G-8H的方法和/或过程。尽管将参考触敏显示器(其中组合了触敏表面和显示器)上的输入给出随后的示例中的一些示例，但在一些实施方案中，设备检测与显示器450分开的触敏表面451上的输入，如图4B所示。8G-8H are flowcharts illustrating a method 875 of obscuring portions of a graphical user interface, according to some embodiments. Figures 6P-6Y are used to illustrate the method and/or process of Figures 8G-8H. Although some of the examples that follow will be given with reference to an input on a touch-sensitive display (where the touch-sensitive surface and display are combined), in some implementations, the device detects an input on a touch-sensitive surface 451 that is separate from the display 450 , as shown in Figure 4B.

在一些实施方案中，方法875由电子设备(例如，便携式多功能设备100，图1A)和/或该电子设备的一个或多个部件(例如，I/O子系统106、操作系统126等)执行。在一些实施方案中，方法875通过被存储在非暂态计算机可读存储介质中并且由设备的一个或多个处理器诸如设备100的一个或多个处理器122(图1A)执行的指令来支配。为了容易解释，下文描述了由设备100执行的方法875。In some embodiments, method 875 is performed by an electronic device (e.g., portable multifunction device 100, FIG. 1A ) and/or one or more components of the electronic device (e.g., I/O subsystem 106, operating system 126, etc.) implement. In some embodiments, method 875 is implemented by instructions stored in a non-transitory computer-readable storage medium and executed by one or more processors of a device, such as one or more processors 122 of device 100 (FIG. 1A ). dominate. For ease of explanation, the method 875 performed by the device 100 is described below.

如下所述，方法875(和相关联的界面)减少了来自用户的输入的数量、范围和/或性质，并且产生更有效的人机界面，从而为用户提供易于使用且直观的方式来与用户界面进行交互。对于电池操作的电子设备，方法875允许通过提供节省功率并且增大电池两次充电之间的时间间隔的容易理解且信息性的音频输出(例如，通过减少对耗尽电池功率的密集且低效的用户交互的需求)来实现有效、无缝且快速的交互。As described below, the method 875 (and associated interface) reduces the number, scope, and/or nature of input from the user and produces a more efficient human-machine interface, thereby providing the user with an easy-to-use and intuitive way to communicate with the user. Interface to interact. For battery-operated electronic devices, method 875 allows for an easily comprehensible and informative audio output that conserves power and increases the time between battery charges (e.g., by reducing intensive and inefficient response to draining battery power user interaction requirements) to achieve effective, seamless and fast interaction.

在一些实施方案中，在该设备在显示器上显示包括应用程序的第一交互区域(例如，交互区域642-2，图6Q)和应用程序的第二交互区域(例如，交互区域642-1，图6Q)的用户界面(876)时，方法875开始。在一些实施方案中，交互区域为用户能够诸如通过提供多种输入手势(轻扫、轻击等)中的任一个输入手势而与其进行交互的用户界面区域，这些输入手势在被检测到时，使得电子设备根据这些手势来修改用户界面。换句话讲，交互区域并不是被显示在顶部部分的用户不能通过其来执行任何交的状态栏。In some embodiments, the display on the device includes a first interactive area of the application (e.g., interactive area 642-2, FIG. 6Q ) and a second interactive area of the application (e.g., interactive area 642-1, 6Q) user interface (876), method 875 begins. In some embodiments, an interaction area is a user interface area that a user can interact with, such as by providing any of a variety of input gestures (swipe, tap, etc.) that, when detected, The electronic device is caused to modify the user interface according to these gestures. In other words, the interaction area is not the status bar displayed at the top part through which the user cannot perform any interaction.

在一些实施方案中，第二交互区域为用于主应用程序(例如，即时消息应用程序)的用户界面(877)。第一交互区域被配置为显示来自被配置为在主应用程序内操作的不同微型应用程序的内容，并且在第一交互区域中所显示的微型应用程序基于设备处的用户输入来进行选择(例如，用户可在第一交互区域中执行轻扫手势，这些轻扫手势在被检测到时，使得该设备针对第一交互区域中的不同微型应用程序而显示用户界面，如参考指示器654(图6Q)所述的)。例如，即时消息应用程序可显示与设备媒体(例如，音乐、视频)、设备相机、绘文字艺术等进行交互的微型应用程序。In some implementations, the second interaction area is a user interface (877) for the host application (eg, an instant messaging application). The first interactive area is configured to display content from different mini-apps configured to operate within the main application, and the mini-apps displayed in the first interactive area are selected based on user input at the device (e.g. , the user may perform swipe gestures in the first interaction area that, when detected, cause the device to display a user interface for a different mini-app in the first interaction area, such as reference indicator 654 (FIG. 6Q) above). For example, an instant messaging application may display mini-apps that interact with device media (eg, music, video), device camera, emoji art, etc.

在一些实施方案中，第二交互区域为包括会话参与者之间的会话中的多个消息的会话区域(878)(例如，第二交互区域为即时消息应用程序(诸如具有电子设备的用户与至少一个其他用户之间的消息的会话记录)内的会话区域诸如交互区域642-1(图6Q)。第一交互区域为用于选择媒体以便在会话参与者之间的会话中共享的媒体选择区域(例如，即时消息应用程序内的音乐共享区域诸如交互区域642-2(图6Q)，该音乐共享区域包括可滚动表单，该可滚动表单显示该设备上最近显示的30首歌曲的表示)。In some embodiments, the second interaction area is a conversation area (878) that includes a plurality of messages in a conversation between conversation participants (e.g., the second interaction area is an instant messaging application such as a user with an electronic device interacting with Conversational area such as interaction area 642-1 (FIG. 6Q) within at least one other user's conversation record of the message). The first interaction area is the media selection for selecting media to share in the conversation between conversational participants area (e.g., a music sharing area within an instant messaging application such as interactive area 642-2 (FIG. 6Q), which includes a scrollable form that displays a representation of the last 30 songs displayed on the device) .

在显示用户界面时，该设备在显示器上在与第一交互区域中的第一用户界面元素对应的位置处检测通过触敏表面上的接触(例如，手指接触或触笔接触)导致的第一输入(879)。例如，图6R示出了用户输入660的开始，该用户输入是在表示658-2上检测到的按压并保持手势。While displaying the user interface, the device detects, on the display at a location corresponding to the first user interface element in the first interaction area, a first contact caused by a contact (eg, finger contact or stylus contact) on the touch-sensitive surface. Enter (879). For example, FIG. 6R shows the beginning of user input 660, which is a detected press and hold gesture on representation 658-2.

响应于检测到通过接触导致的第一输入：根据确定第一输入满足基于强度的激活标准，其中基于强度的激活标准要求接触具有高于第一强度阈值(例如，IT_H，图6R)的特征强度以便满足基于强度的激活标准，除第一用户界面元素之外，该设备隐藏应用程序的第一交互区域，而无需隐藏应用程序的第二交互区域(例如，使交互区域642-2模糊，而无需使表示658-2模糊，图6R-6V)(880)。除第一用户界面元素之外，遮蔽应用程序的第一交互区域，而无需遮蔽应用程序的第二交互区域，向用户指示他或她正在例如微型应用程序内启用基于强度的设备功能。这减少了用户将使得该设备执行不必要的操作的机会，这(例如，通过帮助用户在操作该设备/与该设备交互时提供正确的输入并且减少用户错误)增强了设备的可操作性并使得用户设备界面更有效，这另外还通过使用户能够更快速且更有效地使用该设备而减少了功率使用并且改善了该设备的电池寿命。Responsive to detecting a first input caused by a contact: upon determining that the first input satisfies an intensity-based activation criterion, wherein the intensity-based activation criterion requires that the contact be characterized by a first intensity threshold (e.g., IT_H , FIG. 6R ) Intensity In order to meet the intensity-based activation criteria, the device hides the first interactive area of the application except for the first user interface element without hiding the second interactive area of the application (e.g., blurring the interactive area 642-2, Without obscuring representation 658-2, FIGS. 6R-6V) (880). In addition to the first user interface element, obscuring a first interaction area of the application without obscuring a second interaction area of the application indicates to the user that he or she is enabling an intensity-based device function, eg, within the mini-app. This reduces the chance that the user will cause the device to perform unnecessary operations, which (for example, by helping the user provide correct input and reducing user error when operating/interacting with the device) enhances the operability of the device and Making the user device interface more efficient additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and efficiently.

根据确定第一输入满足第一选择标准，其中第一选择标准不要求接触具有高于第一强度阈值的特征强度以便满足选择标准(例如，第一输入为轻击手势)，该设备执行与第一用户界面元素对应的第一选择操作，而无需遮蔽应用程序的第一交互区域。在一些实施方案中，用于遮蔽上述交互区域的第一选择操作基于接触位置(例如，根据确定在与第一媒体项对应的位置处检测到接触)而确定，以上步骤针对第一媒体项而执行，并且根据确定在与第二媒体项对应的位置处检测到接触，以上步骤针对第二媒体项而不是针对第一媒体项而执行。例如，如果在表示658-1而不是表示658-2上检测到图6R中的输入660，则设备100隐藏交互区域642-2中的除表示658-1之外的任何事物，而不是隐藏交互区域642-2中的除表示658-2之外的任何事物。Based on determining that the first input satisfies a first selection criterion, wherein the first selection criterion does not require the contact to have a characteristic intensity above a first intensity threshold in order to satisfy the selection criterion (e.g., the first input is a tap gesture), the device performs the same operation as the first selection criterion. The first selection operation corresponding to a user interface element does not need to cover the first interaction area of the application program. In some embodiments, the first selection operation for obscuring the aforementioned interaction area is determined based on the location of the contact (e.g., based on a determination that a contact was detected at a location corresponding to the first media item for which the above steps are performed) Executed, and based on determining that a contact is detected at a location corresponding to the second media item, the above steps are performed for the second media item rather than for the first media item. For example, if input 660 in FIG. 6R is detected on representation 658-1 but not representation 658-2, device 100 hides anything in interaction area 642-2 except representation 658-1, rather than hiding the interaction Anything in area 642-2 except representation 658-2.

在一些实施方案中，当第一输入为轻击手势时满足第一选择标准(881)。In some implementations, the first selection criterion is met when the first input is a tap gesture (881).

在一些实施方案中，遮蔽应用程序的第一交互区域包括随着接触的特征强度改变而动态地遮蔽应用程序的第一交互区域(882)(例如，随着接触的特征强度通过多个值改变，应用于第一交互区域的模糊的模糊半径通过多个值而改变)。在一些实施方案中，第一交互区域的模糊的模糊半径随着接触的特征强度的增大而增大并且随着接触的特征强度的减小而减小。例如，随着输入660的强度从图6R到图6S增大，交互区域642-2模糊，其模糊半径与接触660的强度成比例(这些附图中，增大的模糊半径通过覆盖交互区域642-2的图案的减小的透明度而示意性地表示)。In some embodiments, shading the first interaction area of the application includes dynamically shading (882) the first interaction area of the application as the characteristic intensity of the contact changes (e.g., as the characteristic intensity of the contact changes by a plurality of values , the blur radius of the blur applied to the first interaction area varies by multiple values). In some embodiments, the blur radius of the blur of the first interaction region increases as the characteristic intensity of the contact increases and decreases as the characteristic intensity of the contact decreases. For example, as the intensity of input 660 increases from FIG. 6R to FIG. 6S , interaction region 642-2 blurs with a blur radius proportional to the intensity of contact 660 (in these figures, the increasing blur radius by covering interaction region 642 -2 is schematically represented by the reduced transparency of the pattern).

在一些实施方案中，在显示用户界面时，该设备在显示器上在与第二交互区域中的第二用户界面元素对应的位置处检测通过触敏表面上的第二接触导致的第二输入(883)。响应于检测到通过第二接触导致的第二输入：根据确定第二输入满足第二基于强度的激活标准，其中第二基于强度的激活标准要求第二接触具有高于第二强度阈值的特征强度以便满足第二基于强度的激活标准，除第二用户界面元素在之外，该设备隐藏应用程序的第一交互区域和应用程序的第二交互区域(例如，该设备使第一交互区域和第二交互区域的内容淡出，同时预览与未模糊的第二用户界面元素对应的媒体项)。在一些实施方案中，第二强度阈值与第一强度阈值相同。在一些实施方案中，第二强度阈值与第一强度阈值不同。除第二用户界面元素之外，遮蔽应用程序的第一交互区域和应用程序的第二交互区域，向用户指示他或她正在例如主应用程序内相对于第二用户界面元素而启用基于强度的设备功能。这减少了用户将使得该设备执行不必要的操作的机会，这(例如，通过帮助用户在操作该设备/与该设备交互时提供正确的输入并且减少用户错误)增强了设备的可操作性并使得用户设备界面更有效，这另外还通过使用户能够更快速且更有效地使用该设备而减少了功率使用并且改善了该设备的电池寿命。In some embodiments, while displaying the user interface, the device detects, at a location on the display corresponding to a second user interface element in the second interaction area, a second input resulting from a second contact on the touch-sensitive surface ( 883). In response to detecting a second input caused by a second contact: upon determining that the second input satisfies a second intensity-based activation criterion, wherein the second intensity-based activation criterion requires that the second contact have a characteristic intensity above a second intensity threshold In order to satisfy the second intensity-based activation criterion, the device hides the first interaction area of the application and the second interaction area of the application except for the second user interface element (e.g., the device hides the first interaction area and the second interaction area of the application). The content of the second interaction area fades out while previewing the media item corresponding to the unblurred second user interface element). In some embodiments, the second intensity threshold is the same as the first intensity threshold. In some embodiments, the second intensity threshold is different than the first intensity threshold. In addition to the second user interface element, shade the first interaction area of the application and the second interaction area of the application, indicating to the user that he or she is enabling intensity-based interaction with respect to the second user interface element, for example, within the main application. device capabilities. This reduces the chance that the user will cause the device to perform unnecessary operations, which (for example, by helping the user provide correct input and reducing user error when operating/interacting with the device) enhances the operability of the device and Making the user device interface more efficient additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and efficiently.

例如，如图6Y所示，基础674具有超过IT_H的强度，并且因此交互区域642-1和642-2两者被模糊化。因此，在微型应用程序中的媒体项的表示上的高于IT_H的按压输入导致仅微型应用程序模糊，而在与主应用程序对应的交互区域(例如，即时消息应用程序的会话区域)中的媒体项的表示上的类似的按压输入导致包括这两个交互区域642的整个用户界面模糊。For example, as shown in FIG. 6Y , base 674 has an intensity exceeding IT_H , and thus both interaction regions 642-1 and 642-2 are blurred. Thus, a press input above IT_H on a representation of a media item in a mini-app causes only the mini-app to be blurred, while in the interaction area corresponding to the main application (e.g., the conversation area of an instant messaging application) A similar press input on the representation of the media item in causes the entire user interface including these two interaction areas 642 to blur.

根据确定第二输入满足第二选择标准，其中第二选择标准不要求第二接触具有高于第二强度阈值的特征强度以便满足第二选择标准(例如，第二输入为轻击手势)，该设备执行与第二用户界面元素对应的第二选择操作，而无需遮蔽应用程序的第二交互区域并且无需遮蔽应用程序的第一交互区域(例如，该设备重放与第二元素对应的媒体项)。例如，图6Q中的表示658的任一表示上的轻击手势将媒体项放置在架650中(或根据一些实施方案，将这些媒体项直接发送到会话)。Based on determining that the second input satisfies a second selection criterion, wherein the second selection criterion does not require the second contact to have a characteristic intensity above a second intensity threshold in order to satisfy the second selection criterion (e.g., the second input is a tap gesture), the The device performs a second selection operation corresponding to the second user interface element without obscuring the second interaction area of the application and without obscuring the first interaction area of the application (e.g., the device plays back a media item corresponding to the second element ). For example, a tap gesture on any of the representations 658 in FIG. 6Q places media items in the shelf 650 (or sends the media items directly to the session, according to some embodiments).

在一些实施方案中，在第一用户界面元素基于第一交互区域中的输入而被添加到第二用户交互区域之后，第二用户界面元素对应于第一用户界面元素(884)(例如，第二用户界面元素对应于从第一交互区域中选择的并且被添加到第二交互区域中显示的会话的媒体项)。例如，在图6Q-6Y中，音频消息644-4和表示658-2两者对应于同一媒体项(例如，Brahms’s Alto Rhapsody)。In some embodiments, after the first user interface element is added to the second user interaction area based on the input in the first interaction area, the second user interface element corresponds to the first user interface element (884) (e.g., The second user interface element corresponds to a media item selected from the first interactive area and added to the session displayed in the second interactive area). For example, in FIGS. 6Q-6Y, both audio message 644-4 and representation 658-2 correspond to the same media item (eg, Brahms' Alto Rhapsody).

在一些实施方案中，当除第一用户界面元素之外遮蔽应用程序的第一交互区域(而无需遮蔽应用程序的第二交互区域)时，该设备检测到接触的特征强度的增大。响应于检测到接触的特征强度的增大：根据确定接触的特征强度满足第三选择标准，该第三选择标准包括当接触的特征强度大于比第一强度阈值更高的选择强度阈值(例如，IT_D，图6V)时满足的标准，该设备对第一用户界面元素执行第三选择操作并且停止遮蔽应用程序的第一交互区域。因此，由于接触660在图6V中超过IT_D，因此表示658-2弹出到架650中(其中将该表示重新标示为音频消息644-4)。根据确定接触的特征强度满足第三选择标准而对第一用户界面元素执行选择操作并且停止遮蔽应用程序的第一交互区域允许用户一旦用户已发起上述基于强度的操作便选择第一用户界面元素，从而消除了对选择第一用户界面元素并且使得人机界面更有效的附加用户输入的需要。In some embodiments, the device detects an increase in the characteristic intensity of the contact when a first interaction area of the application is obscured in addition to the first user interface element (without obscuring a second interaction area of the application). Responsive to detecting an increase in the characteristic intensity of the contact: upon determining that the characteristic intensity of the contact satisfies a third selection criterion, the third selection criterion comprising when the characteristic intensity of the contact is greater than a selected intensity threshold higher than the first intensity threshold (e.g., IT_D , FIG. 6V ), the device performs a third selection operation on the first user interface element and stops obscuring the first interaction area of the application. Accordingly, since contact 660 passes IT_D in FIG. 6V , representation 658-2 pops into shelf 650 (where the representation is relabeled as audio message 644-4). performing a selection operation on the first user interface element upon determining that the characteristic intensity of the contact satisfies a third selection criterion and ceasing to obscure the first interaction area of the application allows the user to select the first user interface element once the user has initiated the intensity-based operation described above, The need for additional user input to select the first user interface element and make the human-machine interface more efficient is thereby eliminated.

根据确定输入的特征强度不满足第三选择标准，除第一用户界元素之外，该设备继续隐藏应用程序的第一交互区域，而无需隐藏应用程序的第二交互区域。Based on determining that the characteristic strength of the input does not satisfy the third selection criterion, the device continues to hide the first interaction area of the application, without hiding the second interaction area of the application, except for the first user interface element.

应当理解，对图8G-8H中已进行描述的操作的具体次序仅是一个示例，并非旨在表明所述次序是可执行这些操作的唯一次序。本领域的普通技术人员会想到多种方式来对本文所述的操作进行重新排序。在一些实施方案中，将方法875的一个或多个操作与本文所述其他方法(例如，方法840、840、854和/或900)的一个或多个操作组合或由它们补充或替代。It should be understood that the specific order in which the operations have been described in Figures 8G-8H is an example only, and is not intended to imply that the described order is the only order in which these operations may be performed. There are many ways to reorder the operations described herein will occur to those of ordinary skill in the art. In some embodiments, one or more operations of method 875 are combined with, supplemented with, or replaced by one or more operations of other methods described herein (eg, methods 840, 840, 854, and/or 900).

图9A-9C是示出根据一些实施方案的动态地调整音频输出呈现的方法900的流程图。图7A-7G用于示出图9A-9C的方法和/或过程。尽管将参考触敏显示器(其中组合了触敏表面和显示器)上的输入给出随后的示例中的一些示例，但在一些实施方案中，设备检测与显示器450分开的触敏表面451上的输入，如图4B所示。9A-9C are flowcharts illustrating a method 900 of dynamically adjusting audio output presentation according to some embodiments. Figures 7A-7G are used to illustrate the method and/or process of Figures 9A-9C. Although some of the examples that follow will be given with reference to an input on a touch-sensitive display (where the touch-sensitive surface and display are combined), in some implementations, the device detects an input on a touch-sensitive surface 451 that is separate from the display 450 , as shown in Figure 4B.

在一些实施方案中，方法900由电子设备(例如，便携式多功能设备100，图1A)和/或该电子设备的一个或多个部件(例如，I/O子系统106、操作系统126等)执行。在一些实施方案中，方法900通过被存储在非暂态计算机可读存储介质中且由设备的一个或多个处理器诸如设备100的一个或多个处理器122(图1A)执行的指令来支配。为了容易解释，下文描述了由设备100执行的方法900。在一些实施方案中，参考图1A，方法900的操作至少部分地由音频配置文件402、音频更改模块163-3和触敏显示器(例如，触敏显示系统112)执行或至少部分地对其进行使用。方法900中的一些操作任选地被组合，和/或一些操作的顺序任选地被改变。In some embodiments, method 900 is performed by an electronic device (e.g., portable multifunction device 100, FIG. 1A ) and/or one or more components of the electronic device (e.g., I/O subsystem 106, operating system 126, etc.) implement. In some embodiments, method 900 is implemented by instructions stored in a non-transitory computer-readable storage medium and executed by one or more processors of a device, such as one or more processors 122 of device 100 (FIG. 1A ). dominate. For ease of explanation, the method 900 performed by the device 100 is described below. In some embodiments, referring to FIG. 1A , the operations of method 900 are performed at least in part by or at least in part performed by audio profile 402, audio modification module 163-3, and a touch-sensitive display (e.g., touch-sensitive display system 112). use. Some operations in method 900 are optionally combined, and/or the order of some operations is optionally changed.

如下所述，方法900(和相关联的界面)减少了来自用户的输入的数量、范围和/或性质，并且产生更有效的人机界面，从而为用户提供易于使用且直观的方式来与用户界面进行交互。对于电池操作的电子设备，方法900允许通过提供节省功率并且增大电池两次充电之间的时间间隔的容易理解且信息性的音频输出(例如，通过减少对耗尽电池功率的密集且低效的用户交互的需求)来实现有效、无缝且快速的交互。As described below, the method 900 (and associated interface) reduces the number, scope, and/or nature of input from the user and produces a more efficient human-machine interface, thereby providing the user with an easy-to-use and intuitive way to communicate with the user. Interface to interact. For battery-operated electronic devices, method 900 allows for an easily comprehensible and informative audio output that conserves power and increases the time between battery charges (e.g., by reducing intensive and inefficient response to draining battery power user interaction requirements) to achieve effective, seamless and fast interaction.

在一些实施方案中，在与显示器和音频系统进行通信的电子设备上执行方法900(902)，同时在电子设备的显示器上显示包括一组一个或多个示能表示的用户界面(例如，所显示的用户界面包括虚拟数字键盘并且这组一个或多个示能表示包括被显示在该虚拟数字键盘上的数字键，图7A-7G)。方法900开始时，设备在第一时间点检测到指向这组一个或多个示能表示中的第一示能表示的第一输入(例如，检测到虚拟键盘中的第一键上的轻击，诸如图7B中的“5”键上的轻击)(904)。In some embodiments, method 900 (902) is performed on an electronic device in communication with a display and an audio system while a user interface comprising a set of one or more affordances (e.g., the The displayed user interface includes a virtual numeric keypad and the set of one or more affordances includes numeric keys displayed on the virtual numeric keypad, FIGS. 7A-7G ). Method 900 begins when the device detects, at a first point in time, a first input directed to a first affordance in the set of one or more affordances (e.g., detecting a tap on a first key on a virtual keyboard , such as a tap on the "5" key in Figure 7B) (904).

响应于检测到指向第一示能表示的第一输入，该设备开始向音频系统提供第一声音信息以呈现与第一示能表示对应的第一音频输出(906)。第一音频输出具有第一音频配置文件(例如，音频配置文件714-1，图7B-7G)。In response to detecting the first input directed to the first affordance, the device begins providing first sound information to the audio system to present a first audio output corresponding to the first affordance (906). The first audio output has a first audio profile (eg, audio profile 714-1, FIGS. 7B-7G ).

在一些实施方案中，相应的音频配置文件包括随时间变化来支配对应音频输出的一个或多个属性的信息。例如，相应的音频配置文件随时间变化来支配对应音频输出的音高、语音、音色、混响和/或音量(例如，衰减)。在一些实施方案中，相应的音频配置文件还包括支配对应音频输出的一个或多个静态属性的信息。例如，相应的音频配置文件包括用于产生音频输出的信息，该音频输出具有C升音固定音高，其中非静态音量随时间推移而增大随后衰减。In some embodiments, a corresponding audio profile includes information that varies over time to govern one or more properties of the corresponding audio output. For example, the corresponding audio profile varies over time to govern the pitch, voice, timbre, reverberation, and/or volume (eg, decay) of the corresponding audio output. In some embodiments, a corresponding audio profile also includes information governing one or more static properties of the corresponding audio output. For example, a corresponding audio profile includes information for producing an audio output having a C-sharp fixed pitch with a non-static volume that increases and then decays over time.

在一些实施方案中，当音频系统正在呈现第一音频输出时，该设备使得(908)显示器呈现与第一音频输出对应的视觉效应(例如，向外或远离第一示能表示延伸的一个或多个图形如环，诸如涟漪效应，其中环向外或远离显示器上的与第一输入对应的位置而扩展)(例如，图形718、720，图7C-7D)。在一些实施方案中，这些图形包括远离第一示能表示移动的一个或多个图形对象的动画。呈现与第一音频输出对应的视觉效应为用户提供与该音频输出对应的直观视觉提示，这有助于用户与他们的设备进行交互。In some implementations, when the audio system is presenting the first audio output, the device causes (908) the display to present a visual effect corresponding to the first audio output (e.g., one or A plurality of graphics such as rings, such as a ripple effect, where the rings expand outward or away from the location on the display corresponding to the first input) (eg, graphics 718, 720, FIGS. 7C-7D ). In some embodiments, the graphics include animation of one or more graphical objects moving away from the first affordance. Presenting a visual effect corresponding to the first audio output provides the user with an intuitive visual cue corresponding to the audio output, which facilitates the user's interaction with their device.

该设备在第一时间点之后的第二时间点检测到指向这组一个或多个示能表示中的第二示能表示的第二输入(例如，检测到虚拟键盘中的第二键上的轻击，诸如图7C中的“2”键上的轻击)(910)。在一些实施方案中，第二示能表示和第一示能表示为同一示能表示(例如，所显示的用户界面内的所显示的键盘上的同一键)(912)。在一些实施方案中，第一示能表示与第二示能表示不同(例如，所显示的用户界面内的所显示的键盘上的不同键)(914)。The device detects, at a second point in time subsequent to the first point in time, a second input directed to a second affordance in the set of one or more affordances (e.g., detecting a Tap, such as on the "2" key in Figure 7C) (910). In some implementations, the second affordance and the first affordance are the same affordance (eg, the same key on the displayed keyboard within the displayed user interface) (912). In some implementations, the first affordance is different than the second affordance (eg, a different key on a displayed keyboard within the displayed user interface) (914).

响应于检测到指向第二示能表示的第二输入并且根据确定满足音频更改标准，该设备(916)：In response to detecting the second input directed to the second affordance and based on determining that the audio modification criteria are met, the device (916):

(i)使得音频系统呈现与第一示能表示对应的更改第一音频输出，而不是继续利用第一音频配置文件来呈现第一音频输出，其中经更改的第一音频输出具有与第一音频配置文件不同的经更改的音频配置文件。在一些实施方案中，使得音频系统呈现经更改的第一音频输出包括向音频系统提供信息诸如指令和/或声音数据，以使音频系统能够呈现经更改的第一音频输出；以及(i) Cause the audio system to present a modified first audio output corresponding to the first affordance, instead of continuing to render the first audio output using the first audio profile, wherein the modified first audio output has the same Altered audio profiles with different profiles. In some embodiments, causing the audio system to present the modified first audio output includes providing information, such as instructions and/or sound data, to the audio system to enable the audio system to present the modified first audio output; and

(ii)向音频系统提供第二声音信息，以呈现与第二示能表示对应的第二音频输出，其中第二音频输出具有第二音频配置文件。在一些实施方案中，第二音频配置文件与第一音频配置文件相同(例如，每个音频配置文件为响应于示能表示的激活而产生的音频输出的默认音频配置文件)。在一些实施方案中，第二音频配置文件与第一音频配置文件不同。呈现与第一示能表示对应的第一音频输出，而不是继续利用第一音频配置文件来呈现第一音频输出，这使音频输出较少干扰用户并且是用户更期望的。这继而使用户更有可能利用音频输出功能。(ii) providing the second sound information to the audio system to present a second audio output corresponding to the second affordance, wherein the second audio output has a second audio profile. In some implementations, the second audio profile is the same as the first audio profile (eg, each audio profile is a default audio profile for audio output generated in response to activation of an affordance). In some implementations, the second audio profile is different than the first audio profile. Presenting the first audio output corresponding to the first affordance, rather than continuing to present the first audio output with the first audio profile, makes the audio output less intrusive and more desirable to the user. This in turn makes it more likely for users to take advantage of the audio output functionality.

在一些实施方案中，响应于检测到指向第二示能表示的第二输入，该设备确定经由音频系统在第二时间点呈现的第一音频输出是否满足音频更改标准(918)。In some implementations, in response to detecting the second input directed to the second affordance, the device determines whether the first audio output presented via the audio system at the second point in time satisfies the audio modification criteria (918).

在一些实施方案中，音频更改标准包括当第一时间点与第二时间点之间的经过的时间量小于预先确定的时间量(例如，T_阈值，图7B-7G)时满足的标准(920)。In some embodiments, the audio modification criteria include criteria met when the amount of time elapsed between the first point in time and the second point in time is less than a predetermined amount of time (e.g., T_threshold , FIGS. 7B-7G ) (920 ).

在一些实施方案中，该音频更改标准包括在自发起第一音频输出以来所经过的时间小于预先确定的时间量时满足的标准(922)。In some implementations, the audio modification criteria include a criterion met when the time elapsed since the first audio output was initiated is less than a predetermined amount of time (922).

在一些实施方案中，该音频更改标准包括当第一音频输出的量值在第二时间点下降到低于预先确定的量值时满足的标准(924)。在一些实施方案中，确定第一音频输出是否满足音频更改标准包括在预先确定的时间(例如，在第二时间点或接近第二时间点的时间)确定自发起第一音频输出以来所经过的时间是否小于预先确定的时间阈值和/或确定第一音频输出的量值是否下降到低于预先确定的量值(例如，第一音频输出的初始量值或最大量值的一半)。In some embodiments, the audio modification criteria include a criterion met when the magnitude of the first audio output falls below a predetermined magnitude at the second point in time (924). In some embodiments, determining whether the first audio output meets the audio modification criteria includes determining at a predetermined time (e.g., at or near a second point in time) the time elapsed since initiating the first audio output. time is less than a predetermined time threshold and/or determine whether the magnitude of the first audio output has dropped below a predetermined magnitude (eg, half of the initial magnitude or maximum magnitude of the first audio output).

响应于检测到指向第二示能表示的第二输入并且根据确定未满足音频更改标准，该设备(926)：In response to detecting the second input directed to the second affordance and based on the determination that the audio modification criteria are not met, the device (926):

(i)使得音频系统继续呈现与第一示能表示对应的并且具有第一音频配置文件的第一音频输出；以及(i) causing the audio system to continue presenting the first audio output corresponding to the first affordance and having the first audio profile; and

(ii)向音频系统提供第二声音信息以呈现与第二示能表示对应的第三音频输出，其中第三音频输出具有第三音频配置文件。在一些实施方案中，第三音频配置文件与第二音频配置文件相同。在一些实施方案中，第二音频配置文件与第三音频配置文件不同。(ii) providing the second sound information to the audio system to present a third audio output corresponding to the second affordance, wherein the third audio output has a third audio profile. In some implementations, the third audio profile is the same as the second audio profile. In some implementations, the second audio profile is different than the third audio profile.

在一些实施方案中，经更改的音频配置文件具有与当音频系统继续使用第一音频配置文件呈现第一音频输出时产生第一音频输出的音量相比减小的音量(例如，经更改的音频配置文件包括增大第一音频输出的衰减)。在一些实施方案中，经更改的音频配置文件包括经更改的第一音频输出的音高。在一些实施方案中，经更改的音频配置文件在该设备检测到第二输入之后的至少一个时间段内具有非零音量。In some implementations, the altered audio profile has a reduced volume (e.g., the altered audio profile includes increasing the attenuation of the first audio output). In some implementations, the altered audio profile includes an altered pitch of the first audio output. In some embodiments, the altered audio profile has a non-zero volume for at least one time period after the device detects the second input.

应当理解，对图9A-9C中已进行描述的操作的具体次序仅是一个示例，并非旨在表明所述次序是可执行这些操作的唯一次序。本领域的普通技术人员会想到多种方式来对本文所述的操作进行重新排序。在一些实施方案中，将方法900的一个或多个操作与本文所述其他方法(例如，方法800、840、854和/或875)的一个或多个操作组合或由它们补充或替代。It should be understood that the specific order in which the operations have been described in Figures 9A-9C is an example only, and is not intended to imply that the described order is the only order in which these operations may be performed. There are many ways to reorder the operations described herein will occur to those of ordinary skill in the art. In some embodiments, one or more operations of method 900 are combined with, supplemented with, or replaced by one or more operations of other methods described herein (eg, methods 800, 840, 854, and/or 875).

出于解释的目的，前面的描述是通过参考具体实施方案来描述的。然而，上面的例示性论述并非旨在是穷尽的或将本发明限制为所公开的精确形式。根据以上教导内容，很多修改形式和变型形式都是可能的。选择和描述实施方案是为了最佳地阐明本发明的原理及其实际应用，以便由此使得本领域的其他技术人员能够最佳地使用具有适合于所构想的特定用途的各种修改的本发明以及各种所描述的实施方案。The foregoing description, for purposes of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in light of the above teaching. The embodiment was chosen and described in order to best explain the principles of the invention and its practical application, to thereby enable others skilled in the art to best utilize the invention with various modifications as are suited to the particular use contemplated. and various described embodiments.

Claims (40)

Translated fromChinese

1.一种方法，包括：1. A method comprising:在具有显示器、触敏表面和被配置为检测与所述触敏表面的接触的强度的一个或多个传感器的电子设备处：At an electronic device having a display, a touch-sensitive surface, and one or more sensors configured to detect the intensity of contact with the touch-sensitive surface:在所述显示器上显示包括应用程序的第一交互区域和所述应用程序的第二交互区域的用户界面；displaying on the display a user interface comprising a first interaction area of an application and a second interaction area of the application;其特征在于It is characterized by在显示所述用户界面时，在所述显示器上在与所述第一交互区域中的第一用户界面元素对应的位置处检测通过所述触敏表面上的接触导致的第一输入；detecting, while the user interface is displayed, a first input by contact on the touch-sensitive surface at a location on the display corresponding to a first user interface element in the first interaction area;响应于检测到通过所述接触导致的所述第一输入：In response to detecting the first input caused by the contact:根据确定所述第一输入满足基于强度的激活标准，其中所述基于强度的激活标准要求所述接触具有高于第一强度阈值的特征强度以便满足所述基于强度的激活标准，除所述第一用户界面元素之外，遮蔽所述应用程序的所述第一交互区域，而无需遮蔽所述应用程序的所述第二交互区域；以及，Based on determining that the first input satisfies an intensity-based activation criterion, wherein the intensity-based activation criterion requires the contact to have a characteristic intensity above a first intensity threshold in order to satisfy the intensity-based activation criterion, except that the second A user interface element that obscures the first interaction area of the application without obscuring the second interaction area of the application; and,根据确定所述第一输入满足第一选择标准，其中所述第一选择标准不要求所述接触具有高于所述第一强度阈值的特征强度以便满足所述选择标准，执行与所述第一用户界面元素对应的第一选择操作，而无需遮蔽所述应用程序的所述第一交互区域。Based on a determination that the first input satisfies a first selection criterion, wherein the first selection criterion does not require the contact to have a characteristic intensity above the first intensity threshold in order to satisfy the selection criterion, performing an operation with the first The first selection operation corresponding to the user interface element does not need to cover the first interaction area of the application program.2.根据权利要求1所述的方法，其中遮蔽所述应用程序的所述第一交互区域包括随着所述接触的所述特征强度改变而动态地遮蔽所述应用程序的所述第一交互区域。2. The method of claim 1, wherein shading the first interaction area of the application comprises dynamically shading the first interaction area of the application as the characteristic strength of the contact changes area.3.根据权利要求1-2中任一项所述的方法，包括：3. The method according to any one of claims 1-2, comprising:在显示所述用户界面时，在所述显示器上在与所述第二交互区域中的第二用户界面元素对应的位置处检测通过所述触敏表面上的第二接触导致的第二输入；detecting, while the user interface is displayed, a second input caused by a second contact on the touch-sensitive surface at a location on the display corresponding to a second user interface element in the second interaction area;响应于检测到通过所述第二接触导致的所述第二输入：In response to detecting the second input caused by the second contact:根据确定所述第二输入满足第二基于强度的激活标准，其中所述第二基于强度的激活标准要求所述第二接触具有高于第二强度阈值的特征强度以便满足所述第二基于强度的激活标准，除所述第二用户界面元素之外，遮蔽所述应用程序的所述第一交互区域和所述应用程序的所述第二交互区域；以及Based on determining that the second input satisfies a second intensity-based activation criterion, wherein the second intensity-based activation criterion requires the second contact to have a characteristic intensity above a second intensity threshold in order to satisfy the second intensity-based activation criterion activation criteria for, except for the second user interface element, obscuring the first interaction area of the application and the second interaction area of the application; and根据确定所述第二输入满足第二选择标准，其中所述第二选择标准不要求所述第二接触具有高于所述第二强度阈值的特征强度以便满足所述第二选择标准，执行与所述第二用户界面元素对应的第二选择操作，而无需遮蔽所述应用程序的所述第二交互区域并且无需遮蔽所述应用程序的所述第一交互区域。Based on a determination that the second input satisfies a second selection criterion, wherein the second selection criterion does not require the second contact to have a characteristic intensity above the second intensity threshold in order to satisfy the second selection criterion, performing an AND The second selection operation corresponding to the second user interface element does not need to cover the second interaction area of the application program and does not need to cover the first interaction area of the application program.4.根据权利要求3所述的方法，其中在所述第一用户界面元素基于所述第一交互区域中的输入而被添加到所述第二交互区域之后，所述第二用户界面元素对应于所述第一用户界面元素。4. The method of claim 3, wherein after the first user interface element is added to the second interactive area based on input in the first interactive area, the second user interface element corresponds to on the first UI element.5.根据权利要求1-4中任一项所述的方法，其中：5. The method according to any one of claims 1-4, wherein:所述第二交互区域为包括会话参与者之间的会话中的多个消息的会话区域；并且The second interaction area is a conversation area that includes a plurality of messages in a conversation between conversation participants; and所述第一交互区域为用于选择媒体以在所述会话参与者之间的所述会话中共享的媒体选择区域。The first interaction area is a media selection area for selecting media to share in the session between the session participants.6.根据权利要求1-5中任一项所述的方法，其中：6. The method according to any one of claims 1-5, wherein:所述第二交互区域为用于主应用程序的用户界面；并且the second interaction area is a user interface for the main application; and所述第一交互区域被配置为显示来自被配置为在所述主应用程序内进行操作的不同微型应用程序的内容，并且在所述第一交互区域中所显示的所述微型应用程序基于所述设备处的用户输入而被选择。The first interactive area is configured to display content from different mini-apps configured to operate within the main application, and the mini-apps displayed in the first interactive area are based on the selected by user input at the device.7.根据权利要求1-6中任一项所述的方法，包括：7. The method according to any one of claims 1-6, comprising:除所述第一用户界面元素之外，当遮蔽所述应用程序的所述第一交互区域而无需遮蔽所述应用程序的所述第二交互区域时，检测所述接触的所述特征强度的增大；以及In addition to the first user interface element, detecting the intensity of the characteristic of the contact when the first interaction area of the application is obscured without obscuring the second interaction area of the application increase; and响应于检测到所述接触的所述特征强度的所述增大：In response to detecting the increase in the characteristic strength of the contact:根据确定所述接触的所述特征强度满足第三选择标准，所述第三选择标准包括当所述接触的所述特征强度大于比所述第一强度阈值更高的选择强度阈值时满足的标准，对所述第一用户界面元素执行第三选择操作并且停止遮蔽所述应用程序的所述第一交互区域；以及Based on determining that said characteristic intensity of said contact satisfies a third selection criterion, said third selection criterion comprising a criterion satisfied when said characteristic intensity of said contact is greater than a selection intensity threshold higher than said first intensity threshold , performing a third selection operation on the first user interface element and stopping obscuring the first interaction area of the application; and根据确定所述输入的所述特征强度不满足所述第三选择标准，除所述第一用户界元素之外，继续遮蔽所述应用程序的所述第一交互区域，而无需遮蔽所述应用程序的所述第二交互区域。Continuing to shade the first interaction area of the application without shading the application upon determining that the characteristic strength of the input does not satisfy the third selection criterion, except for the first user interface element The second interactive area of the program.8.根据权利要求1-7中任一项所述的方法，其中在所述第一输入为轻击手势时满足所述第一选择标准。8. The method of any one of claims 1-7, wherein the first selection criterion is satisfied when the first input is a tap gesture.9.一种电子设备，包括：9. An electronic device comprising:显示器；monitor;触敏表面；touch-sensitive surfaces;一个或多个传感器，所述一个或多个传感器被配置为检测与所述触敏表面的接触的强度；one or more sensors configured to detect the intensity of contact with the touch-sensitive surface;一个或多个处理器；one or more processors;存储器；和memory; and一个或多个程序，其中所述一个或多个程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个程序包括用于以下操作的指令：one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for :在所述显示器上显示包括应用程序的第一交互区域和所述应用程序的第二交互区域的用户界面；displaying on the display a user interface comprising a first interaction area of an application and a second interaction area of the application;在显示所述用户界面时，在所述显示器上在与所述第一交互区域中的第一用户界面元素对应的位置处检测通过所述触敏表面上的接触导致的第一输入；detecting, while the user interface is displayed, a first input by contact on the touch-sensitive surface at a location on the display corresponding to a first user interface element in the first interaction area;响应于检测到通过所述接触导致的所述第一输入：In response to detecting the first input caused by the contact:根据确定所述第一输入满足基于强度的激活标准，其中所述基于强度的激活标准要求所述接触具有高于第一强度阈值的特征强度以便满足所述基于强度的激活标准，除所述第一用户界面元素之外，遮蔽所述应用程序的所述第一交互区域，而无需遮蔽所述应用程序的所述第二交互区域；以及，Based on determining that the first input satisfies an intensity-based activation criterion, wherein the intensity-based activation criterion requires the contact to have a characteristic intensity above a first intensity threshold in order to satisfy the intensity-based activation criterion, except that the second A user interface element that obscures the first interaction area of the application without obscuring the second interaction area of the application; and,根据确定所述第一输入满足第一选择标准，其中所述第一选择标准不要求所述接触具有高于所述第一强度阈值的特征强度以便满足所述选择标准，执行与所述第一用户界面元素对应的第一选择操作，而无需遮蔽所述应用程序的所述第一交互区域。Based on a determination that the first input satisfies a first selection criterion, wherein the first selection criterion does not require the contact to have a characteristic intensity above the first intensity threshold in order to satisfy the selection criterion, performing an operation with the first The first selection operation corresponding to the user interface element does not need to cover the first interaction area of the application program.10.根据权利要求9所述的电子设备，其中遮蔽所述应用程序的所述第一交互区域包括随着所述接触的所述特征强度改变而动态地遮蔽所述应用程序的所述第一交互区域。10. The electronic device of claim 9, wherein obscuring the first interaction area of the application comprises dynamically obscuring the first interaction area of the application as the characteristic strength of the contact changes. interactive area.11.根据权利要求9-10中任一项所述的电子设备，所述一个或多个程序包括用于进行以下操作的指令：11. The electronic device of any one of claims 9-10, the one or more programs comprising instructions for:在显示所述用户界面时，在所述显示器上在与所述第二交互区域中的第二用户界面元素对应的位置处检测通过所述触敏表面上的第二接触导致的第二输入；detecting, while the user interface is displayed, a second input caused by a second contact on the touch-sensitive surface at a location on the display corresponding to a second user interface element in the second interaction area;响应于检测到通过所述第二接触导致的所述第二输入：In response to detecting the second input caused by the second contact:根据确定所述第二输入满足第二基于强度的激活标准，其中所述第二基于强度的激活标准要求所述第二接触具有高于第二强度阈值的特征强度以便满足所述第二基于强度的激活标准，除所述第二用户界面元素之外，遮蔽所述应用程序的所述第一交互区域和所述应用程序的所述第二交互区域；以及Based on determining that the second input satisfies a second intensity-based activation criterion, wherein the second intensity-based activation criterion requires the second contact to have a characteristic intensity above a second intensity threshold in order to satisfy the second intensity-based activation criterion activation criteria for, except for the second user interface element, obscuring the first interaction area of the application and the second interaction area of the application; and根据确定所述第二输入满足第二选择标准，其中所述第二选择标准不要求所述第二接触具有高于所述第二强度阈值的特征强度以便满足所述第二选择标准，执行与所述第二用户界面元素对应的第二选择操作，而无需遮蔽所述应用程序的所述第二交互区域并且无需遮蔽所述应用程序的所述第一交互区域。Based on a determination that the second input satisfies a second selection criterion, wherein the second selection criterion does not require the second contact to have a characteristic intensity above the second intensity threshold in order to satisfy the second selection criterion, performing an AND The second selection operation corresponding to the second user interface element does not need to cover the second interaction area of the application program and does not need to cover the first interaction area of the application program.12.根据权利要求11所述的电子设备，其中在所述第一用户界面元素基于所述第一交互区域中的输入而被添加到所述第二交互区域之后，所述第二用户界面元素对应于所述第一用户界面元素。12. The electronic device of claim 11 , wherein after the first user interface element is added to the second interactive area based on an input in the first interactive area, the second user interface element Corresponds to the first user interface element.13.根据权利要求9-12中任一项所述的电子设备，其中：13. The electronic device according to any one of claims 9-12, wherein:所述第二交互区域为包括会话参与者之间的会话中的多个消息的会话区域；并且The second interaction area is a conversation area that includes a plurality of messages in a conversation between conversation participants; and所述第一交互区域为用于选择媒体以在所述会话参与者之间的所述会话中共享的媒体选择区域。The first interaction area is a media selection area for selecting media to share in the session between the session participants.14.根据权利要求9-13中任一项所述的电子设备，其中：14. The electronic device according to any one of claims 9-13, wherein:所述第二交互区域为用于主应用程序的用户界面；并且the second interaction area is a user interface for the main application; and所述第一交互区域被配置为显示来自被配置为在所述主应用程序内进行操作的不同微型应用程序的内容，并且在所述第一交互区域中所显示的所述微型应用程序基于所述设备处的用户输入而被选择。The first interactive area is configured to display content from different mini-apps configured to operate within the main application, and the mini-apps displayed in the first interactive area are based on the selected by user input at the device.15.根据权利要求9-14中任一项所述的电子设备，所述一个或多个程序包括用于进行以下操作的指令：15. The electronic device of any one of claims 9-14, the one or more programs comprising instructions for:除所述第一用户界面元素之外，当遮蔽所述应用程序的所述第一交互区域而无需遮蔽所述应用程序的所述第二交互区域时，检测所述接触的所述特征强度的增大；以及In addition to the first user interface element, detecting the intensity of the characteristic of the contact when the first interaction area of the application is obscured without obscuring the second interaction area of the application increase; and响应于检测到所述接触的所述特征强度的所述增大：In response to detecting the increase in the characteristic strength of the contact:根据确定所述接触的所述特征强度满足第三选择标准，所述第三选择标准包括当所述接触的所述特征强度大于比所述第一强度阈值更高的选择强度阈值时满足的标准，对所述第一用户界面元素执行第三选择操作并且停止遮蔽所述应用程序的所述第一交互区域；以及Based on determining that said characteristic intensity of said contact satisfies a third selection criterion, said third selection criterion comprising a criterion satisfied when said characteristic intensity of said contact is greater than a selection intensity threshold higher than said first intensity threshold , performing a third selection operation on the first user interface element and stopping obscuring the first interaction area of the application; and根据确定所述输入的所述特征强度不满足所述第三选择标准，除所述第一用户界元素之外，继续遮蔽所述应用程序的所述第一交互区域，而无需遮蔽所述应用程序的所述第二交互区域。Continuing to shade the first interaction area of the application without shading the application upon determining that the characteristic strength of the input does not satisfy the third selection criterion, except for the first user interface element The second interactive area of the program.16.根据权利要求9-15中任一项所述的电子设备，其中在所述第一输入为轻击手势时满足所述第一选择标准。16. The electronic device according to any one of claims 9-15, wherein the first selection criterion is satisfied when the first input is a tap gesture.17.一种存储一个或多个程序的计算机可读存储介质，所述一个或多个程序包括指令，所述指令当由具有显示器、触敏表面和被配置为检测与所述触敏表面的接触的强度的一个或多个传感器的电子设备执行时使得所述设备：17. A computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that when executed by a computer having a display, a touch-sensitive surface, and a device configured to detect contact with the touch-sensitive surface One or more sensors of the intensity of contact performed by the electronic device such that the device:在所述显示器上显示包括应用程序的第一交互区域和所述应用程序的第二交互区域的用户界面；displaying on the display a user interface comprising a first interaction area of an application and a second interaction area of the application;在显示所述用户界面时，在所述显示器上在与所述第一交互区域中的第一用户界面元素对应的位置处检测通过所述触敏表面上的接触导致的第一输入；detecting, while the user interface is displayed, a first input by contact on the touch-sensitive surface at a location on the display corresponding to a first user interface element in the first interaction area;响应于检测到通过所述接触导致的所述第一输入：In response to detecting the first input caused by the contact:根据确定所述第一输入满足基于强度的激活标准，其中所述基于强度的激活标准要求所述接触具有高于第一强度阈值的特征强度以便满足所述基于强度的激活标准，除所述第一用户界面元素之外，遮蔽所述应用程序的所述第一交互区域，而无需遮蔽所述应用程序的所述第二交互区域；以及，Based on determining that the first input satisfies an intensity-based activation criterion, wherein the intensity-based activation criterion requires the contact to have a characteristic intensity above a first intensity threshold in order to satisfy the intensity-based activation criterion, except that the second A user interface element that obscures the first interaction area of the application without obscuring the second interaction area of the application; and,根据确定所述第一输入满足第一选择标准，其中所述第一选择标准不要求所述接触具有高于所述第一强度阈值的特征强度以便满足所述选择标准，执行与所述第一用户界面元素对应的第一选择操作，而无需遮蔽所述应用程序的所述第一交互区域。Based on a determination that the first input satisfies a first selection criterion, wherein the first selection criterion does not require the contact to have a characteristic intensity above the first intensity threshold in order to satisfy the selection criterion, performing an operation with the first The first selection operation corresponding to the user interface element does not need to cover the first interaction area of the application program.18.根据权利要求17所述的计算机可读存储介质，其中隐藏所述应用程序的所述第一交互区域包括随着所述接触的所述特征强度改变而动态地遮蔽所述应用程序的所述第一交互区域。18. The computer-readable storage medium of claim 17 , wherein hiding the first interaction area of the application comprises dynamically obscuring all of the application's interaction areas as the characteristic strength of the contact changes. the first interaction area.19.根据权利要求17-18中任一项所述的计算机可读存储介质，包括当由所述电子设备执行时使得所述电子设备进行以下操作的指令：19. The computer-readable storage medium of any one of claims 17-18, comprising instructions that, when executed by the electronic device, cause the electronic device to:在显示所述用户界面时，在所述显示器上在与所述第二交互区域中的第二用户界面元素对应的位置处检测通过所述触敏表面上的第二接触导致的第二输入；detecting, while the user interface is displayed, a second input caused by a second contact on the touch-sensitive surface at a location on the display corresponding to a second user interface element in the second interaction area;响应于检测到通过所述第二接触导致的所述第二输入：In response to detecting the second input caused by the second contact:根据确定所述第二输入满足第二基于强度的激活标准，其中所述第二基于强度的激活标准要求所述第二接触具有高于第二强度阈值的特征强度以便满足所述第二基于强度的激活标准，除所述第二用户界面元素之外，遮蔽所述应用程序的所述第一交互区域和所述应用程序的所述第二交互区域；以及Based on determining that the second input satisfies a second intensity-based activation criterion, wherein the second intensity-based activation criterion requires the second contact to have a characteristic intensity above a second intensity threshold in order to satisfy the second intensity-based activation criterion activation criteria for, except for the second user interface element, obscuring the first interaction area of the application and the second interaction area of the application; and根据确定所述第二输入满足第二选择标准，其中所述第二选择标准不要求所述第二接触具有高于所述第二强度阈值的特征强度以便满足所述第二选择标准，执行与所述第二用户界面元素对应的第二选择操作，而无需遮蔽所述应用程序的所述第二交互区域并且无需遮蔽所述应用程序的所述第一交互区域。Based on a determination that the second input satisfies a second selection criterion, wherein the second selection criterion does not require the second contact to have a characteristic intensity above the second intensity threshold in order to satisfy the second selection criterion, performing an AND The second selection operation corresponding to the second user interface element does not need to cover the second interaction area of the application program and does not need to cover the first interaction area of the application program.20.根据权利要求19所述的计算机可读存储介质，其中在所述第一用户界面元素基于所述第一交互区域中的输入而被添加到所述第二交互区域之后，所述第二用户界面元素对应于所述第一用户界面元素。20. The computer-readable storage medium of claim 19 , wherein after the first user interface element is added to the second interactive area based on input in the first interactive area, the second A user interface element corresponds to said first user interface element.21.根据权利要求17-20中任一项所述的计算机可读存储介质，其中：21. The computer-readable storage medium of any one of claims 17-20, wherein:所述第二交互区域为包括会话参与者之间的会话中的多个消息的会话区域；并且The second interaction area is a conversation area that includes a plurality of messages in a conversation between conversation participants; and所述第一交互区域为用于选择媒体以在所述会话参与者之间的所述会话中共享的媒体选择区域。The first interaction area is a media selection area for selecting media to share in the session between the session participants.22.根据权利要求17-21中任一项所述的计算机可读存储介质，其中：22. The computer-readable storage medium of any one of claims 17-21, wherein:所述第二交互区域为用于主应用程序的用户界面；并且the second interaction area is a user interface for the main application; and所述第一交互区域被配置为显示来自被配置为在所述主应用程序内进行操作的不同微型应用程序的内容，并且在所述第一交互区域中所显示的所述微型应用程序基于所述设备处的用户输入而被选择。The first interactive area is configured to display content from different mini-apps configured to operate within the main application, and the mini-apps displayed in the first interactive area are based on the selected by user input at the device.23.根据权利要求17-22中任一项所述的计算机可读存储介质，包括当由所述电子设备执行时使得所述电子设备进行以下操作的指令：23. The computer-readable storage medium of any one of claims 17-22, comprising instructions that, when executed by the electronic device, cause the electronic device to:除所述第一用户界面元素之外，当遮蔽所述应用程序的所述第一交互区域而无需遮蔽所述应用程序的所述第二交互区域时，检测所述接触的所述特征强度的增大；以及In addition to the first user interface element, detecting the intensity of the characteristic of the contact when the first interaction area of the application is obscured without obscuring the second interaction area of the application increase; and响应于检测到所述接触的所述特征强度的所述增大：In response to detecting the increase in the characteristic strength of the contact:根据确定所述接触的所述特征强度满足第三选择标准，所述第三选择标准包括当所述接触的所述特征强度大于比所述第一强度阈值更高的选择强度阈值时满足的标准，对所述第一用户界面元素执行第三选择操作并且停止隐藏所述应用程序的所述第一交互区域；以及Based on determining that said characteristic intensity of said contact satisfies a third selection criterion, said third selection criterion comprising a criterion satisfied when said characteristic intensity of said contact is greater than a selection intensity threshold higher than said first intensity threshold , performing a third selection operation on the first user interface element and stopping hiding the first interaction area of the application; and根据确定所述输入的所述特征强度不满足所述第三选择标准，除所述第一用户界元素之外，继续遮蔽所述应用程序的所述第一交互区域，而无需遮蔽所述应用程序的所述第二交互区域。Continuing to shade the first interaction area of the application without shading the application upon determining that the characteristic strength of the input does not satisfy the third selection criterion, except for the first user interface element The second interactive area of the program.24.根据权利要求17-23中任一项所述的计算机可读存储介质，其中在所述第一输入为轻击手势时满足所述第一选择标准。24. The computer-readable storage medium of any one of claims 17-23, wherein the first selection criterion is satisfied when the first input is a tap gesture.25.一种电子设备，包括：25. An electronic device comprising:显示器；monitor;触敏表面；touch-sensitive surfaces;一个或多个传感器，所述一个或多个传感器被配置为检测与所述触敏表面的接触的强度；和one or more sensors configured to detect the intensity of contact with the touch-sensitive surface; and用于在所述显示器上显示包括应用程序的第一交互区域和所述应用程序的第二交互区域的用户界面的装置；means for displaying on said display a user interface comprising a first interactive area of an application and a second interactive area of said application;在显示所述用户界面时启用的用于在所述显示器上在与所述第一交互区域中的第一用户界面元素对应的位置处检测通过所述触敏表面上的接触导致的第一输入的装置；for detecting a first input caused by a contact on the touch-sensitive surface at a location on the display corresponding to a first user interface element in the first interaction area when the user interface is displayed s installation;响应于检测到通过所述接触导致的所述第一输入而启用的用于进行以下操作的装置：means for: actuated in response to detecting said first input by said contact:根据确定所述第一输入满足基于强度的激活标准，其中所述基于强度的激活标准要求所述接触具有高于第一强度阈值的特征强度以便满足所述基于强度的激活标准，除所述第一用户界面元素之外，遮蔽所述应用程序的所述第一交互区域，而无需遮蔽所述应用程序的所述第二交互区域；以及，Based on determining that the first input satisfies an intensity-based activation criterion, wherein the intensity-based activation criterion requires the contact to have a characteristic intensity above a first intensity threshold in order to satisfy the intensity-based activation criterion, except that the second A user interface element that obscures the first interaction area of the application without obscuring the second interaction area of the application; and,根据确定所述第一输入满足第一选择标准，其中所述第一选择标准不要求所述接触具有高于所述第一强度阈值的特征强度以便满足所述选择标准，执行与所述第一用户界面元素对应的第一选择操作，而无需遮蔽所述应用程序的所述第一交互区域。Based on a determination that the first input satisfies a first selection criterion, wherein the first selection criterion does not require the contact to have a characteristic intensity above the first intensity threshold in order to satisfy the selection criterion, performing an operation with the first The first selection operation corresponding to the user interface element does not need to cover the first interaction area of the application program.26.根据权利要求25所述的电子设备，其中用于遮蔽所述应用程序的所述第一交互区域的所述装置包括用于随着所述接触的所述特征强度改变而动态地遮蔽所述应用程序的所述第一交互区域的装置。26. The electronic device of claim 25 , wherein said means for shading said first interaction area of said application program comprises means for dynamically shading said first interaction area as said characteristic strength of said contact changes. means for said first interactive area of said application.27.根据权利要求25-26中任一项所述的电子设备，包括：27. The electronic device according to any one of claims 25-26, comprising:在显示所述用户界面时启用的用于在所述显示器上在与所述第二交互区域中的第二用户界面元素对应的位置处检测通过所述触敏表面上的第二接触导致的第二输入的装置；for detecting, at a location on the display corresponding to a second user interface element in the second interaction area, a first contact caused by a second contact on the touch-sensitive surface when the user interface is displayed. Two input devices;响应于检测到通过所述第二接触导致的所述第二输入而启动的用于进行以下操作的装置：means for: actuated in response to detecting said second input caused by said second contact:根据确定所述第二输入满足第二基于强度的激活标准，其中所述第二基于强度的激活标准要求所述第二接触具有高于第二强度阈值的特征强度以便满足所述第二基于强度的激活标准，除所述第二用户界面元素之外，遮蔽所述应用程序的所述第一交互区域和所述应用程序的所述第二交互区域；以及Based on determining that the second input satisfies a second intensity-based activation criterion, wherein the second intensity-based activation criterion requires the second contact to have a characteristic intensity above a second intensity threshold in order to satisfy the second intensity-based activation criterion activation criteria for, except for the second user interface element, obscuring the first interaction area of the application and the second interaction area of the application; and根据确定所述第二输入满足第二选择标准，其中所述第二选择标准不要求所述第二接触具有高于所述第二强度阈值的特征强度以便满足所述第二选择标准，执行与所述第二用户界面元素对应的第二选择操作，而无需遮蔽所述应用程序的所述第二交互区域并且无需遮蔽所述应用程序的所述第一交互区域。Based on a determination that the second input satisfies a second selection criterion, wherein the second selection criterion does not require the second contact to have a characteristic intensity above the second intensity threshold in order to satisfy the second selection criterion, performing an AND The second selection operation corresponding to the second user interface element does not need to cover the second interaction area of the application program and does not need to cover the first interaction area of the application program.28.根据权利要求27所述的电子设备，其中在所述第一用户界面元素基于所述第一交互区域中的输入而被添加到所述第二交互区域之后，28. The electronic device of claim 27, wherein after the first user interface element is added to the second interaction area based on an input in the first interaction area,所述第二用户界面元素对应于所述第一用户界面元素。The second user interface element corresponds to the first user interface element.29.根据权利要求25-28中任一项所述的电子设备，其中：29. The electronic device according to any one of claims 25-28, wherein:所述第二交互区域为包括会话参与者之间的会话中的多个消息的会话区域；并且The second interaction area is a conversation area that includes a plurality of messages in a conversation between conversation participants; and所述第一交互区域为用于选择媒体以在所述会话参与者之间的所述会话中共享的媒体选择区域。The first interaction area is a media selection area for selecting media to share in the session between the session participants.30.根据权利要求25-29中任一项所述的电子设备，其中：30. The electronic device according to any one of claims 25-29, wherein:所述第二交互区域为用于主应用程序的用户界面；并且the second interaction area is a user interface for the main application; and所述第一交互区域被配置为显示来自被配置为在所述主应用程序内进行操作的不同微型应用程序的内容，并且在所述第一交互区域中所显示的所述微型应用程序基于所述设备处的用户输入而被选择。The first interactive area is configured to display content from different mini-apps configured to operate within the main application, and the mini-apps displayed in the first interactive area are based on the selected by user input at the device.31.根据权利要求25-30中任一项所述的电子设备，包括：31. The electronic device according to any one of claims 25-30, comprising:除所述第一用户界面元素之外，当遮蔽所述应用程序的所述第一交互区域而无需遮蔽所述应用程序的所述第二交互区域时启用的用于检测所述接触的所述特征强度的增大的装置；和In addition to the first user interface element, the means for detecting the contact enabled when the first interaction area of the application is obscured without obscuring the second interaction area of the application means of increased characteristic strength; and响应于检测到所述接触的所述特征强度的所述增大而启用的用于进行以下操作的装置：means for, enabled in response to detecting said increase in said characteristic intensity of said contact:根据确定所述接触的所述特征强度满足第三选择标准，所述第三选择标准包括当所述接触的所述特征强度大于比所述第一强度阈值更高的选择强度阈值时满足的标准，对所述第一用户界面元素执行第三选择操作并且停止遮蔽所述应用程序的所述第一交互区域；以及Based on determining that said characteristic intensity of said contact satisfies a third selection criterion, said third selection criterion comprising a criterion satisfied when said characteristic intensity of said contact is greater than a selection intensity threshold higher than said first intensity threshold , performing a third selection operation on the first user interface element and stopping obscuring the first interaction area of the application; and根据确定所述输入的所述特征强度不满足所述第三选择标准，除所述第一用户界元素之外，继续遮蔽所述应用程序的所述第一交互区域，而无需遮蔽所述应用程序的所述第二交互区域。Continuing to shade the first interaction area of the application without shading the application upon determining that the characteristic strength of the input does not satisfy the third selection criterion, except for the first user interface element The second interactive area of the program.32.根据权利要求25-31中任一项所述的电子设备，其中在所述第一输入为轻击手势时满足所述第一选择标准。32. The electronic device of any one of claims 25-31, wherein the first selection criterion is satisfied when the first input is a tap gesture.33.一种电子设备，包括：33. An electronic device comprising:显示单元，所述显示单元被配置为显示用户界面；a display unit configured to display a user interface;触敏表面单元，所述触敏表面单元被配置为接收接触；a touch-sensitive surface unit configured to receive a contact;一个或多个传感器单元，所述一个或多个传感器单元被配置为检测与所述触敏表面单元的接触的强度；和one or more sensor units configured to detect the intensity of contact with the touch-sensitive surface unit; and与所述显示单元、所述触敏表面单元和所述一个或多个传感器单元耦接的处理单元，所述处理单元被配置为：a processing unit coupled to the display unit, the touch-sensitive surface unit and the one or more sensor units, the processing unit configured to:在所述显示单元上显示包括应用程序的第一交互区域和所述应用程序的第二交互区域的用户界面；displaying on the display unit a user interface including a first interaction area of an application and a second interaction area of the application;在显示所述用户界面时，在所述显示单元上在与所述第一交互区域中的第一用户界面元素对应的位置处检测通过所述触敏表面单元上的接触导致的第一输入；detecting a first input by contact on the touch-sensitive surface unit at a location on the display unit corresponding to a first user interface element in the first interaction area while the user interface is displayed;响应于检测到通过所述接触导致的所述第一输入：In response to detecting the first input caused by the contact:根据确定所述第一输入满足基于强度的激活标准，其中所述基于强度的激活标准要求所述接触具有高于第一强度阈值的特征强度以便满足所述基于强度的激活标准，除所述第一用户界面元素之外，遮蔽所述应用程序的所述第一交互区域，而无需遮蔽所述应用程序的所述第二交互区域；以及，Based on determining that the first input satisfies an intensity-based activation criterion, wherein the intensity-based activation criterion requires the contact to have a characteristic intensity above a first intensity threshold in order to satisfy the intensity-based activation criterion, except that the second A user interface element that obscures the first interaction area of the application without obscuring the second interaction area of the application; and,根据确定所述第一输入满足第一选择标准，其中所述第一选择标准不要求所述接触具有高于所述第一强度阈值的特征强度以便满足所述选择标准，执行与所述第一用户界面元素对应的第一选择操作，而无需遮蔽所述应用程序的所述第一交互区域。Based on a determination that the first input satisfies a first selection criterion, wherein the first selection criterion does not require the contact to have a characteristic intensity above the first intensity threshold in order to satisfy the selection criterion, performing an operation with the first The first selection operation corresponding to the user interface element does not need to cover the first interaction area of the application program.34.根据权利要求33所述的电子设备，其中遮蔽所述应用程序的所述第一交互区域包括随着所述接触的所述特征强度改变而动态地遮蔽所述应用程序的所述第一交互区域。34. The electronic device of claim 33 , wherein obscuring the first interaction area of the application comprises dynamically obscuring the first interaction area of the application as the characteristic strength of the contact changes. interactive area.35.根据权利要求33-34中任一项所述的电子设备，所述处理单元被进一步配置为：35. The electronic device according to any one of claims 33-34, the processing unit is further configured to:在显示所述用户界面时，在所述显示单元上在与所述第二交互区域中的第二用户界面元素对应的位置处检测通过所述触敏表面单元上的第二接触导致的第二输入；While the user interface is being displayed, a second contact caused by a second contact on the touch-sensitive surface unit is detected on the display unit at a location corresponding to a second user interface element in the second interaction area. enter;响应于检测到通过所述第二接触导致的所述第二输入：In response to detecting the second input caused by the second contact:根据确定所述第二输入满足第二基于强度的激活标准，其中所述第二基于强度的激活标准要求所述第二接触具有高于第二强度阈值的特征强度以便满足所述第二基于强度的激活标准，除所述第二用户界面元素之外，遮蔽所述应用程序的所述第一交互区域和所述应用程序的所述第二交互区域；以及Based on determining that the second input satisfies a second intensity-based activation criterion, wherein the second intensity-based activation criterion requires the second contact to have a characteristic intensity above a second intensity threshold in order to satisfy the second intensity-based activation criterion activation criteria for, except for the second user interface element, obscuring the first interaction area of the application and the second interaction area of the application; and根据确定所述第二输入满足第二选择标准，其中所述第二选择标准不要求所述第二接触具有高于所述第二强度阈值的特征强度以便满足所述第二选择标准，执行与所述第二用户界面元素对应的第二选择操作，而无需遮蔽所述应用程序的所述第二交互区域并且无需遮蔽所述应用程序的所述第一交互区域。Based on a determination that the second input satisfies a second selection criterion, wherein the second selection criterion does not require the second contact to have a characteristic intensity above the second intensity threshold in order to satisfy the second selection criterion, performing an AND The second selection operation corresponding to the second user interface element does not need to cover the second interaction area of the application program and does not need to cover the first interaction area of the application program.36.根据权利要求35所述的电子设备，其中在所述第一用户界面元素基于所述第一交互区域中的输入而被添加到所述第二交互区域之后，所述第二用户界面元素对应于所述第一用户界面元素。36. The electronic device of claim 35 , wherein after the first user interface element is added to the second interactive area based on input in the first interactive area, the second user interface element Corresponds to the first user interface element.37.根据权利要求33-36中任一项所述的电子设备，其中：37. The electronic device according to any one of claims 33-36, wherein:所述第二交互区域为包括会话参与者之间的会话中的多个消息的会话区域；并且The second interaction area is a conversation area that includes a plurality of messages in a conversation between conversation participants; and所述第一交互区域为用于选择媒体以在所述会话参与者之间的所述会话中共享的媒体选择区域。The first interaction area is a media selection area for selecting media to share in the session between the session participants.38.根据权利要求33-37中任一项所述的电子设备，其中：38. The electronic device according to any one of claims 33-37, wherein:所述第二交互区域为用于主应用程序的用户界面；并且the second interaction area is a user interface for the main application; and所述第一交互区域被配置为显示来自被配置为在所述主应用程序内进行操作的不同微型应用程序的内容，并且在所述第一交互区域中所显示的所述微型应用程序基于所述设备处的用户输入而被选择。The first interactive area is configured to display content from different mini-apps configured to operate within the main application, and the mini-apps displayed in the first interactive area are based on the selected by user input at the device.39.根据权利要求33-38中任一项所述的电子设备，其中所述处理单元被进一步配置为：39. The electronic device according to any one of claims 33-38, wherein the processing unit is further configured to:除所述第一用户界面元素之外，当遮蔽所述应用程序的所述第一交互区域而无需遮蔽所述应用程序的所述第二交互区域时，检测所述接触的所述特征强度的增大；以及In addition to the first user interface element, detecting the intensity of the characteristic of the contact when the first interaction area of the application is obscured without obscuring the second interaction area of the application increase; and响应于检测到所述接触的所述特征强度的所述增大：In response to detecting the increase in the characteristic strength of the contact:根据确定所述接触的所述特征强度满足第三选择标准，所述第三选择标准包括当所述接触的所述特征强度大于比所述第一强度阈值更高的选择强度阈值时满足的标准，对所述第一用户界面元素执行第三选择操作并且停止遮蔽所述应用程序的所述第一交互区域；以及Based on determining that said characteristic intensity of said contact satisfies a third selection criterion, said third selection criterion comprising a criterion satisfied when said characteristic intensity of said contact is greater than a selection intensity threshold higher than said first intensity threshold , performing a third selection operation on the first user interface element and stopping obscuring the first interaction area of the application; and根据确定所述输入的所述特征强度不满足所述第三选择标准，除所述第一用户界元素之外，继续遮蔽所述应用程序的所述第一交互区域，而无需遮蔽所述应用程序的所述第二交互区域。Continuing to shade the first interaction area of the application without shading the application upon determining that the characteristic strength of the input does not satisfy the third selection criterion, except for the first user interface element The second interactive area of the program.40.根据权利要求33-39中任一项所述的电子设备，其中在所述第一输入为轻击手势时满足所述第一选择标准。40. The electronic device of any one of claims 33-39, wherein the first selection criterion is satisfied when the first input is a tap gesture.